From martin at v.loewis.de Thu Jul 1 00:01:10 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 01 Jul 2010 00:01:10 +0200 Subject: [Python-Dev] OS X buildbots: why am I skipping these tests? In-Reply-To: References: <71728.1277866512@parc.com> <4C2AD511.5020709@v.loewis.de> <68821.1277913795@parc.com> <4C2BA0A4.9070002@v.loewis.de> Message-ID: <4C2BBEA6.9080303@v.loewis.de> >> When Tim Peters added it, he wanted it to tell him whether he did the >> Windows build correctly, INCLUDING ALL OPTIONAL PACKAGES that can >> possibly work on Windows. If you try to generalize this beyond Windows, >> then the only skips that are expected are the ones for tests that >> absolutely cannot work on the platform - i.e. Unix tests on Windows, >> and Windows tests on Unix. Otherwise, if you can get it to pass by >> installing additional software, Tim did *not* mean this to be an >> expected skip. > > Interesting. Do you use it that way when you make the Windows build? You want a honest answer? I usually don't run the test suite on Windows, and trust that the packaging will tell me if an extension module failed to build (and otherwise trust that if the setup worked for the release candidate, it will also work for the final release). Independent of that, I also decided to entirely ignore the notion of expected skipped test (so even if I would run the test suite, I wouldn't bother if one was reported as skipped). Regards, Martin From brett at python.org Thu Jul 1 00:06:21 2010 From: brett at python.org (Brett Cannon) Date: Wed, 30 Jun 2010 15:06:21 -0700 Subject: [Python-Dev] OS X buildbots: why am I skipping these tests? In-Reply-To: <4C2BBEA6.9080303@v.loewis.de> References: <71728.1277866512@parc.com> <4C2AD511.5020709@v.loewis.de> <68821.1277913795@parc.com> <4C2BA0A4.9070002@v.loewis.de> <4C2BBEA6.9080303@v.loewis.de> Message-ID: On Wed, Jun 30, 2010 at 15:01, "Martin v. L?wis" wrote: >>> When Tim Peters added it, he wanted it to tell him whether he did the >>> Windows build correctly, INCLUDING ALL OPTIONAL PACKAGES that can >>> possibly work on Windows. If you try to generalize this beyond Windows, >>> then the only skips that are expected are the ones for tests that >>> absolutely cannot work on the platform - i.e. Unix tests on Windows, >>> and Windows tests on Unix. Otherwise, if you can get it to pass by >>> installing additional software, Tim did *not* mean this to be an >>> expected skip. >> >> Interesting. Do you use it that way when you make the Windows build? > > You want a honest answer? Of course. You're amongst friends. =) > I usually don't run the test suite on Windows, > and trust that the packaging will tell me if an extension module failed > to build (and otherwise trust that if the setup worked for the release > candidate, it will also work for the final release). Seems reasonable. > > Independent of that, I also decided to entirely ignore the notion of > expected skipped test (so even if I would run the test suite, I wouldn't > bother if one was reported as skipped). Yeah, I remember you bringing this up years ago. I think that "unexpected" is a bad term and should be renamed to reflect the fact that the test was skipped because an optional third-party package was not included. Or simply output why a certain test was skipped at the end of the test suite run (I know it is outputted when the individual test is skipped, but it would be easier to read that info at the end during the summary of results). From barry at python.org Thu Jul 1 00:10:24 2010 From: barry at python.org (Barry Warsaw) Date: Wed, 30 Jun 2010 18:10:24 -0400 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <4C2BAAAC.5090101@egenix.com> References: <20100624115048.4fd152e3@heresy> <20100624170944.7e68ad21@heresy> <4C23D3C2.1060500@scottdial.com> <4C23DD99.9050604@egenix.com> <20100630150610.7ae4ac6a@heresy> <4C2BAAAC.5090101@egenix.com> Message-ID: <20100630181024.1e46b667@heresy> On Jun 30, 2010, at 10:35 PM, M.-A. Lemburg wrote: >The Python default installation dir for >libs (including site-packages) is $prefix/lib/pythonX.X, so you >already have separate and properly versioned directory paths. > >What difference would the extra version on the .so file make in >such a setup ? So on Debian (inherted by Ubuntu), that would be /usr/lib/pythonX.Y and in fact if you look there, you'll see the normal standard library layout you'd expect. There is no site-packages under there though because we install add-ons to dist-packages[*] but functionally it's what you'd expect. However, if you look inside dist-packages, you'll see something a little weird. All the .py files .egg-info files will actually be symlinks into a parallel hierarchy under /usr/share/pyshared. Also, under dist-packages you'll see a directory layout that looks like it normally would under a standard site-packages layout, except that again, all the .py files are symlinked to the pyshared location. The .so and .pyc files in dist-packages will be the actual .so and .pyc files. Why is that? Well, it's because the .py files can be shared but the .pyc and .so files can't. Getting rid of these crufty symlink farms was the initial motivation for PEP 3147. In a PEP 3147 world, pure-Python packages don't need the directory /usr/lib/pythonX.Y/dist-packages. We can just install packages to /usr/share/pyshared and none of the .pyc files will collide between Python versions. It makes distro packaging simpler (no symlinks to keep track of or post-install linking and post-removal clean up to do) and smaller (only one .py file for all supported Python versions). All we have to do is a post-install byte-compilation and we're done. When we remove such a package we only *have* to remove the .py source file because PEP 3147 will not load a __pycache__ pyc if the source is missing (though it's easy enough to clean up the pyc files too). Throw extension modules into the mix and we have the file collision problem again. We can't install _foo.so into /usr/share/pyshared because that's going to be Python ABI-specific. If we can ABI-tag the .so and ensure it will get imported, then we're back to the simpler layout where /usr/share/pyshared is the destination for all installed Python packages. -Barry [*] That's done as a compromise between Debian's interpretation of the FHS and the conflict of that interpretation to from-source installs of Python. Specifically, Debian puts /usr/local/lib/pythonX.Y/dist-packages on the *system* Python's path because that's where it expects system administrators to put their own add-ons to the system Python. That used to be /usr/local/lib/pythonX.Y/site-packages but then if you did a from-source altinstall for example of the same Python version, it would be possible for a "/usr/local/bin/pythonX.Y setup.py install" of a third-party package to break your system Python. Not good! (And yes, it happened to me :). -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From ncoghlan at gmail.com Thu Jul 1 00:20:16 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 1 Jul 2010 08:20:16 +1000 Subject: [Python-Dev] OS X buildbots: why am I skipping these tests? In-Reply-To: References: <71728.1277866512@parc.com> <4C2AD511.5020709@v.loewis.de> <68821.1277913795@parc.com> <4C2BA0A4.9070002@v.loewis.de> Message-ID: On Thu, Jul 1, 2010 at 7:55 AM, Brett Cannon wrote: > So it isn't that it's "unexpected", it's that a dependency is missing. > So it seems the terminology needs to get tweaked. More that the phrase "expected skip" isn't clearly defined and people sometimes guess wrong as to what it means. As Martin pointed out, there are two possible meanings: "will never work on this OS" and "won't work with just the base OS install". Currently, the "expected skip" list is based purely on the former, but developers occasionally interpret it as the latter (as Bill did in this case). I will note that the first list is much easier to keep up to date, since the latter may vary significantly based on vendor decisions as to what they install by default (a fairly significant factor in the Linux and *BSD worlds). Adding "(Were all optional modules built successfully?)" to the end of the "skips were unexpected" line in the regrtest output may be enough to eliminate the confusion. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From barry at python.org Thu Jul 1 00:22:46 2010 From: barry at python.org (Barry Warsaw) Date: Wed, 30 Jun 2010 18:22:46 -0400 Subject: [Python-Dev] OS X buildbots: why am I skipping these tests? In-Reply-To: References: <71728.1277866512@parc.com> <4C2AD511.5020709@v.loewis.de> <68821.1277913795@parc.com> <4C2BA0A4.9070002@v.loewis.de> Message-ID: <20100630182246.1ecdedde@heresy> On Jul 01, 2010, at 07:52 AM, Nick Coghlan wrote: >Note that it works this way on Linux as well. On Kubuntu (for example) >you need another half dozen or so additional *-dev packages installed >to avoid unexpected test skips. > >Cheers, >Nick. > >P.S. For anyone curious, I posted the list of extra packages you need >here: >http://boredomandlaziness.blogspot.com/2010/01/kubuntu-dev-packages-to-build-python.html As an aside, I wonder if it would make sense to create an Ubuntu/Debian package that essentially just depended on all those -dev packages. That way you could install this python-uber-dev package and get everything you needed to build a sumo Python. Hmm, /me goes to hack on his PPA. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From brett at python.org Thu Jul 1 00:45:35 2010 From: brett at python.org (Brett Cannon) Date: Wed, 30 Jun 2010 15:45:35 -0700 Subject: [Python-Dev] OS X buildbots: why am I skipping these tests? In-Reply-To: References: <71728.1277866512@parc.com> <4C2AD511.5020709@v.loewis.de> <68821.1277913795@parc.com> <4C2BA0A4.9070002@v.loewis.de> Message-ID: On Wed, Jun 30, 2010 at 15:20, Nick Coghlan wrote: > On Thu, Jul 1, 2010 at 7:55 AM, Brett Cannon wrote: >> So it isn't that it's "unexpected", it's that a dependency is missing. >> So it seems the terminology needs to get tweaked. > > More that the phrase "expected skip" isn't clearly defined and people > sometimes guess wrong as to what it means. As Martin pointed out, > there are two possible meanings: "will never work on this OS" and > "won't work with just the base OS install". Currently, the "expected > skip" list is based purely on the former, but developers occasionally > interpret it as the latter (as Bill did in this case). > > I will note that the first list is much easier to keep up to date, > since the latter may vary significantly based on vendor decisions as > to what they install by default (a fairly significant factor in the > Linux and *BSD worlds). > > Adding "(Were all optional modules built successfully?)" to the end of > the "skips were unexpected" line in the regrtest output may be enough > to eliminate the confusion. Probably. So I would still want to shift the test-specific info into the tests instead of regrtest and raise a subclass od SkippedTest (or whatever the name of the exception is) to signify that there is a difference. This would also do away with the possibility of having a test get silently skipped by an ImportError even though the module should definitely be available (didn't that bite you once, or was that someone else?). From steve at holdenweb.com Thu Jul 1 01:04:59 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 30 Jun 2010 19:04:59 -0400 Subject: [Python-Dev] OS X buildbots: why am I skipping these tests? In-Reply-To: References: <71728.1277866512@parc.com> <4C2AD511.5020709@v.loewis.de> <68821.1277913795@parc.com> <4C2BA0A4.9070002@v.loewis.de> <4C2BBEA6.9080303@v.loewis.de> Message-ID: Brett Cannon wrote: > On Wed, Jun 30, 2010 at 15:01, "Martin v. L?wis" > wrote: >>>> When Tim Peters added it, he wanted it to tell him whether he >>>> did the Windows build correctly, INCLUDING ALL OPTIONAL >>>> PACKAGES that can possibly work on Windows. If you try to >>>> generalize this beyond Windows, then the only skips that are >>>> expected are the ones for tests that absolutely cannot work on >>>> the platform - i.e. Unix tests on Windows, and Windows tests on >>>> Unix. Otherwise, if you can get it to pass by installing >>>> additional software, Tim did *not* mean this to be an expected >>>> skip. >>> Interesting. Do you use it that way when you make the Windows >>> build? >> You want a honest answer? > > Of course. You're amongst friends. =) > >> I usually don't run the test suite on Windows, and trust that the >> packaging will tell me if an extension module failed to build (and >> otherwise trust that if the setup worked for the release candidate, >> it will also work for the final release). > > Seems reasonable. > >> Independent of that, I also decided to entirely ignore the notion >> of expected skipped test (so even if I would run the test suite, I >> wouldn't bother if one was reported as skipped). > > Yeah, I remember you bringing this up years ago. I think that > "unexpected" is a bad term and should be renamed to reflect the fact > that the test was skipped because an optional third-party package was > not included. A "dependency skip", perhaps? > Or simply output why a certain test was skipped at the end of the > test suite run (I know it is outputted when the individual test is > skipped, but it would be easier to read that info at the end during > the summary of results). That would be a good idea, I think. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From daniel.buch at gmail.com Thu Jul 1 02:01:40 2010 From: daniel.buch at gmail.com (Dan Buch) Date: Wed, 30 Jun 2010 20:01:40 -0400 Subject: [Python-Dev] Taking over the Mercurial Migration In-Reply-To: References: <4C2AD80E.9010404@v.loewis.de> Message-ID: <20100701000137.GA5297@gmail.com> /me throws hat into ring. I'm in the middle of migrating fairly large chunks of an overgrown codebase from Subversion to Mercurial, so I might actually have worthwhile input :) -- ~Dan On Wed, Jun 30, 2010 at 10:41:51AM +0200, Georg Brandl wrote: > Am 30.06.2010 07:37, schrieb "Martin v. L?wis": > > It seems that both Dirkjan and Brett are very caught up > > with real life for the coming months. So I suggest that > > some other committer who favors the Mercurial transition > > steps forward and takes over this project. > > > > If nobody volunteers, I propose that we release 3.2 > > from Subversion, and reconsider Mercurial migration > > next year. > > IIUC, Dirkjan is only caught up for another month. I have > no problems with releasing a first 3.2 alpha from SVN and > then switching, so I propose that we target the migration > for August -- I can help in the second half of August if > needed. > > Georg > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/daniel.buch%40gmail.com -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 230 bytes Desc: not available URL: From cournape at gmail.com Thu Jul 1 04:11:55 2010 From: cournape at gmail.com (David Cournapeau) Date: Thu, 1 Jul 2010 11:11:55 +0900 Subject: [Python-Dev] How are the bdist_wininst binaries built ? Message-ID: Hi, I would like to modify the code of the bdist installers, but I don't see any VS project for VS 9.0. How are the wininst-9.0*exe built ? thanks, David From martin at v.loewis.de Thu Jul 1 06:22:01 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 01 Jul 2010 06:22:01 +0200 Subject: [Python-Dev] How are the bdist_wininst binaries built ? In-Reply-To: References: Message-ID: <4C2C17E9.4000906@v.loewis.de> > I would like to modify the code of the bdist installers, but I don't > see any VS project for VS 9.0. How are the wininst-9.0*exe built ? See PC/bdist_wininst. HTH, Martin From martin at v.loewis.de Thu Jul 1 06:37:22 2010 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 01 Jul 2010 06:37:22 +0200 Subject: [Python-Dev] Taking over the Mercurial Migration In-Reply-To: <20100701000137.GA5297@gmail.com> References: <4C2AD80E.9010404@v.loewis.de> <20100701000137.GA5297@gmail.com> Message-ID: <4C2C1B82.4010602@v.loewis.de> Am 01.07.2010 02:01, schrieb Dan Buch: > /me throws hat into ring. I'm in the middle of migrating fairly > large chunks of an overgrown codebase from Subversion to Mercurial, > so I might actually have worthwhile input :) To all the volunteers: an issue that apparently requires immediate attention is that r21112 fails to convert. For details, see http://mail.python.org/pipermail/python-committers/2010-June/000977.html So somebody please reproduce the problem, propose a patch, and get in contact with Dirkjan to integrate it. Regards, Martin From cournape at gmail.com Thu Jul 1 06:53:14 2010 From: cournape at gmail.com (David Cournapeau) Date: Thu, 1 Jul 2010 13:53:14 +0900 Subject: [Python-Dev] How are the bdist_wininst binaries built ? In-Reply-To: <4C2C17E9.4000906@v.loewis.de> References: <4C2C17E9.4000906@v.loewis.de> Message-ID: On Thu, Jul 1, 2010 at 1:22 PM, "Martin v. L?wis" wrote: >> I would like to modify the code of the bdist installers, but I don't >> see any VS project for VS 9.0. How are the wininst-9.0*exe built ? > > See PC/bdist_wininst. Hm, my question may not have been clear: *how* is the wininst-9.0 built from the bdist_wininst sources ? I see 6, 7.0, 7.1 and 8.0 versions of the visual studio build scripts, but nothing for VS 9.0. cheers, David From martin at v.loewis.de Thu Jul 1 07:00:42 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 01 Jul 2010 07:00:42 +0200 Subject: [Python-Dev] How are the bdist_wininst binaries built ? In-Reply-To: References: <4C2C17E9.4000906@v.loewis.de> Message-ID: <4C2C20FA.7090400@v.loewis.de> >> See PC/bdist_wininst. > > Hm, my question may not have been clear: *how* is the wininst-9.0 > built from the bdist_wininst sources ? I see 6, 7.0, 7.1 and 8.0 > versions of the visual studio build scripts, but nothing for VS 9.0. Ah. See PCbuild/bdist_wininst.vcproj. Regards, Martin From merwok at netwok.org Thu Jul 1 07:41:53 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Thu, 01 Jul 2010 07:41:53 +0200 Subject: [Python-Dev] OS X buildbots: why am I skipping these tests? In-Reply-To: <20100630182246.1ecdedde@heresy> References: <71728.1277866512@parc.com> <4C2AD511.5020709@v.loewis.de> <68821.1277913795@parc.com> <4C2BA0A4.9070002@v.loewis.de> <20100630182246.1ecdedde@heresy> Message-ID: <4C2C2AA1.705@netwok.org> [Barry Warsaw] > As an aside, I wonder if it would make sense to create an Ubuntu/Debian > package that essentially just depended on all those -dev packages. That way > you could install this python-uber-dev package and get everything you needed > to build a sumo Python. I don?t know. Any Debian advanced user wanting to build a package can just run ?aptitude build-dep python2.6?. Aptitude has the nice property of asking for a confirmation, so it?s even possible to cherry-pick from the lists of deps (press ?e? at the confirmation prompt). E.g., I choose readline but not tk. So it seems to me that the information and the command are already here. From anders.sandvig at gmail.com Thu Jul 1 10:33:30 2010 From: anders.sandvig at gmail.com (Anders Sandvig) Date: Thu, 1 Jul 2010 10:33:30 +0200 Subject: [Python-Dev] Regarding socket timeouts in httplib Message-ID: Consider the following code for retreieving a web page using httplib: ? ?def get_url(hostname, port, url, timeout=5): ? ? ? ?con = httplib.HTTPConnection(hostname, port, timeout) ? ? ? ?con.request("GET", url) ? ? ? ?res = con.getresponse() ? ? ? ?data = res.read() ? ? ? ?return res, data As expected, this will raise a socket.error if the client is unable to connect before the timeout has expired. However, once the connection has been made, the timeout parameter no longer has any effect and con.getresponse() will block forever if the server does not send any data. I think the reason for this is that the socket object created in HTTPConnection.connect() has a default timeout of 0 (i.e. it is never set explicitly): ? ?def connect(self): ? ? ? ?"""Connect to the host and port specified in __init__.""" ? ? ? ?self.sock = socket.create_connection((self.host,self.port), ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? self.timeout) For now I have been able to work around this by manually setting the timeout of the (private) socket object after calling con.request(), like this: ? ? ? ?... ? ? ? ?con.request("GET", url) ? ? ? ?con.sock.settimeout(timeout) ? ? ? ?res = con.getresponse() ? ? ? ?... However, I don't think this is a very good solution as it relies on knowledge about the inner workings of httplib (and I had to read the library source code to come up with it). >From the top of my head, I can come up with three (four) ways of properly solving the issue: 1) Documenting the timeout behavior and describing the above hack in the httplib documentation. 2) Modify HTTPConnection.connect() to set the timeout on the socket object after it has been ?created (using the same timeout as given on the HTTPConnection constructor). 3) Adding (optional) timeout parameters to HTTPConnection.getresponse() and HTTPResponse.read() (and possibly other functions with the same problem). 4) A combination of 2) and 3). Any thoughts on this? BTW: Once I figure out how, I wouldn't mind submitting a patch for either 2), 3) or 4), but personally I don't like 1). Anders From hodgestar+pythondev at gmail.com Thu Jul 1 10:51:48 2010 From: hodgestar+pythondev at gmail.com (Simon Cross) Date: Thu, 1 Jul 2010 10:51:48 +0200 Subject: [Python-Dev] Regarding socket timeouts in httplib In-Reply-To: References: Message-ID: On Thu, Jul 1, 2010 at 10:33 AM, Anders Sandvig wrote: > 2) Modify HTTPConnection.connect() to set the timeout on the socket > object after it has been ?created (using the same timeout as given on > the HTTPConnection constructor). It looks like urllib2 in trunk and urllib.request in py3k are also affected by this oddity. My vote is for option 2 -- i.e. consider it a bug that the timeout wasn't consistently applied. Schiavo Simon From fijall at gmail.com Thu Jul 1 12:02:31 2010 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 1 Jul 2010 04:02:31 -0600 Subject: [Python-Dev] [pypy-dev] PyPy 1.3 released In-Reply-To: References: Message-ID: On Wed, Jun 30, 2010 at 3:24 PM, Phyo Arkar wrote: > So far , python-mysql still not working.. > > Anyone had sucessfully got it work? Hey. I'm not aware of anyone who had any success. You can come to #pypy on irc.freenode.net and we can see how to solve the problem. > > On Fri, Jun 25, 2010 at 11:27 PM, Maciej Fijalkowski > wrote: >> >> ======================= >> PyPy 1.3: Stabilization >> ======================= >> >> Hello. >> >> We're please to announce release of PyPy 1.3. This release has two major >> improvements. First of all, we stabilized the JIT compiler since 1.2 >> release, >> answered user issues, fixed bugs, and generally improved speed. >> >> We're also pleased to announce alpha support for loading CPython extension >> modules written in C. While the main purpose of this release is increased >> stability, this feature is in alpha stage and it is not yet suited for >> production environments. >> >> Highlights of this release >> ========================== >> >> * We introduced support for CPython extension modules written in C. As of >> now, >> ?this support is in alpha, and it's very unlikely unaltered C extensions >> will >> ?work out of the box, due to missing functions or refcounting details. The >> ?support is disable by default, so you have to do:: >> >> ? import cpyext >> >> ?before trying to import any .so file. Also, libraries are >> source-compatible >> ?and not binary-compatible. That means you need to recompile binaries, >> using >> ?for example:: >> >> ? python setup.py build >> >> ?Details may vary, depending on your build system. Make sure you include >> ?the above line at the beginning of setup.py or put it in your >> PYTHONSTARTUP. >> >> ?This is alpha feature. It'll likely segfault. You have been warned! >> >> * JIT bugfixes. A lot of bugs reported for the JIT have been fixed, and >> its >> ?stability greatly improved since 1.2 release. >> >> * Various small improvements have been added to the JIT code, as well as a >> great >> ?speedup of compiling time. >> >> Cheers, >> Maciej Fijalkowski, Armin Rigo, Alex Gaynor, Amaury Forgeot d'Arc and >> the PyPy team >> _______________________________________________ >> pypy-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/pypy-dev > > From scott+python-dev at scottdial.com Thu Jul 1 13:02:25 2010 From: scott+python-dev at scottdial.com (Scott Dial) Date: Thu, 01 Jul 2010 07:02:25 -0400 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <20100630145329.736f2aab@heresy> References: <20100624115048.4fd152e3@heresy> <20100624170944.7e68ad21@heresy> <4C23D3C2.1060500@scottdial.com> <4C246E81.3020302@scottdial.com> <4C265DC6.4080600@ubuntu.com> <4C268433.30405@scottdial.com> <20100630145329.736f2aab@heresy> Message-ID: <4C2C75C1.8020708@scottdial.com> On 6/30/2010 2:53 PM, Barry Warsaw wrote: > It might be amazing, but it's still a significant overhead. As I've > described, multiply that by all the py files in all the distro packages > containing Python source code, and then still try to fit it on a CDROM. I decided to prove to myself that it was not a significant issue to have parallel directory structures in a .tar.bz2, and I was surprised to find it much worse at that then I had imagined. For example, # cd /usr/lib/python2.6/site-packages # tar --exclude="*.pyc" --exclude="*.pyo" \ -cjf mercurial.tar.bz2 mercurial # du -h mercurial.tar.bz2 640K mercurial.tar.bz2 # cp -a mercurial mercurial2 # tar --exclude="*.pyc" --exclude="*.pyo" \ -cjf mercurial2.tar.bz2 mercurial mercurial2 # du -h mercurial.tar.bz2 1.3M mercurial2.tar.bz2 So, I was definitely wrong in saying that you do better than doubling. >> What happens to the distro packaging if a python package splits the >> codebase between 2.x and 3.x (meaning they have distinct .py files)? > > The Debian/Ubuntu approach to Python 2/3 support is to provide them in > separate distro packages. E.g. for Python package foo, you would have Debuntu > package python-foo (for the Python 2.x version) and python3-foo. We do not > share source between Python 2 and 3 versions, at least not yet . I had asked this question to point out that you will still need to accommodate some form of version-specific packages (I am not a debuntu expert by any means). And, I think your response is an acknowledgment of that fact, however it's certainly true that there are few examples, as you said. I appreciate all your replies. I am not sure a PEP is really needed here, but to having had all of this discussed and explained on the mailing list is certainly useful. I trust that yourself and the debuntu python group will end up chasing down and taking care of any quirks that this change might cause, so I am not worried about it. :D -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From daniel.buch at gmail.com Thu Jul 1 14:09:28 2010 From: daniel.buch at gmail.com (Dan Buch) Date: Thu, 1 Jul 2010 08:09:28 -0400 Subject: [Python-Dev] Taking over the Mercurial Migration In-Reply-To: <4C2C1B82.4010602@v.loewis.de> References: <4C2AD80E.9010404@v.loewis.de> <20100701000137.GA5297@gmail.com> <4C2C1B82.4010602@v.loewis.de> Message-ID: <20100701120926.GA6460@gmail.com> On Thu, Jul 01, 2010 at 06:37:22AM +0200, "Martin v. L?wis" wrote: > Am 01.07.2010 02:01, schrieb Dan Buch: > > /me throws hat into ring. I'm in the middle of migrating fairly > > large chunks of an overgrown codebase from Subversion to Mercurial, > > so I might actually have worthwhile input :) > > To all the volunteers: an issue that apparently requires immediate > attention is that r21112 fails to convert. For details, see > > http://mail.python.org/pipermail/python-committers/2010-June/000977.html > > So somebody please reproduce the problem, propose a patch, and get in > contact with Dirkjan to integrate it. Does anybody know if there's already an issue tracking the failure so that volunteers can better reproduce the issue? Is a full checkout of /projects/python via hgsubversion all that's required, perhaps? -- ~Dan > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/daniel.buch%40gmail.com -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 230 bytes Desc: not available URL: From rdmurray at bitdance.com Thu Jul 1 14:16:41 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 01 Jul 2010 08:16:41 -0400 Subject: [Python-Dev] Regarding socket timeouts in httplib In-Reply-To: References: Message-ID: <20100701121641.40F031FE314@kimball.webabinitio.net> On Thu, 01 Jul 2010 10:33:30 +0200, Anders Sandvig wrote: > >From the top of my head, I can come up with three (four) ways of > properly solving the issue: > > 1) Documenting the timeout behavior and describing the above hack in > the httplib documentation. > > 2) Modify HTTPConnection.connect() to set the timeout on the socket > object after it has been created (using the same timeout as given on > the HTTPConnection constructor). > > 3) Adding (optional) timeout parameters to > HTTPConnection.getresponse() and HTTPResponse.read() (and possibly > other functions with the same problem). > > 4) A combination of 2) and 3). > > Any thoughts on this? > > BTW: Once I figure out how, I wouldn't mind submitting a patch for > either 2), 3) or 4), but personally I don't like 1). FYI there's an open bug about this (or at least related to it): http://bugs.python.org/issue8595 -- R. David Murray www.bitdance.com From dirkjan at ochtman.nl Thu Jul 1 14:19:06 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Thu, 1 Jul 2010 14:19:06 +0200 Subject: [Python-Dev] Taking over the Mercurial Migration In-Reply-To: <20100701120926.GA6460@gmail.com> References: <4C2AD80E.9010404@v.loewis.de> <20100701000137.GA5297@gmail.com> <4C2C1B82.4010602@v.loewis.de> <20100701120926.GA6460@gmail.com> Message-ID: On Thu, Jul 1, 2010 at 14:09, Dan Buch wrote: > Does anybody know if there's already an issue tracking the failure > so that volunteers can better reproduce the issue? ?Is a full checkout > of /projects/python via hgsubversion all that's required, perhaps? I work from a full svnsync of the projects repo (including python). I can probably put up a tarball somewhere. Cheers, Dirkjan From techtonik at gmail.com Thu Jul 1 14:52:24 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Thu, 1 Jul 2010 15:52:24 +0300 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) Message-ID: After reading PEP 384 and PEP 385 (finally) I got a strong impression that they are not ready for the change (read below the line for details), because they do not propose any workflow. So, instead of rushing with migration I'd like to propose incremental change rather than revolutionary that will allow people to taste what it like to work with Mercurial and update all relevant papers on the website. I propose to setup workflow that will include pulling changes to Mercurial master clone from current Subversion repository and pushing them back to Subversion using Mercurial. This will be similar to http://mercurial.selenic.com/wiki/DeveloperRepos scheme, but with hg-stable replaced with svn. I am willing to collaborate for getting this system in place before EuroPython even if I won't find a chance to get there. --- the line --- I didn' t have time to follow Mercurial discussion, so I expected PEP 384 and PEP 385 to be exhaustive reference on both the migration and workflow before and after the switch. Unfortunately, both are too vague in this respect and my opinion that migration now will do more harm than good. Primary concern is that will happen with central Subversion repository. There are a plenty of private tools and automated scripts that were written to work with central Subversion repository, so it is important that central Subversion repository continue to operate and be synchronized. However, PEP 385 "The Future of Subversion" [1] does not propose any way to keep Subversion repository in sync. "it will probably live on for a bit" phrase looks like Subversion will be immediately dead after switching. The second concern is workflow. Even after getting my hands dirty I still can't find the best solution for working with Mercurial copy of Python. In Subversion I do shallow checkouts of interesting features when I run into one bug or another. Not all patches are ready, so they stay in those checkouts as uncommitted diffs. To be prepared for conflicts I review code with `svn log -vr base:head` before updating. But with Mercurial I see two major problems with my workflow (which I am unlikely to change for the next few weeks due to heavy automation): 1. No shallow checkouts - that means that I have to copy 200Mb+ for every patch I have 2. No `hg log` command to see changes in my branch from remote repository before fetching them (I used to invoke `hg incoming`, but ended up with fetching everything with `hg pull` and then using `hg log -vr .:tip`, because `hg inc` didn't show changes that were already pulled, but not updated in wc, and `hg log` could not show incoming) 3. There is no clear branch separation in my working copy. I don't know how to diff files in different branches without copying 200Mb+ clone. I don't know how to diff my files with remote repository without pulling it. 4. Even if I make patch in my Mercurial clone - you still can't pull it and I have to attach it to tracker. No gain. So from these PEPs I can't really see how Mercurial solves problems with development. I can feel that it doesn't solve my problems, but I see no way where I can submit my user story. I would put accent on keeping mirror of Subversion as easy way to contribute for those who are not yet ready for DVCS. Subversion also provides greater interoperability. Assuming that any modern DVCS tool may act as Subversion client, we will gain more contributors if we won't try to force people use Python and Mercurial. [1] http://www.python.org/dev/peps/pep-0385/#the-future-of-subversion -- anatoly t. From daniel.buch at gmail.com Thu Jul 1 15:23:58 2010 From: daniel.buch at gmail.com (Dan Buch) Date: Thu, 1 Jul 2010 09:23:58 -0400 Subject: [Python-Dev] Taking over the Mercurial Migration In-Reply-To: References: <4C2AD80E.9010404@v.loewis.de> <20100701000137.GA5297@gmail.com> <4C2C1B82.4010602@v.loewis.de> <20100701120926.GA6460@gmail.com> Message-ID: <20100701132357.GB4351@gmail.com> Dirkjan, I assume having such a tarball available would be a good thing, but what do I know!? :) Are your steps for reproducing the referenced problem with cvs2svn-generated revs available on an issue, wiki page or PEP? -- ~Dan On Thu, Jul 01, 2010 at 02:19:06PM +0200, Dirkjan Ochtman wrote: > On Thu, Jul 1, 2010 at 14:09, Dan Buch wrote: > > Does anybody know if there's already an issue tracking the failure > > so that volunteers can better reproduce the issue? ?Is a full checkout > > of /projects/python via hgsubversion all that's required, perhaps? > > I work from a full svnsync of the projects repo (including python). I > can probably put up a tarball somewhere. > > Cheers, > > Dirkjan > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/daniel.buch%40gmail.com -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 230 bytes Desc: not available URL: From dirkjan at ochtman.nl Thu Jul 1 15:51:06 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Thu, 1 Jul 2010 15:51:06 +0200 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: On Thu, Jul 1, 2010 at 14:52, anatoly techtonik wrote: > Primary concern is that will happen with central Subversion > repository. There are a plenty of private tools and automated scripts > that were written to work with central Subversion repository, so it is > important that central Subversion repository continue to operate and > be synchronized. > > However, PEP 385 "The Future of Subversion" [1] does not propose any > way to keep Subversion repository in sync. "it will probably live on > for a bit" phrase looks like Subversion will be immediately dead after > switching. It would certainly be possible to have snapshots of the hg repo be committed into SVN, but I think people will actually notice sooner that it's dead if we don't update it at all anymore. > The second concern is workflow. Even after getting my hands dirty I > still can't find the best solution for working with Mercurial copy of > Python. Mercurial is relatively flexible in that there are lots of ways you can do this, and so I find that, certainly for the local parts of the workflow (e.g. those that don't directly affect the repo that everyone sees), I don't want to mandate a certain workflow. On the other hand, there will certainly be documentation on a very basic workflow to get people started. Inevitably, in order to get the most out of the tool, people will have to investigate some different ways of doing things (e.g., mq, which is one of my favorite workflows -- I'd probably include a basic description of it in the docs for python-dev). Cheers, Dirkjan From dirkjan at ochtman.nl Thu Jul 1 16:14:16 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Thu, 1 Jul 2010 16:14:16 +0200 Subject: [Python-Dev] Taking over the Mercurial Migration In-Reply-To: <20100701132357.GB4351@gmail.com> References: <4C2AD80E.9010404@v.loewis.de> <20100701000137.GA5297@gmail.com> <4C2C1B82.4010602@v.loewis.de> <20100701120926.GA6460@gmail.com> <20100701132357.GB4351@gmail.com> Message-ID: On Thu, Jul 1, 2010 at 15:23, Dan Buch wrote: > I assume having such a tarball available would be a good thing, but what > do I know!? :) I'm putting one up. I'll email you the address privately in order to preserve some bandwidth. Anyone else who wants a copy: just email me. > Are your steps for reproducing the referenced problem with > cvs2svn-generated revs available on an issue, wiki page or PEP? Install hgsubversion (tip) and get the pymigr repo, then just run this: $ hg clone -A pymigr/author-map --branchmap pymigr/branchmap file:///path/to/svn-repo/python Cheers, Dirkjan From daniel.buch at gmail.com Thu Jul 1 16:22:53 2010 From: daniel.buch at gmail.com (Dan Buch) Date: Thu, 1 Jul 2010 10:22:53 -0400 Subject: [Python-Dev] Taking over the Mercurial Migration In-Reply-To: References: <4C2AD80E.9010404@v.loewis.de> <20100701000137.GA5297@gmail.com> <4C2C1B82.4010602@v.loewis.de> <20100701120926.GA6460@gmail.com> <20100701132357.GB4351@gmail.com> Message-ID: <20100701142252.GD4351@gmail.com> Excellent! Much thanks, Dirkjan. -- ~Dan On Thu, Jul 01, 2010 at 04:14:16PM +0200, Dirkjan Ochtman wrote: > On Thu, Jul 1, 2010 at 15:23, Dan Buch wrote: > > I assume having such a tarball available would be a good thing, but what > > do I know!? :) > > I'm putting one up. I'll email you the address privately in order to > preserve some bandwidth. Anyone else who wants a copy: just email me. > > > Are your steps for reproducing the referenced problem with > > cvs2svn-generated revs available on an issue, wiki page or PEP? > > Install hgsubversion (tip) and get the pymigr repo, then just run this: > > $ hg clone -A pymigr/author-map --branchmap pymigr/branchmap > file:///path/to/svn-repo/python > > Cheers, > > Dirkjan -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 230 bytes Desc: not available URL: From daniel at stutzbachenterprises.com Thu Jul 1 16:31:13 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Thu, 1 Jul 2010 09:31:13 -0500 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: On Thu, Jul 1, 2010 at 7:52 AM, anatoly techtonik wrote: > 4. Even if I make patch in my Mercurial clone - you still can't pull > it and I have to attach it to tracker. No gain. > Was there ever any discussion about hosting the central repository on a site such as bitbucket or github? I tried searching the python-dev archives but was unable to find much. Anyway... assuming there's a at least a clone of the central repository on one of those sites, you can fork it and work on your patch there. A core developer can then pull your patch to their local repository, tweak it as needed, then commit it to the central repository. > I would put accent on keeping mirror of Subversion as easy way to > contribute for those who are not yet ready for DVCS. Subversion also > provides greater interoperability. Assuming that any modern DVCS tool > may act as Subversion client, we will gain more contributors if we > won't try to force people use Python and Mercurial. > The hg-git tool allows Mercurial and git to interoperate, so that's not as much of an issue as it once was. It's geared toward using a Mercurial client to talk to a git server, but I'm told it can work the other way around with a bit of work. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdmurray at bitdance.com Thu Jul 1 16:31:29 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 01 Jul 2010 10:31:29 -0400 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: <20100701143129.E63FE219BAA@kimball.webabinitio.net> On Thu, 01 Jul 2010 15:51:06 +0200, Dirkjan Ochtman wrote: > On Thu, Jul 1, 2010 at 14:52, anatoly techtonik wrote: > > Primary concern is that will happen with central Subversion > > repository. There are a plenty of private tools and automated scripts > > that were written to work with central Subversion repository, so it is > > important that central Subversion repository continue to operate and > > be synchronized. > > > > However, PEP 385 "The Future of Subversion" [1] does not propose any > > way to keep Subversion repository in sync. "it will probably live on > > for a bit" phrase looks like Subversion will be immediately dead after > > switching. > > It would certainly be possible to have snapshots of the hg repo be > committed into SVN, but I think people will actually notice sooner > that it's dead if we don't update it at all anymore. I remember the decision as being that SVN would be read-only after the switch. Which will cause some pain, yes, but probably not as much pain as trying to keep both workflows available would. > > The second concern is workflow. Even after getting my hands dirty I > > still can't find the best solution for working with Mercurial copy of > > Python. > > Mercurial is relatively flexible in that there are lots of ways you > can do this, and so I find that, certainly for the local parts of the > workflow (e.g. those that don't directly affect the repo that everyone > sees), I don't want to mandate a certain workflow. On the other hand, And this is the reason that Martin says that a *working* (final configuration) repo must be available for some time before the transition takes place, so that we can all experiment and figure out what external workflow is needed, and document what people find to be useful internal workflows. I think someone needs to say, "OK, what's up is how it is going to be, you've got X weeks to make sure you can work with it and then we cut over" before the balance of the dev community is going to actually try working with it. But one or more volunteers could make the first cut at the workflow docs right now, I would think, and that would be a big help for this process I know I would have preferred to have such docs before I gave it a try...but I need to be able to talk intelligently about HG at PyOhio, so I need to start learning it this week. -- R. David Murray www.bitdance.com From dirkjan at ochtman.nl Thu Jul 1 16:38:50 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Thu, 1 Jul 2010 16:38:50 +0200 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: On Thu, Jul 1, 2010 at 16:31, Daniel Stutzbach wrote: > The hg-git tool allows Mercurial and git to interoperate, so that's not as > much of an issue as it once was. ?It's geared toward using a Mercurial > client to talk to a git server, but I'm told it can work the other way > around with a bit of work. There's also heechee, which could be interesting for those who really love SVN. http://bitbucket.org/andrewgodwin/heechee Cheers, Dirkjan From dirkjan at ochtman.nl Thu Jul 1 16:39:40 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Thu, 1 Jul 2010 16:39:40 +0200 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: On Thu, Jul 1, 2010 at 16:31, Daniel Stutzbach wrote: > Was there ever any discussion about hosting the central repository on a site > such as bitbucket or github? ?I tried searching the python-dev archives but > was unable to find much. > Anyway... assuming there's a at least a clone of the central repository on > one of those sites, you can fork it and work on your patch there. ?A core > developer can then pull your patch to their local repository, tweak it as > needed, then commit it to the central repository. I'm pretty sure bitbucket will do a mirror once hg is up and running. Cheers, Dirkjan From ctb at msu.edu Thu Jul 1 16:49:35 2010 From: ctb at msu.edu (C. Titus Brown) Date: Thu, 1 Jul 2010 07:49:35 -0700 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: <20100701144935.GB5727@idyll.org> On Thu, Jul 01, 2010 at 09:31:13AM -0500, Daniel Stutzbach wrote: > On Thu, Jul 1, 2010 at 7:52 AM, anatoly techtonik wrote: > > > 4. Even if I make patch in my Mercurial clone - you still can't pull > > it and I have to attach it to tracker. No gain. > > > > Was there ever any discussion about hosting the central repository on a site > such as bitbucket or github? I tried searching the python-dev archives but > was unable to find much. We discussed this briefly at the Python language summit at PyCon '10, the discussion intersected and/or merged with the whole infrastructure discussion, got borg-ified by that, and I think discussions continue(d) on pydotorg-www in general. Personally I think it makes a great deal of sense to pay someone to manage this for us, but there were concerns about customizability and control that warrant further investigation. Since it's DVCS, it's much easier to change the "official" repo, but it still incurs costs in terms of documentation, workflow changes, etc. cheers, --titus -- C. Titus Brown, ctb at msu.edu From jnoller at gmail.com Thu Jul 1 17:16:35 2010 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 1 Jul 2010 11:16:35 -0400 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: On Thu, Jul 1, 2010 at 8:52 AM, anatoly techtonik wrote: > After reading PEP 384 and PEP 385 (finally) I got a strong impression > that they are not ready for the change (read below the line for > details), because they do not propose any workflow. So, instead of > rushing with migration I'd like to propose incremental change rather > than revolutionary that will allow people to taste what it like to > work with Mercurial and update all relevant papers on the website. > This migration is far from "rushed". Workflow will need to be documented and we need a working hg setup a little while before the official migration. Both of those said, I personally think this has dragged on long enough. jesse From cournape at gmail.com Thu Jul 1 17:44:51 2010 From: cournape at gmail.com (David Cournapeau) Date: Fri, 2 Jul 2010 00:44:51 +0900 Subject: [Python-Dev] How are the bdist_wininst binaries built ? In-Reply-To: <4C2C20FA.7090400@v.loewis.de> References: <4C2C17E9.4000906@v.loewis.de> <4C2C20FA.7090400@v.loewis.de> Message-ID: On Thu, Jul 1, 2010 at 2:00 PM, "Martin v. L?wis" wrote: >>> See PC/bdist_wininst. >> >> Hm, my question may not have been clear: *how* is the wininst-9.0 >> built from the bdist_wininst sources ? I see 6, 7.0, 7.1 and 8.0 >> versions of the visual studio build scripts, but nothing for VS 9.0. > > Ah. See PCbuild/bdist_wininst.vcproj. I thought I checked there, but I obviously missed it. thanks, David From brett at python.org Thu Jul 1 21:58:46 2010 From: brett at python.org (Brett Cannon) Date: Thu, 1 Jul 2010 12:58:46 -0700 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: <20100701143129.E63FE219BAA@kimball.webabinitio.net> References: <20100701143129.E63FE219BAA@kimball.webabinitio.net> Message-ID: On Thu, Jul 1, 2010 at 07:31, R. David Murray wrote: > On Thu, 01 Jul 2010 15:51:06 +0200, Dirkjan Ochtman wrote: >> On Thu, Jul 1, 2010 at 14:52, anatoly techtonik wrote: >> > Primary concern is that will happen with central Subversion >> > repository. There are a plenty of private tools and automated scripts >> > that were written to work with central Subversion repository, so it is >> > important that central Subversion repository continue to operate and >> > be synchronized. >> > >> > However, PEP 385 "The Future of Subversion" [1] does not propose any >> > way to keep Subversion repository in sync. "it will probably live on >> > for a bit" phrase looks like Subversion will be immediately dead after >> > switching. >> >> It would certainly be possible to have snapshots of the hg repo be >> committed into SVN, but I think people will actually notice sooner >> that it's dead if we don't update it at all anymore. > > I remember the decision as being that SVN would be read-only after the > switch. ?Which will cause some pain, yes, but probably not as much > pain as trying to keep both workflows available would. > >> > The second concern is workflow. Even after getting my hands dirty I >> > still can't find the best solution for working with Mercurial copy of >> > Python. >> >> Mercurial is relatively flexible in that there are lots of ways you >> can do this, and so I find that, certainly for the local parts of the >> workflow (e.g. those that don't directly affect the repo that everyone >> sees), I don't want to mandate a certain workflow. On the other hand, > > And this is the reason that Martin says that a *working* (final > configuration) repo must be available for some time before the transition > takes place, so that we can all experiment and figure out what external > workflow is needed, and document what people find to be useful internal > workflows. > > I think someone needs to say, "OK, what's up is how it is going to be, > you've got X weeks to make sure you can work with it and then we cut > over" before the balance of the dev community is going to actually try > working with it. ?But one or more volunteers could make the first cut > at the workflow docs right now, I would think, and that would be a big > help for this process > > I know I would have preferred to have such docs before I gave it a > try...but I need to be able to talk intelligently about HG at PyOhio, > so I need to start learning it this week. Here is a *really* quick-and-dirty approach for non-committers to create a patch they can submit. This is not extensively tested so some other Hg expert should back me up on this before telling anyone that this is the simplest way. I am also not saying this is how we will want people to contribute in the long run, but this does work and matches how svn does things well enough that people shouldn't get thrown by the details. 1. Contributor clones the repo 2. Contributor makes changes, committing as they go 3. Contributor runs ``hg outgoing --patch --git > patch.diff`` 4. Committer runs ``hg patch --no-commit patch.diff`` 5. Committer does the usual review->commit thing Basically this creates git-style diffs that one can shuttle around. I think you can also use ``patch -p1`` or git-apply to apply the patch generated by Mercurial. The other option is using mq (Mercurial Queues), but I personally have not gotten around to learning how to use them so I can't tell you any details. But if the Hg transition is taking too long for you, mq can be used on top of svn w/o issue: http://mercurial.selenic.com/wiki/WorkingWithSubversion#With_MQ_only . This obviously does not cover how committers should backport. That will very between whether it is within 3.x or 3.x to 2.x (or vice-versa). That will require a bit more thought and discussion. And finally, http://hginit.com/ is one of the best Hg intros I have come across for explaining fundamentals. From merwok at netwok.org Thu Jul 1 22:59:24 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Thu, 01 Jul 2010 22:59:24 +0200 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: <20100701143129.E63FE219BAA@kimball.webabinitio.net> Message-ID: <4C2D01AC.5030909@netwok.org> The workflow described by Brett looks like a patch-and-email one. It?s used by the Mercurial team itself, and has the advantage of not cluttering the history with lots of changes, since you only apply one patch in the end. However, in the not-so-long run I feel that a simpler and more mercurialic workflow would be a changesets-and-repos one. 1. Contributor clones the repo. 2. Contributor makes changes, committing as needed. 3. Contributor pushes to some public location and asks for pulling. 4. Core dev reviews and pulls from that repo into the official one. (optional step) Core dev may make additional edits (e.g. to NEWS and ACKS) in another changeset. One advantage of DVCS is that it separates the step of getting ideas from brain to code (commit) from the step of approving code and putting it into the official repo (pull). Having both the contributor and the core dev doing commits seems counter-natural to me. Advanced users may use MQ or pbranches or bookmarks to refine patches, but basic usage (commit and pull) still seems worthwhile to me. > And finally, http://hginit.com/ is one of the best Hg intros I have > come across for explaining fundamentals. Agreed, although I?m not sure whether I?d call it a long introduction or a short tuto :) The ?reeducation for Subversion users? page is helpful. hgtip.com has bite-sized tips for daily use, and http://hgbook.red-bean.com/ is an excellent read. Regards From doug.hellmann at gmail.com Thu Jul 1 23:25:23 2010 From: doug.hellmann at gmail.com (Doug Hellmann) Date: Thu, 1 Jul 2010 17:25:23 -0400 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: <36B1DF58-223A-4ED7-B0C3-CFB5CE36D17E@gmail.com> On Jul 1, 2010, at 10:31 AM, Daniel Stutzbach wrote: > On Thu, Jul 1, 2010 at 7:52 AM, anatoly techtonik > wrote: > 4. Even if I make patch in my Mercurial clone - you still can't pull > it and I have to attach it to tracker. No gain. > > Was there ever any discussion about hosting the central repository > on a site such as bitbucket or github? I tried searching the python- > dev archives but was unable to find much. I just set up a PSF account on BitBucket for hosting the Pulse newsletter repository. I can assist with setting up a public repository for the source under that account, too. Doug -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Thu Jul 1 23:57:34 2010 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 1 Jul 2010 22:57:34 +0100 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: <20100701143129.E63FE219BAA@kimball.webabinitio.net> Message-ID: On 1 July 2010 20:58, Brett Cannon wrote: > Here is a *really* quick-and-dirty approach for non-committers to > create a patch they can submit. This is not extensively tested so some > other Hg expert should back me up on this before telling anyone that > this is the simplest way. I am also not saying this is how we will > want people to contribute in the long run, but this does work and > matches how svn does things well enough that people shouldn't get > thrown by the details. > > 1. Contributor clones the repo > 2. Contributor makes changes, committing as they go > 3. Contributor runs ``hg outgoing --patch --git > patch.diff`` > 4. Committer runs ``hg patch --no-commit patch.diff`` > 5. Committer does the usual review->commit thing > > Basically this creates git-style diffs that one can shuttle around. I > think you can also use ``patch -p1`` or git-apply to apply the patch > generated by Mercurial. I'd suggest the patchbomb extension (distributed with Mercurial) hg email --outgoing --to dest at somewhere sends a series of patches to the given email address. This is what the Mercurial developers use (with the to address being the mercurial-dev list). Or maybe better, hg email --outgoing --bundle which sends a binary bundle of all outgoing changesets. You can use --to to send the email to something like roundup (will Roundup extract an attachment from an email and add it to the issue as a file? That would be particularly neat...) Paul. From barry at python.org Fri Jul 2 00:07:45 2010 From: barry at python.org (Barry Warsaw) Date: Thu, 1 Jul 2010 18:07:45 -0400 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: <20100701143129.E63FE219BAA@kimball.webabinitio.net> Message-ID: <20100701180745.264898de@heresy> On Jul 01, 2010, at 10:57 PM, Paul Moore wrote: >On 1 July 2010 20:58, Brett Cannon wrote: >> Here is a *really* quick-and-dirty approach for non-committers to >> create a patch they can submit. This is not extensively tested so >> some other Hg expert should back me up on this before telling anyone >> that this is the simplest way. I am also not saying this is how we >> will want people to contribute in the long run, but this does work >> and matches how svn does things well enough that people shouldn't get >> thrown by the details. >> >> 1. Contributor clones the repo >> 2. Contributor makes changes, committing as they go >> 3. Contributor runs ``hg outgoing --patch --git > patch.diff`` >> 4. Committer runs ``hg patch --no-commit patch.diff`` >> 5. Committer does the usual review->commit thing >> >> Basically this creates git-style diffs that one can shuttle around. I >> think you can also use ``patch -p1`` or git-apply to apply the patch >> generated by Mercurial. > >I'd suggest the patchbomb extension (distributed with Mercurial) > >hg email --outgoing --to dest at somewhere sends a series of patches to >the given email address. This is what the Mercurial developers use >(with the to address being the mercurial-dev list). Or maybe better, >hg email --outgoing --bundle which sends a binary bundle of all >outgoing changesets. You can use --to to send the email to something >like roundup (will Roundup extract an attachment from an email and add >it to the issue as a file? That would be particularly neat...) Wouldn't it be cool if we could hook this up to Rietveld? Other than that, while I sometimes review patches in email, I do not think patches in a tracker are the best way to manage these. A dvcs's biggest strength is in branches, so we should use those as much as possible. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From martin at v.loewis.de Fri Jul 2 00:08:45 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 02 Jul 2010 00:08:45 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <20100701143129.E63FE219BAA@kimball.webabinitio.net> Message-ID: <4C2D11ED.6020802@v.loewis.de> Am 01.07.2010 23:57, schrieb Paul Moore: > will Roundup extract an attachment from an email and add > it to the issue as a file? That would be particularly neat... It actually does. Feel free to try it out on the life system (i.e. not worrying about bogus issues - we have several test issues alread). Regards, Martin From brett at python.org Fri Jul 2 00:26:12 2010 From: brett at python.org (Brett Cannon) Date: Thu, 1 Jul 2010 15:26:12 -0700 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: <20100701180745.264898de@heresy> References: <20100701143129.E63FE219BAA@kimball.webabinitio.net> <20100701180745.264898de@heresy> Message-ID: On Thu, Jul 1, 2010 at 15:07, Barry Warsaw wrote: > On Jul 01, 2010, at 10:57 PM, Paul Moore wrote: > >>On 1 July 2010 20:58, Brett Cannon wrote: >>> Here is a *really* quick-and-dirty approach for non-committers to >>> create a patch they can submit. This is not extensively tested so >>> some other Hg expert should back me up on this before telling anyone >>> that this is the simplest way. I am also not saying this is how we >>> will want people to contribute in the long run, but this does work >>> and matches how svn does things well enough that people shouldn't get >>> thrown by the details. >>> >>> 1. Contributor clones the repo >>> 2. Contributor makes changes, committing as they go >>> 3. Contributor runs ``hg outgoing --patch --git > patch.diff`` >>> 4. Committer runs ``hg patch --no-commit patch.diff`` >>> 5. Committer does the usual review->commit thing >>> >>> Basically this creates git-style diffs that one can shuttle around. I >>> think you can also use ``patch -p1`` or git-apply to apply the patch >>> generated by Mercurial. >> >>I'd suggest the patchbomb extension (distributed with Mercurial) >> >>hg email --outgoing --to dest at somewhere sends a series of patches to >>the given email address. This is what the Mercurial developers use >>(with the to address being the mercurial-dev list). Or maybe better, >>hg email --outgoing --bundle which sends a binary bundle of all >>outgoing changesets. You can use --to to send the email to something >>like roundup (will Roundup extract an attachment from an email and add >>it to the issue as a file? That would be particularly neat...) > > Wouldn't it be cool if we could hook this up to Rietveld? Don't see why not; it already has Hg support, IIRC. > > Other than that, while I sometimes review patches in email, I do not think > patches in a tracker are the best way to manage these. ?A dvcs's biggest > strength is in branches, so we should use those as much as possible. As I said, quick-and-dirty. It will take discussion to decide what we want to ask non-committers to do, weighing cost of complexity of the approach for people new to DVCS in general and Hg in particular, resource access such as a publicly accessible place to push repos, what data format we prefer, etc. I just wanted to give David something to go to PyOhio with that I know will be simple to explain and will always work. From solipsis at pitrou.net Fri Jul 2 01:05:13 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 2 Jul 2010 01:05:13 +0200 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) References: <20100701143129.E63FE219BAA@kimball.webabinitio.net> <20100701180745.264898de@heresy> Message-ID: <20100702010513.6d106e48@pitrou.net> On Thu, 1 Jul 2010 15:26:12 -0700 Brett Cannon wrote: > > As I said, quick-and-dirty. It will take discussion to decide what we > want to ask non-committers to do, I don't think we have to ask them to do anything special, as long as they can submit their contributions under the form of a patch. Whether they use named branches, separate clones, mercurial queues, the pbranch extension, or even the basic hg-as-an-svn-synonym workflow you suggested should be none of our business, IMO. A DVCS allows non-committers to manage (augment, fix, synchronize) their patches more easily until they get committed. I don't think it will change the committers' work a lot, though. I don't know how Mercurial could make the task of committing an outsider's patch significantly simpler. Regards Antoine. From timothy.c.delaney at gmail.com Fri Jul 2 01:08:13 2010 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Fri, 2 Jul 2010 09:08:13 +1000 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: <20100701180745.264898de@heresy> References: <20100701143129.E63FE219BAA@kimball.webabinitio.net> <20100701180745.264898de@heresy> Message-ID: On 2 July 2010 08:07, Barry Warsaw wrote: > > Other than that, while I sometimes review patches in email, I do not think > patches in a tracker are the best way to manage these. A dvcs's biggest > strength is in branches, so we should use those as much as possible. > > I changed my team over from ClearCase to Mercurial in the last 3 months (just before we were made redundant ... :) Obviously our usage was coloured by our previous use of ClearCase, but the workflow I found that worked best was along the lines of the following. 1. Central server for the team, with a repository for each version (e.g. 2.6, 2.7, 3.2) where the default branch is the "trunk". Later versions get cloned from a previous version at a particular tag, and merges between versions are always forwards (if there's a need to merge backwards, we cherry-pick the change by doing an hg export of the appropriate versions). 2. Developer made at least one clone of the appropriate repo, but could have as many as they liked. Generally would have one local repo for production work, and others for prototyping, exploratory work, etc - stuff that you may not want in the central repo. 3. Each change is developed in its own named branch, named for the task number. Local commits are performed as often as possible - no requirement that any particular commit was compilable so long as it was on the task's named branch. 4. Changesets are pushed to the central repository (requires forcing, as Hg doesn't like you pushing new named branches). The named branch should be compilable, pass unit tests, etc at this point. Generally preferred not to edit the history or anything, but if there was something egregious 5. Other developers pulled from the central repository and reviewed (integration with ReviewBoard or Reitveld was planned, but ran out of time). This often led to a pair programming session on the reviewers machine where comments were addressed directly. 6. Named branch was closed and merged to the main/trunk branch (in our case, default). In our case this was usually done by the author - for Python this would be done by a committer. 7. Merged default branch was pushed to the central repo, which triggered a continuous build. This approach is quite contrary to that favoured by the Mercurial developers, but we found it had the following advantages: a. Central team repo made backup and continuous build simple. b. Named branches for tasks made the whole "what do I do with these unfinished changes?" question moot - you commit them to your named branch. c. Switching between tasks is incredibly simple - you already have your workspace set up (e.g. Eclipse) and just update to the appropriate named branch. No need to set up multiple workspaces. d. Similarly, switching to someone else's named branch is just as easy (for review, pair programming, etc). e. Named branches make it very obvious what needs to be merged for any particular task. f. Easier to follow in the log/graphical log as everything was "tagged" with what task it was against. The only issue was that if a branch was abandoned, it would show up in "hg branches" unless it was closed - but closing wasn't a problem since you can always reopen if necessary. Tim Delaney -------------- next part -------------- An HTML attachment was scrubbed... URL: From timothy.c.delaney at gmail.com Fri Jul 2 01:10:52 2010 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Fri, 2 Jul 2010 09:10:52 +1000 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: <20100701143129.E63FE219BAA@kimball.webabinitio.net> <20100701180745.264898de@heresy> Message-ID: On 2 July 2010 09:08, Tim Delaney wrote: > On 2 July 2010 08:07, Barry Warsaw wrote: > >> >> Other than that, while I sometimes review patches in email, I do not think >> patches in a tracker are the best way to manage these. A dvcs's biggest >> strength is in branches, so we should use those as much as possible. >> >> > 7. Merged default branch was pushed to the central repo, which triggered a > continuous build. > > Clarification here - I mean that a committer would merge it to default, then pull it into the main python repo - there would be one that anyone could push changes to, and those named branches would then be cherry-picked to be merged and pulled into the main repo by committers. Or something along those lines. Tim Delaney -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Fri Jul 2 01:52:43 2010 From: barry at python.org (Barry Warsaw) Date: Thu, 1 Jul 2010 19:52:43 -0400 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: <20100701143129.E63FE219BAA@kimball.webabinitio.net> <20100701180745.264898de@heresy> Message-ID: <20100701195243.352756ea@heresy> On Jul 01, 2010, at 03:26 PM, Brett Cannon wrote: >On Thu, Jul 1, 2010 at 15:07, Barry Warsaw wrote: >> Other than that, while I sometimes review patches in email, I do not >> think patches in a tracker are the best way to manage these. ?A >> dvcs's biggest strength is in branches, so we should use those as >> much as possible. > >As I said, quick-and-dirty. It will take discussion to decide what we >want to ask non-committers to do, weighing cost of complexity of the >approach for people new to DVCS in general and Hg in particular, >resource access such as a publicly accessible place to push repos, >what data format we prefer, etc. I just wanted to give David something >to go to PyOhio with that I know will be simple to explain and will >always work. That's cool. I've had pretty good luck asking contributors on other project to create branches and merge proposals (a Launchpad feature that triggers code reviews), even when they first posted patches to a tracker or mailing list. I'm confident most of our non-committers will find it pretty easy, especially if we can provide a guide for the basics. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From stephen at xemacs.org Fri Jul 2 02:08:02 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 02 Jul 2010 09:08:02 +0900 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: <874ogibva5.fsf@uwakimon.sk.tsukuba.ac.jp> anatoly techtonik writes: > After reading PEP 384 and PEP 385 (finally) I got a strong impression > that they are not ready for the change (read below the line for > details), because they do not propose any workflow. This was deliberate. There are a lot of different workflows, and we are not in a position to tell people to change them. > So, instead of rushing with migration We are hardly rushing. You're just late to the party. Catch your breathe, get a beer, have a good time. No? > I'd like to propose incremental change rather than revolutionary > that will allow people to taste what it like to work with Mercurial > and update all relevant papers on the website. Been there, done that, if that's the way it's gonna be, I won't touch it. It's much too painful for those who are doing the work, compared to the small benefits received by the community as a whole. > I am willing to collaborate for getting this system in place before > EuroPython even if I won't find a chance to get there. Are you also willing to maintain the system indefinitely? I haven't yet seen such a system that really works without regular attention from a human. > However, PEP 385 "The Future of Subversion" [1] does not propose any > way to keep Subversion repository in sync. "it will probably live on > for a bit" phrase looks like Subversion will be immediately dead after > switching. Yes, that is the intent. For old versions, the Subversion repo will be available read-only. For casual contributors, there's a one-time set up cost of pulling a local mirror from the public repository. For those familiar with Mercurial, there is nothing but joy to be found in the transition. For folks like you, heavily invested in Subversion-based private automation, yes, there will be some pain. > The second concern is workflow. Even after getting my hands dirty I > still can't find the best solution for working with Mercurial copy of > Python. In Subversion I do shallow checkouts of interesting features > when I run into one bug or another. Not all patches are ready, so they > stay in those checkouts as uncommitted diffs. For this kind of workflow, Mercurial queues (mq) is probably the best solution. Unfortunately, it will require you to write some additional scripts to get the same level of convenience you have now. > To be prepared for conflicts I review code with `svn log -vr > base:head` before updating. But with Mercurial I see two major > problems with my workflow (which I am unlikely to change for the > next few weeks due to heavy automation): > 1. No shallow checkouts - that means that I have to copy 200Mb+ for > every patch I have No, you don't. You make links to 200MB+ (unless you're on Windows, where I don't know how this works). This is much cheaper than copying, though not as cheap as in git. I don't hesitate to make branches in Mercurial. Alternatively, for localized bugfixes, doc typos, and the like, one "little-fixes" branch with an mq patch or two for each is a reasonable way to work. > 2. No `hg log` command to see changes in my branch from remote > repository before fetching them > (I used to invoke `hg incoming`, but ended up with fetching everything > with `hg pull` and then using `hg log -vr .:tip`, because `hg inc` > didn't show changes that were already pulled, but not updated in wc, > and `hg log` could not show incoming) > 3. There is no clear branch separation in my working copy. I don't > know how to diff files in different branches without copying 200Mb+ > clone. I don't know how to diff my files with remote repository > without pulling it. If you're at all serious about working with Mercurial, you don't do any operations except pull (and push, if you're a committer) against the remote. You keep a local pristine mirror of the remote repository, which you update frequently, and all of your work revolves around that rather than around contact with upstream. This does mean that when upstream uses a repo-per-branch model you have to keep multiple mirrors. On the other hand, you can work directly in branches that you work on only infrequently, and also use Mercurial queues to avoid making branches if you don't want to. > 4. Even if I make patch in my Mercurial clone - you still can't pull > it and I have to attach it to tracker. No gain. No loss, either, since that's the current workflow, and maintaining as much of the current workflow (possibly with "new spellings" of the commands implementing it) was an explicit goal for PEP 374. I assume that carries over to PEP 385. If you and some others working on a common project want to work with branches rather than patches (email or tracker), then if there's a committer in the group, you can set up a branch on code.python.org. If not, you can use bitbucket or something. Yes, it's a bit of extra effort for you. > So from these PEPs I can't really see how Mercurial solves problems > with development. I can feel that it doesn't solve my problems, but I > see no way where I can submit my user story. It doesn't solve problems with development, and that is not the intent of the PEPs AIUI. IMO, the theme of these PEPs is to maintain current practices and gives a platform for improved practices in the future. > I would put accent on keeping mirror of Subversion as easy way to > contribute for those who are not yet ready for DVCS. "Accent" won't cut it. Effort is needed. That effort won't be mine, I don't know how anybody else likely to be involved feels about it. And in fact, Mercurial makes it quite a bit *easier* for *casual* contributors to work on the code base. *All* of my work based on projects using Subversion (or CVS) is done from colocated DVCS branches. The only difference from my basic DVCS workflow is (1) the synch to upstream command is "svn up" rather than "hg pull -u", and (2) those changes must be committed to the DVCS (automated by script). It also is clearly going to make more effective workflows for many of the core developers. AFAIK, assuming the issues that have been raised in PEP 385 and the tracker are resolved, other core developers agree that the transition will have an acceptably low impact on them. Admittedly, it seems this transition is going to affect you adversely. That's unfortunate, but AFAIK you are in a small minority. Note that this is my current understanding of the situation. As the transition proceeds (and we are still very much in a planning phase, where anything could be changed), it may become clear that active developers in your situation are more numerous than I (we?) think. Also, any suggestions you make, including maintaining the Svn repository, will be taken seriously ... but many of them, specifically the decision to freeze the Svn repo once we cut over, have already been thought through carefully. Doesn't mean that they can't be changed, but you do have to accept that you're going to need to provide evidence that changing the decisions will benefit the community as a whole. From daniel.buch at gmail.com Fri Jul 2 03:34:01 2010 From: daniel.buch at gmail.com (Dan Buch) Date: Thu, 1 Jul 2010 21:34:01 -0400 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <20100701180745.264898de@heresy> References: <20100701143129.E63FE219BAA@kimball.webabinitio.net> <20100701180745.264898de@heresy> Message-ID: <20100702013349.GB10165@gmail.com> > Wouldn't it be cool if we could hook this up to Rietveld? I can also highly recommend ReviewBoard_ since it already speaks Mercurial. My company has been using it since Dec. 2009 as an invaluable addition to in-person code reviews for both Subversion and Mercurial repos. FWIW, I've got an instance at this address for anybody who wants to try it out: http://reviews.meatballhat.com Cheers, -- ~ Dan .. _ReviewBoard: http://www.reviewboard.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 230 bytes Desc: not available URL: From steve at pearwood.info Fri Jul 2 06:28:47 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 2 Jul 2010 14:28:47 +1000 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? Message-ID: <201007021428.48162.steve@pearwood.info> This question was inspired by something asked on #python today. Consider it a hypothetical, not a serious proposal. We know that many semantic errors in Python lead to runtime errors, e.g. 1 + "1". If an implementation rejected them at compile time, would it still be Python? E.g. if the keyhole optimizer raised SyntaxError (or some other exception) on seeing this: def f(): return 1 + "1" instead of compiling something which can't fail to raise an exception, would that still be a legal Python implementation? -- Steven D'Aprano From martin at v.loewis.de Fri Jul 2 06:42:31 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 02 Jul 2010 06:42:31 +0200 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: <201007021428.48162.steve@pearwood.info> References: <201007021428.48162.steve@pearwood.info> Message-ID: <4C2D6E37.8080209@v.loewis.de> > We know that many semantic errors in Python lead to runtime errors, e.g. > 1 + "1". If an implementation rejected them at compile time, would it > still be Python? E.g. if the keyhole optimizer raised SyntaxError (or > some other exception) on seeing this: > > def f(): > return 1 + "1" > > instead of compiling something which can't fail to raise an exception, > would that still be a legal Python implementation? I'd say "no". Any syntactically correct module should start executing, and type errors are only a runtime concept. If you were to reject code at startup more restrictively, you might end up rejecting the standard library, as it contains syntax errors in code that isn't being imported normally (test/badsyntax*). Regards, Martin From glyph at twistedmatrix.com Fri Jul 2 06:43:54 2010 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Fri, 2 Jul 2010 00:43:54 -0400 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: <201007021428.48162.steve@pearwood.info> References: <201007021428.48162.steve@pearwood.info> Message-ID: <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> On Jul 2, 2010, at 12:28 AM, Steven D'Aprano wrote: > This question was inspired by something asked on #python today. Consider > it a hypothetical, not a serious proposal. > > We know that many semantic errors in Python lead to runtime errors, e.g. > 1 + "1". If an implementation rejected them at compile time, would it > still be Python? E.g. if the keyhole optimizer raised SyntaxError (or > some other exception) on seeing this: > > def f(): > return 1 + "1" > > instead of compiling something which can't fail to raise an exception, > would that still be a legal Python implementation? I'd say "no". Python has defined semantics in this situation: a TypeError is raised. To me, this seems akin to a keyhole optimizer arbitrarily deciding that raise TypeError() should cause the compiler to abort. If this type of expression were common, it would be within the rights of, for example, a Python JIT to generate a fast path through 'f' that wouldn't bother to actually invoke its 'int' type's '__add__' method, since there is no possible way for a Python program to tell the difference, since int.__add__ is immutable. From stefan_ml at behnel.de Fri Jul 2 08:31:21 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 02 Jul 2010 08:31:21 +0200 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: Glyph Lefkowitz, 02.07.2010 06:43: > On Jul 2, 2010, at 12:28 AM, Steven D'Aprano wrote: > >> This question was inspired by something asked on #python today. Consider >> it a hypothetical, not a serious proposal. >> >> We know that many semantic errors in Python lead to runtime errors, e.g. >> 1 + "1". If an implementation rejected them at compile time, would it >> still be Python? E.g. if the keyhole optimizer raised SyntaxError (or >> some other exception) on seeing this: >> >> def f(): >> return 1 + "1" >> >> instead of compiling something which can't fail to raise an exception, >> would that still be a legal Python implementation? > > I'd say "no". Python has defined semantics in this situation: a TypeError is raised. So, would it still be Python if it folded 1 + "1" into raise TypeError() at compile time? Stefan From fijall at gmail.com Fri Jul 2 08:38:11 2010 From: fijall at gmail.com (Maciej Fijalkowski) Date: Fri, 2 Jul 2010 00:38:11 -0600 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: On Fri, Jul 2, 2010 at 12:31 AM, Stefan Behnel wrote: > Glyph Lefkowitz, 02.07.2010 06:43: >> >> On Jul 2, 2010, at 12:28 AM, Steven D'Aprano wrote: >> >>> This question was inspired by something asked on #python today. Consider >>> it a hypothetical, not a serious proposal. >>> >>> We know that many semantic errors in Python lead to runtime errors, e.g. >>> 1 + "1". If an implementation rejected them at compile time, would it >>> still be Python? E.g. if the keyhole optimizer raised SyntaxError (or >>> some other exception) on seeing this: >>> >>> def f(): >>> ? ?return 1 + "1" >>> >>> instead of compiling something which can't fail to raise an exception, >>> would that still be a legal Python implementation? >> >> I'd say "no". ?Python has defined semantics in this situation: a TypeError >> is raised. > > So, would it still be Python if it folded > > ? ?1 + "1" > > into > > ? ?raise TypeError() > > at compile time? > > Stefan > This question has an easy answer - can you possibly tell the difference? Cheers, fijal From craigcitro at gmail.com Fri Jul 2 08:55:10 2010 From: craigcitro at gmail.com (Craig Citro) Date: Thu, 1 Jul 2010 23:55:10 -0700 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: > This question has an easy answer - can you possibly tell the difference? > Ok, I'm obviously being silly here, but sure you can: >>> dis.dis("raise TypeError()") 0 <114> 26977 3 <115> 8293 6 IMPORT_STAR 7 SETUP_EXCEPT 25968 (to 25978) 10 <69> 11 <114> 28530 14 <114> 10536 >>> dis.dis("1 + '1'") 0 <49> 1 SLICE+2 2 STORE_SLICE+3 3 SLICE+2 4 <39> 5 <49> 6 <39> That said, I agree with the point you're making -- they have the same semantics, so you should be fine substituting one for the other. Honestly, though, I'd come down on the side of letting the compiler raise an error -- while I understand that it means you have *different* behavior, I think it's *preferable* behavior. -cc From martin at v.loewis.de Fri Jul 2 09:20:18 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 02 Jul 2010 09:20:18 +0200 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: <4C2D9332.9010002@v.loewis.de> Am 02.07.2010 08:55, schrieb Craig Citro: >> This question has an easy answer - can you possibly tell the difference? >> > > Ok, I'm obviously being silly here, but sure you can: The dis module is deliberately (*) not part of the Python language and standard library; it's an implementation detail (as is the func_code attribute, and the code object). So the question really is: can you tell the difference, using only mechanisms not explicitly documented as implementation-specific? Regards, Martin (*) Unfortunately, the documentation fails to mention that, probably because it's too obvious. From dickinsm at gmail.com Fri Jul 2 09:22:43 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 2 Jul 2010 08:22:43 +0100 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: On Fri, Jul 2, 2010 at 7:55 AM, Craig Citro wrote: > > Ok, I'm obviously being silly here, but sure you can: > >>>> dis.dis("raise TypeError()") > ? ? ? ? ?0 <114> ? ? ? ? ? 26977 > ? ? ? ? ?3 <115> ? ? ? ? ? ?8293 > ? ? ? ? ?6 IMPORT_STAR > ? ? ? ? ?7 SETUP_EXCEPT ? ?25968 (to 25978) > ? ? ? ? 10 <69> > ? ? ? ? 11 <114> ? ? ? ? ? 28530 > ? ? ? ? 14 <114> ? ? ? ? ? 10536 >>>> dis.dis("1 + '1'") > ? ? ? ? ?0 <49> > ? ? ? ? ?1 SLICE+2 > ? ? ? ? ?2 STORE_SLICE+3 > ? ? ? ? ?3 SLICE+2 > ? ? ? ? ?4 <39> > ? ? ? ? ?5 <49> > ? ? ? ? ?6 <39> Whoa. That's very peculiar looking bytecode. Is dis.dis behaving as it should here? BTW, I think you want 'raise TypeError', not 'raise TypeError()'. Mark From techtonik at gmail.com Fri Jul 2 09:27:13 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Fri, 2 Jul 2010 10:27:13 +0300 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: On Thu, Jul 1, 2010 at 6:16 PM, Jesse Noller wrote: > > This migration is far from "rushed". Workflow will need to be > documented and we need a working hg setup a little while before the > official migration. Both of those said, I personally think this has > dragged on long enough. So, if I understand correctly - there are no Mercurial mirrors for testing at the moment, so the current step is to create these mirrors to try releasing 3.2 from them. Development will continue in SVN repository until everybody is ready for final migration in X weeks later. Is that right? -- anatoly t. From dickinsm at gmail.com Fri Jul 2 09:29:05 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 2 Jul 2010 08:29:05 +0100 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: On Fri, Jul 2, 2010 at 8:22 AM, Mark Dickinson wrote: > On Fri, Jul 2, 2010 at 7:55 AM, Craig Citro wrote: >> >>>>> dis.dis("raise TypeError()") >> ? ? ? ? ?0 <114> ? ? ? ? ? 26977 >> ? ? ? ? ?3 <115> ? ? ? ? ? ?8293 >> ? ? ? ? ?6 IMPORT_STAR >> ? ? ? ? ?7 SETUP_EXCEPT ? ?25968 (to 25978) >> ? ? ? ? 10 <69> >> ? ? ? ? 11 <114> ? ? ? ? ? 28530 >> ? ? ? ? 14 <114> ? ? ? ? ? 10536 >>>>> dis.dis("1 + '1'") >> ? ? ? ? ?0 <49> >> ? ? ? ? ?1 SLICE+2 >> ? ? ? ? ?2 STORE_SLICE+3 >> ? ? ? ? ?3 SLICE+2 >> ? ? ? ? ?4 <39> >> ? ? ? ? ?5 <49> >> ? ? ? ? ?6 <39> > > Whoa. ?That's very peculiar looking bytecode. ?Is dis.dis behaving as > it should here? Ah. I see. It looks like the string "raise TypeError()" is being interpreted *directly* as Python bytecode, with no intermediate compilation. I don't think this is what you intended. Try: >>> dis.dis(compile("raise TypeError", "", "exec")) 1 0 LOAD_NAME 0 (TypeError) 3 RAISE_VARARGS 1 6 LOAD_CONST 0 (None) 9 RETURN_VALUE >>> dis.dis(compile("1 + '1'", "", "exec")) 1 0 LOAD_CONST 0 (1) 3 LOAD_CONST 1 ('1') 6 BINARY_ADD 7 POP_TOP 8 LOAD_CONST 2 (None) 11 RETURN_VALUE Mark From dirkjan at ochtman.nl Fri Jul 2 09:25:37 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Fri, 2 Jul 2010 09:25:37 +0200 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: <874ogibva5.fsf@uwakimon.sk.tsukuba.ac.jp> References: <874ogibva5.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Fri, Jul 2, 2010 at 02:08, Stephen J. Turnbull wrote: > No, you don't. ?You make links to 200MB+ (unless you're on Windows, > where I don't know how this works). ?This is much cheaper than > copying, though not as cheap as in git. ?I don't hesitate to make > branches in Mercurial. It can still do hardlinks on Windows, provided the repo is on NTFS (and I think NTFS supports only hardlinks within the same partition). Cheers, Dirkjan From mail at timgolden.me.uk Fri Jul 2 09:40:47 2010 From: mail at timgolden.me.uk (Tim Golden) Date: Fri, 02 Jul 2010 08:40:47 +0100 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <874ogibva5.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4C2D97FF.6070400@timgolden.me.uk> On 02/07/2010 08:25, Dirkjan Ochtman wrote: > On Fri, Jul 2, 2010 at 02:08, Stephen J. Turnbull wrote: >> No, you don't. You make links to 200MB+ (unless you're on Windows, >> where I don't know how this works). This is much cheaper than >> copying, though not as cheap as in git. I don't hesitate to make >> branches in Mercurial. > > It can still do hardlinks on Windows, provided the repo is on NTFS > (and I think NTFS supports only hardlinks within the same partition). Strictly: on the same volume. (Which will generally but not always amount to: under the same drive letter) TJG From g.brandl at gmx.net Fri Jul 2 10:30:18 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 02 Jul 2010 10:30:18 +0200 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: <36B1DF58-223A-4ED7-B0C3-CFB5CE36D17E@gmail.com> References: <36B1DF58-223A-4ED7-B0C3-CFB5CE36D17E@gmail.com> Message-ID: Am 01.07.2010 23:25, schrieb Doug Hellmann: > > On Jul 1, 2010, at 10:31 AM, Daniel Stutzbach wrote: > >> On Thu, Jul 1, 2010 at 7:52 AM, anatoly techtonik > > wrote: >> >> 4. Even if I make patch in my Mercurial clone - you still can't pull >> it and I have to attach it to tracker. No gain. >> >> >> Was there ever any discussion about hosting the central repository on >> a site such as bitbucket or github? I tried searching the python-dev >> archives but was unable to find much. > > I just set up a PSF account on BitBucket for hosting the Pulse > newsletter repository. I can assist with setting up a public repository > for the source under that account, too. Bitbucket usually uses the "mirror" account for pure mirror repos; but we can probably use our own as well :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Fri Jul 2 10:33:06 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 02 Jul 2010 10:33:06 +0200 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: Am 02.07.2010 09:27, schrieb anatoly techtonik: > On Thu, Jul 1, 2010 at 6:16 PM, Jesse Noller wrote: >> >> This migration is far from "rushed". Workflow will need to be >> documented and we need a working hg setup a little while before the >> official migration. Both of those said, I personally think this has >> dragged on long enough. > > So, if I understand correctly - there are no Mercurial mirrors for > testing at the moment, There are repositories at http://hg.python.org/; the "cpython" one represents the result of conversion at some point in time. > so the current step is to create these mirrors > to try releasing 3.2 from them. I don't know about "try" -- personally I don't see a difference for the release procedure, no matter where the source comes from. I also have no problems releasing a first alpha from SVN, and the following from Hg. > Development will continue in SVN > repository until everybody is ready for final migration in X weeks > later. Is that right? No; as soon as we switch, SVN will be read-only. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From fijall at gmail.com Fri Jul 2 10:39:29 2010 From: fijall at gmail.com (Maciej Fijalkowski) Date: Fri, 2 Jul 2010 02:39:29 -0600 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: <4C2D9332.9010002@v.loewis.de> References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> <4C2D9332.9010002@v.loewis.de> Message-ID: On Fri, Jul 2, 2010 at 1:20 AM, "Martin v. L?wis" wrote: > Am 02.07.2010 08:55, schrieb Craig Citro: >>> This question has an easy answer - can you possibly tell the difference? >>> >> >> Ok, I'm obviously being silly here, but sure you can: > > The dis module is deliberately (*) not part of the Python language and > standard library; it's an implementation detail (as is the func_code > attribute, and the code object). > > So the question really is: can you tell the difference, using only > mechanisms not explicitly documented as implementation-specific? > > Regards, > Martin > > (*) Unfortunately, the documentation fails to mention that, probably > because it's too obvious. > Even more, Jython and IronPython don't have Python bytecode at all and they're considered python implementations. Cheers, fijal From g.brandl at gmx.net Fri Jul 2 10:38:55 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 02 Jul 2010 10:38:55 +0200 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: <20100702010513.6d106e48@pitrou.net> References: <20100701143129.E63FE219BAA@kimball.webabinitio.net> <20100701180745.264898de@heresy> <20100702010513.6d106e48@pitrou.net> Message-ID: Am 02.07.2010 01:05, schrieb Antoine Pitrou: > On Thu, 1 Jul 2010 15:26:12 -0700 > Brett Cannon wrote: >> >> As I said, quick-and-dirty. It will take discussion to decide what we >> want to ask non-committers to do, > > I don't think we have to ask them to do anything special, as long as > they can submit their contributions under the form of a patch. > Whether they use named branches, separate clones, mercurial queues, the > pbranch extension, or even the basic hg-as-an-svn-synonym workflow you > suggested should be none of our business, IMO. > > A DVCS allows non-committers to manage (augment, fix, synchronize) their > patches more easily until they get committed. I don't think it will > change the committers' work a lot, though. I don't know how Mercurial > could make the task of committing an outsider's patch significantly > simpler. I do: if the outsider creates a patch in a personal branch, he is more likely to merge with the trunk every now and then as if he creates a patch once and then puts it on the tracker. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From steve at pearwood.info Fri Jul 2 13:25:14 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 2 Jul 2010 21:25:14 +1000 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> Message-ID: <201007022125.15024.steve@pearwood.info> On Fri, 2 Jul 2010 04:55:10 pm Craig Citro wrote: > > This question has an easy answer - can you possibly tell the > > difference? > > Ok, I'm obviously being silly here, but sure you can: > >>> dis.dis("raise TypeError()") > > 0 <114> 26977 > 3 <115> 8293 > 6 IMPORT_STAR > 7 SETUP_EXCEPT 25968 (to 25978) > 10 <69> > 11 <114> 28530 > 14 <114> 10536 Craig, what are you using to get that? When I try it in Python 3.1, I get: TypeError: don't know how to disassemble str objects How do you get that result? -- Steven D'Aprano From dickinsm at gmail.com Fri Jul 2 13:39:18 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 2 Jul 2010 12:39:18 +0100 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: <201007022125.15024.steve@pearwood.info> References: <201007021428.48162.steve@pearwood.info> <201007022125.15024.steve@pearwood.info> Message-ID: On Fri, Jul 2, 2010 at 12:25 PM, Steven D'Aprano wrote: > Craig, what are you using to get that? When I try it in Python 3.1, I > get: > > TypeError: don't know how to disassemble str objects > > How do you get that result? As I just discovered (see above), dis.dis is happy to interpret byte strings (i.e., strings in 2.x, bytes in 3.x) as direct representations of Python bytecode. There's also an open feature request[1] to allow text strings as input in py3k, doing an automatic compile before passing the result to dis.dis. Mark [1] http://bugs.python.org/issue6507 From ncoghlan at gmail.com Fri Jul 2 13:47:18 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 2 Jul 2010 21:47:18 +1000 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: On Fri, Jul 2, 2010 at 4:55 PM, Craig Citro wrote: > Honestly, though, I'd come down on the side of letting the compiler > raise an error -- while I understand that it means you have > *different* behavior, I think it's *preferable* behavior. But you would be taking a module that will compile and making it uncompilable. The Python code: def f(): return 1 + "1" has fully defined runtime semantics: when f() is called, it will raise TypeError. A module containing this code is still perfectly valid Python (e.g. the Python test suite does that kind of thing a lot in tests of the core interpreter semantics). A Python implementation issuing a SyntaxWarning over this would be fine, but triggering a SyntaxError would not be valid. However, I'd be inclined to leave this kind of check to tools like pychecker and pylint. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From thomas at jollybox.de Fri Jul 2 13:47:35 2010 From: thomas at jollybox.de (Thomas Jollans) Date: Fri, 02 Jul 2010 13:47:35 +0200 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: <4C2DD1D7.6040109@jollybox.de> On 07/02/2010 10:33 AM, Georg Brandl wrote: > Am 02.07.2010 09:27, schrieb anatoly techtonik: >> On Thu, Jul 1, 2010 at 6:16 PM, Jesse Noller wrote: >>> >>> This migration is far from "rushed". Workflow will need to be >>> documented and we need a working hg setup a little while before the >>> official migration. Both of those said, I personally think this has >>> dragged on long enough. >> >> So, if I understand correctly - there are no Mercurial mirrors for >> testing at the moment, > > There are repositories at http://hg.python.org/; the "cpython" one > represents the result of conversion at some point in time. http://code.python.org/hg/ looks like a working mercurial mirror of current CPython svn. It's probably nothing you'd like to switch to as such since it doesn't replace svn usernames with names/e-mail addressed (as I gather was planned), but it does get updated. As one of those potential contributors that are new to the game and actually know and love Mercurial, I've never actually checked out a Python subversion branch, and have based the few patches I have written on http://code.python.org/hg/branches/py3k/ Thomas From merwok at netwok.org Fri Jul 2 14:02:06 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Fri, 02 Jul 2010 14:02:06 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <4C2DD1D7.6040109@jollybox.de> References: <4C2DD1D7.6040109@jollybox.de> Message-ID: <4C2DD53E.7040107@netwok.org> hg.python.org/cpython is a test setup for people working on the transition. It is not guaranteed to be usable, it usually lags, and it will be rewritten before the real switch IIRC. code.python.org/hg is a mirror kept in sync for use by contributors who don?t want to use Subversion now. After the switch, hg.python.org/cpython will be the official repo, and code.python.org/hg will probably be closed. HTH Regards From techtonik at gmail.com Fri Jul 2 14:22:44 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Fri, 2 Jul 2010 15:22:44 +0300 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <4C2DD53E.7040107@netwok.org> References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> Message-ID: On Fri, Jul 2, 2010 at 3:02 PM, ?ric Araujo wrote: > hg.python.org/cpython is a test setup for people working on the > transition. It is not guaranteed to be usable, it usually lags, and it > will be rewritten before the real switch IIRC. > > code.python.org/hg is a mirror kept in sync for use by contributors who > don?t want to use Subversion now. > > After the switch, hg.python.org/cpython will be the official repo, and > code.python.org/hg will probably be closed. Why this transition is not described in PEP? How code.python.org/hg is synchronized with Subversion? Why it is not possible to leave code.python.org/hg as is in slave mode and then realtime replication is ready just switch master/slave over? -- anatoly t. From solipsis at pitrou.net Fri Jul 2 14:34:09 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 2 Jul 2010 14:34:09 +0200 Subject: [Python-Dev] Mercurial migration readiness References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> Message-ID: <20100702143409.67f805a4@pitrou.net> On Fri, 2 Jul 2010 15:22:44 +0300 anatoly techtonik wrote: > On Fri, Jul 2, 2010 at 3:02 PM, ?ric Araujo wrote: > > hg.python.org/cpython is a test setup for people working on the > > transition. It is not guaranteed to be usable, it usually lags, and it > > will be rewritten before the real switch IIRC. > > > > code.python.org/hg is a mirror kept in sync for use by contributors who > > don?t want to use Subversion now. > > > > After the switch, hg.python.org/cpython will be the official repo, and > > code.python.org/hg will probably be closed. > > Why this transition is not described in PEP? Because it's not a transition. It's a mirror. It was put in place before the hg migration plan was accepted, IIRC. > How code.python.org/hg is synchronized with Subversion? What does your question mean exactly? It's a mirror (well, a set of mirrors) and is synchronized roughly every 5 minutes. > Why it is not possible to leave code.python.org/hg as is in slave mode > and then realtime replication is ready just switch master/slave over? The two sets of repositories use different conversion tools and rules. They have nothing in common (different changeset IDs, different metadata, different branch/clone layout). Regards Antoine. From fdrake at acm.org Fri Jul 2 15:09:48 2010 From: fdrake at acm.org (Fred Drake) Date: Fri, 2 Jul 2010 09:09:48 -0400 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <20100702143409.67f805a4@pitrou.net> References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> Message-ID: On Fri, Jul 2, 2010 at 8:34 AM, Antoine Pitrou wrote: > The two sets of repositories use different conversion tools and rules. > They have nothing in common (different changeset IDs, different > metadata, different branch/clone layout). I'd love to see a more detailed description of this, including why someone new to Mercurial would choose one over the other. This information really belongs in www.python.org/dev/ rather than only in the mailing list. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein From stephen at xemacs.org Fri Jul 2 15:06:03 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 02 Jul 2010 22:06:03 +0900 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> Message-ID: <87630yxcck.fsf@uwakimon.sk.tsukuba.ac.jp> anatoly techtonik writes: > Why this transition is not described in PEP? Please reread the whole thread, and the PEP. PEP 385 is *incomplete* (see the red Warning at the top), and the original proponent *is not going to have enough time to complete it soon*. That being the case, Martin suggested that either we find a new proponent or postpone the transition to next year. We're currently in the process of finding a new proponent and supporting cast, and determining what, if anything, we're going to do about the transition over the next few months. There is no reason at this point to suppose the transition can't be complete by the end of summer. However, as always, the devil is in the details, and one of them may be a showstopper. We'll just have to see about that. From solipsis at pitrou.net Fri Jul 2 15:21:35 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 2 Jul 2010 15:21:35 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> Message-ID: <20100702152135.32ef38b3@pitrou.net> On Fri, 2 Jul 2010 09:09:48 -0400 Fred Drake wrote: > On Fri, Jul 2, 2010 at 8:34 AM, Antoine Pitrou wrote: > > The two sets of repositories use different conversion tools and rules. > > They have nothing in common (different changeset IDs, different > > metadata, different branch/clone layout). > > I'd love to see a more detailed description of this, including why > someone new to Mercurial would choose one over the other. > > This information really belongs in www.python.org/dev/ rather than > only in the mailing list. I'm not sure. Contributors don't really have to know about the specifics of the temporary repositories used for conversion tests. www.python.org/dev/ already points to the stable, up-to-date mirrors; I don't think it's worth complicating that page. Regards Antoine. From techtonik at gmail.com Fri Jul 2 15:48:44 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Fri, 2 Jul 2010 16:48:44 +0300 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: On Fri, Jul 2, 2010 at 11:33 AM, Georg Brandl wrote: >> >> So, if I understand correctly - there are no Mercurial mirrors for >> testing at the moment, > > There are repositories at http://hg.python.org/; the "cpython" one > represents the result of conversion at some point in time. What is the problem with realtime synchronization and working with already up to date Mercurial mirror of central SVN repository? >> Development will continue in SVN >> repository until everybody is ready for final migration in X weeks >> later. Is that right? > > No; as soon as we switch, SVN will be read-only. Why don't allow people who already know Mercurial use Mercurial and those who prefer Subversion use that. If Mercurial allows to submit to Subversion - why people can't use that while we writing tutorials and answering question about workflow? -- anatoly t. From g.brandl at gmx.net Fri Jul 2 15:53:14 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 02 Jul 2010 15:53:14 +0200 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: Am 02.07.2010 15:48, schrieb anatoly techtonik: > On Fri, Jul 2, 2010 at 11:33 AM, Georg Brandl wrote: >>> >>> So, if I understand correctly - there are no Mercurial mirrors for >>> testing at the moment, >> >> There are repositories at http://hg.python.org/; the "cpython" one >> represents the result of conversion at some point in time. > > What is the problem with realtime synchronization and working with > already up to date Mercurial mirror of central SVN repository? The specifics of the conversion process are not nailed down yet. Therefore, the exact translation of SVN to Hg commits will change, and with it the Mercurial revision IDs, for example. >>> Development will continue in SVN >>> repository until everybody is ready for final migration in X weeks >>> later. Is that right? >> >> No; as soon as we switch, SVN will be read-only. > > Why don't allow people who already know Mercurial use Mercurial and > those who prefer Subversion use that. If Mercurial allows to submit to > Subversion - why people can't use that while we writing tutorials and > answering question about workflow? I don't think that we have enough manpower to maintain such a bridge indefinitely. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From dirkjan at ochtman.nl Fri Jul 2 16:04:53 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Fri, 2 Jul 2010 16:04:53 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <20100702152135.32ef38b3@pitrou.net> References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <20100702152135.32ef38b3@pitrou.net> Message-ID: On Fri, Jul 2, 2010 at 15:21, Antoine Pitrou wrote: >> I'd love to see a more detailed description of this, including why >> someone new to Mercurial would choose one over the other. >> >> This information really belongs in www.python.org/dev/ rather than >> only in the mailing list. > > I'm not sure. Contributors don't really have to know about the > specifics of the temporary repositories used for conversion tests. Very much agreed. Cheers, Dirkjan From techtonik at gmail.com Fri Jul 2 16:17:36 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Fri, 2 Jul 2010 17:17:36 +0300 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: On Fri, Jul 2, 2010 at 4:53 PM, Georg Brandl wrote: >> >> What is the problem with realtime synchronization and working with >> already up to date Mercurial mirror of central SVN repository? > > The specifics of the conversion process are not nailed down yet. > Therefore, the exact translation of SVN to Hg commits will change, > and with it the Mercurial revision IDs, for example. Does anybody here know how Mercurial calculates the IDs? From that I remember it is author + message + diff content. What can change there? >>>> Development will continue in SVN >>>> repository until everybody is ready for final migration in X weeks >>>> later. Is that right? >>> >>> No; as soon as we switch, SVN will be read-only. >> >> Why don't allow people who already know Mercurial use Mercurial and >> those who prefer Subversion use that. If Mercurial allows to submit to >> Subversion - why people can't use that while we writing tutorials and >> answering question about workflow? > > I don't think that we have enough manpower to maintain such a bridge > indefinitely. It doesn't require manpower. It requires automation. Considering that the biggest problem now is to get sane lossless conversion, we should elaborate on getting this in place. After that I would still follow the path of setting realtime mirror for X weeks that could be replicated by bitbucket, launchpad and other services to see how people pickup the changes. As PEP 384 says - the transition is mostly to make lives of outside contributors easier. Core developers can wait for a while. -- anatoly t. From techtonik at gmail.com Fri Jul 2 16:29:02 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Fri, 2 Jul 2010 17:29:02 +0300 Subject: [Python-Dev] Are you ready for Mercurial migration? Message-ID: To shed some light on the readiness of Python community for the switch I've opened public Google Wave. Please add your opinion if you can and send this link to other contributors you may know: https://wave.google.com/wave/waveref/googlewave.com/w+G12NYQbDA -- anatoly t. From dirkjan at ochtman.nl Fri Jul 2 16:29:23 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Fri, 2 Jul 2010 16:29:23 +0200 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: On Fri, Jul 2, 2010 at 16:17, anatoly techtonik wrote: > As PEP 384 says - the transition is mostly to make lives of outside > contributors easier. Core developers can wait for a while. I think a lot of the core developers also want this because it makes their lives better. Also, several people so far have stated that they wouldn't be interested in a SVN bridge -- AFAICT you are the only one. If you really want this, maybe you can set it up and maintain it yourself? Everything about the repo is freely available, so you can just get it going. Cheers, Dirkjan From jnoller at gmail.com Fri Jul 2 16:31:46 2010 From: jnoller at gmail.com (Jesse Noller) Date: Fri, 2 Jul 2010 10:31:46 -0400 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: On Fri, Jul 2, 2010 at 10:17 AM, anatoly techtonik wrote: > On Fri, Jul 2, 2010 at 4:53 PM, Georg Brandl wrote: >>> >>> What is the problem with realtime synchronization and working with >>> already up to date Mercurial mirror of central SVN repository? >> >> The specifics of the conversion process are not nailed down yet. >> Therefore, the exact translation of SVN to Hg commits will change, >> and with it the Mercurial revision IDs, for example. > > Does anybody here know how Mercurial calculates the IDs? From that I > remember it is author + message + diff content. What can change there? > >>>>> Development will continue in SVN >>>>> repository until everybody is ready for final migration in X weeks >>>>> later. Is that right? >>>> >>>> No; as soon as we switch, SVN will be read-only. >>> >>> Why don't allow people who already know Mercurial use Mercurial and >>> those who prefer Subversion use that. If Mercurial allows to submit to >>> Subversion - why people can't use that while we writing tutorials and >>> answering question about workflow? >> >> I don't think that we have enough manpower to maintain such a bridge >> indefinitely. > > It doesn't require manpower. It requires automation. Considering that > the biggest problem now is to get sane lossless conversion, we should > elaborate on getting this in place. After that I would still follow > the path of setting realtime mirror for X weeks that could be > replicated by bitbucket, launchpad and other services to see how > people pickup the changes. > > As PEP 384 says - the transition is mostly to make lives of outside > contributors easier. Core developers can wait for a while. No, we can't. We *have* been waiting. From craigcitro at gmail.com Fri Jul 2 16:44:08 2010 From: craigcitro at gmail.com (Craig Citro) Date: Fri, 2 Jul 2010 07:44:08 -0700 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: > Whoa. ?That's very peculiar looking bytecode. ?Is dis.dis behaving as > it should here? > BTW, I think you want 'raise TypeError', not 'raise TypeError()'. > Yep, that's embarrassing. I was being lazy: I was expecting different bytecodes, and I got it ... so I apparently didn't bother to actually read the bytecodes? It's interesting -- if I'd just had TypeError instead of TypeError(), I would have found this out, because "raise TypeError" is not the bytes representation of a valid sequence of bytecodes. ;) Anyway, here's what I was going for: >>> def foo(): ... return 1+'1' ... >>> def bar(): ... raise TypeError ... >>> dis.dis(foo) 2 0 LOAD_CONST 1 (1) 3 LOAD_CONST 2 ('1') 6 BINARY_ADD 7 RETURN_VALUE >>> dis.dis(bar) 2 0 LOAD_GLOBAL 0 (TypeError) 3 RAISE_VARARGS 1 6 LOAD_CONST 0 (None) 9 RETURN_VALUE That said, I totally concede Martin's point -- this is an implementation-specific thing. It happens that all the major Python implementations compile to some VM, and I'd bet that these two compile to different bytecodes on any of them, but that doesn't preclude another implementation from making a different choice there. -cc From merwok at netwok.org Fri Jul 2 16:52:57 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Fri, 02 Jul 2010 16:52:57 +0200 Subject: [Python-Dev] Are you ready for Mercurial migration? In-Reply-To: References: Message-ID: <4C2DFD49.70507@netwok.org> Hello Anatoly I?m thankful that you give time and energy to the issues raised by the migration. As a minor contributor, I?m eager for the migration, and I also find healthy that there is discussion about these things. However, I think that there is too much discussion right now. I offer some facts and opinions. 1) The PEP is accepted. Python is migrating to Mercurial and closing the Subversion repository after lengthy discussion and rounds of feedback. This is beating a dead horse. 2) Python mailing lists are effective, open to anyone with an email address, and work with all Web and email clients. Google Wave requires an account, JavaScript, and learning a new thing. That doesn?t help a situation where we already lack people and time. 2a) Old tools are not always bad. (Emails works.) 2b) New tools can be helpful, but do not magically solve issues. I can?t see python-dev moving to Trac or Wave in the coming years, so fighting for them is not productive. In a community, it?s important to assume that people have reasons for their choices, research before jumping to conclusions (e.g. hg uses the parent changeset ID too to compute the ID of a changeset, so the current mirror and the new, clean official repo will be incompatible), and say that everything sucks. To sum up my opinion: The Mercurial migration will proceed as explained in the PEP, there are still some areas to discuss, the transition will be delayed if there?s no committer willing to lead the effort. Debating agreed issues does not help solving open issues. Regards From g.brandl at gmx.net Fri Jul 2 17:03:12 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 02 Jul 2010 17:03:12 +0200 Subject: [Python-Dev] Are you ready for Mercurial migration? In-Reply-To: References: Message-ID: Am 02.07.2010 16:29, schrieb anatoly techtonik: > To shed some light on the readiness of Python community for the switch > I've opened public Google Wave. Please add your opinion if you can and > send this link to other contributors you may know: tl;dr: I failed forcing my notions upon the participants in one particular medium, let's switch to another and try again. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From solipsis at pitrou.net Fri Jul 2 17:08:19 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 2 Jul 2010 17:08:19 +0200 Subject: [Python-Dev] Are you ready for Mercurial migration? References: Message-ID: <20100702170819.793047d4@pitrou.net> On Fri, 2 Jul 2010 17:29:02 +0300 anatoly techtonik wrote: > To shed some light on the readiness of Python community for the switch > I've opened public Google Wave. Please add your opinion if you can and > send this link to other contributors you may know: Am I the only one to think this should really stop? By "this", I mean the flow of complaints and dubious "recommendations" you send here and on the bug tracker. We are volunteers, we don't need a boss, especially not one who prefers arguing about workflow rather than addressing concrete issues. I'm not sure if other people are finding those (Anatoly's) messages constructive and insightful. To me it looks like they are wasting the aggregate signal to noise ratio. Antoine. From tseaver at palladion.com Fri Jul 2 17:09:16 2010 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 02 Jul 2010 11:09:16 -0400 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Georg Brandl wrote: > Am 02.07.2010 15:48, schrieb anatoly techtonik: >> On Fri, Jul 2, 2010 at 11:33 AM, Georg Brandl wrote: >>>> So, if I understand correctly - there are no Mercurial mirrors for >>>> testing at the moment, >>> There are repositories at http://hg.python.org/; the "cpython" one >>> represents the result of conversion at some point in time. >> What is the problem with realtime synchronization and working with >> already up to date Mercurial mirror of central SVN repository? > > The specifics of the conversion process are not nailed down yet. > Therefore, the exact translation of SVN to Hg commits will change, > and with it the Mercurial revision IDs, for example. > >>>> Development will continue in SVN >>>> repository until everybody is ready for final migration in X weeks >>>> later. Is that right? >>> No; as soon as we switch, SVN will be read-only. >> Why don't allow people who already know Mercurial use Mercurial and >> those who prefer Subversion use that. If Mercurial allows to submit to >> Subversion - why people can't use that while we writing tutorials and >> answering question about workflow? > > I don't think that we have enough manpower to maintain such a bridge > indefinitely. The code.python.org/hg mirror has been running for many months now: As a non-committer, I have been able to use its 2.6 branch and the trunk with hg to test / develop / submit patches. Can somebody comment on how much ongoing effort is required to keep that mirror running? I know that the hg / git / bzr mirrors I set up for the repoze SVN repository haven't require any ongoing effort, after the initial day or two of setup effort. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkwuARwACgkQ+gerLs4ltQ6rUgCfU2NSdjz+pIY1I95OnpRMU5Fx 22gAn1Zu+KGOaNgYJYJ7c9NKzr3ICxS+ =wIB0 -----END PGP SIGNATURE----- From g.brandl at gmx.net Fri Jul 2 17:05:37 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 02 Jul 2010 17:05:37 +0200 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: Am 02.07.2010 16:17, schrieb anatoly techtonik: > On Fri, Jul 2, 2010 at 4:53 PM, Georg Brandl wrote: >>> >>> What is the problem with realtime synchronization and working with >>> already up to date Mercurial mirror of central SVN repository? >> >> The specifics of the conversion process are not nailed down yet. >> Therefore, the exact translation of SVN to Hg commits will change, >> and with it the Mercurial revision IDs, for example. > > Does anybody here know how Mercurial calculates the IDs? From that I > remember it is author + message + diff content. What can change there? The parents IDs also are part of that hash. Apart from that, author and message can change as well, since SVN/CVS-style committer names will be mapped to Mercurial-style name + email, and the message can have "[rev ...]" appended or prepended or not. >>>>> Development will continue in SVN >>>>> repository until everybody is ready for final migration in X weeks >>>>> later. Is that right? >>>> >>>> No; as soon as we switch, SVN will be read-only. >>> >>> Why don't allow people who already know Mercurial use Mercurial and >>> those who prefer Subversion use that. If Mercurial allows to submit to >>> Subversion - why people can't use that while we writing tutorials and >>> answering question about workflow? >> >> I don't think that we have enough manpower to maintain such a bridge >> indefinitely. > > It doesn't require manpower. It requires automation. And who, do you think, is going to implement that automation? > Considering that the biggest problem now is to get sane lossless > conversion, we should elaborate on getting this in place. Ah, the mysterious "we". > After that I would still follow > the path of setting realtime mirror for X weeks that could be > replicated by bitbucket, launchpad and other services to see how > people pickup the changes. > > As PEP 384 says - the transition is mostly to make lives of outside > contributors easier. Core developers can wait for a while. Anatoly, I don't know what you are trying to achieve here. The decision to switch to Mercurial as soon as possible given our manpower restriction has been made at PyCon 2009. Since then, there has been progress, albeit slow, towards that switch, and now that has come into reach, you start questioning anything and everything. This is neither productive, nor have you shown any willingness to actually *do* something to help. You claim you do not have enough time for that; looking at the multitude of topics you're trying to force a discussion about, I wonder if you couldn't write one decent patch instead of ten complaining mails, and make both you and us happier in the process. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From techtonik at gmail.com Fri Jul 2 17:16:09 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Fri, 2 Jul 2010 18:16:09 +0300 Subject: [Python-Dev] Are you ready for Mercurial migration? In-Reply-To: <4C2DFD49.70507@netwok.org> References: <4C2DFD49.70507@netwok.org> Message-ID: ?ric, you letter is discouraging. This is not for coredevs, who already "decided" - it for the rest of the world. I should clarify it in the first place, but I would like to avoid lengthy debates outside of the Wave. > As a minor contributor, I?m eager for the migration, It is not the question about do you like it or not. It is the question "Are You ready?" That means: Have you tried Mercurial? Do you understand how it works? Do you have a workflow ready and tested to start commiting right away? or Do you plan to read ~100 page long hginit.com tutorial after *it happens? -- anatoly From barry at python.org Fri Jul 2 17:28:11 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 2 Jul 2010 11:28:11 -0400 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: <20100702112811.45b81c72@heresy> On Jul 02, 2010, at 11:09 AM, Tres Seaver wrote: >Can somebody comment on how much ongoing effort is required to keep >that mirror running? I'm guess "zero". Other than uploading new ssh keys, I think our svn master has been humming along pretty well without intervention. I know that the bzr mirrors on Launchpad have been zero effort as well. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From brian.curtin at gmail.com Fri Jul 2 17:28:53 2010 From: brian.curtin at gmail.com (Brian Curtin) Date: Fri, 2 Jul 2010 10:28:53 -0500 Subject: [Python-Dev] Are you ready for Mercurial migration? In-Reply-To: References: <4C2DFD49.70507@netwok.org> Message-ID: On Fri, Jul 2, 2010 at 10:16, anatoly techtonik wrote: > It is not the question about do you like it or not. It is the question > > "Are You ready?" > > That means: > Have you tried Mercurial? > Yes. > Do you understand how it works? Yes. > Do you have a workflow ready and tested to start commiting right away? > My mercurial experience is much less than that of others around here and with much smaller teams, but I'm comfortable with a few workflows and am open to learn more. > or > Do you plan to read ~100 page long hginit.com tutorial after *it happens? > That tutorial is not ~100 pages. It's actually a good tutorial. Feel free to copy that into Google Wave. -------------- next part -------------- An HTML attachment was scrubbed... URL: From merwok at netwok.org Fri Jul 2 17:42:41 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Fri, 02 Jul 2010 17:42:41 +0200 Subject: [Python-Dev] Are you ready for Mercurial migration? In-Reply-To: References: <4C2DFD49.70507@netwok.org> Message-ID: <4C2E08F1.4080403@netwok.org> > ?ric, you letter is discouraging. That?s the right term. I wanted to discourage you from fruitless endeavors. > This is not for coredevs, who already "decided" - it for the rest of > the world. I don?t think there is a discrepancy between the core devs that want to keep the power and do what they please, and the community that is never consulted and unable to express itself. I believe that the core devs are the first circle of the community and that they strive to make open technical dicussions and reach decisions for the good of everyone. I find them open and kind, even if sometimes necessarily stern (e.g. when answering the same question for the umpteenth time). > I should clarify it in the first place, but I would like to avoid > lengthy debates outside of the Wave. And I would like to keep Python-related discussions on open official fora that have been working for decades. >> As a minor contributor, I?m eager for the migration, > It is not the question about do you like it or not. That paragraph was just me trying to be positive before expressing my negative comments. > It is the question > "Are You ready?" > That means: > Have you tried Mercurial? Yes. > Do you understand how it works? Yes. > Do you have a workflow ready and tested to start commiting right away? Yes. I have a handful of workflows in my belt to address different needs (one-shot typo fix, feature development, local editions, etc.) > Do you plan to read ~100 page long hginit.com tutorial after *it happens? Done. I have read the hg book, information on the wiki, made a lot of tests, read the PEPs, read the short hginit tutorial, and am willing to review Python-centered docs. Regards From techtonik at gmail.com Fri Jul 2 17:49:27 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Fri, 2 Jul 2010 18:49:27 +0300 Subject: [Python-Dev] Are you ready for Mercurial migration? In-Reply-To: <20100702170819.793047d4@pitrou.net> References: <20100702170819.793047d4@pitrou.net> Message-ID: Antoine, I value you contribution to `hgsvn` project, and this thread is not a personal accusation of anybody in making proper transition - please understand that I would like to see the opinion of people who preferred not to be involved in lengthy discussions. For personal pretensions against me - please start a new thread and no hijack. I am trying to solve the same problem as you are - make the Mercurial migration, but make it as painless as possible. On Fri, Jul 2, 2010 at 6:08 PM, Antoine Pitrou wrote: > >> To shed some light on the readiness of Python community for the switch >> I've opened public Google Wave. Please add your opinion if you can and >> send this link to other contributors you may know: > > Am I the only one to think this should really stop? > > By "this", I mean the flow of complaints and dubious "recommendations" > you send here and on the bug tracker. We are volunteers, we don't need > a boss, especially not one who prefers arguing about workflow rather > than addressing concrete issues. > > I'm not sure if other people are finding those (Anatoly's) messages > constructive and insightful. To me it looks like they are wasting the > aggregate signal to noise ratio. > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/techtonik%40gmail.com > From g.brandl at gmx.net Fri Jul 2 17:47:16 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 02 Jul 2010 17:47:16 +0200 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: Am 02.07.2010 17:09, schrieb Tres Seaver: >>>>> Development will continue in SVN >>>>> repository until everybody is ready for final migration in X weeks >>>>> later. Is that right? >>>> No; as soon as we switch, SVN will be read-only. >>> Why don't allow people who already know Mercurial use Mercurial and >>> those who prefer Subversion use that. If Mercurial allows to submit to >>> Subversion - why people can't use that while we writing tutorials and >>> answering question about workflow? > >> I don't think that we have enough manpower to maintain such a bridge >> indefinitely. > > The code.python.org/hg mirror has been running for many months now: As a > non-committer, I have been able to use its 2.6 branch and the trunk with > hg to test / develop / submit patches. > > Can somebody comment on how much ongoing effort is required to keep that > mirror running? I know that the hg / git / bzr mirrors I set up for the > repoze SVN repository haven't require any ongoing effort, after the > initial day or two of setup effort. As Barry says, that's not a problem. What's problematic is the continued use of the current Subversion setup as a read-write bridge to the Mercurial repo, once the switch to Mercurial as the main platform has happened. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From status at bugs.python.org Fri Jul 2 18:07:38 2010 From: status at bugs.python.org (Python tracker) Date: Fri, 2 Jul 2010 18:07:38 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20100702160738.42AD878066@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2010-06-25 - 2010-07-02) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2796 open (+44) / 18167 closed (+20) / 20963 total (+64) Open issues with patches: 1126 Average duration of open issues: 709 days. Median duration of open issues: 504 days. Open Issues Breakdown open 2763 (+44) languishing 14 ( +0) pending 18 ( +0) Issues Created Or Reopened (70) _______________________________ assumption about unsigned long byte size in struct module usag 2010-06-25 CLOSED http://bugs.python.org/issue1789 reopened mark.dickinson patch, easy optparse doesn???t disallow adding one-dash long options (???- 2010-06-26 http://bugs.python.org/issue4640 reopened haypo mimetypes.guess_extension result changes after mimetypes.init( 2010-06-26 http://bugs.python.org/issue4963 reopened r.david.murray patch, easy PyDateTime_IMPORT macro incorrectly marked up 2010-06-26 CLOSED http://bugs.python.org/issue9024 reopened belopolsky patch test_sysconfig failure 2010-06-25 CLOSED http://bugs.python.org/issue9081 created michael.foord warnings.filterwarnings doesn't work with -O 2010-06-25 http://bugs.python.org/issue9082 created michael.foord At least some Tools utilities are still Python 2 2010-06-25 http://bugs.python.org/issue9083 created holdenweb vimrc: use matchall() instead of ":match" to allow multiple ma 2010-06-26 http://bugs.python.org/issue9084 created blueyed patch, needs review Version number inconsistency in email package 2010-06-28 http://bugs.python.org/issue9085 reopened r.david.murray Wrong linking terminology in windows FAQ 2010-06-26 http://bugs.python.org/issue9086 created john.miller json docstrings on 3.x still use 'unicode' and 'str' 2010-06-26 http://bugs.python.org/issue9087 created ezio.melotti revert distutils to its 3.1 state 2010-06-26 http://bugs.python.org/issue9088 created tarek PyNumber_Int is still mentioned in number protocol docs 2010-06-27 CLOSED http://bugs.python.org/issue9089 created belopolsky Error code 10035 calling socket.recv() on a socket with a time 2010-06-27 http://bugs.python.org/issue9090 created ehohenstein patch str.capitalize() should not lower the rest of the string 2010-06-27 CLOSED http://bugs.python.org/issue9091 created eka static behavior of local variables 2010-06-27 CLOSED http://bugs.python.org/issue9092 created Aslanidi Tools/README is out of date 2010-06-27 http://bugs.python.org/issue9093 created belopolsky Make python-m pickletools do something useful 2010-06-27 http://bugs.python.org/issue9094 created belopolsky patch, easy patchcheck should handle extraneous whitespace in .rst files 2010-06-27 http://bugs.python.org/issue9095 created brett.cannon easy importlib doesn't handle valid marshalled data with invalid se 2010-06-28 http://bugs.python.org/issue9096 created benjamin.peterson os.chdir(path) to return current dir 2010-06-28 CLOSED http://bugs.python.org/issue9097 created techtonik MSYS build fails with `S_IXGRP' undeclared 2010-06-28 http://bugs.python.org/issue9098 created Martin.Ellison multiprocessing/win32: WindowsError: [Error 0] Success on Pipe 2010-06-28 http://bugs.python.org/issue9099 created honeyman test_sysconfig fails (test_user_similar) 2010-06-28 http://bugs.python.org/issue9100 created csernazs reference json format in file formats chapter 2010-06-28 http://bugs.python.org/issue9101 created techtonik pybench: Cannot compare 2.x and 3.x benchmarks 2010-06-28 http://bugs.python.org/issue9102 created belopolsky API keyword 2010-06-28 CLOSED http://bugs.python.org/issue9103 created techtonik test_exceptions does not test pickling with pickle.py 2010-06-28 http://bugs.python.org/issue9104 created belopolsky patch pickle security note should be more prominent 2010-06-28 http://bugs.python.org/issue9105 created techtonik easy remove numbers from 3-.. level entries in docs toc 2010-06-28 CLOSED http://bugs.python.org/issue9106 created techtonik PyModule_Create not working properly 2010-06-28 CLOSED http://bugs.python.org/issue9107 created Krauzi list object variable assign with multiplication sign 2010-06-28 CLOSED http://bugs.python.org/issue9108 created muzuiget absolute import cleanups for Python 3 2010-06-28 http://bugs.python.org/issue9109 created nas contextlib.ContextDecorator 2010-06-28 CLOSED http://bugs.python.org/issue9110 created michael.foord patch cmd.do_help documentation should mention docstrings 2010-06-29 http://bugs.python.org/issue9111 created tebeka argparse missing documentation for error() method 2010-06-29 http://bugs.python.org/issue9112 created mark Tools/msi still uses string exceptions 2010-06-29 CLOSED http://bugs.python.org/issue9113 created doko br"" string literals don't appear to be documented 2010-06-29 CLOSED http://bugs.python.org/issue9114 created mark test_site: support for systems without unsetenv 2010-06-29 http://bugs.python.org/issue9115 created csernazs patch test_capi.test_no_FatalError_infinite_loop crash on Windows 2010-06-29 http://bugs.python.org/issue9116 created brian.curtin class syntax not fully documented in reference manual 2010-06-29 http://bugs.python.org/issue9117 created mark.dickinson patch help() on a property descriptor launches interactive help 2010-06-29 http://bugs.python.org/issue9118 created jackdied patch Python download page needs to mention crypto code in Windows i 2010-06-29 http://bugs.python.org/issue9119 created lemburg Reduce pickle size for an empty set 2010-06-29 http://bugs.python.org/issue9120 created belopolsky patch Glossary entry for nested scope incorrect 2010-06-29 CLOSED http://bugs.python.org/issue9121 created stoneleaf Problems with multiprocessing, Python embedding and Windows 2010-06-29 http://bugs.python.org/issue9122 created honeyman insecure os.urandom on VMS 2010-06-30 http://bugs.python.org/issue9123 created zooko Mailbox module demonstrates infeasibly slow performance 2010-06-30 http://bugs.python.org/issue9124 created holdenweb parser module doesn't like except-as syntax. 2010-06-30 CLOSED http://bugs.python.org/issue9125 created mark.dickinson patch errors='replace' does not work at Windows command line 2010-06-30 CLOSED http://bugs.python.org/issue9126 created jvanpraag subprocess.Popen.communicate() and SIGCHLD handlers 2010-06-30 http://bugs.python.org/issue9127 created zarkdav Parser module fails to validate class decorators correctly. 2010-06-30 http://bugs.python.org/issue9128 created mark.dickinson patch DoS smtpd module vulnerability 2010-06-30 http://bugs.python.org/issue9129 created giampaolo.rodola Parser module fails on short relative imports 2010-06-30 http://bugs.python.org/issue9130 created mark.dickinson patch test_set_reprs in test_pprint is fragile 2010-06-30 http://bugs.python.org/issue9131 created stutzbach Documentation for comparing dictionaries is out of date 2010-06-30 http://bugs.python.org/issue9132 created stutzbach easy Invalid UTF8 Byte sequence not raising exception/being substit 2010-06-30 CLOSED http://bugs.python.org/issue9133 created Mike.Lewis sre bug: lastmark_save/restore 2010-06-30 http://bugs.python.org/issue9134 created arigo XMLRPC documentation binary file example does not execute. 2010-06-30 CLOSED http://bugs.python.org/issue9135 created Delaney.Gillilan RuntimeError when profiling Decimal 2010-06-30 http://bugs.python.org/issue9136 created ubershmekel patch x.update(red=5, blue=6, other=7) doesn't work, where x is a Mu 2010-07-01 http://bugs.python.org/issue9137 created stutzbach patch Tutorial: classes intro paragraph icky 2010-07-01 http://bugs.python.org/issue9138 created aahz Add examples for str.format() 2010-07-02 http://bugs.python.org/issue9139 created ezio.melotti patch SocketServer.BaseRequestHandler not a new-style class? 2010-07-02 CLOSED http://bugs.python.org/issue9140 created phf Allow objects to decide if they can be collected by GC 2010-07-02 http://bugs.python.org/issue9141 created krisvale patch, patch, needs review while executing python 2010-07-02 CLOSED http://bugs.python.org/issue9142 created rajesh.chandran test_ctypes: fails on sparc64 long double 2010-07-02 CLOSED http://bugs.python.org/issue9143 created jengelh Import error for multiprocessing in 2.7rc2 on Windows 2010-07-02 http://bugs.python.org/issue9144 created ot tracebacks eat up memory by holding references to locals and g 2010-06-25 http://bugs.python.org/issue1565525 reopened ghazel recursive urlparse 2010-06-30 CLOSED http://bugs.python.org/issue1643370 reopened techtonik easy Issues Now Closed (74) ______________________ pickle's documentation is severely outdated 940 days http://bugs.python.org/issue1536 alexandre.vassalotti assumption about unsigned long byte size in struct module usag 4 days http://bugs.python.org/issue1789 mark.dickinson patch, easy Backport: Mixing default keyword arguments with *args 886 days http://bugs.python.org/issue1909 ezio.melotti ctypes.util fails to find libc in some environments 709 days http://bugs.python.org/issue3383 tjreedy patch cPickle to pickle conversion in py3k missing methods 712 days http://bugs.python.org/issue3385 alexandre.vassalotti patch httplib.HTTPMessage undocumented 707 days http://bugs.python.org/issue3428 orsenthil easy ssl.wrap_socket() is incompatible with servers that drop privi 658 days http://bugs.python.org/issue3823 pitrou 3.0rc2.msi Install Fails (Error Code 2755) 592 days http://bugs.python.org/issue4299 mark.dickinson Add a warnings.showwarning replacement for logging 586 days http://bugs.python.org/issue4384 merwok easy SequenceMatcher bug with long sequences 562 days http://bugs.python.org/issue4622 tjreedy 2.6.1 build issues on solaris with SunStudio 12 530 days http://bugs.python.org/issue4919 benjamin.peterson Duplicate UTF-16 BOM if a file is open in append mode 527 days http://bugs.python.org/issue5006 Conrad.Irwin patch [reopening] native build of python win32 using msys under both 520 days http://bugs.python.org/issue5026 tjreedy RFC: improve distutils bdist_rpm so it builds pure python modu 471 days http://bugs.python.org/issue5482 tjreedy patch os.getpwent returns unsigned 32bit value, os.setuid refuses it 297 days http://bugs.python.org/issue5705 ezio.melotti patch [Distutils][PATCH] Add bdist_rpm option to select the name of 339 days http://bugs.python.org/issue6546 tjreedy patch smtpd.SMTPServer can cause asyncore.loop to enter infinite eve 338 days http://bugs.python.org/issue6589 giampaolo.rodola patch deepcopy erroneously doesn't call __setstate__ if __getstate__ 300 days http://bugs.python.org/issue6827 r.david.murray easy Silence DeprecationWarning by default 63 days http://bugs.python.org/issue7319 benjamin.peterson patch setup.py ignores SDK root on OSX 161 days http://bugs.python.org/issue7724 ronaldoussoren patch, needs review Piped parent's multiprocessing.Process children cannot write t 118 days http://bugs.python.org/issue8056 Vojtech.Fried pep-0370 on osx duplicates existing functionality 50 days http://bugs.python.org/issue8084 ronaldoussoren patch, needs review, buildbot buildbot: DeprecationWarning not raised for icglue (test_py3kw 70 days http://bugs.python.org/issue8446 ronaldoussoren patch, buildbot buildbot: test_urllib2_localnet failures (Connection refused) 69 days http://bugs.python.org/issue8455 ronaldoussoren patch, buildbot test_distutils fails if srcdir != builddir 12 days http://bugs.python.org/issue8577 ronaldoussoren patch uuid.uuid4() generates non-unique values on OSX 6 days http://bugs.python.org/issue8621 ronaldoussoren patch urlparse.urlparse/urlsplit doc missing 52 days http://bugs.python.org/issue8653 orsenthil Link to PEP 3147 from importlib docs 50 days http://bugs.python.org/issue8667 brett.cannon patch, easy, needs review Proxy exception lookup fails on MacOS in urllib. 24 days http://bugs.python.org/issue8883 ronaldoussoren patch email.encoders.encode_base64 sets payload to bytes, should set 22 days http://bugs.python.org/issue8896 merwok patch Provide as_integer_ratio() method to Decimal 21 days http://bugs.python.org/issue8947 mark.dickinson patch PyDateTime_IMPORT macro incorrectly marked up 1 days http://bugs.python.org/issue9024 belopolsky patch add test_support.run_unittest command line options and argumen 9 days http://bugs.python.org/issue9027 georg.brandl errors='replace' works in IDLE, fails at Windows command line. 12 days http://bugs.python.org/issue9029 benjamin.peterson using MIMEApplication to attach a PDF raises a TypeError excep 5 days http://bugs.python.org/issue9040 r.david.murray 2to3 doesn't handle byte comparison well 4 days http://bugs.python.org/issue9043 tjreedy [optparse] confusion over an option and its value without any 6 days http://bugs.python.org/issue9044 georg.brandl Python 2.7rc2 doesn't build on Mac OS X 10.4 6 days http://bugs.python.org/issue9046 ronaldoussoren Backwards compatibility 4 days http://bugs.python.org/issue9059 georg.brandl pdb enhancement up/down traversals 4 days http://bugs.python.org/issue9064 georg.brandl patch test_float failure on Solaris 3 days http://bugs.python.org/issue9069 mark.dickinson TarFile doesn't support member files with a leading "./" 5 days http://bugs.python.org/issue9071 niemeyer ssl module sets "debug" flag on SSL struct 5 days http://bugs.python.org/issue9075 srid argparse does not handle arguments correctly after -- 4 days http://bugs.python.org/issue9077 georg.brandl Fix C API documentation of unicode 2 days http://bugs.python.org/issue9078 georg.brandl patch test_sysconfig failure 0 days http://bugs.python.org/issue9081 pitrou PyNumber_Int is still mentioned in number protocol docs 1 days http://bugs.python.org/issue9089 mark.dickinson str.capitalize() should not lower the rest of the string 0 days http://bugs.python.org/issue9091 orsenthil static behavior of local variables 0 days http://bugs.python.org/issue9092 mark.dickinson os.chdir(path) to return current dir 2 days http://bugs.python.org/issue9097 techtonik API keyword 1 days http://bugs.python.org/issue9103 merwok remove numbers from 3-.. level entries in docs toc 1 days http://bugs.python.org/issue9106 techtonik PyModule_Create not working properly 0 days http://bugs.python.org/issue9107 amaury.forgeotdarc list object variable assign with multiplication sign 0 days http://bugs.python.org/issue9108 ezio.melotti contextlib.ContextDecorator 2 days http://bugs.python.org/issue9110 michael.foord patch Tools/msi still uses string exceptions 0 days http://bugs.python.org/issue9113 loewis br"" string literals don't appear to be documented 0 days http://bugs.python.org/issue9114 merwok Glossary entry for nested scope incorrect 0 days http://bugs.python.org/issue9121 benjamin.peterson parser module doesn't like except-as syntax. 0 days http://bugs.python.org/issue9125 mark.dickinson patch errors='replace' does not work at Windows command line 1 days http://bugs.python.org/issue9126 r.david.murray Invalid UTF8 Byte sequence not raising exception/being substit 0 days http://bugs.python.org/issue9133 ezio.melotti XMLRPC documentation binary file example does not execute. 0 days http://bugs.python.org/issue9135 ezio.melotti SocketServer.BaseRequestHandler not a new-style class? 0 days http://bugs.python.org/issue9140 phf while executing python 0 days http://bugs.python.org/issue9142 mark.dickinson test_ctypes: fails on sparc64 long double 0 days http://bugs.python.org/issue9143 mark.dickinson cPickle not always same as pickle 2749 days http://bugs.python.org/issue655802 alexandre.vassalotti Generate rpm filelist including directories 2567 days http://bugs.python.org/issue755286 tjreedy patch bdist_rpm failure when no setup.py 2439 days http://bugs.python.org/issue828743 tjreedy Specify a source baseurl for bdist_rpm. 2111 days http://bugs.python.org/issue1028432 tjreedy patch Add New RPM-friendly record option to setup.py 2098 days http://bugs.python.org/issue1035576 tjreedy patch difflib.SequenceMatcher.find_longest_match() wrong result 1432 days http://bugs.python.org/issue1528074 tjreedy Py3k: Fix pybench so it runs 1262 days http://bugs.python.org/issue1634499 merwok patch recursive urlparse 2 days http://bugs.python.org/issue1643370 r.david.murray easy New byte packing format for the struct module 1088 days http://bugs.python.org/issue1749662 mark.dickinson patch Top Issues Most Discussed (10) ______________________________ 25 RuntimeError when profiling Decimal 2 days open http://bugs.python.org/issue9136 22 test_float failure on Solaris 3 days closed http://bugs.python.org/issue9069 17 3.1 cannot unpickle 2.7-created pickle 510 days open http://bugs.python.org/issue5180 16 Add pure Python implementation of datetime module to CPython 130 days open http://bugs.python.org/issue7989 12 test_exceptions does not test pickling with pickle.py 4 days open http://bugs.python.org/issue9104 11 Reduce pickle size for an empty set 3 days open http://bugs.python.org/issue9120 11 tracebacks eat up memory by holding references to locals and gl 7 days open http://bugs.python.org/issue1565525 9 remove numbers from 3-.. level entries in docs toc 1 days closed http://bugs.python.org/issue9106 9 mimetypes.guess_extension result changes after mimetypes.init() 6 days open http://bugs.python.org/issue4963 8 Instance methods are misreporting the number of arguments 824 days open http://bugs.python.org/issue2516 From techtonik at gmail.com Fri Jul 2 18:12:28 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Fri, 2 Jul 2010 19:12:28 +0300 Subject: [Python-Dev] Are you ready for Mercurial migration? In-Reply-To: References: <4C2DFD49.70507@netwok.org> Message-ID: On Fri, Jul 2, 2010 at 6:28 PM, Brian Curtin wrote: >> Do you plan to read ~100 page long hginit.com tutorial after *it happens? > > That tutorial is not ~100 pages. It's actually a good tutorial. That's why I posted it here, but it still >80 pages in my browser. > Feel free to copy that into Google Wave. Thanks for feedback. It appears that I overestimated the ubiquitousness of Google Wave. It already has some valuable feedback and I would really prefer to carry out all discussions there. In particular Gnome project members estimated as very important the point of publishing instructions about how to use new DVCS workflow _before_ migration. We don't have this for the moment. -- anatoly t. From dickinsm at gmail.com Fri Jul 2 18:26:55 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 2 Jul 2010 17:26:55 +0100 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: On Fri, Jul 2, 2010 at 3:44 PM, Craig Citro wrote: >> Whoa. ?That's very peculiar looking bytecode. ?Is dis.dis behaving as >> it should here? >> BTW, I think you want 'raise TypeError', not 'raise TypeError()'. >> > > Yep, that's embarrassing. I was being lazy: I was expecting different > bytecodes, and I got it ... so I apparently didn't bother to actually > read the bytecodes? It's interesting -- if I'd just had TypeError > instead of TypeError(), I would have found this out, because "raise > TypeError" is not the bytes representation of a valid sequence of > bytecodes. ;) Ah; interesting. Well clearly that's exactly what I meant when I said 'raise TypeError' was better than 'raise TypeError()'. ;-) (Actually, I confess to being similarly embarrassed: I have no idea *what* I was thinking there, since 'raise TypeError()' is just as valid, if a little less idiomatic.) > That said, I totally concede Martin's point -- this is an > implementation-specific thing. It happens that all the major Python > implementations compile to some VM, and I'd bet that these two compile > to different bytecodes on any of them, but that doesn't preclude > another implementation from making a different choice there. Agreed. Mark From craigcitro at gmail.com Fri Jul 2 18:51:08 2010 From: craigcitro at gmail.com (Craig Citro) Date: Fri, 2 Jul 2010 09:51:08 -0700 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: > But you would be taking a module that will compile and making it uncompilable. > You're absolutely right, and since I definitely *don't* think that the program "raise TypeError" should cause a CompileError, you could say it's safer to have a simple rule like "vaild syntax => will compile" -- it's probably a slippery slope once you start deciding which bits of semantics raise CompileErrors and which don't. However, in this particular case, here's a question: *why* would someone write "return 1 + '1'"? Did they do it *knowing* what would happen, or because they just didn't realize it was just an error? * If they knew what it was going to do, then I'd say shame on them -- they should have just raised a TypeError instead, and anyone who comes along to read or maintain that code would thank them for the change. My impression is that we generally try to discourage people from writing "tricky" code with Python, and this would do exactly that. * If, on the other hand, it's an accident, then I think it's a service to the user to let them know as soon as possible. Here I'm thinking both of someone new to Python, or even a seasoned pro who makes a few "quick fixes" before sending some code to someone, and doesn't happen to test that code path before handing it off. Either way, I personally prefer the CompileError -- it helps me catch a stupid mistake if I've made one, and it prevents other people from writing code I find less clear. My real motive, though, is that I'd like to have more freedom for Python implementations, *especially* things that let you make more decisions at compile-time. (This is no doubt influenced by the fact that I've spent a lot of time thinking about Cython lately.) In this case, I see it as a win-win -- it gives more freedom to the folks writing the implementation, and (personally) I find it more pleasing as a user. Again, I don't think this *particular* case allows us to do something awesome behind the scenes with Cython -- but the community starting to consider changes of this ilk *would* be a big win, I think. -cc From g.brandl at gmx.net Fri Jul 2 18:54:14 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 02 Jul 2010 18:54:14 +0200 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: Am 02.07.2010 18:51, schrieb Craig Citro: >> But you would be taking a module that will compile and making it uncompilable. >> > > You're absolutely right, and since I definitely *don't* think that the > program "raise TypeError" should cause a CompileError, you could say > it's safer to have a simple rule like "vaild syntax => will compile" > -- it's probably a slippery slope once you start deciding which bits > of semantics raise CompileErrors and which don't. > > However, in this particular case, here's a question: *why* would > someone write "return 1 + '1'"? Did they do it *knowing* what would > happen, or because they just didn't realize it was just an error? > > * If they knew what it was going to do, then I'd say shame on them -- > they should have just raised a TypeError instead, and anyone who comes > along to read or maintain that code would thank them for the change. "1/0" is much faster to type than "raise SomeError" and serves the same purpose sometimes for debugging purposes. Let's not forget that not all code is written for eternity :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From steve at holdenweb.com Fri Jul 2 18:55:56 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 02 Jul 2010 12:55:56 -0400 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> Message-ID: Fred Drake wrote: > On Fri, Jul 2, 2010 at 8:34 AM, Antoine Pitrou wrote: >> The two sets of repositories use different conversion tools and rules. >> They have nothing in common (different changeset IDs, different >> metadata, different branch/clone layout). > > I'd love to see a more detailed description of this, including why > someone new to Mercurial would choose one over the other. > > This information really belongs in www.python.org/dev/ rather than > only in the mailing list. > +1 As does a recent essay by Eli Bendersky, IMO. I believe it could lower the barriers for entry into the "non-committer" class of developer. This should make it easier for people to adapt Python to their own purposes whether or not they want to contribute to the open source code base, but also encourage people to investigate the compiler's innards. http://eli.thegreenplace.net/2010/06/30/python-internals-adding-a-new-statement-to-python/ regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From ncoghlan at gmail.com Fri Jul 2 19:00:20 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 3 Jul 2010 03:00:20 +1000 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: On Sat, Jul 3, 2010 at 2:51 AM, Craig Citro wrote: >> But you would be taking a module that will compile and making it uncompilable. >> > > You're absolutely right, and since I definitely *don't* think that the > program "raise TypeError" should cause a CompileError, you could say > it's safer to have a simple rule like "vaild syntax => will compile" > -- it's probably a slippery slope once you start deciding which bits > of semantics raise CompileErrors and which don't. > > However, in this particular case, here's a question: *why* would > someone write "return 1 + '1'"? Did they do it *knowing* what would > happen, or because they just didn't realize it was just an error? To test that adding a string to an integer raises TypeError at runtime. That is, something along the lines of: with self.assertRaises(TypeError): 1 + "1" If an end user is doing it rather than an implementation's own test suite... well, I have no idea why anyone else would want to do that :) > My real motive, though, is that I'd like to have more freedom for > Python implementations, *especially* things that let you make more > decisions at compile-time. (This is no doubt influenced by the fact > that I've spent a lot of time thinking about Cython lately.) In this > case, I see it as a win-win -- it gives more freedom to the folks > writing the implementation, and (personally) I find it more pleasing > as a user. Again, I don't think this *particular* case allows us to do > something awesome behind the scenes with Cython -- but the community > starting to consider changes of this ilk *would* be a big win, I > think. It's definitely something of a grey area. The existing test suite would likely fail if "obviously insane" operations between literals started throwing SyntaxError, but it would be possible to classify some of those tests as implementation specific. However, an implementation that did that would need to completely fork any affected parts of the test suite, since the implementation specific test decorators won't help protect against failures to compile the code. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From steve at holdenweb.com Fri Jul 2 19:01:55 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 02 Jul 2010 13:01:55 -0400 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <87630yxcck.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <87630yxcck.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: Stephen J. Turnbull wrote: > anatoly techtonik writes: > > > Why this transition is not described in PEP? > > Please reread the whole thread, and the PEP. > > PEP 385 is *incomplete* (see the red Warning at the top), and the > original proponent *is not going to have enough time to complete it > soon*. That being the case, Martin suggested that either we find a > new proponent or postpone the transition to next year. We're > currently in the process of finding a new proponent and supporting > cast, and determining what, if anything, we're going to do about the > transition over the next few months. > > There is no reason at this point to suppose the transition can't be > complete by the end of summer. However, as always, the devil is in > the details, and one of them may be a showstopper. We'll just have to > see about that. > And, as always, a way to get better insight and help speed the process along is to join the cast ... regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From solipsis at pitrou.net Fri Jul 2 19:12:31 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 2 Jul 2010 19:12:31 +0200 Subject: [Python-Dev] Mercurial migration readiness References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> Message-ID: <20100702191231.3317486d@pitrou.net> On Fri, 02 Jul 2010 12:55:56 -0400 Steve Holden wrote: > Fred Drake wrote: > > On Fri, Jul 2, 2010 at 8:34 AM, Antoine Pitrou wrote: > >> The two sets of repositories use different conversion tools and rules. > >> They have nothing in common (different changeset IDs, different > >> metadata, different branch/clone layout). > > > > I'd love to see a more detailed description of this, including why > > someone new to Mercurial would choose one over the other. > > > > This information really belongs in www.python.org/dev/ rather than > > only in the mailing list. > > > +1 > > As does a recent essay by Eli Bendersky, IMO. I believe it could lower > the barriers for entry into the "non-committer" class of developer. This > should make it easier for people to adapt Python to their own purposes > whether or not they want to contribute to the open source code base, but > also encourage people to investigate the compiler's innards. With the moratorium on language constructs and builtins (not only in letter until 3.3, but more generally in spirit), I don't think we should encourage it at all, for such contributions would surely be rejected. Rather than fancy syntax propositions or wild semantic changes/enhancements (such as have often been proposed on python-ideas), what we need is humbler but more useful work on stdlib bugs and improvements, as well as documentation and tutorials. (what's more, those two kinds of contributions are likely to attract two different kinds of people, which means that people whose syntax patches get refused won't necessarily start contributing stdlib or documentation patches...) Regards Antoine. From steve at holdenweb.com Fri Jul 2 19:12:27 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 02 Jul 2010 13:12:27 -0400 Subject: [Python-Dev] Mercurial migration readiness (was: Taking over the Mercurial Migration) In-Reply-To: References: Message-ID: Georg Brandl wrote: > Am 02.07.2010 16:17, schrieb anatoly techtonik: >> On Fri, Jul 2, 2010 at 4:53 PM, Georg Brandl wrote: >>>> What is the problem with realtime synchronization and working with >>>> already up to date Mercurial mirror of central SVN repository? >>> The specifics of the conversion process are not nailed down yet. >>> Therefore, the exact translation of SVN to Hg commits will change, >>> and with it the Mercurial revision IDs, for example. >> Does anybody here know how Mercurial calculates the IDs? From that I >> remember it is author + message + diff content. What can change there? > > The parents IDs also are part of that hash. Apart from that, author and > message can change as well, since SVN/CVS-style committer names will be > mapped to Mercurial-style name + email, and the message can have > "[rev ...]" appended or prepended or not. > >>>>>> Development will continue in SVN >>>>>> repository until everybody is ready for final migration in X weeks >>>>>> later. Is that right? >>>>> No; as soon as we switch, SVN will be read-only. >>>> Why don't allow people who already know Mercurial use Mercurial and >>>> those who prefer Subversion use that. If Mercurial allows to submit to >>>> Subversion - why people can't use that while we writing tutorials and >>>> answering question about workflow? >>> I don't think that we have enough manpower to maintain such a bridge >>> indefinitely. >> It doesn't require manpower. It requires automation. > > And who, do you think, is going to implement that automation? > >> Considering that the biggest problem now is to get sane lossless >> conversion, we should elaborate on getting this in place. > > Ah, the mysterious "we". > >> After that I would still follow >> the path of setting realtime mirror for X weeks that could be >> replicated by bitbucket, launchpad and other services to see how >> people pickup the changes. >> >> As PEP 384 says - the transition is mostly to make lives of outside >> contributors easier. Core developers can wait for a while. > > Anatoly, I don't know what you are trying to achieve here. The decision > to switch to Mercurial as soon as possible given our manpower restriction > has been made at PyCon 2009. Since then, there has been progress, albeit > slow, towards that switch, and now that has come into reach, you start > questioning anything and everything. This is neither productive, nor > have you shown any willingness to actually *do* something to help. You > claim you do not have enough time for that; looking at the multitude of > topics you're trying to force a discussion about, I wonder if you couldn't > write one decent patch instead of ten complaining mails, and make both > you and us happier in the process. > +1 That's the spirit, Georg. Anatoly, I had an email along these lines saved pending transmission, but I have deleted it now. Let's make things better rather than dwell on how rotten everything is and how it could be /so/ much better if "we" would get off our butts and do {stuff}. ;-) The more {stuff} done I can point to as PSF chairman the easier it will be to justify raising funds to help make things better still. I'd rather see devs developing than administering systems. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From craigcitro at gmail.com Fri Jul 2 19:13:05 2010 From: craigcitro at gmail.com (Craig Citro) Date: Fri, 2 Jul 2010 10:13:05 -0700 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: > "1/0" is much faster to type than "raise SomeError" and serves the same > purpose sometimes for debugging purposes. ?Let's not forget that not > all code is written for eternity :) > Doesn't "raise" do the same thing for just two extra characters? I agree that not all code lives forever -- but I bet we all have stories about "debugging code" living on a lot longer than it should have. ;) -cc From jnoller at gmail.com Fri Jul 2 19:16:35 2010 From: jnoller at gmail.com (Jesse Noller) Date: Fri, 2 Jul 2010 13:16:35 -0400 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <20100702191231.3317486d@pitrou.net> References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <20100702191231.3317486d@pitrou.net> Message-ID: On Fri, Jul 2, 2010 at 1:12 PM, Antoine Pitrou wrote: > On Fri, 02 Jul 2010 12:55:56 -0400 > Steve Holden wrote: >> Fred Drake wrote: >> > On Fri, Jul 2, 2010 at 8:34 AM, Antoine Pitrou wrote: >> >> The two sets of repositories use different conversion tools and rules. >> >> They have nothing in common (different changeset IDs, different >> >> metadata, different branch/clone layout). >> > >> > I'd love to see a more detailed description of this, including why >> > someone new to Mercurial would choose one over the other. >> > >> > This information really belongs in www.python.org/dev/ rather than >> > only in the mailing list. >> > >> +1 >> >> As does a recent essay by Eli Bendersky, IMO. I believe it could lower >> the barriers for entry into the "non-committer" class of developer. This >> should make it easier for people to adapt Python to their own purposes >> whether or not they want to contribute to the open source code base, but >> also encourage people to investigate the compiler's innards. > > With the moratorium on language constructs and builtins (not only in > letter until 3.3, but more generally in spirit), I don't think we should > encourage it at all, for such contributions would surely be rejected. > > Rather than fancy syntax propositions or wild semantic > changes/enhancements (such as have often been proposed on > python-ideas), what we need is humbler but more useful work on stdlib > bugs and improvements, as well as documentation and tutorials. > > (what's more, those two kinds of contributions are likely to attract > two different kinds of people, which means that people whose syntax > patches get refused won't necessarily start contributing stdlib or > documentation patches...) > > Regards > > Antoine. I completely and wholeheartedly agree with Antoine. I'm hoping the sprints thing will help with this - the stuff outside of the "deep core" needs a lot more help and TLC. jesse From steve at holdenweb.com Fri Jul 2 19:17:59 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 02 Jul 2010 13:17:59 -0400 Subject: [Python-Dev] Are you ready for Mercurial migration? In-Reply-To: References: <4C2DFD49.70507@netwok.org> Message-ID: anatoly techtonik wrote: > On Fri, Jul 2, 2010 at 6:28 PM, Brian Curtin wrote: >>> Do you plan to read ~100 page long hginit.com tutorial after *it happens? >> That tutorial is not ~100 pages. It's actually a good tutorial. > > That's why I posted it here, but it still >80 pages in my browser. > >> Feel free to copy that into Google Wave. > > Thanks for feedback. It appears that I overestimated the > ubiquitousness of Google Wave. It already has some valuable feedback > and I would really prefer to carry out all discussions there. > > In particular Gnome project members estimated as very important the > point of publishing instructions about how to use new DVCS workflow > _before_ migration. We don't have this for the moment. My own experience (I was pretty "meh" about it when people were tweeting "tweet me for an invite to the wave") is that you have to persist, and learn different habits for each set of collaborators. I suppose I should approve of it on account of how it improves geek social skills. If the wave were to result in good documentation about how to *get* ready that would be an amazingly useful contribution. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Fri Jul 2 19:20:00 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 02 Jul 2010 13:20:00 -0400 Subject: [Python-Dev] Are you ready for Mercurial migration? In-Reply-To: References: <20100702170819.793047d4@pitrou.net> Message-ID: <4C2E1FC0.6080402@holdenweb.com> Perhaps at this stage you could actually start producing, then, and use up a bit less bandwidth on this channel (on this matter, at least) until you have results to report? regards Steve anatoly techtonik wrote: > Antoine, I value you contribution to `hgsvn` project, and this thread > is not a personal accusation of anybody in making proper transition - > please understand that I would like to see the opinion of people who > preferred not to be involved in lengthy discussions. For personal > pretensions against me - please start a new thread and no hijack. > > I am trying to solve the same problem as you are - make the Mercurial > migration, but make it as painless as possible. > > On Fri, Jul 2, 2010 at 6:08 PM, Antoine Pitrou wrote: >>> To shed some light on the readiness of Python community for the switch >>> I've opened public Google Wave. Please add your opinion if you can and >>> send this link to other contributors you may know: >> Am I the only one to think this should really stop? >> >> By "this", I mean the flow of complaints and dubious "recommendations" >> you send here and on the bug tracker. We are volunteers, we don't need >> a boss, especially not one who prefers arguing about workflow rather >> than addressing concrete issues. >> >> I'm not sure if other people are finding those (Anatoly's) messages >> constructive and insightful. To me it looks like they are wasting the >> aggregate signal to noise ratio. >> >> Antoine. >> >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/techtonik%40gmail.com >> -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Fri Jul 2 19:20:00 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 02 Jul 2010 13:20:00 -0400 Subject: [Python-Dev] Are you ready for Mercurial migration? In-Reply-To: References: <20100702170819.793047d4@pitrou.net> Message-ID: <4C2E1FC0.6080402@holdenweb.com> Perhaps at this stage you could actually start producing, then, and use up a bit less bandwidth on this channel (on this matter, at least) until you have results to report? regards Steve anatoly techtonik wrote: > Antoine, I value you contribution to `hgsvn` project, and this thread > is not a personal accusation of anybody in making proper transition - > please understand that I would like to see the opinion of people who > preferred not to be involved in lengthy discussions. For personal > pretensions against me - please start a new thread and no hijack. > > I am trying to solve the same problem as you are - make the Mercurial > migration, but make it as painless as possible. > > On Fri, Jul 2, 2010 at 6:08 PM, Antoine Pitrou wrote: >>> To shed some light on the readiness of Python community for the switch >>> I've opened public Google Wave. Please add your opinion if you can and >>> send this link to other contributors you may know: >> Am I the only one to think this should really stop? >> >> By "this", I mean the flow of complaints and dubious "recommendations" >> you send here and on the bug tracker. We are volunteers, we don't need >> a boss, especially not one who prefers arguing about workflow rather >> than addressing concrete issues. >> >> I'm not sure if other people are finding those (Anatoly's) messages >> constructive and insightful. To me it looks like they are wasting the >> aggregate signal to noise ratio. >> >> Antoine. >> >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/techtonik%40gmail.com >> -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From ncoghlan at gmail.com Fri Jul 2 19:28:44 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 3 Jul 2010 03:28:44 +1000 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: On Sat, Jul 3, 2010 at 3:13 AM, Craig Citro wrote: >> "1/0" is much faster to type than "raise SomeError" and serves the same >> purpose sometimes for debugging purposes. ?Let's not forget that not >> all code is written for eternity :) >> > > Doesn't "raise" do the same thing for just two extra characters? No, raise on its own is only valid in an exception handler. Writing "1/0" is at least somewhat common as an idiom for forcing a ZeroDivisionError in examples and in test harnesses (I know I have used it for both of those things many times). Given the diverse range of uses Python is put to, moving things from runtime to compile time can definitely have significant unexpected consequences (hence why many of us would be hesitant to consider an implementation that made such changes to be an actual Python implementation). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From craigcitro at gmail.com Fri Jul 2 19:28:52 2010 From: craigcitro at gmail.com (Craig Citro) Date: Fri, 2 Jul 2010 10:28:52 -0700 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: > To test that adding a string to an integer raises TypeError at > runtime. That is, something along the lines of: > > ?with self.assertRaises(TypeError): > ? ? 1 + "1" > Well, this would just mean the test suite would have to change -- that test would become something like with self.assertRaises(TypeError): import operator operator.add(1, "1") Of course, checking that the literal syntax now raises a compile error could be ugly: with self.assertRaises(CompileError): eval('1 + "1"') ... or it could move to test_parser. ;) > If an end user is doing it rather than an implementation's own test > suite... well, I have no idea why anyone else would want to do that :) > Exactly -- and if it's a clear win for users, I don't think "it makes test-writing harder but not impossible" should really be a counter-argument. Of course, "there's lots of existing code it would break" is a very good counter-argument ... maybe Py4k. ;) > It's definitely something of a grey area. The existing test suite > would likely fail if "obviously insane" operations between literals > started throwing SyntaxError, but it would be possible to classify > some of those tests as implementation specific. However, an > implementation that did that would need to completely fork any > affected parts of the test suite, since the implementation specific > test decorators won't help protect against failures to compile the > code. > Well, I think there's some momentum towards splitting some of the tests into Python-standard and implementation-specific things, so that they can get shared between implementations, right? As long as clear lines are drawn, isn't it just a question of which bucket these tests go into? -cc From guido at python.org Fri Jul 2 19:44:05 2010 From: guido at python.org (Guido van Rossum) Date: Fri, 2 Jul 2010 10:44:05 -0700 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: On Fri, Jul 2, 2010 at 10:28 AM, Nick Coghlan wrote: > On Sat, Jul 3, 2010 at 3:13 AM, Craig Citro wrote: >>> "1/0" is much faster to type than "raise SomeError" and serves the same >>> purpose sometimes for debugging purposes. ?Let's not forget that not >>> all code is written for eternity :) >>> >> >> Doesn't "raise" do the same thing for just two extra characters? > > No, raise on its own is only valid in an exception handler. Writing > "1/0" is at least somewhat common as an idiom for forcing a > ZeroDivisionError in examples and in test harnesses (I know I have > used it for both of those things many times). > > Given the diverse range of uses Python is put to, moving things from > runtime to compile time can definitely have significant unexpected > consequences (hence why many of us would be hesitant to consider an > implementation that made such changes to be an actual Python > implementation). +1 on not changing this. For one, this will most likely break a large amount of 3rd party and stdlib software -- there are tons of statements like this that are practically unreachable or intentional. Second, I don't think it's going to make the kind of difference the OP is thinking of. Since Python is totally dynamic, and doesn't have macros, the only cases that would be caught would be things you are unlikely to type by accident -- like 1/0 or 1+"1". In other languages that have this behavior, there is usually a benefit where the arguments involved are *variables* whose type is known to the compiler, so it will catch things like (simple C example) #define FOO 0 main() { printf("%d\n", 1/FOO); } However the equivalent Python FOO = 0 def main(): print 1/FOO cannot be rejected at compile time because there is insufficient evidence that the value of FOO won't be changed before main() is called. I even reject the substitution of "raise ZeroDivisionError" for "1/0" since (a) nobody cares about such an optimization, and (b) it would break introspection and invalidate tests. (We have a long history of constant propagation in expressions causing subtle bugs. This could be worse.) -- --Guido van Rossum (python.org/~guido) From cesare.di.mauro at gmail.com Fri Jul 2 19:50:55 2010 From: cesare.di.mauro at gmail.com (Cesare Di Mauro) Date: Fri, 2 Jul 2010 19:50:55 +0200 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: 2010/7/2 Guido van Rossum > On Fri, Jul 2, 2010 at 10:28 AM, Nick Coghlan wrote: > > On Sat, Jul 3, 2010 at 3:13 AM, Craig Citro > wrote: > >>> "1/0" is much faster to type than "raise SomeError" and serves the same > >>> purpose sometimes for debugging purposes. Let's not forget that not > >>> all code is written for eternity :) > >>> > >> > >> Doesn't "raise" do the same thing for just two extra characters? > > > > No, raise on its own is only valid in an exception handler. Writing > > "1/0" is at least somewhat common as an idiom for forcing a > > ZeroDivisionError in examples and in test harnesses (I know I have > > used it for both of those things many times). > > > > Given the diverse range of uses Python is put to, moving things from > > runtime to compile time can definitely have significant unexpected > > consequences (hence why many of us would be hesitant to consider an > > implementation that made such changes to be an actual Python > > implementation). > > +1 on not changing this. > > For one, this will most likely break a large amount of 3rd party and > stdlib software -- there are tons of statements like this that are > practically unreachable or intentional. > > Second, I don't think it's going to make the kind of difference the OP > is thinking of. Since Python is totally dynamic, and doesn't have > macros, the only cases that would be caught would be things you are > unlikely to type by accident -- like 1/0 or 1+"1". In other languages > that have this behavior, there is usually a benefit where the > arguments involved are *variables* whose type is known to the > compiler, so it will catch things like (simple C example) > > #define FOO 0 > main() { > printf("%d\n", 1/FOO); > } > > However the equivalent Python > > FOO = 0 > def main(): > print 1/FOO > > cannot be rejected at compile time because there is insufficient > evidence that the value of FOO won't be changed before main() is > called. > > I even reject the substitution of "raise ZeroDivisionError" for "1/0" > since (a) nobody cares about such an optimization, and (b) it would > break introspection and invalidate tests. (We have a long history of > constant propagation in expressions causing subtle bugs. This could be > worse.) > > -- > --Guido van Rossum (python.org/~guido) > from __future__ import compile_checks Cesare Di Mauro -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Fri Jul 2 20:37:54 2010 From: brett at python.org (Brett Cannon) Date: Fri, 2 Jul 2010 11:37:54 -0700 Subject: [Python-Dev] Are you ready for Mercurial migration? In-Reply-To: References: <20100702170819.793047d4@pitrou.net> Message-ID: On Fri, Jul 2, 2010 at 08:49, anatoly techtonik wrote: > Antoine, I value you contribution to `hgsvn` project, and this thread > is not a personal accusation of anybody in making proper transition - > please understand that I would like to see the opinion of people who > preferred not to be involved in lengthy discussions. People who choose not to be involved in a lengthy discussion on a topic and thus most likely be ill-informed should not participate and expect to be taken seriously. Making statements without having some decent amount of knowledge on the subject just leads to bad decisions. In other words, I don't care about the opinion of uninformed individuals (and I doubt anyone else does either) so you trying to bring them into this is not helping matters. This transition has been in the works since roughly December 2008 when I started evaluating whether moving to a DVCS was beneficial. That has been in the open either through PEPs or this mailing list, even a lightning talk at PyCon 2009. When Dirkjan took on the task (all by himself) to handle the transition that discussion has stayed public, mostly through emails on python-dev. This has been vetted as publicly as possible. So people who wanted to participate when decisions were being made could have. But having you come in here and say that this needs to be discussed again is at least unproductive, if not insulting. This transition will happen once a proper conversion is up and has been pounded on, as Martin has suggested. And the proper documentation will be in place before we hit the switch and turn off svn. Yes, it has been slow, but considering a single individual has been handling all technical aspects it is to be expected. And in case I have not said it publicly, I thank Dirkjan for sticking with this after all this time. > For personal > pretensions against me - please start a new thread and no hijack. Yes hijack. In case you have not noticed practically every reply on this email thread has said "stop stirring up trouble and try to help." All you are doing is muckraking at this point. If you want to help, be constructive instead of trying to drag this transition out and throw up barriers. We are beyond the point of bringing up alternatives to how to handle this. Now we just need manpower to fix the final details so we can get this finished. Letting this thread just go on is not helping anyone, so we are all trying to kill it before any more people waste their time with that wave of yours. > > I am trying to solve the same problem as you are - make the Mercurial > migration, but make it as painless as possible. Then help! Fix the hgsubversion bug that is blocking the transition. Port the dev FAQ over to Mercurial if you don't want to code. Do something constructive! But trying to cause us to change plans that have been moving forward on the scale of years suddenly is not helping make the transition any less painless. All you are doing is pissing people off and causing us to have to waste our time telling you to stop doing what you are doing. To be totally frank, Anatoly, you need to contribute in a positive way and without sounding demanding. Maybe you don't mean to sound demanding, but your emails sometimes come off like you think we should listen to all of your ideas with no backing of experience here or enough evidence to support your ideas. And then rehashing old decisions like this transition, arguing for svn bridges and the like, isn't helping improve people's perception of you. It's gotten to the point where I don't want to read your emails. If you keep up with the way you are interacting with people on this list I for one will add you to my python-dev blacklist filter (which currently consists of a single person, so it is not easy to get put on that list) so that I don't feel angry or frustrated every time I see an email from you. -Brett > > On Fri, Jul 2, 2010 at 6:08 PM, Antoine Pitrou wrote: >> >>> To shed some light on the readiness of Python community for the switch >>> I've opened public Google Wave. Please add your opinion if you can and >>> send this link to other contributors you may know: >> >> Am I the only one to think this should really stop? >> >> By "this", I mean the flow of complaints and dubious "recommendations" >> you send here and on the bug tracker. We are volunteers, we don't need >> a boss, especially not one who prefers arguing about workflow rather >> than addressing concrete issues. >> >> I'm not sure if other people are finding those (Anatoly's) messages >> constructive and insightful. To me it looks like they are wasting the >> aggregate signal to noise ratio. >> >> Antoine. >> >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/techtonik%40gmail.com >> > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From steve at holdenweb.com Fri Jul 2 21:19:42 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 02 Jul 2010 15:19:42 -0400 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <20100702191231.3317486d@pitrou.net> References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <20100702191231.3317486d@pitrou.net> Message-ID: Antoine Pitrou wrote: > On Fri, 02 Jul 2010 12:55:56 -0400 > Steve Holden wrote: >> Fred Drake wrote: >>> On Fri, Jul 2, 2010 at 8:34 AM, Antoine Pitrou wrote: >>>> The two sets of repositories use different conversion tools and rules. >>>> They have nothing in common (different changeset IDs, different >>>> metadata, different branch/clone layout). >>> I'd love to see a more detailed description of this, including why >>> someone new to Mercurial would choose one over the other. >>> >>> This information really belongs in www.python.org/dev/ rather than >>> only in the mailing list. >>> >> +1 >> >> As does a recent essay by Eli Bendersky, IMO. I believe it could lower >> the barriers for entry into the "non-committer" class of developer. This >> should make it easier for people to adapt Python to their own purposes >> whether or not they want to contribute to the open source code base, but >> also encourage people to investigate the compiler's innards. > > With the moratorium on language constructs and builtins (not only in > letter until 3.3, but more generally in spirit), I don't think we should > encourage it at all, for such contributions would surely be rejected. > > Rather than fancy syntax propositions or wild semantic > changes/enhancements (such as have often been proposed on > python-ideas), what we need is humbler but more useful work on stdlib > bugs and improvements, as well as documentation and tutorials. > > (what's more, those two kinds of contributions are likely to attract > two different kinds of people, which means that people whose syntax > patches get refused won't necessarily start contributing stdlib or > documentation patches...) > The point was not to encourage experimentation with wild syntax variants (and Bendersky underlines that he knows perfectly well that Python does not need an "until" statement. THe point is to encourage more people to understand that what goes on under the hood is comprehensible. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From techtonik at gmail.com Fri Jul 2 21:25:08 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Fri, 2 Jul 2010 22:25:08 +0300 Subject: [Python-Dev] SVN <-> HG workflow to split Python Library by Module Message-ID: I planned to publish this proposal when it is finally ready and tested with an assumption that Subversion repository will be online and up-to-date after Mercurial migration. But recent threads showed that currently there is no tested mechanism to sync Subversion repository back with Mercurial, so it will probably quickly outdate, and the proposal won't have a chance to be evaluated. So now is better than never. So, this is a way to split modules from monolithic Subversion repository into several Mercurial mirrors - one mirror for each module (or whatever directory structure you like). This will allow to concentrate your work on only one module at a time ("distutils", "CGIHTTPServer" etc.) without caring much about anything else. Exceptionally useful for occasional external "contributors" like me, and folks on Windows, who don't possess Visual Studio to compile Python and are forced to use whatever version they have installed to create and test patches. Here is a picture if you feel bored - https://docs.google.com/drawings/edit?id=1c9FDQ27BnaIew_1T7Tr-rFg1OCdPVS9w3TQREOkzyjk&hl=en An example of the split distutils module - http://bitbucket.org/techtonik/distutils The split is not perfect, but the process can be polished - it is the first version I managed to get only this morning. More important is that HG repository is incrementally synchronized. The split is not perfect, because in particular I see that documentation dir is not sucked in. But it is a working proof on concept you can test yourself using the code from: http://bitbucket.org/techtonik/python-split You will also need patched version of `hgsvn` from http://bitbucket.org/techtonik/hgsvn How does it work ------------------------- The module is described as a series of paths inside typical Subversion checkout. On the first run `refresh.py` script from `python-split` creates shallow SVN checkout with only required files using distutils.module.def module definition Second run of `refresh.py` imports shallow checkout into Mercurial And the third run imports the rest of the history pulling only changesets relevant to given paths. Workflow ------------- Diagram showed patches that are pulled from local clones of split repositories to master Mercurial mirror, but it won't work this way, because hashes of revisions in direct mirror wont't match hashes in split repositories - that's why some hash lookup/sync procedure is needed to correctly process incoming patches. This workflow works with hash sync only when changes are pushed back to central Subversion repository from local clones (possibly through another intermediate normalizing repository). Changes pushed this way are streamlined and could be downloaded into stable branch of other mirrors as a single line of development. I borrowed streamlining concept from Go contribution guide as it really helps to review chaotic Mercurial commits. http://golang.org/doc/contribute.html#Code_review Maintaining centralized Subversion repository will require additional properties to be set, but this is doable. I don't how to make module split with Mercurial alone. http://mercurial.selenic.com/wiki/ShallowClone is still a draft (and complicated one) and Mercurial 1.6 that released today doesn't contain anything revolutionary to propose an alternative. I am exhausted. -- anatoly t. From jnoller at gmail.com Fri Jul 2 22:01:48 2010 From: jnoller at gmail.com (Jesse Noller) Date: Fri, 2 Jul 2010 16:01:48 -0400 Subject: [Python-Dev] SVN <-> HG workflow to split Python Library by Module In-Reply-To: References: Message-ID: On Fri, Jul 2, 2010 at 3:25 PM, anatoly techtonik wrote: > I planned to publish this proposal when it is finally ready and tested > with an assumption that Subversion repository will be online and > up-to-date after Mercurial migration. But recent threads showed that > currently there is no tested mechanism to sync Subversion repository > back with Mercurial, so it will probably quickly outdate, and the > proposal won't have a chance to be evaluated. So now is better than > never. > > So, this is a way to split modules from monolithic Subversion > repository into several Mercurial mirrors - one mirror for each module > (or whatever directory structure you like). This will allow to > concentrate your work on only one module at a time ("distutils", > "CGIHTTPServer" etc.) without caring much about anything else. > Exceptionally useful for occasional external "contributors" like me, > and folks on Windows, who don't possess Visual Studio to compile > Python and are forced to use whatever version they have installed to > create and test patches. > > Here is a picture if you feel bored - > https://docs.google.com/drawings/edit?id=1c9FDQ27BnaIew_1T7Tr-rFg1OCdPVS9w3TQREOkzyjk&hl=en > An example of the split distutils module - > http://bitbucket.org/techtonik/distutils > > The split is not perfect, but the process can be polished - it is the > first version I managed to get only this morning. More important is > that HG repository is incrementally synchronized. The split is not > perfect, because in particular I see that documentation dir is not > sucked in. But it is a working proof on concept you can test yourself > using the code from: > http://bitbucket.org/techtonik/python-split > You will also need patched version of `hgsvn` from > http://bitbucket.org/techtonik/hgsvn > > How does it work > ------------------------- > The module is described as a series of paths inside typical Subversion checkout. > On the first run `refresh.py` script from `python-split` creates > shallow SVN checkout with only required files using > distutils.module.def module definition > Second run of `refresh.py` imports shallow checkout into Mercurial > And the third run imports the rest of the history pulling only > changesets relevant to given paths. > > Workflow > ------------- > Diagram showed patches that are pulled from local clones of split > repositories to master Mercurial mirror, but it won't work this way, > because hashes of revisions in direct mirror wont't match hashes in > split repositories - that's why some hash lookup/sync procedure is > needed to correctly process incoming patches. This workflow works with > hash sync only when changes are pushed back to central Subversion > repository from local clones (possibly through another intermediate > normalizing repository). Changes pushed this way are streamlined and > could be downloaded into stable branch of other mirrors as a single > line of development. I borrowed streamlining concept from Go > contribution guide as it really helps to review chaotic Mercurial > commits. http://golang.org/doc/contribute.html#Code_review > > Maintaining centralized Subversion repository will require additional > properties to be set, but this is doable. I don't how to make module > split with Mercurial alone. > http://mercurial.selenic.com/wiki/ShallowClone is still a draft (and > complicated one) and Mercurial 1.6 that released today doesn't contain > anything revolutionary to propose an alternative. > > I am exhausted. fwiw - there is a/are plan(s) to break out the stdlib from "core" once the transition is complete, to better allow re-use between the various interpreters. I do not think that "lots of small mirrors/repos" for each library is a net gain. Once the stdlib is broken out; the same thing you have proposed is achievable (within reason) without a proliferation of mirrors/repos/etc. Also, we're not staying on subversion - well, as far I know. jesse From dstanek at dstanek.com Fri Jul 2 22:09:22 2010 From: dstanek at dstanek.com (David Stanek) Date: Fri, 2 Jul 2010 16:09:22 -0400 Subject: [Python-Dev] SVN <-> HG workflow to split Python Library by Module In-Reply-To: References: Message-ID: On Fri, Jul 2, 2010 at 3:25 PM, anatoly techtonik wrote: > I planned to publish this proposal when it is finally ready and tested > with an assumption that Subversion repository will be online and > up-to-date after Mercurial migration. But recent threads showed that > currently there is no tested mechanism to sync Subversion repository > back with Mercurial, so it will probably quickly outdate, and the > proposal won't have a chance to be evaluated. So now is better than > never. > I don't think it would be very hard to update a read-only version of Subversion. It just gets complicated if you plan on committing to both Subversion and Mercurial. Does having many little repos add more value than the pain it creates? -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek From tjreedy at udel.edu Fri Jul 2 22:44:55 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 02 Jul 2010 16:44:55 -0400 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: On 7/2/2010 12:43 AM, Glyph Lefkowitz wrote: >> def f(): return 1 + "1" >> >> instead of compiling something which can't fail to raise an >> exception, would that still be a legal Python implementation? > > I'd say "no". Python has defined semantics in this situation: a > TypeError is raised. The manuals are rather inconsistent about defining the exception semantics. Some parts define the exception returned. Other, equivalent parts, do not. I should start a separate thread on this when I find the examples I once had. -- Terry Jan Reedy From brett at python.org Fri Jul 2 23:37:08 2010 From: brett at python.org (Brett Cannon) Date: Fri, 2 Jul 2010 14:37:08 -0700 Subject: [Python-Dev] SVN <-> HG workflow to split Python Library by Module In-Reply-To: References: Message-ID: On Fri, Jul 2, 2010 at 12:25, anatoly techtonik wrote: > I planned to publish this proposal when it is finally ready and tested > with an assumption that Subversion repository will be online and > up-to-date after Mercurial migration. But recent threads showed that > currently there is no tested mechanism to sync Subversion repository > back with Mercurial, so it will probably quickly outdate, and the > proposal won't have a chance to be evaluated. So now is better than > never. > > So, this is a way to split modules from monolithic Subversion > repository into several Mercurial mirrors - one mirror for each module > (or whatever directory structure you like). This will allow to > concentrate your work on only one module at a time ("distutils", > "CGIHTTPServer" etc.) without caring much about anything else. > Exceptionally useful for occasional external "contributors" like me, > and folks on Windows, who don't possess Visual Studio to compile > Python and are forced to use whatever version they have installed to > create and test patches. But modules do not live in an isolated world; they are dependent on changes made to other modules. Isolating them from other modules whose semantics change during development will lead to skew and improper patches. As for Windows users who don't have Visual Studio, there is a free version that compiles Python fine: http://www.python.org/dev/faq/#id8 . -Brett > > Here is a picture if you feel bored - > https://docs.google.com/drawings/edit?id=1c9FDQ27BnaIew_1T7Tr-rFg1OCdPVS9w3TQREOkzyjk&hl=en > An example of the split distutils module - > http://bitbucket.org/techtonik/distutils > > The split is not perfect, but the process can be polished - it is the > first version I managed to get only this morning. More important is > that HG repository is incrementally synchronized. The split is not > perfect, because in particular I see that documentation dir is not > sucked in. But it is a working proof on concept you can test yourself > using the code from: > http://bitbucket.org/techtonik/python-split > You will also need patched version of `hgsvn` from > http://bitbucket.org/techtonik/hgsvn > > How does it work > ------------------------- > The module is described as a series of paths inside typical Subversion checkout. > On the first run `refresh.py` script from `python-split` creates > shallow SVN checkout with only required files using > distutils.module.def module definition > Second run of `refresh.py` imports shallow checkout into Mercurial > And the third run imports the rest of the history pulling only > changesets relevant to given paths. > > Workflow > ------------- > Diagram showed patches that are pulled from local clones of split > repositories to master Mercurial mirror, but it won't work this way, > because hashes of revisions in direct mirror wont't match hashes in > split repositories - that's why some hash lookup/sync procedure is > needed to correctly process incoming patches. This workflow works with > hash sync only when changes are pushed back to central Subversion > repository from local clones (possibly through another intermediate > normalizing repository). Changes pushed this way are streamlined and > could be downloaded into stable branch of other mirrors as a single > line of development. I borrowed streamlining concept from Go > contribution guide as it really helps to review chaotic Mercurial > commits. http://golang.org/doc/contribute.html#Code_review > > Maintaining centralized Subversion repository will require additional > properties to be set, but this is doable. I don't how to make module > split with Mercurial alone. > http://mercurial.selenic.com/wiki/ShallowClone is still a draft (and > complicated one) and Mercurial 1.6 that released today doesn't contain > anything revolutionary to propose an alternative. > > I am exhausted. > -- > anatoly t. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From g.brandl at gmx.net Fri Jul 2 23:57:15 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 02 Jul 2010 23:57:15 +0200 Subject: [Python-Dev] SVN <-> HG workflow to split Python Library by Module In-Reply-To: References: Message-ID: Am 02.07.2010 22:01, schrieb Jesse Noller: >> I am exhausted. > > fwiw - there is a/are plan(s) to break out the stdlib from "core" once > the transition is complete, to better allow re-use between the various > interpreters. I do not think that "lots of small mirrors/repos" for > each library is a net gain. Once the stdlib is broken out; the same > thing you have proposed is achievable (within reason) without a > proliferation of mirrors/repos/etc. > > Also, we're not staying on subversion - well, as far I know. At least not parallel to Mercurial. Definitely. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From jnoller at gmail.com Sat Jul 3 00:03:37 2010 From: jnoller at gmail.com (Jesse Noller) Date: Fri, 2 Jul 2010 18:03:37 -0400 Subject: [Python-Dev] SVN <-> HG workflow to split Python Library by Module In-Reply-To: References: Message-ID: On Fri, Jul 2, 2010 at 5:57 PM, Georg Brandl wrote: > Am 02.07.2010 22:01, schrieb Jesse Noller: > >>> I am exhausted. >> >> fwiw - there is a/are plan(s) to break out the stdlib from "core" once >> the transition is complete, to better allow re-use between the various >> interpreters. I do not think that "lots of small mirrors/repos" for >> each library is a net gain. Once the stdlib is broken out; the same >> thing you have proposed is achievable (within reason) without a >> proliferation of mirrors/repos/etc. >> >> Also, we're not staying on subversion - well, as far I know. > > At least not parallel to Mercurial. ?Definitely. > > Georg Well, I was worried I missed a meeting in the secret clubhouse. From mg at lazybytes.net Sat Jul 3 00:31:03 2010 From: mg at lazybytes.net (Martin Geisler) Date: Sat, 03 Jul 2010 00:31:03 +0200 Subject: [Python-Dev] Mercurial migration readiness References: <874ogibva5.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87vd8xzfbs.fsf@hbox.dyndns.org> "Stephen J. Turnbull" writes: > anatoly techtonik writes: > > > To be prepared for conflicts I review code with `svn log -vr > > base:head` before updating. But with Mercurial I see two major > > problems with my workflow (which I am unlikely to change for the > > next few weeks due to heavy automation): > > 1. No shallow checkouts - that means that I have to copy 200Mb+ for > > every patch I have > > No, you don't. You make links to 200MB+ (unless you're on Windows, > where I don't know how this works). Surprisingly, it works there too due to NTFS hardlinks :-) > This is much cheaper than copying, though not as cheap as in git. I > don't hesitate to make branches in Mercurial. You can have branches inside your repository with Mercurial too, just like in Git. The bookmarks extension is for that. With Mercurial 1.6, you can push and pull bookmarks -- they are no longer local-only. I may work more sequentially than most, but I do all my work on Mercurial using a single clone. > > 3. There is no clear branch separation in my working copy. I don't > > know how to diff files in different branches without copying 200Mb+ > > clone. I don't know how to diff my files with remote repository > > without pulling it. > > If you're at all serious about working with Mercurial, you don't do > any operations except pull (and push, if you're a committer) against > the remote. You keep a local pristine mirror of the remote > repository, which you update frequently, and all of your work revolves > around that rather than around contact with upstream. > > This does mean that when upstream uses a repo-per-branch model you > have to keep multiple mirrors. On the other hand, you can work > directly in branches that you work on only infrequently, and also use > Mercurial queues to avoid making branches if you don't want to. That is one option and I think "The Definitive Guide" advocates it. However, you are free to mix the upstream clones into a single clone if you want (with 'hg pull' from different upstream repositories), or you can keep them separate. If the upstream uses a single repository with multiple named brancehs, then you can still maintain separate clones locally, say one per named branch (with 'hg clone http://...#branch' style clones). There is really no contraints on how you can setup your local repositories. -- Martin Geisler Mercurial links: http://mercurial.ch/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From steve at pearwood.info Sat Jul 3 01:38:13 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 3 Jul 2010 09:38:13 +1000 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> Message-ID: <201007030938.13617.steve@pearwood.info> Wow! I didn't expect anywhere near this amount of interest. Thanks to all who responded. One small comment follows: On Sat, 3 Jul 2010 03:44:05 am Guido van Rossum wrote: > On Fri, Jul 2, 2010 at 10:28 AM, Nick Coghlan wrote: > > Given the diverse range of uses Python is put to, moving things > > from runtime to compile time can definitely have significant > > unexpected consequences (hence why many of us would be hesitant to > > consider an implementation that made such changes to be an actual > > Python implementation). > > +1 on not changing this. > > For one, this will most likely break a large amount of 3rd party and > stdlib software -- there are tons of statements like this that are > practically unreachable or intentional. > > Second, I don't think it's going to make the kind of difference the > OP is thinking of. As I said in my initial post, this was a hypothetical, not a serious suggestion for a change, so I'm not pushing for it. I'm not even sure I would vote for change -- somebody asked on #python whether implementations were free to reject semantically impossible code at compile-time, my first instinct was to say Yes, and then I thought about it a bit more and thought "maybe not" and decided to ask here. I'm glad I did, because I've learned a lot. *If* such a change was made, I think it would have to be controlled by a command-line switch or environmental variable, like -O, and documented as potentially changing the behaviour of the program. But given how few accidental errors are likely to be caught by this, I doubt it would be of any real benefit. Thanks to all who answered! -- Steven D'Aprano From martin at v.loewis.de Sat Jul 3 01:54:31 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 03 Jul 2010 01:54:31 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: Message-ID: <4C2E7C37.1050006@v.loewis.de> > I don't know about "try" -- personally I don't see a difference for > the release procedure, no matter where the source comes from. I guess you haven't done a release yet, then :-) Assuming you are going to use https://svn.python.org/projects/sandbox/trunk/release/release.py then you'll have to port that to Mercurial, first. Or, you write your own release tool, as any serious release manager did so far :-) As for the Windows build, I don't know how to do "externals", yet. In particular, I don't think it is a good idea to have all externals in a single Mercurial clone: due to versioning, we have multiple copies of them, which will take quite some space. As a minor problem: I currently have a batch file head.bat, which updates sysmodule.c after a switch to expand HeadURL correctly. Not sure whether this will be needed on Mercurial, or whether the build identification patch is able to learn about tags in the first place. Finally, I wonder how the documentation build process will continue to integrate subversion. I guess you just need to have an svn binary around when building the documentation. Regards, Martin From ben+python at benfinney.id.au Sat Jul 3 01:55:30 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 03 Jul 2010 09:55:30 +1000 Subject: [Python-Dev] Are you ready for Mercurial migration? References: <4C2DFD49.70507@netwok.org> Message-ID: <871vblh219.fsf@benfinney.id.au> anatoly techtonik writes: > Thanks for feedback. It appears that I overestimated the > ubiquitousness of Google Wave. It already has some valuable feedback > and I would really prefer to carry out all discussions there. And yet it excludes anyone who doesn't want an account with a specific organisation. A barrier that is not present for discussions in this forum. -- \ Lucifer: ?Just sign the Contract, sir, and the Piano is yours.? | `\ Ray: ?Sheesh! This is long! Mind if I sign it now and read it | _o__) later?? ?http://www.achewood.com/ | Ben Finney From martin at v.loewis.de Sat Jul 3 01:57:26 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 03 Jul 2010 01:57:26 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> Message-ID: <4C2E7CE6.9060501@v.loewis.de> Am 02.07.2010 15:09, schrieb Fred Drake: > On Fri, Jul 2, 2010 at 8:34 AM, Antoine Pitrou wrote: >> The two sets of repositories use different conversion tools and rules. >> They have nothing in common (different changeset IDs, different >> metadata, different branch/clone layout). > > I'd love to see a more detailed description of this, including why > someone new to Mercurial would choose one over the other. I think someone new to Mercurial shouldn't choose either one. Just sit back and wait for the real migration to happen. Regards, Martin From martin at v.loewis.de Sat Jul 3 02:06:14 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 03 Jul 2010 02:06:14 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: Message-ID: <4C2E7EF6.3030005@v.loewis.de> > Can somebody comment on how much ongoing effort is required to keep that > mirror running? As everybody else indicated: none (I believe). FWIW, the bzr mirror wasn't that self-maintaining: the process started to consume too much memory and got killed; the cron jobs broke, and so on, so we finally switched it of (in favor of the Launchpad mirror that also existed). Regards, Martin From cournape at gmail.com Sat Jul 3 02:17:15 2010 From: cournape at gmail.com (David Cournapeau) Date: Sat, 3 Jul 2010 09:17:15 +0900 Subject: [Python-Dev] SVN <-> HG workflow to split Python Library by Module In-Reply-To: References: Message-ID: On Sat, Jul 3, 2010 at 6:37 AM, Brett Cannon wrote: > On Fri, Jul 2, 2010 at 12:25, anatoly techtonik wrote: >> I planned to publish this proposal when it is finally ready and tested >> with an assumption that Subversion repository will be online and >> up-to-date after Mercurial migration. But recent threads showed that >> currently there is no tested mechanism to sync Subversion repository >> back with Mercurial, so it will probably quickly outdate, and the >> proposal won't have a chance to be evaluated. So now is better than >> never. >> >> So, this is a way to split modules from monolithic Subversion >> repository into several Mercurial mirrors - one mirror for each module >> (or whatever directory structure you like). This will allow to >> concentrate your work on only one module at a time ("distutils", >> "CGIHTTPServer" etc.) without caring much about anything else. >> Exceptionally useful for occasional external "contributors" like me, >> and folks on Windows, who don't possess Visual Studio to compile >> Python and are forced to use whatever version they have installed to >> create and test patches. > > But modules do not live in an isolated world; they are dependent on > changes made to other modules. Isolating them from other modules whose > semantics change during development will lead to skew and improper > patches. I cannot comment on the original proposal, but this issue has known solutions in git, in the form of submodules. I believe hg has something similar with the forest extension http://mercurial.selenic.com/wiki/ForestExtension David From martin at v.loewis.de Sat Jul 3 02:17:40 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 03 Jul 2010 02:17:40 +0200 Subject: [Python-Dev] Are you ready for Mercurial migration? In-Reply-To: <20100702170819.793047d4@pitrou.net> References: <20100702170819.793047d4@pitrou.net> Message-ID: <4C2E81A4.8060608@v.loewis.de> Am 02.07.2010 17:08, schrieb Antoine Pitrou: > On Fri, 2 Jul 2010 17:29:02 +0300 > anatoly techtonik wrote: >> To shed some light on the readiness of Python community for the switch >> I've opened public Google Wave. Please add your opinion if you can and >> send this link to other contributors you may know: > > Am I the only one to think this should really stop? You are certainly not alone. However, I've given up responding to anatoly techtonik, knowing that it will be futile, and a waste of time. Regards, Martin From brett at python.org Sat Jul 3 02:34:33 2010 From: brett at python.org (Brett Cannon) Date: Fri, 2 Jul 2010 17:34:33 -0700 Subject: [Python-Dev] SVN <-> HG workflow to split Python Library by Module In-Reply-To: References: Message-ID: On Fri, Jul 2, 2010 at 17:17, David Cournapeau wrote: > On Sat, Jul 3, 2010 at 6:37 AM, Brett Cannon wrote: >> On Fri, Jul 2, 2010 at 12:25, anatoly techtonik wrote: >>> I planned to publish this proposal when it is finally ready and tested >>> with an assumption that Subversion repository will be online and >>> up-to-date after Mercurial migration. But recent threads showed that >>> currently there is no tested mechanism to sync Subversion repository >>> back with Mercurial, so it will probably quickly outdate, and the >>> proposal won't have a chance to be evaluated. So now is better than >>> never. >>> >>> So, this is a way to split modules from monolithic Subversion >>> repository into several Mercurial mirrors - one mirror for each module >>> (or whatever directory structure you like). This will allow to >>> concentrate your work on only one module at a time ("distutils", >>> "CGIHTTPServer" etc.) without caring much about anything else. >>> Exceptionally useful for occasional external "contributors" like me, >>> and folks on Windows, who don't possess Visual Studio to compile >>> Python and are forced to use whatever version they have installed to >>> create and test patches. >> >> But modules do not live in an isolated world; they are dependent on >> changes made to other modules. Isolating them from other modules whose >> semantics change during development will lead to skew and improper >> patches. > > I cannot comment on the original proposal, but this issue has known > solutions in git, in the form of submodules. I believe hg has > something similar with the forest extension > > http://mercurial.selenic.com/wiki/ForestExtension Mercurial has subrepo support, but that doesn't justify the need to have every module in its own repository so they can be checked out individually. From greg.ewing at canterbury.ac.nz Sat Jul 3 03:39:07 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 03 Jul 2010 13:39:07 +1200 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: <4C2E94BB.1090901@canterbury.ac.nz> Stefan Behnel wrote: > So, would it still be Python if it folded > > 1 + "1" > > into > > raise TypeError() > > at compile time? It would have to be raise TypeError("Exactly the message that would have been produced at run time") That might be acceptable, but then you have to ask, is it really worth performing this optimisation? The overhead of raising and handling the exception is likely to completely swamp that of executing the original code. -- Greg From greg.ewing at canterbury.ac.nz Sat Jul 3 03:43:04 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 03 Jul 2010 13:43:04 +1200 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: <4C2E95A8.9070209@canterbury.ac.nz> Craig Citro wrote: > Ok, I'm obviously being silly here, but sure you can: > >>>>dis.dis("raise TypeError()") If producing different bytecode were considered a reason against performing an optimisation, then no code optimisations would be permissible at all! -- Greg From cournape at gmail.com Sat Jul 3 03:45:54 2010 From: cournape at gmail.com (David Cournapeau) Date: Sat, 3 Jul 2010 10:45:54 +0900 Subject: [Python-Dev] SVN <-> HG workflow to split Python Library by Module In-Reply-To: References: Message-ID: On Sat, Jul 3, 2010 at 9:34 AM, Brett Cannon wrote: > On Fri, Jul 2, 2010 at 17:17, David Cournapeau wrote: >> On Sat, Jul 3, 2010 at 6:37 AM, Brett Cannon wrote: >>> On Fri, Jul 2, 2010 at 12:25, anatoly techtonik wrote: >>>> I planned to publish this proposal when it is finally ready and tested >>>> with an assumption that Subversion repository will be online and >>>> up-to-date after Mercurial migration. But recent threads showed that >>>> currently there is no tested mechanism to sync Subversion repository >>>> back with Mercurial, so it will probably quickly outdate, and the >>>> proposal won't have a chance to be evaluated. So now is better than >>>> never. >>>> >>>> So, this is a way to split modules from monolithic Subversion >>>> repository into several Mercurial mirrors - one mirror for each module >>>> (or whatever directory structure you like). This will allow to >>>> concentrate your work on only one module at a time ("distutils", >>>> "CGIHTTPServer" etc.) without caring much about anything else. >>>> Exceptionally useful for occasional external "contributors" like me, >>>> and folks on Windows, who don't possess Visual Studio to compile >>>> Python and are forced to use whatever version they have installed to >>>> create and test patches. >>> >>> But modules do not live in an isolated world; they are dependent on >>> changes made to other modules. Isolating them from other modules whose >>> semantics change during development will lead to skew and improper >>> patches. >> >> I cannot comment on the original proposal, but this issue has known >> solutions in git, in the form of submodules. I believe hg has >> something similar with the forest extension >> >> http://mercurial.selenic.com/wiki/ForestExtension > > Mercurial has subrepo support, but that doesn't justify the need to > have every module in its own repository so they can be checked out > individually. It does not justify it, but it makes it possible to keep several repositories in sync, and that you get a consistent state when cloning the top repo. If there is a need to often move code from one repo to the other, or if a change in one repo often cause a change in another one, then certainly that's a sign that they should be in the same repo. But for the windows issue, using subrepo so that when you clone python repo, you get the exact same versions of C libraries as used for the official msi (tk, tcl, openssl, bzip2, etc...), that would be very useful. At least I would have prefered this to the current situation when I need to build python myself on windows. David From steve at holdenweb.com Sat Jul 3 05:24:08 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 02 Jul 2010 23:24:08 -0400 Subject: [Python-Dev] SVN <-> HG workflow to split Python Library by Module In-Reply-To: References: Message-ID: Jesse Noller wrote: > On Fri, Jul 2, 2010 at 5:57 PM, Georg Brandl wrote: >> Am 02.07.2010 22:01, schrieb Jesse Noller: >> >>>> I am exhausted. >>> fwiw - there is a/are plan(s) to break out the stdlib from "core" once >>> the transition is complete, to better allow re-use between the various >>> interpreters. I do not think that "lots of small mirrors/repos" for >>> each library is a net gain. Once the stdlib is broken out; the same >>> thing you have proposed is achievable (within reason) without a >>> proliferation of mirrors/repos/etc. >>> >>> Also, we're not staying on subversion - well, as far I know. >> At least not parallel to Mercurial. Definitely. >> >> Georg > > Well, I was worried I missed a meeting in the secret clubhouse. I've told you before about mentioning the club house on public mailing lists. Remember, there IS NO PSU ... -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Sat Jul 3 05:26:43 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 02 Jul 2010 23:26:43 -0400 Subject: [Python-Dev] SVN <-> HG workflow to split Python Library by Module In-Reply-To: References: Message-ID: <4C2EADF3.5040201@holdenweb.com> David Cournapeau wrote: > On Sat, Jul 3, 2010 at 9:34 AM, Brett Cannon wrote: >> On Fri, Jul 2, 2010 at 17:17, David Cournapeau wrote: >>> On Sat, Jul 3, 2010 at 6:37 AM, Brett Cannon wrote: >>>> On Fri, Jul 2, 2010 at 12:25, anatoly techtonik wrote: >>>>> I planned to publish this proposal when it is finally ready and tested >>>>> with an assumption that Subversion repository will be online and >>>>> up-to-date after Mercurial migration. But recent threads showed that >>>>> currently there is no tested mechanism to sync Subversion repository >>>>> back with Mercurial, so it will probably quickly outdate, and the >>>>> proposal won't have a chance to be evaluated. So now is better than >>>>> never. >>>>> >>>>> So, this is a way to split modules from monolithic Subversion >>>>> repository into several Mercurial mirrors - one mirror for each module >>>>> (or whatever directory structure you like). This will allow to >>>>> concentrate your work on only one module at a time ("distutils", >>>>> "CGIHTTPServer" etc.) without caring much about anything else. >>>>> Exceptionally useful for occasional external "contributors" like me, >>>>> and folks on Windows, who don't possess Visual Studio to compile >>>>> Python and are forced to use whatever version they have installed to >>>>> create and test patches. >>>> But modules do not live in an isolated world; they are dependent on >>>> changes made to other modules. Isolating them from other modules whose >>>> semantics change during development will lead to skew and improper >>>> patches. >>> I cannot comment on the original proposal, but this issue has known >>> solutions in git, in the form of submodules. I believe hg has >>> something similar with the forest extension >>> >>> http://mercurial.selenic.com/wiki/ForestExtension >> Mercurial has subrepo support, but that doesn't justify the need to >> have every module in its own repository so they can be checked out >> individually. > > It does not justify it, but it makes it possible to keep several > repositories in sync, and that you get a consistent state when cloning > the top repo. If there is a need to often move code from one repo to > the other, or if a change in one repo often cause a change in another > one, then certainly that's a sign that they should be in the same > repo. > > But for the windows issue, using subrepo so that when you clone python > repo, you get the exact same versions of C libraries as used for the > official msi (tk, tcl, openssl, bzip2, etc...), that would be very > useful. At least I would have prefered this to the current situation > when I need to build python myself on windows. > It does sound like it would be helpful, though I guess we would have to check each project to ensure we had a license to redistribute under some terms or other ... regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From benjamin at python.org Sat Jul 3 05:28:58 2010 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 2 Jul 2010 22:28:58 -0500 Subject: [Python-Dev] blocking 2.7 Message-ID: This is just a note that we have one bug blocking 2.7 final at the moment: http://bugs.python.org/issue9144 -- Regards, Benjamin From steve at holdenweb.com Sat Jul 3 05:26:43 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 02 Jul 2010 23:26:43 -0400 Subject: [Python-Dev] SVN <-> HG workflow to split Python Library by Module In-Reply-To: References: Message-ID: <4C2EADF3.5040201@holdenweb.com> David Cournapeau wrote: > On Sat, Jul 3, 2010 at 9:34 AM, Brett Cannon wrote: >> On Fri, Jul 2, 2010 at 17:17, David Cournapeau wrote: >>> On Sat, Jul 3, 2010 at 6:37 AM, Brett Cannon wrote: >>>> On Fri, Jul 2, 2010 at 12:25, anatoly techtonik wrote: >>>>> I planned to publish this proposal when it is finally ready and tested >>>>> with an assumption that Subversion repository will be online and >>>>> up-to-date after Mercurial migration. But recent threads showed that >>>>> currently there is no tested mechanism to sync Subversion repository >>>>> back with Mercurial, so it will probably quickly outdate, and the >>>>> proposal won't have a chance to be evaluated. So now is better than >>>>> never. >>>>> >>>>> So, this is a way to split modules from monolithic Subversion >>>>> repository into several Mercurial mirrors - one mirror for each module >>>>> (or whatever directory structure you like). This will allow to >>>>> concentrate your work on only one module at a time ("distutils", >>>>> "CGIHTTPServer" etc.) without caring much about anything else. >>>>> Exceptionally useful for occasional external "contributors" like me, >>>>> and folks on Windows, who don't possess Visual Studio to compile >>>>> Python and are forced to use whatever version they have installed to >>>>> create and test patches. >>>> But modules do not live in an isolated world; they are dependent on >>>> changes made to other modules. Isolating them from other modules whose >>>> semantics change during development will lead to skew and improper >>>> patches. >>> I cannot comment on the original proposal, but this issue has known >>> solutions in git, in the form of submodules. I believe hg has >>> something similar with the forest extension >>> >>> http://mercurial.selenic.com/wiki/ForestExtension >> Mercurial has subrepo support, but that doesn't justify the need to >> have every module in its own repository so they can be checked out >> individually. > > It does not justify it, but it makes it possible to keep several > repositories in sync, and that you get a consistent state when cloning > the top repo. If there is a need to often move code from one repo to > the other, or if a change in one repo often cause a change in another > one, then certainly that's a sign that they should be in the same > repo. > > But for the windows issue, using subrepo so that when you clone python > repo, you get the exact same versions of C libraries as used for the > official msi (tk, tcl, openssl, bzip2, etc...), that would be very > useful. At least I would have prefered this to the current situation > when I need to build python myself on windows. > It does sound like it would be helpful, though I guess we would have to check each project to ensure we had a license to redistribute under some terms or other ... regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From greg.ewing at canterbury.ac.nz Sat Jul 3 03:31:18 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 03 Jul 2010 13:31:18 +1200 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: <201007021428.48162.steve@pearwood.info> References: <201007021428.48162.steve@pearwood.info> Message-ID: <4C2E92E6.6000807@canterbury.ac.nz> Steven D'Aprano wrote: > if the keyhole optimizer raised SyntaxError (or > some other exception) on seeing this: > > def f(): > return 1 + "1" That might break code that was deliberately trying to raise an exception. Sometimes you see things like try: 1/0 except Exception, e: ... Usually this kind of thing is only done in test code or illustrative snippets, but even so, it should work as expected. -- Greg From ncoghlan at gmail.com Sat Jul 3 05:40:28 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 3 Jul 2010 13:40:28 +1000 Subject: [Python-Dev] blocking 2.7 In-Reply-To: References: Message-ID: On Sat, Jul 3, 2010 at 1:28 PM, Benjamin Peterson wrote: > This is just a note that we have one bug blocking 2.7 final at the > moment: http://bugs.python.org/issue9144 I added Jesse to the nosy list for that as well. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From steve at pearwood.info Sat Jul 3 06:35:09 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 3 Jul 2010 14:35:09 +1000 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: <4C2E94BB.1090901@canterbury.ac.nz> References: <201007021428.48162.steve@pearwood.info> <4C2E94BB.1090901@canterbury.ac.nz> Message-ID: <201007031435.09229.steve@pearwood.info> On Sat, 3 Jul 2010 11:39:07 am Greg Ewing wrote: > Stefan Behnel wrote: > > So, would it still be Python if it folded > > > > ? ? 1 + "1" > > > > into > > > > ? ? raise TypeError() > > > > at compile time? > > It would have to be > > ? ? raise TypeError("Exactly the message that would have been > produced at run time") Python doesn't make any guarantees about the message that exceptions display, so I don't think you need to match the message, just the exception. Anyone testing for specific exception messages is living in a state of sin and shouldn't complain when their code stops working. An implementation might choose to raise TypeError('is this the right place for an argument?') on alternate Tuesdays, and it would still meet the promises made by the language. -- Steven D'Aprano From greg.ewing at canterbury.ac.nz Sat Jul 3 06:42:42 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 03 Jul 2010 16:42:42 +1200 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: References: <201007021428.48162.steve@pearwood.info> <3EB9ED56-3355-44F4-AD0C-B90E2137F531@twistedmatrix.com> Message-ID: <4C2EBFC2.4070106@canterbury.ac.nz> Craig Citro wrote: > However, in this particular case, here's a question: *why* would > someone write "return 1 + '1'"? They might not intend to execute the code at all -- e.g. they may want to pass the compiled code to dis() to find out what bytecode gets generated. Having it refuse to compile would be annoying in that case. -- Greg From eliben at gmail.com Sat Jul 3 07:22:26 2010 From: eliben at gmail.com (Eli Bendersky) Date: Sat, 3 Jul 2010 07:22:26 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <20100702191231.3317486d@pitrou.net> References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <20100702191231.3317486d@pitrou.net> Message-ID: > > > This information really belongs in www.python.org/dev/ rather than > > > only in the mailing list. > > > > > +1 > > > > As does a recent essay by Eli Bendersky, IMO. I believe it could lower > > the barriers for entry into the "non-committer" class of developer. This > > should make it easier for people to adapt Python to their own purposes > > whether or not they want to contribute to the open source code base, but > > also encourage people to investigate the compiler's innards. > > With the moratorium on language constructs and builtins (not only in > letter until 3.3, but more generally in spirit), I don't think we should > encourage it at all, for such contributions would surely be rejected. > > Rather than fancy syntax propositions or wild semantic > changes/enhancements (such as have often been proposed on > python-ideas), what we need is humbler but more useful work on stdlib > bugs and improvements, as well as documentation and tutorials. > > (what's more, those two kinds of contributions are likely to attract > two different kinds of people, which means that people whose syntax > patches get refused won't necessarily start contributing stdlib or > documentation patches...) > > Regards > > Antoine. > > Antoine, I wrote that article, and I agree with everything you say here. I just don't want my intentions to be understood incorrectly. Many readers stopped reading after the title and jumped to conclusions which simply aren't true (as some of the comments on the blog and Reddit demonstrate). Just a couple of quotes from the article itself to show that breaking the moratorium (with which I wholeheartedly agree) is far from my intentions: ----------------------------- This article is an attempt to better understand how the front-end of Python works. Just reading documentation and source code may be a bit boring, so I?m taking a hands-on approach here: I?m going to add an until statement to Python. [...] This article doesn?t attempt to suggest the addition of an until statement to Python. Although I think such a statement would make some code clearer, and this article displays how easy it is to add, I completely respect Python?s philosophy of minimalism. All I?m trying to do here, really, is gain some insight into the inner workings of Python. ----------------------------- Kind regards, Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From reid.kleckner at gmail.com Sat Jul 3 07:26:38 2010 From: reid.kleckner at gmail.com (Reid Kleckner) Date: Fri, 2 Jul 2010 22:26:38 -0700 Subject: [Python-Dev] More detailed build instructions for Windows Message-ID: Hey folks, I'm trying to test out a patch to add a timeout in subprocess.py on Windows, so I need to build Python with Visual Studio. The docs say the files in PCBuild/ work with VC 9 and newer. I downloaded Visual C++ 2010 Express, and it needs to convert the .vcproj files into .vcxproj files, but it fails. I can't figure out where to get VC 9, all I see is 2008 and 2010. Can someone with experience share the best practices for building Python on Windows? In particular, what is the most recent compiler known to work and where can I download it? Thanks, Reid From stefan_ml at behnel.de Sat Jul 3 08:38:37 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 03 Jul 2010 08:38:37 +0200 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: <201007031435.09229.steve@pearwood.info> References: <201007021428.48162.steve@pearwood.info> <4C2E94BB.1090901@canterbury.ac.nz> <201007031435.09229.steve@pearwood.info> Message-ID: Steven D'Aprano, 03.07.2010 06:35: > On Sat, 3 Jul 2010 11:39:07 am Greg Ewing wrote: >> Stefan Behnel wrote: >>> So, would it still be Python if it folded >>> >>> 1 + "1" >>> >>> into >>> >>> raise TypeError() >>> >>> at compile time? >> >> It would have to be >> >> raise TypeError("Exactly the message that would have been >> produced at run time") > > Python doesn't make any guarantees about the message that exceptions > display, so I don't think you need to match the message, just the > exception. Anyone testing for specific exception messages is living in > a state of sin and shouldn't complain when their code stops working. Yep, the Cython project keeps running into this all over the place. When we reimplement CPython functionality for performance reasons, we make sure we match the behaviour exactly, which usually means that we try to match the exceptions and their messages as well. Since we use doctests for our test suite, this means that we need to special case some Python versions in the test suite to make sure we test our code as good as possible without letting the tests break across CPython versions. This can be really tricky at times. The general trend seems to be that modified exception messages become more exact and telling over time, so we tend to follow Python 3.x in our own choice of error messages. Stefan From cournape at gmail.com Sat Jul 3 08:39:01 2010 From: cournape at gmail.com (David Cournapeau) Date: Sat, 3 Jul 2010 15:39:01 +0900 Subject: [Python-Dev] More detailed build instructions for Windows In-Reply-To: References: Message-ID: On Sat, Jul 3, 2010 at 2:26 PM, Reid Kleckner wrote: > Hey folks, > > I'm trying to test out a patch to add a timeout in subprocess.py on > Windows, so I need to build Python with Visual Studio. ?The docs say > the files in PCBuild/ work with VC 9 and newer. ?I downloaded Visual > C++ 2010 Express, and it needs to convert the .vcproj files into > .vcxproj files, but it fails. > > I can't figure out where to get VC 9, all I see is 2008 and 2010. VS 2008 == VC 9 == MSVC 15 David From martin at v.loewis.de Sat Jul 3 09:00:37 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 03 Jul 2010 09:00:37 +0200 Subject: [Python-Dev] More detailed build instructions for Windows In-Reply-To: References: Message-ID: <4C2EE015.4050305@v.loewis.de> > I'm trying to test out a patch to add a timeout in subprocess.py on > Windows, so I need to build Python with Visual Studio. The docs say > the files in PCBuild/ work with VC 9 and newer. Which docs did you look at specifically that said "and newer"? That would be a bug. > I downloaded Visual > C++ 2010 Express, and it needs to convert the .vcproj files into > .vcxproj files, but it fails. Please take a look at PCbuild/readme.txt. It tells you to install one of the Visual Studio 2008 editions. Regards, Martin From stephen at xemacs.org Sat Jul 3 11:31:24 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 03 Jul 2010 18:31:24 +0900 Subject: [Python-Dev] Let's get you ready for Mercurial migration! In-Reply-To: References: <4C2DFD49.70507@netwok.org> Message-ID: <87wrtcx66r.fsf@uwakimon.sk.tsukuba.ac.jp> Steve Holden writes: > If the wave were to result in good documentation about how to *get* > ready that would be an amazingly useful contribution. I'm a coauthor of PEP 374 and of http://emacswiki.org/BzrForEmacsDevs. I think that I can have a document adapted from the Python dev FAQ, possibly integrating parts of BzrForEmacsDevs (style, ideas, not literal commands of course -- sorry, Barry) and the hginit.com tutorial (assuming it's free enough, will check) by July 21. If anybody thinks they'd do a better job, or simply that I'm not appropriate, let me know (I have a thick skin and plenty of other work to do :-). If it sounds like a good idea, let me know where and how to submit (patch to dev FAQ on the tracker is the default). Of course, collaborators welcome, mail me off list. From dickinsm at gmail.com Sat Jul 3 12:17:16 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 3 Jul 2010 11:17:16 +0100 Subject: [Python-Dev] blocking 2.7 In-Reply-To: References: Message-ID: On Sat, Jul 3, 2010 at 4:28 AM, Benjamin Peterson wrote: > This is just a note that we have one bug blocking 2.7 final at the > moment: http://bugs.python.org/issue9144 I've just made http://bugs.python.org/issue7673 a release blocker too, I'm afraid. It's a potential security vulnerability in the audioop module. (CVE-2010-2089). It's got a reviewed patch, and is ready to be committed, but if you're not comfortable with fixing it this late then that's completely understandable. Mark From solipsis at pitrou.net Sat Jul 3 12:43:16 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 3 Jul 2010 12:43:16 +0200 Subject: [Python-Dev] blocking 2.7 References: Message-ID: <20100703124316.3c6877ff@pitrou.net> On Sat, 3 Jul 2010 11:17:16 +0100 Mark Dickinson wrote: > On Sat, Jul 3, 2010 at 4:28 AM, Benjamin Peterson wrote: > > This is just a note that we have one bug blocking 2.7 final at the > > moment: http://bugs.python.org/issue9144 > > I've just made http://bugs.python.org/issue7673 a release blocker too, > I'm afraid. It's a potential security vulnerability in the audioop > module. (CVE-2010-2089). It's got a reviewed patch, and is ready to > be committed, but if you're not comfortable with fixing it this late > then that's completely understandable. Interestingly, Victor filed both the issue and the initial patch five months before the CVE alert. Well done Victor! From stephen at xemacs.org Sat Jul 3 12:53:07 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 03 Jul 2010 19:53:07 +0900 Subject: [Python-Dev] SVN <-> HG workflow to split Python Library by Module In-Reply-To: References: Message-ID: <87vd8wx2ek.fsf@uwakimon.sk.tsukuba.ac.jp> Brett Cannon writes: > Mercurial has subrepo support, but that doesn't justify the need to > have every module in its own repository so they can be checked out > individually. The point of submodules a la git is subtly different. It is that you can mix and match *known versions* of the modules. So, eg, in order to work on recent urllib, maybe you need a recent *but stable* email but you don't want any of the work being done on removing the GIL because it's somewhat unstable. Of course it doesn't guarantee that your choice will actually work. The point is to make it (a) convenient to explore and (b) easy to exactly reproduce a successful configuration. More or less David C's reply but with a different slant. From dirkjan at ochtman.nl Sat Jul 3 13:05:49 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Sat, 3 Jul 2010 13:05:49 +0200 Subject: [Python-Dev] SVN <-> HG workflow to split Python Library by Module In-Reply-To: <87vd8wx2ek.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87vd8wx2ek.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Sat, Jul 3, 2010 at 12:53, Stephen J. Turnbull wrote: > The point of submodules a la git is subtly different. ?It is that you > can mix and match *known versions* of the modules. ?So, eg, in order > to work on recent urllib, maybe you need a recent *but stable* email > but you don't want any of the work being done on removing the GIL > because it's somewhat unstable. This sounds like it could also be done with good branching + merging. Cheers, Dirkjan From victor.stinner at haypocalc.com Sat Jul 3 14:26:53 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Sat, 3 Jul 2010 14:26:53 +0200 Subject: [Python-Dev] blocking 2.7 In-Reply-To: References: Message-ID: <201007031426.53880.victor.stinner@haypocalc.com> Le samedi 03 juillet 2010 12:17:16, Mark Dickinson a ?crit : > On Sat, Jul 3, 2010 at 4:28 AM, Benjamin Peterson wrote: > > This is just a note that we have one bug blocking 2.7 final at the > > moment: http://bugs.python.org/issue9144 > > I've just made http://bugs.python.org/issue7673 a release blocker too, > I'm afraid. It's a potential security vulnerability in the audioop > module. (CVE-2010-2089) At least, Fedora consider it as a security vulnerability: https://bugzilla.redhat.com/show_bug.cgi?id=598197 I agree because the crash is caused by the input data. > It's got a reviewed patch, and is ready to be committed Thanks because my first patch was incomplete :-) > but if you're not comfortable with fixing it this late > then that's completely understandable. In the worst case, a function rejects valid data. If I have to choose, I prefer to reject valid data than a security vulnerability. But audioop has tests and I don't think that my patch breaks anything :-) -- Victor Stinner http://www.haypocalc.com/ From jnoller at gmail.com Sat Jul 3 14:28:01 2010 From: jnoller at gmail.com (Jesse Noller) Date: Sat, 3 Jul 2010 08:28:01 -0400 Subject: [Python-Dev] blocking 2.7 In-Reply-To: References: Message-ID: On Fri, Jul 2, 2010 at 11:40 PM, Nick Coghlan wrote: > On Sat, Jul 3, 2010 at 1:28 PM, Benjamin Peterson wrote: >> This is just a note that we have one bug blocking 2.7 final at the >> moment: http://bugs.python.org/issue9144 > > I added Jesse to the nosy list for that as well. > > Cheers, > Nick. Committed the patch for 9144 - Brett broke mah imports! From jnoller at gmail.com Sat Jul 3 14:30:13 2010 From: jnoller at gmail.com (Jesse Noller) Date: Sat, 3 Jul 2010 08:30:13 -0400 Subject: [Python-Dev] SVN <-> HG workflow to split Python Library by Module In-Reply-To: References: <87vd8wx2ek.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Sat, Jul 3, 2010 at 7:05 AM, Dirkjan Ochtman wrote: > On Sat, Jul 3, 2010 at 12:53, Stephen J. Turnbull wrote: >> The point of submodules a la git is subtly different. ?It is that you >> can mix and match *known versions* of the modules. ?So, eg, in order >> to work on recent urllib, maybe you need a recent *but stable* email >> but you don't want any of the work being done on removing the GIL >> because it's somewhat unstable. > > This sounds like it could also be done with good branching + merging. > > Cheers, > > Dirkjan What Dirkjan said; since using mercurial more and more lately, local branches are an insanely good tool and method of coordinating work From victor.stinner at haypocalc.com Sat Jul 3 14:40:57 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Sat, 3 Jul 2010 14:40:57 +0200 Subject: [Python-Dev] blocking 2.7 In-Reply-To: <201007031426.53880.victor.stinner@haypocalc.com> References: <201007031426.53880.victor.stinner@haypocalc.com> Message-ID: <201007031440.57602.victor.stinner@haypocalc.com> Le samedi 03 juillet 2010 14:26:53, Victor Stinner a ?crit : > In the worst case, a function rejects valid data. If I have to choose, I > prefer to reject valid data than a security vulnerability. But audioop has > tests and I don't think that my patch breaks anything :-) I checked the test suite: *all* audioop function are tested at least once. So, can I / should I commit the patch? -- Victor Stinner http://www.haypocalc.com/ From solipsis at pitrou.net Sat Jul 3 14:55:21 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 3 Jul 2010 14:55:21 +0200 Subject: [Python-Dev] blocking 2.7 References: <201007031426.53880.victor.stinner@haypocalc.com> <201007031440.57602.victor.stinner@haypocalc.com> Message-ID: <20100703145521.4e3b6ee9@pitrou.net> On Sat, 3 Jul 2010 14:40:57 +0200 Victor Stinner wrote: > Le samedi 03 juillet 2010 14:26:53, Victor Stinner a ?crit : > > In the worst case, a function rejects valid data. If I have to choose, I > > prefer to reject valid data than a security vulnerability. But audioop has > > tests and I don't think that my patch breaks anything :-) > > I checked the test suite: *all* audioop function are tested at least once. > > So, can I / should I commit the patch? In my opinion you can. From lists at cheimes.de Sat Jul 3 16:34:07 2010 From: lists at cheimes.de (Christian Heimes) Date: Sat, 03 Jul 2010 16:34:07 +0200 Subject: [Python-Dev] More detailed build instructions for Windows In-Reply-To: <4C2EE015.4050305@v.loewis.de> References: <4C2EE015.4050305@v.loewis.de> Message-ID: Am 03.07.2010 09:00, schrieb "Martin v. L?wis": >> I'm trying to test out a patch to add a timeout in subprocess.py on >> Windows, so I need to build Python with Visual Studio. The docs say >> the files in PCBuild/ work with VC 9 and newer. > > Which docs did you look at specifically that said "and newer"? That > would be a bug. The readme.txt in the PCbuild directory contains the sentence "Microsoft Visual C++ 2008 Express Edition is required at the very least". The wording could be interpreted as "2008 at very least" instead of "Express Edition at very least". Christian From reid.kleckner at gmail.com Sat Jul 3 16:46:48 2010 From: reid.kleckner at gmail.com (Reid Kleckner) Date: Sat, 3 Jul 2010 07:46:48 -0700 Subject: [Python-Dev] More detailed build instructions for Windows In-Reply-To: <4C2EE015.4050305@v.loewis.de> References: <4C2EE015.4050305@v.loewis.de> Message-ID: On Sat, Jul 3, 2010 at 12:00 AM, "Martin v. L?wis" wrote: >> I'm trying to test out a patch to add a timeout in subprocess.py on >> Windows, so I need to build Python with Visual Studio. ?The docs say >> the files in PCBuild/ work with VC 9 and newer. > > Which docs did you look at specifically that said "and newer"? That > would be a bug. On the developer FAQ page it says: http://www.python.org/dev/faq/#id8 "For VC 9 and newer, the PCbuild directory contains the build files." But I'll go get 2008. Thanks! Reid From g.brandl at gmx.net Sat Jul 3 17:18:05 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 03 Jul 2010 17:18:05 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <4C2E7C37.1050006@v.loewis.de> References: <4C2E7C37.1050006@v.loewis.de> Message-ID: Am 03.07.2010 01:54, schrieb "Martin v. L?wis": >> I don't know about "try" -- personally I don't see a difference for >> the release procedure, no matter where the source comes from. > > I guess you haven't done a release yet, then :-) That's possible :) > Assuming you are going to use > > https://svn.python.org/projects/sandbox/trunk/release/release.py > > then you'll have to port that to Mercurial, first. True, but it doesn't look too hard. > Or, you write your own release tool, as any serious release manager did > so far :-) > > As for the Windows build, I don't know how to do "externals", yet. > In particular, I don't think it is a good idea to have all externals > in a single Mercurial clone: due to versioning, we have multiple copies > of them, which will take quite some space. True. > As a minor problem: I currently have a batch file head.bat, which > updates sysmodule.c after a switch to expand HeadURL correctly. > Not sure whether this will be needed on Mercurial, or whether the > build identification patch is able to learn about tags in the first > place. I have no solution to that from the top of my head, but it will be dealt with > Finally, I wonder how the documentation build process will continue to > integrate subversion. I guess you just need to have an svn binary around > when building the documentation. Since I'm giving specific versions in the makefiles anyway, the easiest solution would be to put them as tarfiles/zipfiles up for download somewhere on python.org, and have a small script that gets them via urllib. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From tseaver at palladion.com Sat Jul 3 17:51:01 2010 From: tseaver at palladion.com (Tres Seaver) Date: Sat, 03 Jul 2010 11:51:01 -0400 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <4C2E7EF6.3030005@v.loewis.de> References: <4C2E7EF6.3030005@v.loewis.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin v. L?wis wrote: >> Can somebody comment on how much ongoing effort is required to keep that >> mirror running? > > As everybody else indicated: none (I believe). OK, cool. I have certainly had no issues using it when working as a non-committer to verify patches, etc., for bugs.python.org issues. > FWIW, the bzr mirror wasn't that self-maintaining: the process started > to consume too much memory and got killed; the cron jobs broke, and so > on, so we finally switched it of (in favor of the Launchpad mirror that > also existed). Yup, we saw that to, and have the loggerhead server running under supervisor, with a configuration to restart it it exceeds a given amount of RAM. In our case, the cron jobs to update the mirror were actually more easier to set up for bzr (and more stable) than for hg or git, but the hg server was better behaved (the git web bit runs as CGI, IIRC). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkwvXGAACgkQ+gerLs4ltQ55kwCgmO5Z1coKDcRUs/n/MvLVfW9t ul8AmwfnorXyVznEDNyxIX5f2/zNGurI =TIqB -----END PGP SIGNATURE----- From tseaver at palladion.com Sat Jul 3 18:36:12 2010 From: tseaver at palladion.com (Tres Seaver) Date: Sat, 03 Jul 2010 12:36:12 -0400 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <4C2E7CE6.9060501@v.loewis.de> References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin v. L?wis wrote: > Am 02.07.2010 15:09, schrieb Fred Drake: >> On Fri, Jul 2, 2010 at 8:34 AM, Antoine Pitrou wrote: >>> The two sets of repositories use different conversion tools and rules. >>> They have nothing in common (different changeset IDs, different >>> metadata, different branch/clone layout). >> I'd love to see a more detailed description of this, including why >> someone new to Mercurial would choose one over the other. > > I think someone new to Mercurial shouldn't choose either one. > > Just sit back and wait for the real migration to happen. I would say that using the SVN mirror is a fine way to experiment with using hg against the Python sources to develop and test patches. Here is the setup I have used for work against trunk (I have a parallel pair of repositories for the release2.6-maint branch): - - Create a "pristine" clone of the trunk (one where I never commit any changes): $ cd $python_repo $ hg clone http://code.python.org/hg/trunk/ pytrunk-upstream - - Create a local clone from that repository: $ hg clone pytrunk-upstream pytrunk-work $ ./configure && make Before working on a patch, I refresh the upstream repository: $ cd $python_repo/pytrunk-upstream && hg pull and pull any changes into the local working repository: $ cd $python_repo/pytrunk-work $ hg pull -u $ make I make a branch per roundup issue number in the working repository (in order to avoid needing to rebuild the world for each issue): $ cd $python_repo/pytrunk-work $ hg branch issue4265-shutil_copyfile and then test / tweak the patch on that branch, committing as I go: $ patch -p0 < /tmp/shutil_copyfile.patch $ hg commit $ make $ ./python -E -tt -m test.regrtest test_shutil etc. If I have tweaked the patch, I can export a new version of the patch: $ hg log -p -b issue4265-shutil_copyfile > /tmp/updated_fix.patch I haven't tried yet, but I imagine I could use 'hg email' to send the patch as well. Once the migration is done, of course, users who have been working with that mirror will need to set up new clones for the upstream and re-clone the working trees, as the revision IDs won't match, etc. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkwvZvgACgkQ+gerLs4ltQ71GACfTw4kNDm+IOv7Jvq8XFtCu6XD rXoAn3G7i+emGwgp9bxhaHQ+gctIXilA =am67 -----END PGP SIGNATURE----- From tjreedy at udel.edu Sat Jul 3 20:06:05 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 03 Jul 2010 14:06:05 -0400 Subject: [Python-Dev] More detailed build instructions for Windows In-Reply-To: References: <4C2EE015.4050305@v.loewis.de> Message-ID: On 7/3/2010 10:34 AM, Christian Heimes wrote: >> Which docs did you look at specifically that said "and newer"? That >> would be a bug. > > The readme.txt in the PCbuild directory contains the sentence "Microsoft > Visual C++ 2008 Express Edition is required at the very least". The > wording could be interpreted as "2008 at very least" That I how I would first read it, not already knowing differently. > instead of "Express Edition at very least". I did not know that there was anything 'less' than Express Edition. Something like "Building Pythonx.y requires at least the Express Edition of Visual C++ 2008." or "Building Pyx.y requires Visual C++ 2008. Use the free Express Edition or one of its paid upgrades." -- Terry Jan Reedy From tjreedy at udel.edu Sat Jul 3 20:16:08 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 03 Jul 2010 14:16:08 -0400 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> Message-ID: On 7/3/2010 12:36 PM, Tres Seaver wrote: > I would say that using the SVN mirror is a fine way to experiment with > using hg against the Python sources to develop and test patches. Here > is the setup I have used for work against trunk (I have a parallel pair > of repositories for the release2.6-maint branch): > > - - Create a "pristine" clone of the trunk (one where I never commit any > changes): > > $ cd $python_repo > $ hg clone http://code.python.org/hg/trunk/ pytrunk-upstream > > - - Create a local clone from that repository: > > $ hg clone pytrunk-upstream pytrunk-work > $ ./configure&& make > > Before working on a patch, I refresh the upstream repository: > > $ cd $python_repo/pytrunk-upstream&& hg pull > > and pull any changes into the local working repository: > > $ cd $python_repo/pytrunk-work > $ hg pull -u > $ make This is perhaps a naive question, but hat do you gain with the intermediate mirror clone of upstream? (Other than filling more of your disk?) -- Terry Jan Reedy From solipsis at pitrou.net Sat Jul 3 20:28:40 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 3 Jul 2010 20:28:40 +0200 Subject: [Python-Dev] Mercurial migration readiness References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> Message-ID: <20100703202840.77bed5e9@pitrou.net> On Sat, 03 Jul 2010 14:16:08 -0400 Terry Reedy wrote: > > This is perhaps a naive question, but hat do you gain with the > intermediate mirror clone of upstream? (Other than filling more of your > disk?) Filling less of your disk, actually, since local clones use hardlinks. Also, pulling less data from the network might be interesting if you have a slow connection, or pull from a very heavy or very active repo (which CPython isn't, though). Regards Antoine. From tseaver at palladion.com Sat Jul 3 21:16:58 2010 From: tseaver at palladion.com (Tres Seaver) Date: Sat, 03 Jul 2010 15:16:58 -0400 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Terry Reedy wrote: > This is perhaps a naive question, but hat do you gain with the > intermediate mirror clone of upstream? (Other than filling more of your > disk?) I gain having my local changes be in a "scratchpad" repsitory, which I can discard at will without requiring a re-fetch of the whole repository from the mirror server, which takes a loooong time and (apparently) puts significant load on that server. As Antoine noted, hg shares the disk space via hard links where possible; where not, what's a few hundred megabytes, more or less? Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkwvjKoACgkQ+gerLs4ltQ7aUgCdHeUOkSTDQYefjih8WMH+OXkL 3E4AoLjk7mtG3E8ayxK1NDMlbkQweIiY =40ji -----END PGP SIGNATURE----- From daniel at stutzbachenterprises.com Sat Jul 3 23:51:58 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sat, 3 Jul 2010 16:51:58 -0500 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> Message-ID: On Sat, Jul 3, 2010 at 11:36 AM, Tres Seaver wrote: > - - Create a "pristine" clone of the trunk (one where I never commit any > changes): > > $ cd $python_repo > $ hg clone http://code.python.org/hg/trunk/ pytrunk-upstream > > - - Create a local clone from that repository: > > $ hg clone pytrunk-upstream pytrunk-work > $ ./configure && make > My question is basically the same as Terry Reedy's, but I'm going to phrase it a bit differently: This is perhaps a naive question, but why do you create a second local clone instead of just creating a branch? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Sun Jul 4 00:40:47 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 04 Jul 2010 00:40:47 +0200 Subject: [Python-Dev] More detailed build instructions for Windows In-Reply-To: References: <4C2EE015.4050305@v.loewis.de> Message-ID: <4C2FBC6F.4020706@v.loewis.de> Am 03.07.2010 16:34, schrieb Christian Heimes: > Am 03.07.2010 09:00, schrieb "Martin v. L?wis": >>> I'm trying to test out a patch to add a timeout in subprocess.py on >>> Windows, so I need to build Python with Visual Studio. The docs say >>> the files in PCBuild/ work with VC 9 and newer. >> >> Which docs did you look at specifically that said "and newer"? That >> would be a bug. > > The readme.txt in the PCbuild directory contains the sentence "Microsoft > Visual C++ 2008 Express Edition is required at the very least". The > wording could be interpreted as "2008 at very least" instead of "Express > Edition at very least". That's what I expected to get as an answer - interestingly enough, Reid was looking at some other place (the FAQ) that literally said "and newer". I'd call that an error, even though, technically, converting the projects to VS 2010 "ought to work" (i.e. it did after I committed some fixes to make it work). Regards, Martin From martin at v.loewis.de Sun Jul 4 00:51:58 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 04 Jul 2010 00:51:58 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> Message-ID: <4C2FBF0E.6020302@v.loewis.de> >>> I'd love to see a more detailed description of this, including why >>> someone new to Mercurial would choose one over the other. > >> I think someone new to Mercurial shouldn't choose either one. > >> Just sit back and wait for the real migration to happen. > > I would say that using the SVN mirror is a fine way to experiment with > using hg against the Python sources to develop and test patches. I think your description already falls into the "advanced user" category. The new-to-mercurial committer should (IMO) use a "what if it still was svn" workflow, which uses hg pull/up/commit/push. That can only work if pushing really has the desired effect, which isn't the case for any of the installations that we operate. When the test-for-two-weeks installation becomes available, the new-to-hg users will have a chance to learn how to use it without fear of breaking anything. Until then, they should just remain patient. Regards, Martin From solipsis at pitrou.net Sun Jul 4 00:56:27 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 4 Jul 2010 00:56:27 +0200 Subject: [Python-Dev] Mercurial migration readiness References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> <4C2FBF0E.6020302@v.loewis.de> Message-ID: <20100704005627.3ee97894@pitrou.net> On Sun, 04 Jul 2010 00:51:58 +0200 "Martin v. L?wis" wrote: > >>> I'd love to see a more detailed description of this, including why > >>> someone new to Mercurial would choose one over the other. > > > >> I think someone new to Mercurial shouldn't choose either one. > > > >> Just sit back and wait for the real migration to happen. > > > > I would say that using the SVN mirror is a fine way to experiment with > > using hg against the Python sources to develop and test patches. > > I think your description already falls into the "advanced user" > category. The new-to-mercurial committer should (IMO) use a "what if it > still was svn" workflow, which uses hg pull/up/commit/push. This assumes they are accustomed to SVN. I guess not all people are, although it is certainly a common skill. Regards Antoine. From martin at v.loewis.de Sun Jul 4 00:59:58 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 04 Jul 2010 00:59:58 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> Message-ID: <4C2FC0EE.1060702@v.loewis.de> > This is perhaps a naive question, but hat do you gain with the > intermediate mirror clone of upstream? (Other than filling more of your > disk?) In addition to the answer you got: this way of working is also the process that I arrived at, independently. I see two uses, both based around the problem "creating a full clone will take a long time - much longer than a subversion checkout". 1. if I want several local checkouts (e.g. for testing separate features), I can clone them all from the local copy (thereby also preserving space, compared to independent clones) 2, throwing away local changes is not that easy in Mercurial, if you have committed them already. There are extensions to uncommit, but they are discouraged and have limitations. So it's best to throw away everything and start over fresh, which is faster if you have a pristine clone. In either case, you keep pulling into the pristine clone from python.org, and then uppdate your local clones as you please. When pushing changes, it is best to directly push into the network (rather than going through the pristine clone): if pushing fails due to concurrent updates, you haven't cluttered your pristine copy. Regards, Martin From martin at v.loewis.de Sun Jul 4 01:04:53 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 04 Jul 2010 01:04:53 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> Message-ID: <4C2FC215.2040702@v.loewis.de> > My question is basically the same as Terry Reedy's, but I'm going to > phrase it a bit differently: > > This is perhaps a naive question, but why do you create a second local > clone instead of just creating a branch? IIUC, if you create a named branch, the branch will become globally visible when you push your changes back. I assume people will consider that clutter - it would be sufficient to just push the changes on the branch, along with commit messages, but not the branch itself (which would be only temporary, anyway). I'm not even sure how you pull changes from one branch into another: can somebody kindly explain the commands that would be required? Regards, Martin From martin at v.loewis.de Sun Jul 4 01:05:42 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 04 Jul 2010 01:05:42 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <20100704005627.3ee97894@pitrou.net> References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> <4C2FBF0E.6020302@v.loewis.de> <20100704005627.3ee97894@pitrou.net> Message-ID: <4C2FC246.908@v.loewis.de> Am 04.07.2010 00:56, schrieb Antoine Pitrou: > On Sun, 04 Jul 2010 00:51:58 +0200 > "Martin v. L?wis" wrote: >>>>> I'd love to see a more detailed description of this, including why >>>>> someone new to Mercurial would choose one over the other. >>> >>>> I think someone new to Mercurial shouldn't choose either one. >>> >>>> Just sit back and wait for the real migration to happen. >>> >>> I would say that using the SVN mirror is a fine way to experiment with >>> using hg against the Python sources to develop and test patches. >> >> I think your description already falls into the "advanced user" >> category. The new-to-mercurial committer should (IMO) use a "what if it >> still was svn" workflow, which uses hg pull/up/commit/push. > > This assumes they are accustomed to SVN. I guess not all people are, > although it is certainly a common skill. I thought we were talking about Python committers specifically. Regards, Martin From merwok at netwok.org Sun Jul 4 01:16:54 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sun, 04 Jul 2010 01:16:54 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <4C2FC0EE.1060702@v.loewis.de> References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> <4C2FC0EE.1060702@v.loewis.de> Message-ID: <4C2FC4E6.30906@netwok.org> > 2, throwing away local changes is not that easy in Mercurial, > if you have committed them already. There are extensions to > uncommit, but they are discouraged and have limitations. So it's > best to throw away everything and start over fresh, which is > faster if you have a pristine clone. If you?re using named branches or bookmarks, there is an easy way to throw away the whole branch. Say you have a clone with branches default, fix8888, fix9008, shlex-unicode (default is clean upstream Python 3.2, the other three are branches you made for bug fixes or features). Now you want to discard shlex-unicode. $ cd .. $ mv cpython cpython.old $ hg clone cpython.old cpython -r default -r fix8888 -r fix9008 $ rm -r cpython.old The new repo will only contain the branches you ask for. Giving all names on the command line may be cumbersome if you?re on a lot of branches at one, but it?s okay for a small number. A small bit of shell scripting can automate that easily (get all branches, remove the one given as argument to the shell function, mv, hg clone, rm). Of course, a branch you wan to abandon can stay in your repo for a while if you don?t want to clean up now. HTH. Regards From merwok at netwok.org Sun Jul 4 01:19:15 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sun, 04 Jul 2010 01:19:15 +0200 Subject: [Python-Dev] Are you ready for Mercurial migration? In-Reply-To: References: <4C2DFD49.70507@netwok.org> Message-ID: <4C2FC573.4020603@netwok.org> >> That tutorial is not ~100 pages. It's actually a good tutorial. > That's why I posted it here, but it still >80 pages in my browser. Perhaps you meant 80 screens, or a different tutorial. hginit is a short tutorial useful for Subversion users who don?t have the time to read the hgbook. Regards From techtonik at gmail.com Sun Jul 4 08:19:15 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Sun, 4 Jul 2010 09:19:15 +0300 Subject: [Python-Dev] blocking 2.7 In-Reply-To: <201007031426.53880.victor.stinner@haypocalc.com> References: <201007031426.53880.victor.stinner@haypocalc.com> Message-ID: On Sat, Jul 3, 2010 at 3:26 PM, Victor Stinner wrote: > > In the worst case, a function rejects valid data. If I have to choose, I > prefer to reject valid data than a security vulnerability. But audioop has > tests and I don't think that my patch breaks anything :-) But Python tests lack coverage stats, so it is hard to say anything. Even with tests it is not always possible to be even 80% sure if behavior is complicated or depends on the input data. -- anatoly t. From merwok at netwok.org Sun Jul 4 08:31:57 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sun, 04 Jul 2010 08:31:57 +0200 Subject: [Python-Dev] blocking 2.7 In-Reply-To: References: <201007031426.53880.victor.stinner@haypocalc.com> Message-ID: <4C302ADD.30608@netwok.org> > But Python tests lack coverage stats, so it is hard to say anything. FYI: http://coverage.livinglogic.de/ Regards From ziade.tarek at gmail.com Sun Jul 4 12:51:31 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 4 Jul 2010 12:51:31 +0200 Subject: [Python-Dev] distutils2 checking mails Message-ID: Hi all ! I am sorry about the flood of mails in checkins this morning. It was annoying. It seems to me that this is a general problem that will appear in CPython too once it's in mercurial. I think this can be easily fixed by a patch policy. I have just sent a mail to python-ideas about this, to propose something. Regards Tarek -- Tarek Ziad? | http://ziade.org From g.brandl at gmx.net Sun Jul 4 13:16:53 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 04 Jul 2010 13:16:53 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <4C2FC0EE.1060702@v.loewis.de> References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> <4C2FC0EE.1060702@v.loewis.de> Message-ID: Am 04.07.2010 00:59, schrieb "Martin v. L?wis": >> This is perhaps a naive question, but hat do you gain with the >> intermediate mirror clone of upstream? (Other than filling more of your >> disk?) > > In addition to the answer you got: this way of working is also the > process that I arrived at, independently. > > I see two uses, both based around the problem "creating a full clone > will take a long time - much longer than a subversion checkout". > 1. if I want several local checkouts (e.g. for testing separate > features), I can clone them all from the local copy (thereby > also preserving space, compared to independent clones) > 2, throwing away local changes is not that easy in Mercurial, > if you have committed them already. Of course, in SVN throwing away committed changes is completely impossible :) > There are extensions to > uncommit, but they are discouraged and have limitations. I wouldn't say that. strip works well and it does so logically, once one understands the DAG. The only thing discouraged is to strip changesets once pushed to the public repo, but that holds as well for getting rid of the changesets by making a new clone without them. cheers, Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From merwok at netwok.org Sun Jul 4 13:29:06 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sun, 04 Jul 2010 13:29:06 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> <4C2FC0EE.1060702@v.loewis.de> Message-ID: <4C307082.9030305@netwok.org> The other risk with history-editing extensions is that they use the merge and rebase machinery to do their work, so they require a clean working directory. Otherwise you may loose uncommitted changes. Not requiring any Mercurial extension to contribute to Python would be a nice policy to lower the entry bar. Regards From g.brandl at gmx.net Sun Jul 4 13:37:27 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 04 Jul 2010 13:37:27 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <4C307082.9030305@netwok.org> References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> <4C2FC0EE.1060702@v.loewis.de> <4C307082.9030305@netwok.org> Message-ID: Am 04.07.2010 13:29, schrieb ?ric Araujo: > The other risk with history-editing extensions is that they use the > merge and rebase machinery to do their work, so they require a clean > working directory. Otherwise you may loose uncommitted changes. That's true. > Not requiring any Mercurial extension to contribute to Python would be a > nice policy to lower the entry bar. That is already defeated by the need for the eol extension (which is now built-in in hg 1.6). Activating mq is something every developer should manage... Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From merwok at netwok.org Sun Jul 4 13:53:51 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sun, 04 Jul 2010 13:53:51 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> <4C2FC0EE.1060702@v.loewis.de> <4C307082.9030305@netwok.org> Message-ID: <4C30764F.2020103@netwok.org> >> Not requiring any Mercurial extension to contribute to Python would be a >> nice policy to lower the entry bar. > > That is already defeated by the need for the eol extension (which is now > built-in in hg 1.6). Activating mq is something every developer should > manage... Okay, make that ?not requiring any third-party Mercurial extension? :) Some extensions are too useful to be left out, and since they?re shipped with Mercurial the cost of using them is null (patchbomb, eol, bookmarks, graphlog, progress, relink, crecord, rebase). MQ is another beast, it?s a new way of using Mercurial for middle or advanced users. Cheers From mg at lazybytes.net Sun Jul 4 13:37:11 2010 From: mg at lazybytes.net (Martin Geisler) Date: Sun, 04 Jul 2010 13:37:11 +0200 Subject: [Python-Dev] Mercurial migration readiness References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> <4C2FC215.2040702@v.loewis.de> Message-ID: <87mxu7iil4.fsf@hbox.dyndns.org> "Martin v. L?wis" writes: >> My question is basically the same as Terry Reedy's, but I'm going to >> phrase it a bit differently: >> >> This is perhaps a naive question, but why do you create a second local >> clone instead of just creating a branch? > > IIUC, if you create a named branch, the branch will become globally > visible when you push your changes back. I assume people will consider > that clutter - it would be sufficient to just push the changes on the > branch, along with commit messages, but not the branch itself (which > would be only temporary, anyway). > > I'm not even sure how you pull changes from one branch into another: > can somebody kindly explain the commands that would be required? You don't -- it is tempting to think of a "named branch" as a kind of container for changesets, but that metaphor makes people think that you can put changesets "into" a named branch and takes them "out" again. The opposite is in fact true: the changesets induce the named branch. Each changeset has a field in its metadata that names the branch it is on. So if you do hg branch X hg commit -m "Created X branch" then the newly created changeset has branch="X" in its metadata. The X branch has thus been created because of the changeset. If there are no changesets with branch="X" in your repository, then there is no X branch. A named branch is thus more a labeling system -- changesets belonging to the branch can (in principle) be scattered all over the repository. They are normally a connected sub-graph, though, and Mercurial will complain if you try to re-start a branch name. Since the branch name is embedded into the changesets themselves, you cannot changeset it without changing the identities of the changesets. A branch appears in the output of 'hg branches' if there are any open branch heads (a "branch head" is a changeset with no children on the same named branch). Use the 'hg commit --close-branch' command to make a closed branch head. The branch is considered closed when it only has closed branch heads. Summing up, the notion of named branches correspond quite closely to how people model branches in Subversion: if you make your changes in the directory branches/X/, then we say you "work on the X branch". These revisions will always be on that branch. You can delete the branch by deleting the directory from the HEAD revision. The changes remain in the history and even after merging with trunk/, the old revision will of course show that they were made in the branches/X/ directory. Take a look at the bookmarks extension if you want to work with un-named branches (often called anonymous branches) instead. These branches are just a fork in the history graph, but they have no permanent name. The bookmarks gives you a convenient way to refer to them. Without bookmarks you can of course look at 'hg heads' or some other log viewer. http://mercurial.selenic.com/wiki/BookmarksExtension Since Mercurial 1.6, you can push and pull the bookmarks between repositories. They used to be local only, but this is now solved. See the bottom of the wiki page. -- Martin Geisler Mercurial links: http://mercurial.ch/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From g.brandl at gmx.net Sun Jul 4 14:31:25 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 04 Jul 2010 14:31:25 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <4C30764F.2020103@netwok.org> References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> <4C2FC0EE.1060702@v.loewis.de> <4C307082.9030305@netwok.org> <4C30764F.2020103@netwok.org> Message-ID: Am 04.07.2010 13:53, schrieb ?ric Araujo: >>> Not requiring any Mercurial extension to contribute to Python would be a >>> nice policy to lower the entry bar. >> >> That is already defeated by the need for the eol extension (which is now >> built-in in hg 1.6). Activating mq is something every developer should >> manage... > > Okay, make that ?not requiring any third-party Mercurial extension? :) mq is anything but third-party. > Some extensions are too useful to be left out, and since they?re shipped > with Mercurial the cost of using them is null (patchbomb, eol, > bookmarks, graphlog, progress, relink, crecord, rebase). > > MQ is another beast, it?s a new way of using Mercurial for middle or > advanced users. mq contains the strip command. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From merwok at netwok.org Sun Jul 4 14:41:36 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sun, 04 Jul 2010 14:41:36 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> <4C2FC0EE.1060702@v.loewis.de> <4C307082.9030305@netwok.org> <4C30764F.2020103@netwok.org> Message-ID: <4C308180.40500@netwok.org> >>>> Not requiring any Mercurial extension to contribute to Python would be a >>>> nice policy to lower the entry bar. >>> >>> That is already defeated by the need for the eol extension (which is now >>> built-in in hg 1.6). Activating mq is something every developer should >>> manage... >> >> Okay, make that ?not requiring any third-party Mercurial extension? :) > > mq is anything but third-party. Sorry, I didn?t want to imply that. What I meant was: Since eol is already a requirement, and since extensions provide tremendously useful things, it?s okay to require or recommend some of them. MQ is in another category IMO not because it?s not shipped-with, since it is, but because it makes you think differently that regular Mercurial usage. (Not saying better or worse, just differently, thus a bit more difficult for would-be contributors.) > mq contains the strip command. Which fits into regular Mercurial usage without requiring any MQ knowledge, agreed. By the way, is eol a requirement for people using Windows tools with limitations only, or for every person that will clone Python? Regards From g.brandl at gmx.net Sun Jul 4 14:40:14 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 04 Jul 2010 14:40:14 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <87mxu7iil4.fsf@hbox.dyndns.org> References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> <4C2FC215.2040702@v.loewis.de> <87mxu7iil4.fsf@hbox.dyndns.org> Message-ID: Am 04.07.2010 13:37, schrieb Martin Geisler: > "Martin v. L?wis" writes: > >>> My question is basically the same as Terry Reedy's, but I'm going to >>> phrase it a bit differently: >>> >>> This is perhaps a naive question, but why do you create a second local >>> clone instead of just creating a branch? >> >> IIUC, if you create a named branch, the branch will become globally >> visible when you push your changes back. I assume people will consider >> that clutter - it would be sufficient to just push the changes on the >> branch, along with commit messages, but not the branch itself (which >> would be only temporary, anyway). >> >> I'm not even sure how you pull changes from one branch into another: >> can somebody kindly explain the commands that would be required? > > You don't -- it is tempting to think of a "named branch" as a kind of > container for changesets, but that metaphor makes people think that you > can put changesets "into" a named branch and takes them "out" again. I'm not sure if I misunderstand Martin's intent, but in principle, if you want to merge one changes (not changesets!) branch into another, all you need to do would be "hg merge ". Subsequently committing the merge (after fixing conflicts) creates a new changeset "on" the current branch. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ncoghlan at gmail.com Sun Jul 4 14:48:53 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 4 Jul 2010 22:48:53 +1000 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <4C308180.40500@netwok.org> References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> <4C2FC0EE.1060702@v.loewis.de> <4C307082.9030305@netwok.org> <4C30764F.2020103@netwok.org> <4C308180.40500@netwok.org> Message-ID: On Sun, Jul 4, 2010 at 10:41 PM, ?ric Araujo wrote: > By the way, is eol a requirement for people using Windows tools with > limitations only, or for every person that will clone Python? It's designed so that everyone can run it regardless of platform. If someone chooses not to use it and messes up the line endings then the central repository hooks will reject the commit. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From merwok at netwok.org Sun Jul 4 14:55:50 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sun, 04 Jul 2010 14:55:50 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> <4C2FC215.2040702@v.loewis.de> <87mxu7iil4.fsf@hbox.dyndns.org> Message-ID: <4C3084D6.10703@netwok.org> > I'm not sure if I misunderstand Martin's intent, but in principle, if you > want to merge one changes (not changesets!) branch into another, all you > need to do would be "hg merge ". Subsequently committing > the merge (after fixing conflicts) creates a new changeset "on" the current > branch. Indeed, merges are branch-wise operations, not changeset-wise. You need tricks to copy one changeset from a named branch into another one. Copying changesets between a stable branch and a devel branch is easy if the devel history is a strict superset of the stable history. Every changesets landing into stable can then just be pulled into devel. This means no different named branches for stable and devel. Not sure I?m clear enough, I hope Mercurial experts can chime in. Regards From mg at lazybytes.net Sun Jul 4 16:32:59 2010 From: mg at lazybytes.net (Martin Geisler) Date: Sun, 04 Jul 2010 16:32:59 +0200 Subject: [Python-Dev] Mercurial migration readiness References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> <4C2FC215.2040702@v.loewis.de> <87mxu7iil4.fsf@hbox.dyndns.org> <4C3084D6.10703@netwok.org> Message-ID: <871vbjiag4.fsf@hbox.dyndns.org> ?ric Araujo writes: >> I'm not sure if I misunderstand Martin's intent, but in principle, if >> you want to merge one changes (not changesets!) branch into another, >> all you need to do would be "hg merge ". Subsequently >> committing the merge (after fixing conflicts) creates a new changeset >> "on" the current branch. > > Indeed, merges are branch-wise operations, not changeset-wise. You > need tricks to copy one changeset from a named branch into another > one. You basically need to replay the changes -- the transplant extension can do this for you. This is more or less a wrapper around 'hg export' (gives you a diff) and 'hg import' (applies the diff). > Copying changesets between a stable branch and a devel branch is easy > if the devel history is a strict superset of the stable history. Every > changesets landing into stable can then just be pulled into devel. > This means no different named branches for stable and devel. Not sure > I?m clear enough, I hope Mercurial experts can chime in. It sounds like you are describing how Mercurial itself was developed before we began using named branches. This part of the graph is typical for us then: http://selenic.com/repo/hg/graph/52c5be55af82 Notice the three merges wtih 'crew-stable', which was what we called the clone representing the next stable release. Whenever a bugfix is made, it is immediatedly merged into the 'crew' clone, where normal development takes place. This gives a characteristic flow with two tracks (branches), where one is continuously merged into the other. The two with commit message "Fix typeerror when specifying both --rebase and --pull" (dd1b47e17d7e and aee8455ee8ec) are a typical example of a transplanted changeset: the bug was first fixed in 'crew', but it was then realized that it should also be in the 'crew-stable' repository so that it could be part of the next point release. Today we use a 'stable' branch in addition to the 'default' branch and the graph looks much the same: http://selenic.com/repo/hg/graph/4d3288197717 You'll notice the small labels saying 'stable' on the changesets from that branch -- that is the whole difference. A named branch is made up by the changesets that same the branch name, but the graph itself works the same. The nice thing about our current setup is that I can do hg update stable and the name 'stable' is then interpreted as the tip-most changeset on the stable branch. -- Martin Geisler Mercurial links: http://mercurial.ch/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From benjamin at python.org Sun Jul 4 17:34:57 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 4 Jul 2010 10:34:57 -0500 Subject: [Python-Dev] [RELEASE] Python 2.7 released Message-ID: On behalf of the Python development team, I'm jocund to announce the second release candidate of Python 2.7. Python 2.7 will be the last major version in the 2.x series. However, it will also have an extended period of bugfix maintenance. 2.7 includes many features that were first released in Python 3.1. The faster io module, the new nested with statement syntax, improved float repr, set literals, dictionary views, and the memoryview object have been backported from 3.1. Other features include an ordered dictionary implementation, unittests improvements, a new sysconfig module, auto-numbering of fields in the str/unicode format method, and support for ttk Tile in Tkinter. For a more extensive list of changes in 2.7, see http://doc.python.org/dev/whatsnew/2.7.html or Misc/NEWS in the Python distribution. To download Python 2.7 visit: http://www.python.org/download/releases/2.7/ 2.7 documentation can be found at: http://docs.python.org/2.7/ This is a production release and should be suitable for all libraries and applications. Please report any bugs you find, so they can be fixed in the next maintenance releases. The bug tracker is at: http://bugs.python.org/ Enjoy! -- Benjamin Peterson Release Manager benjamin at python.org (on behalf of the entire python-dev team and 2.7's contributors) From merwok at netwok.org Sun Jul 4 17:56:49 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sun, 04 Jul 2010 17:56:49 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <871vbjiag4.fsf@hbox.dyndns.org> References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> <4C2FC215.2040702@v.loewis.de> <87mxu7iil4.fsf@hbox.dyndns.org> <4C3084D6.10703@netwok.org> <871vbjiag4.fsf@hbox.dyndns.org> Message-ID: <4C30AF41.7090302@netwok.org> [Martin Geisler] > You basically need to replay the changes -- the transplant extension can > do this for you. This is more or less a wrapper around 'hg export' > (gives you a diff) and 'hg import' (applies the diff). Core developers seem to be okay with svnmerge, so perhaps a familiar-looking command like transplant (with ?merges? that are actually replays, and explicit blocking for bookkeeping) will feel right to them. > It sounds like you are describing how Mercurial itself was developed > before we began using named branches. Spot on, sir! > [snip] You'll notice the small labels saying 'stable' on the changesets > from that branch -- that is the whole difference. Thanks for the explanation. The new workflow seems quite good. Regards From benjamin at python.org Sun Jul 4 18:02:55 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 4 Jul 2010 11:02:55 -0500 Subject: [Python-Dev] Thank yous Message-ID: Now that Python 2.7 is out, I'd like to thank a few of the people who made it possible: - Martin for building Windows Installers to my schedule, maintaining the tracker and PyPi, and providing lots of guidance and advice. - Ronald for building Mac installers. - Tarek for picking up the reins of distutils. - Jesse for being responsive to last minute multiprocessing bugs. - Georg for keeping the documentation excellent and numerous bug fixes. - Victor for fixing obscure unicode bugs and segfaults. - Ezio for plugging unicode holes and fixing docs. - Alexander Belopolsky for taking up datetime. - R. David Murray for picking up the email module. - Alexandre Vassalotti for various backports. - Mark for oiling Python's mathematical gears and explaining floating point to me. - Antoine for backporting the improved io module, memoryviews, adopting the ssl module, and numerous bug fixes. - Barry for mentoring me in the release process and testing out release.py. - Dirkjan for always getting us closer to Mercurial migration. - Andrew for writing "What's New in Python 2.7?" That's a lot of changes to keep up with. - And, of course, all of the people who reported bugs or submitted patches. Thank you once again, Benjamin From benjamin at python.org Sun Jul 4 18:03:47 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 4 Jul 2010 11:03:47 -0500 Subject: [Python-Dev] [RELEASE] Python 2.7 released In-Reply-To: References: Message-ID: 2010/7/4 Benjamin Peterson : > On behalf of the Python development team, I'm jocund to announce the second > release candidate of Python 2.7. Arg!!! This should, of course, be "final release". -- Regards, Benjamin From larry at hastings.org Sun Jul 4 18:04:52 2010 From: larry at hastings.org (Larry Hastings) Date: Sun, 04 Jul 2010 09:04:52 -0700 Subject: [Python-Dev] [RELEASE] Python 2.7 released In-Reply-To: References: Message-ID: <4C30B124.9070603@hastings.org> On 07/04/2010 08:34 AM, Benjamin Peterson wrote: > On behalf of the Python development team, I'm jocund to announce the second > release candidate of Python 2.7. > s/second release candidate/release/ *standing ovation* //larry// From solipsis at pitrou.net Sun Jul 4 18:11:38 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 4 Jul 2010 18:11:38 +0200 Subject: [Python-Dev] 2.7 hg mirror References: Message-ID: <20100704181138.36f340b0@pitrou.net> On Sun, 4 Jul 2010 10:34:57 -0500 Benjamin Peterson wrote: > On behalf of the Python development team, I'm jocund to announce the second > release candidate of Python 2.7. > > Python 2.7 will be the last major version in the 2.x series. However, it will > also have an extended period of bugfix maintenance. A Mercurial mirror is also available for the 2.7 maintenance branch: http://code.python.org/hg/branches/release2.7-maint/ As usual, the full list of mirrors can be viewed at: http://code.python.org/hg Regards Antoine. From alexander.belopolsky at gmail.com Sun Jul 4 18:15:14 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 4 Jul 2010 12:15:14 -0400 Subject: [Python-Dev] Thank yous In-Reply-To: References: Message-ID: On Sun, Jul 4, 2010 at 12:02 PM, Benjamin Peterson wrote: .. > - Alexander Belopolsky for taking up datetime. I am honored that my contributions have been noticed, but would not be able to contribute as much without support from Mark Dickinson. From merwok at netwok.org Sun Jul 4 18:58:53 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sun, 04 Jul 2010 18:58:53 +0200 Subject: [Python-Dev] Signs of neglect? In-Reply-To: <4C25E20D.2040007@holdenweb.com> References: <4C25E20D.2040007@holdenweb.com> Message-ID: <4C30BDCD.8010201@netwok.org> Hello pydev I?d like to volunteer to maintain a tool but I?m not sure where I can help. I?m already proposing changes to Brett for Tools/scripts/patchcheck.py, and I have commented on Tools/i18n bugs, but these ones are already maintained by their authors (e.g. Barry is assigned pygettext bugs) and I?m by no means a gettext expert. If I adopt a tool, it will still require some core dev time to review and commit, but I?ll do the bug triage, discussion and coding. I suggest that each adoption starts with a reindent.py run and a PEP 8 compliance check. Let demos and tools become a showcase to be proud of! I have opened a bug to discuss adding a simple sanity test that would just make sure tools and demos can run without syntax errors: http://bugs.python.org/issue9153 There was an open bug about demos, http://bugs.python.org/issue7962, I?ve widened it into a meta-bug for demos and tools. (I hope it is okay.) Please comment, adopt or condemn demos and tools in that report. Hope this helps. Regards From p.f.moore at gmail.com Sun Jul 4 19:16:10 2010 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 4 Jul 2010 18:16:10 +0100 Subject: [Python-Dev] Thank yous In-Reply-To: References: Message-ID: On 4 July 2010 17:02, Benjamin Peterson wrote: > Now that Python 2.7 is out, I'd like to thank a few of the people who > made it possible: And not forgetting Benjamin himself for managing the whole thing! Paul. From ziade.tarek at gmail.com Sun Jul 4 19:26:44 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 4 Jul 2010 19:26:44 +0200 Subject: [Python-Dev] Thank yous In-Reply-To: References: Message-ID: On Sun, Jul 4, 2010 at 7:16 PM, Paul Moore wrote: > On 4 July 2010 17:02, Benjamin Peterson wrote: >> Now that Python 2.7 is out, I'd like to thank a few of the people who >> made it possible: > > And not forgetting Benjamin himself for managing the whole thing! +1. Thanks a lot for your hard work From kristjan at ccpgames.com Sun Jul 4 19:59:47 2010 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Sun, 4 Jul 2010 17:59:47 +0000 Subject: [Python-Dev] issue 9141 Message-ID: <930F189C8A437347B80DF2C156F7EC7F0AABE3370E@exchis.ccp.ad.local> Dear colleagues: I'd like to have your input on http://bugs.python.org/issue9141 "Allow objects to decide if they can be collected by GC." This generalizes the dynamic discovery of finalization state of objects, as already provided for generator objects with PyGen_NeedsFinalizing(),to any object. Kristj?n -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Sun Jul 4 21:20:07 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 04 Jul 2010 15:20:07 -0400 Subject: [Python-Dev] blocking 2.7 In-Reply-To: <4C302ADD.30608@netwok.org> References: <201007031426.53880.victor.stinner@haypocalc.com> <4C302ADD.30608@netwok.org> Message-ID: On 7/4/2010 2:31 AM, ?ric Araujo wrote: >> But Python tests lack coverage stats, so it is hard to say anything. > FYI: http://coverage.livinglogic.de/ Turns out the audioop is one of the best covered modules, at 98% -- Terry Jan Reedy From brett at python.org Sun Jul 4 21:20:37 2010 From: brett at python.org (Brett Cannon) Date: Sun, 4 Jul 2010 12:20:37 -0700 Subject: [Python-Dev] More detailed build instructions for Windows In-Reply-To: <4C2FBC6F.4020706@v.loewis.de> References: <4C2EE015.4050305@v.loewis.de> <4C2FBC6F.4020706@v.loewis.de> Message-ID: On Sat, Jul 3, 2010 at 15:40, "Martin v. L?wis" wrote: > Am 03.07.2010 16:34, schrieb Christian Heimes: >> Am 03.07.2010 09:00, schrieb "Martin v. L?wis": >>>> I'm trying to test out a patch to add a timeout in subprocess.py on >>>> Windows, so I need to build Python with Visual Studio. ?The docs say >>>> the files in PCBuild/ work with VC 9 and newer. >>> >>> Which docs did you look at specifically that said "and newer"? That >>> would be a bug. >> >> The readme.txt in the PCbuild directory contains the sentence "Microsoft >> Visual C++ 2008 Express Edition is required at the very least". The >> wording could be interpreted as "2008 at very least" instead of "Express >> Edition at very least". > > That's what I expected to get as an answer - interestingly enough, Reid > was looking at some other place (the FAQ) that literally said "and > newer". I'd call that an error, even though, technically, converting the > projects to VS 2010 "ought to work" (i.e. it did after I committed some > fixes to make it work). FAQ updated with "and newer" stripped out. -Brett > > Regards, > Martin > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From jnoller at gmail.com Sun Jul 4 22:45:10 2010 From: jnoller at gmail.com (Jesse Noller) Date: Sun, 4 Jul 2010 16:45:10 -0400 Subject: [Python-Dev] Thank yous In-Reply-To: References: Message-ID: On Sun, Jul 4, 2010 at 1:26 PM, Tarek Ziad? wrote: > On Sun, Jul 4, 2010 at 7:16 PM, Paul Moore wrote: >> On 4 July 2010 17:02, Benjamin Peterson wrote: >>> Now that Python 2.7 is out, I'd like to thank a few of the people who >>> made it possible: >> >> And not forgetting Benjamin himself for managing the whole thing! > > +1. Thanks a lot for your hard work Seriously Benjamin, you've done a great job. From g.brandl at gmx.net Mon Jul 5 00:04:26 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 05 Jul 2010 00:04:26 +0200 Subject: [Python-Dev] Thank yous In-Reply-To: References: Message-ID: Am 04.07.2010 22:45, schrieb Jesse Noller: > On Sun, Jul 4, 2010 at 1:26 PM, Tarek Ziad? wrote: >> On Sun, Jul 4, 2010 at 7:16 PM, Paul Moore wrote: >>> On 4 July 2010 17:02, Benjamin Peterson wrote: >>>> Now that Python 2.7 is out, I'd like to thank a few of the people who >>>> made it possible: >>> >>> And not forgetting Benjamin himself for managing the whole thing! >> >> +1. Thanks a lot for your hard work > > Seriously Benjamin, you've done a great job. Totally! *cheer* Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From dickinsm at gmail.com Mon Jul 5 09:46:57 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 5 Jul 2010 08:46:57 +0100 Subject: [Python-Dev] Thank yous In-Reply-To: References: Message-ID: On Sun, Jul 4, 2010 at 9:45 PM, Jesse Noller wrote: > On Sun, Jul 4, 2010 at 1:26 PM, Tarek Ziad? wrote: >> On Sun, Jul 4, 2010 at 7:16 PM, Paul Moore wrote: >>> On 4 July 2010 17:02, Benjamin Peterson wrote: >>>> Now that Python 2.7 is out, I'd like to thank a few of the people who >>>> made it possible: >>> >>> And not forgetting Benjamin himself for managing the whole thing! >> >> +1. Thanks a lot for your hard work > > Seriously Benjamin, you've done a great job. +1. A fantastic job. Mark From orsenthil at gmail.com Mon Jul 5 10:21:32 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Mon, 5 Jul 2010 13:51:32 +0530 Subject: [Python-Dev] Thank yous In-Reply-To: References: Message-ID: <20100705082132.GA2675@remy> On Mon, Jul 05, 2010 at 08:46:57AM +0100, Mark Dickinson wrote: > >>> On 4 July 2010 17:02, Benjamin Peterson wrote: > >>>> Now that Python 2.7 is out, I'd like to thank a few of the people who > >>>> made it possible: > >>> > >>> And not forgetting Benjamin himself for managing the whole thing! > >> > >> +1. Thanks a lot for your hard work > > > > Seriously Benjamin, you've done a great job. > > +1. A fantastic job. > Yeah, there is lot to learn from you in the way you lead the release. Thank you! Senthil From fijall at gmail.com Mon Jul 5 10:44:57 2010 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 5 Jul 2010 02:44:57 -0600 Subject: [Python-Dev] Can Python implementations reject semantically invalid expressions? In-Reply-To: <201007031435.09229.steve@pearwood.info> References: <201007021428.48162.steve@pearwood.info> <4C2E94BB.1090901@canterbury.ac.nz> <201007031435.09229.steve@pearwood.info> Message-ID: On Fri, Jul 2, 2010 at 10:35 PM, Steven D'Aprano wrote: > On Sat, 3 Jul 2010 11:39:07 am Greg Ewing wrote: >> Stefan Behnel wrote: >> > So, would it still be Python if it folded >> > >> > ? ? 1 + "1" >> > >> > into >> > >> > ? ? raise TypeError() >> > >> > at compile time? >> >> It would have to be >> >> ? ? raise TypeError("Exactly the message that would have been >> produced at run time") > > > Python doesn't make any guarantees about the message that exceptions > display, so I don't think you need to match the message, just the > exception. Anyone testing for specific exception messages is living in > a state of sin and shouldn't complain when their code stops working. An > implementation might choose to raise TypeError('is this the right place > for an argument?') on alternate Tuesdays, and it would still meet the > promises made by the language. > In theory it does not, in practice people rely on it. Besides, I would not be overly happy if: def f(): return 1 + "1" and def g(): a = 1 return a + "1" raise different messages From chris at simplistix.co.uk Mon Jul 5 14:34:21 2010 From: chris at simplistix.co.uk (Chris Withers) Date: Mon, 05 Jul 2010 13:34:21 +0100 Subject: [Python-Dev] logging package vs unicode Message-ID: <4C31D14D.5010001@simplistix.co.uk> Hi All, The documentation for the logging package doesn't include any mention of unicode. What's the recommended usage in terms of passing unicode vs encoded strings? How does this differ from Python 2 to Python 3? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From stefan_ml at behnel.de Mon Jul 5 15:06:22 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 05 Jul 2010 15:06:22 +0200 Subject: [Python-Dev] set/dict comprehensions don't leak in Py2.7 - intentional? Message-ID: Hi, I only noticed now that set/dict comprehensions were backported to 2.7 without letting the loop Variables leak into the surrounding scope. So they now behave different from list comprehensions. Is this intentional or just a backporting oversight? I'm asking because we had a discussion about the 'right' semantics several times on the Cython list, and the current behaviour is that all comprehensions leak when compiling Py2 code and the do not leak when compiling Py3 code. I would hate to fix that up to match 2.7 and then see that the behaviour in Py2.7 is to be considered an accident. Stefan From mail at timgolden.me.uk Mon Jul 5 15:12:15 2010 From: mail at timgolden.me.uk (Tim Golden) Date: Mon, 05 Jul 2010 14:12:15 +0100 Subject: [Python-Dev] set/dict comprehensions don't leak in Py2.7 - intentional? In-Reply-To: References: Message-ID: <4C31DA2F.9070503@timgolden.me.uk> On 05/07/2010 14:06, Stefan Behnel wrote: > Hi, > > I only noticed now that set/dict comprehensions were backported to 2.7 > without letting the loop Variables leak into the surrounding scope. So > they now behave different from list comprehensions. Is this intentional > or just a backporting oversight? > > I'm asking because we had a discussion about the 'right' semantics > several times on the Cython list, and the current behaviour is that all > comprehensions leak when compiling Py2 code and the do not leak when > compiling Py3 code. I would hate to fix that up to match 2.7 and then > see that the behaviour in Py2.7 is to be considered an accident. Depends on your definition of "comprehension", but generator expressions have always not leaked, so these are already different in Py2.x: [i for i in range (10)] list (i for i in range (10)) I had understood (without being able to put my finger on a relevant thread) that all comprehensions were going not leak their loop variables in the future. TJG From fuzzyman at voidspace.org.uk Mon Jul 5 15:16:23 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 05 Jul 2010 14:16:23 +0100 Subject: [Python-Dev] set/dict comprehensions don't leak in Py2.7 - intentional? In-Reply-To: <4C31DA2F.9070503@timgolden.me.uk> References: <4C31DA2F.9070503@timgolden.me.uk> Message-ID: <4C31DB27.8040004@voidspace.org.uk> On 05/07/2010 14:12, Tim Golden wrote: > On 05/07/2010 14:06, Stefan Behnel wrote: >> Hi, >> >> I only noticed now that set/dict comprehensions were backported to 2.7 >> without letting the loop Variables leak into the surrounding scope. So >> they now behave different from list comprehensions. Is this intentional >> or just a backporting oversight? >> >> I'm asking because we had a discussion about the 'right' semantics >> several times on the Cython list, and the current behaviour is that all >> comprehensions leak when compiling Py2 code and the do not leak when >> compiling Py3 code. I would hate to fix that up to match 2.7 and then >> see that the behaviour in Py2.7 is to be considered an accident. > > Depends on your definition of "comprehension", but generator expressions > have always not leaked, so these are already different in Py2.x: > > [i for i in range (10)] > > list (i for i in range (10)) > > I had understood (without being able to put my finger on a relevant > thread) > that all comprehensions were going not leak their loop variables in the > future. Similarly my understanding was that list comprehensions leaking was an unfortunate implementation detail and that to depend on it was therefore "a bug". I don't see it as an issue that set and dict comprehensions don't leak in 2.7 (and in fact it is a good thing as it makes it harder to write code that works in 2.7 but fails in 3.x). Michael > > TJG > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From ncoghlan at gmail.com Mon Jul 5 16:04:00 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 6 Jul 2010 00:04:00 +1000 Subject: [Python-Dev] [RELEASE] Python 2.7 released In-Reply-To: References: Message-ID: On Mon, Jul 5, 2010 at 2:03 AM, Benjamin Peterson wrote: > 2010/7/4 Benjamin Peterson : >> On behalf of the Python development team, I'm jocund to announce the second >> release candidate of Python 2.7. > > Arg!!! This should, of course, be "final release". It wouldn't be a proper Python release without a typo *somewhere* in the release announcement :) Well shepherded! Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Mon Jul 5 16:19:31 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 6 Jul 2010 00:19:31 +1000 Subject: [Python-Dev] blocking 2.7 In-Reply-To: References: <201007031426.53880.victor.stinner@haypocalc.com> <4C302ADD.30608@netwok.org> Message-ID: On Mon, Jul 5, 2010 at 5:20 AM, Terry Reedy wrote: > On 7/4/2010 2:31 AM, ?ric Araujo wrote: >>> >>> But Python tests lack coverage stats, so it is hard to say anything. >> >> FYI: http://coverage.livinglogic.de/ > > Turns out the audioop is one of the best covered modules, at 98% Alas, those are only the stats for the audioop test suite. audioop itself is written in C, so the automatic coverage stats generated by livinglogic don't provide any details. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Mon Jul 5 16:25:52 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 6 Jul 2010 00:25:52 +1000 Subject: [Python-Dev] set/dict comprehensions don't leak in Py2.7 - intentional? In-Reply-To: <4C31DA2F.9070503@timgolden.me.uk> References: <4C31DA2F.9070503@timgolden.me.uk> Message-ID: On Mon, Jul 5, 2010 at 11:12 PM, Tim Golden wrote: > I had understood (without being able to put my finger on a relevant thread) > that all comprehensions were going not leak their loop variables in the > future. This understanding is correct (with that future being Python 3.x where all comprehensions have their own private scope). As others have noted, generator expressions in 2.x already didn't leak the iteration variable into the surrounding scope. The only reason list comprehensions continue to leak the iteration variable in 2.6 and 2.7 is that there *is* code out there that relies on that behaviour. However, since legacy code can't include set or dict comprehensions, there was no need for the backported versions to replicate this misfeature of 2.x list comprehensions. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From aahz at pythoncraft.com Mon Jul 5 18:28:56 2010 From: aahz at pythoncraft.com (Aahz) Date: Mon, 5 Jul 2010 09:28:56 -0700 Subject: [Python-Dev] FWD: error: 3.2 release schedule has release in Jan 2010. Message-ID: <20100705162855.GA22056@panix.com> ----- Forwarded message from Ric Johnson ----- > Date: Mon, 5 Jul 2010 08:49:29 -0700 (PDT) > From: Ric Johnson > Subject: error: 3.2 release schedule has release in Jan 2010. > To: webmaster at python.org > > Hey! > There's a typo on http://www.python.org/dev/peps/pep-0392/: > > Release Schedule > > The current schedule is: > > ... > > # 3.2 candidate 2: January 1, 2010 ==> should be 2011 > # 3.2 final: January 15, 2010 ==> should be 2011 > > > I know it is rough schedule but it looks bad ... > > Ric > > -Verba nihil operandi melius est quam absurde. > > > ----- End forwarded message ----- -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you don't know what your program is supposed to do, you'd better not start writing it." --Dijkstra From techtonik at gmail.com Mon Jul 5 20:04:28 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Mon, 5 Jul 2010 21:04:28 +0300 Subject: [Python-Dev] Licensing // PSF // Motion of non-confidence Message-ID: Sorry for touching a sore point of if I sound like a boss to someone. I tried to be as constructive as possible, but politeness was not the point, so I can only hope you understand. I do not think PSF does its job well and here is why. 1. Python licensing terms are explained poorly In order to "advance the Python programming language", and "to support and facilitate the growth of international community" PSF should be clear about the legal stuff with no complications. So that even people without English background can understand the terms. 1.1 Let's start with http://www.python.org/psf/license/ ... Full text of the Python 2.6.2 license ... Q: Why 2.6.2? A: Nobody cares. Inside referenced doc there are several licenses for Python + historical note. I remember I've tried to read this file several times, but only after comparing freeness and beerness of PSF license to GPL in private talk with Eric I was able to go through this seriously. Moving on... 1.2. Mixed content of Python License file Attached or see http://code.python.org/hg/branches/release2.7-maint/annotate/41a3e598052f/LICENSE The file consists of several licenses for multiple versions of Python. It is an unusual mix that negatively affects understanding. For the sake of clarity the LICENSE file scope should only be relevant to the current code. It may include a historical reference to other files if needed - LICENSE-HISTORY if you like. It is important to keep LICENSE terms concise for occasional contributors with skills needed for core development, because for outsiders there are many open source projects out there. _For example_, if I see an interesting project with GPL license (Ruby) and BSD license (Go) and ask myself "What should I choose for the next time I feel too depressive to spend my weekend watching movies?" - I will choose Go. I am sure it won't be a waste of my life time, just because both languages are equally trendy (Ruby sites are truly Gems in design, and Go presentation at Google IO was awesome), so license here is deciding factor. Every language has bugs and it is inevitable that you would like them to be fixed. As it is open source.. yes - you'll be politely asked to go fix them yourself. If you're an expert - you will fix them anyway (whatever you share fixes or not is a different story). Over the time you'll become acquainted with codebase and of course you will want to use this experience in your daily job writing code for customers who pay you money for keeping the code closed. In this case GPL allows you the freedom to rewrite the code from scratch, BSD - to copy/paste with attribution notice. If I'd be an outsider - when I'd see PSF license - I'd think - "or no, thats yet another stupid big license" (YASBL) and move on to the next news item (this time from pure C universe). 1.3 Python License content I believe that contents of attached LICENSE-PYTHON-2.x single copy/paste from LICENSE is enough for understanding and complying for Python 2.1+ series. Correct me if I wrong. But thanks to Berne Convention the wording can and should be improved. I hope the example in 1.2 made it clear that to attract professionals with strong core development skills, who are able to make quality contributions and are usually sought after in commercial projects, it is important to have license terms similar to those people already familiar and comfortable with, i.e. BSD, MIT/X (or more recent ISC variant - http://en.wikipedia.org/wiki/ISC_license) I propose the simplified PSF License v3 that in comparison to PSF v2: 1.3.1 combines point 4 and 5 to standard "THE SOFTWARE IS PROVIDED AS-IS ..." copied verbatim from ISC (MIT equivalent) license 1.3.2 drops point 6 as it lacks the copyright of Captain Obvious 1.3.3 combines points 8, 1 and 2 into standard ISC (simplified MIT) header with "and the following conditions are met:" from BSD license to outline the requirements of PSF in comparison to MIT, BSD and ISC style licenses 1.3.4 replaces point 7 with standard "Neither the name of the ... " from BSD license 1.3.5 replaces point 3 with "Redistributions of derivative works should include a brief summary of the changes made to Python" Full text of license is attached as PSF-LICENSE-v3. For those who complain that I am not sending patches and only direct people what to do - LICENSE.patch contains the patch made with diff.py tool that comes bundled with Python. Please indicate if and where the license became different from legal point of view. 2. PSF fails to comply with the terms of Contributors Agreement http://python.org/psf/contrib/contrib-form/ Note that there are also: http://python.org/psf/contrib/contrib-form-python/ http://python.org/psf/contrib/contrib-form-jython/ which are reachable by "python contributor's agreement" search query and it is doesn't clear why there are multiple terms and why they differ. -- personal intermezzo -- The fact that Contributors Agreements chapter is here is that I didn't care about the license much until people sorted me out from Python community of "those who signed contributor's agreement" into "those other outcasts that don't have the right to speak" category. I started to ask questions about licensing stuff, and received a bunch of negative replies like "why don't you sign the agreement first?", "we've consulted with lawyer - believe us", "if you are not a lawyer - why are you asking?". I do not know how many of these replies were from members of PSF, but they surely were not happy to clarify anything. I stopped sending patches (because I didn't sign this agreement) and was accused in unethical behavior like doing myself nothing and just directing people what to do. To prevent further allegations I hereby claim that PSF is fully responsible in the outcome of current situation and in the people pissed off from Python Community as a result! Just kidding - a little but tired already. -- personal intermezzo -- I assume that pydotorg contributions, Python contributions and Jython are the same contributions, so there is only one Contributors Agreement, but there is no proof. PSF doesn't comply with the terms of agreement, because it requires contributors to submit code under either Apache 2.0 or AFL license. Both these licenses require to include text of license in distributions, but so far no Apache or AFL license text is present in Python source code tree. PSF also fails to explain they need to receive contributions using two very verbose licenses, which are almost identical. AFL in fact was found "redundant" in favor of Apache 2.0 in 2006 by OSI's License Proliferation Committee. PSF can just say: """Contributor understands and agrees that PSF shall have the irrevocable and perpetual right to make and distribute copies of any Contribution, as well as to create and distribute collective works and derivative works of any Contribution.""" That is enough. Why playing strange Apache 2.0 and AFL games if it is clearly written : """ Contributor understands and agrees that PSF shall have ... the right ... to create ... derivative works ... under the Initial License or under any other open source license approved by a unanimous vote of the PSF board """ " Contributor understands and agrees ... " - I do not understand, so I can't sign. Even if I sign CA there is a chance that I (or whoever has more money) can prove that terms impossible to understand. In this case all contributors agreements will be deemed invalid. So Python code is in danger and hence my derived work is in danger too. That's why I asked who supplied PSF lawyer to create such agreement. The argumentation that this agreement is legitimate like "Believe us - we've payed to lawyer" is surely not the thing I expected to hear. If you want international recognition of legitimacy of such agreements. - you need the reputation of some legal authority like Ernst&Young or Deloitte or one of big PSF supporters. In this case PSF will be eligible for trust instead of non-confidence notes. There are other reasons that make me hesitate about signing PSF agreements. For instance PSF doesn't provide any guarantees that: 1. The text of Contributors Agreement won't be changed after I sign it 2. My personal details are properly guarded and won't be shared with any members of PSF committee or partner companies or any third parties without my explicit permission That's all. You may want to reread the first paragraph of the letter before reply. P.S. Sorry for not replying to recent threads (i.e. Mercurial) - writing a letter like these takes from 4 to 12 hours and is very exhausting. -- anatoly t. -------------- next part -------------- A non-text attachment was scrubbed... Name: LICENSE Type: application/octet-stream Size: 14448 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: LICENSE-PYTHON-2.x Type: application/octet-stream Size: 2464 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PSF-LICENSE-v3 Type: application/octet-stream Size: 1182 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: LICENSE.patch Type: application/octet-stream Size: 3930 bytes Desc: not available URL: From benjamin at python.org Mon Jul 5 20:19:07 2010 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 5 Jul 2010 13:19:07 -0500 Subject: [Python-Dev] Licensing // PSF // Motion of non-confidence In-Reply-To: References: Message-ID: 2010/7/5 anatoly techtonik : > Sorry for touching a sore point of if I sound like a boss to someone. > I tried to be as constructive as possible, but politeness was not the > point, so I can only hope you understand. > > I do not think PSF does its job well and here is why. Please contact psf at python.org then. -- Regards, Benjamin From brett at python.org Mon Jul 5 22:04:45 2010 From: brett at python.org (Brett Cannon) Date: Mon, 5 Jul 2010 13:04:45 -0700 Subject: [Python-Dev] Licensing // PSF // Motion of non-confidence In-Reply-To: References: Message-ID: On Mon, Jul 5, 2010 at 11:04, anatoly techtonik wrote: > Sorry for touching a sore point of if I sound like a boss to someone. > I tried to be as constructive as possible, but politeness was not the > point, so I can only hope you understand. > > I do not think PSF does its job well and here is why. > > > 1. Python licensing terms are explained poorly > In order to "advance the Python programming language", and "to support > and facilitate the growth of international community" PSF should be > clear about the legal stuff with no complications. So that even people > without English background can understand the terms. > > 1.1 Let's start with http://www.python.org/psf/license/ > ... > ? ?Full text of the Python 2.6.2 license > ... > Q: Why 2.6.2? > A: Nobody cares. But no one also cares that it's 2.6.2. Someone who has editing writes on the website decided to point the link to then newest version of the license at that time. But the only difference in the license between releases is that there is another line recording a release date. THAT'S IT! So who care if it is 2.6.2 that's listed? That page could get updated by the release manager for every release, but that's just another step in an already involved process. > > Inside referenced doc there are several licenses for Python + > historical note. I remember I've tried to read this file several > times, but only after comparing freeness and beerness of PSF license > to GPL in private talk with Eric I was able to go through this > seriously. > Moving on... > > 1.2. Mixed content of Python License file > Attached or see > http://code.python.org/hg/branches/release2.7-maint/annotate/41a3e598052f/LICENSE > > The file consists of several licenses for multiple versions of Python. > It is an unusual mix that negatively affects understanding. For the > sake of clarity the LICENSE file scope should only be relevant to the > current code. It may include a historical reference to other files if > needed - LICENSE-HISTORY if you like. But the license stack is required. There is code in Python that dates back to the early 1990s and is covered by the oldest license in there. Unless an explicit purge is done to rid Python of all code that falls under one of the licenses in the stack you cannot just remove it because it's old. The license is complicated and long because Python's history is complicated and long. > > It is important to keep LICENSE terms concise for occasional > contributors with skills needed for core development, because for > outsiders there are many open source projects out there. > _For example_, if I see an interesting project with GPL license (Ruby) > and BSD license (Go) and ask myself "What should I choose for the next > time I feel too depressive to spend my weekend watching movies?" - I > will choose Go. I am sure it won't be a waste of my life time, just > because both languages are equally trendy (Ruby sites are truly Gems > in design, and Go presentation at Google IO was awesome), so license > here is deciding factor. Every language has bugs and it is inevitable > that you would like them to be fixed. As it is open source.. yes - > you'll be politely asked to go fix them yourself. If you're an expert > - you will fix them anyway (whatever you share fixes or not is a > different story). Over the time you'll become acquainted with codebase > and of course you will want to use this experience in your daily job > writing code for customers who pay you money for keeping the code > closed. In this case GPL allows you the freedom to rewrite the code > from scratch, BSD - to copy/paste with attribution notice. > > If I'd be an outsider - when I'd see PSF license - I'd think - "or no, > thats yet another stupid big license" (YASBL) and move on to the next > news item (this time from pure C universe). The license is certified by the OSI and is GPL-compatible. If you want to go through the entire codebase of Python and rewrite it so that it can be licensed under a BSD-derived one, go for it. I for one, have better things to do. > > > 1.3 Python License content > > I believe that contents of attached LICENSE-PYTHON-2.x single > copy/paste from LICENSE is enough for understanding and complying for > Python 2.1+ series. Correct me if I wrong. But thanks to Berne > Convention the wording can and should be improved. > > I hope the example in 1.2 made it clear that to attract professionals > with strong core development skills, who are able to make quality > contributions and are usually sought after in commercial projects, it > is important to have license terms similar to those people already > familiar and comfortable with, i.e. BSD, MIT/X (or more recent ISC > variant - http://en.wikipedia.org/wiki/ISC_license) Out of my over 8 years of participating on python-dev, this is the first time ANYONE has brought this up. I do not think we are losing any developers who want to contribute because of the PSF license. > > I propose the simplified PSF License v3 that in comparison to PSF v2: > ?1.3.1 combines point 4 and 5 to standard "THE SOFTWARE IS PROVIDED > AS-IS ..." copied verbatim from ISC (MIT equivalent) license > ?1.3.2 drops point 6 as it lacks the copyright of Captain Obvious > ?1.3.3 combines points 8, 1 and 2 into standard ISC (simplified MIT) > header with "and the following conditions are met:" from BSD license > to outline the requirements of PSF in comparison to MIT, BSD and ISC > style licenses > ?1.3.4 replaces point 7 with standard "Neither the name of the > ... " from BSD license > ?1.3.5 replaces point 3 with "Redistributions of derivative works > should include a brief summary of the changes made to Python" > That's going to require a lawyer to sign off on, and I think our lawyer has better things to do than to worry about trying to make a little bit of legalese a little clearer for the layperson. It has been certified by third parties that the PSF license is a BSD-derived license and is GPL-compatible. That's plenty. > Full text of license is attached as PSF-LICENSE-v3. > For those who complain that I am not sending patches and only direct > people what to do - LICENSE.patch contains the patch made with diff.py > tool that comes bundled with Python. > > Please indicate if and where the license became different from legal > point of view. > > > 2. PSF fails to comply with the terms of Contributors Agreement > http://python.org/psf/contrib/contrib-form/ > > Note that there are also: > http://python.org/psf/contrib/contrib-form-python/ > http://python.org/psf/contrib/contrib-form-jython/ > which are reachable by "python contributor's agreement" search query > and it is doesn't clear why there are multiple terms and why they > differ. It's purely to be nice to the developer if they happen to have some preference for some reason. The licenses listed allow for us to immediately relicense any contributed code under the PSF. Which one makes absolutely no technical difference. > > -- personal intermezzo -- > The fact that Contributors Agreements chapter is here is that I didn't > care about the license much until people sorted me out from Python > community of "those who signed contributor's agreement" into "those > other outcasts that don't have the right to speak" category. Where the hell did you get that POV from? Anyone can sign the agreement. Practically every person who has shown up to a sprint for the core has signed that agreement. Just because you have signed it does not mean you have garnered any special ranking within the Python community and your opinion is held in higher regard. You get listened to by earning people's respect, not by signing some piece of paper. If you feel like people are not taking you seriously enough it's because you have not earned their respect enough. > I started > to ask questions about licensing stuff, and received a bunch of > negative replies like "why don't you sign the agreement first?", > "we've consulted with lawyer - believe us", "if you are not a lawyer - > why are you asking?". I do not know how many of these replies were > from members of PSF, but they surely were not happy to clarify > anything. Because people have better things to do than to try to clarify legal details when they are not lawyers themselves. Deferring to experts is what people do to distribute work, and these people giving you these replies were deferring. > I stopped sending patches (because I didn't sign this > agreement) and was accused in unethical behavior like doing myself > nothing and just directing people what to do. To prevent further > allegations I hereby claim that PSF is fully responsible in the > outcome of current situation and in the people pissed off from Python > Community as a result! > Just kidding - a little but tired already. What?!? If this was meant as a joke it is not funny in the slightest. > -- personal intermezzo -- > > > I assume that pydotorg contributions, Python contributions and Jython > are the same contributions, so there is only one Contributors > Agreement, but there is no proof. > > PSF doesn't comply with the terms of agreement, because it requires > contributors to submit code under either Apache 2.0 or AFL license. > Both these licenses require to include text of license in > distributions, but so far no Apache or AFL license text is present in > Python source code tree. It's not needed because we relicense the code under out license immediately, which is allowed by the AFL. > > PSF also fails to explain they need to receive contributions using two > very verbose licenses, which are almost identical. AFL in fact was > found "redundant" in favor of Apache 2.0 in 2006 by OSI's License > Proliferation Committee. PSF can just say: > > """Contributor understands and agrees that PSF shall have the irrevocable and > perpetual right to make and distribute copies of any Contribution, as well > as to create and distribute collective works and derivative works of any > Contribution.""" > > That is enough. Says you, which is not enough to cause me to want to change how our expertly designed agreement works. > > Why playing strange Apache 2.0 and AFL games if it is clearly written : > """ Contributor understands and agrees that PSF shall have ... the > right ... to create ... derivative works ... under the Initial License > or under any other open source license approved by a unanimous vote of > the PSF board """ > > " Contributor understands and agrees ... " - I do not understand, so I > can't sign. Even if I sign CA there is a chance that I (or whoever has > more money) can prove that terms impossible to understand. In this > case all contributors agreements will be deemed invalid. So Python > code is in danger and hence my derived work is in danger too. > > That's why I asked who supplied PSF lawyer to create such agreement. > The argumentation that this agreement is legitimate like "Believe us - > we've payed to lawyer" is surely not the thing I expected to hear. If > you want international recognition of legitimacy of such agreements. - > you need the reputation of some legal authority like Ernst&Young or > Deloitte or one of big PSF supporters. In this case PSF will be > eligible for trust instead of non-confidence notes. Non-confidence notes? You are the only person who has no faith in our lawyer, Van Lindberg, who is a recognized and respected intellectual property lawyer. > > > There are other reasons that make me hesitate about signing PSF > agreements. For instance PSF doesn't provide any guarantees that: > 1. The text of Contributors Agreement won't be changed after I sign it Of course we don't. We relicense the code for our use. You keep the copyright, we take the license. If the license changes, it changes. If you don't like it then don't contribute. > 2. My personal details are properly guarded and won't be shared with > any members of PSF committee or partner companies or any third parties > without my explicit permission Seriously? You are worried we are going to spam you or something? If you can't trust us then don't sign it. > > That's all. You may want to reread the first paragraph of the letter > before reply. > > > P.S. Sorry for not replying to recent threads (i.e. Mercurial) - > writing a letter like these takes from 4 to 12 hours and is very > exhausting. And you should not have written it. Basically at this point, knowing how you feel, I will never touch any of your code knowing you dislike the idea of it becoming licensed under the PSF license. I have tried to give you the benefit of the doubt, Anatoly, and have tried to overlook your general attitude of being somewhat pushy, but this has pushed me over the edge. If you had some questions about the license, you should have asked the PSF or here on python-dev instead of saying that the PSF is not doing a good job. From where I come from you first try to calmly talk to the people who are doing something that you have questions about before calling for their overthrowing. Considering essentially all of the core developers are PSF members you have just insulted the entire development team by saying they are doing a poor job in shepherding the project they work on in their spare time . Good job. And by the way, some of the core developers also used to be (that's me) or currently are on the board of directors. Well done indeed. You have just landed on my python-dev shit list, doubling its size to two. I hope that some day you can learn how to communicate online more effectively with a group of people, but for now I am simply deleting all of your emails. From tseaver at palladion.com Mon Jul 5 22:12:34 2010 From: tseaver at palladion.com (Tres Seaver) Date: Mon, 05 Jul 2010 16:12:34 -0400 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <4C2FBF0E.6020302@v.loewis.de> References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> <4C2FBF0E.6020302@v.loewis.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin v. L?wis wrote: >>>> I'd love to see a more detailed description of this, including why >>>> someone new to Mercurial would choose one over the other. >>> I think someone new to Mercurial shouldn't choose either one. >>> Just sit back and wait for the real migration to happen. >> I would say that using the SVN mirror is a fine way to experiment with >> using hg against the Python sources to develop and test patches. > > I think your description already falls into the "advanced user" > category. The new-to-mercurial committer should (IMO) use a "what if it > still was svn" workflow, which uses hg pull/up/commit/push. That can > only work if pushing really has the desired effect, which isn't the case > for any of the installations that we operate. I am not a committer myself, and not really familiar with Mercurial (I know CVS/SVN best, and bzr better). I found the current hg mirror of svn quite useful as a way to work on bugs, review patches, etc. without needing any commit access. > When the test-for-two-weeks installation becomes available, the > new-to-hg users will have a chance to learn how to use it without fear > of breaking anything. Until then, they should just remain patient. Experimenting with the mirror *today* without trying to push changes back would give those users a chance to do "familiarization" drills with the majority of mercurial's features, with the exception of the final push. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkwyPKwACgkQ+gerLs4ltQ6zxQCcCRFduUc97jH3g28m/xh3+fTC RtIAniyqilkaNFHS54bW+oF4YXv/cq4l =pdkc -----END PGP SIGNATURE----- From martin at v.loewis.de Mon Jul 5 22:21:58 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 05 Jul 2010 22:21:58 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> <4C2FBF0E.6020302@v.loewis.de> Message-ID: <4C323EE6.6000408@v.loewis.de> > Experimenting with the mirror *today* without trying to push changes > back would give those users a chance to do "familiarization" drills with > the majority of mercurial's features, with the exception of the final push. That's true. However, for those users, I'd rather recommend to use hg on an entirely independent project. That worked for me. Regards, Martin From g.brandl at gmx.net Mon Jul 5 23:00:39 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 05 Jul 2010 23:00:39 +0200 Subject: [Python-Dev] FWD: error: 3.2 release schedule has release in Jan 2010. In-Reply-To: <20100705162855.GA22056@panix.com> References: <20100705162855.GA22056@panix.com> Message-ID: That's now fixed. The time machine disagreed with me. Georg Am 05.07.2010 18:28, schrieb Aahz: > ----- Forwarded message from Ric Johnson ----- > >> Date: Mon, 5 Jul 2010 08:49:29 -0700 (PDT) >> From: Ric Johnson >> Subject: error: 3.2 release schedule has release in Jan 2010. >> To: webmaster at python.org >> >> Hey! >> There's a typo on http://www.python.org/dev/peps/pep-0392/: >> >> Release Schedule >> >> The current schedule is: >> >> ... >> >> # 3.2 candidate 2: January 1, 2010 ==> should be 2011 >> # 3.2 final: January 15, 2010 ==> should be 2011 >> >> >> I know it is rough schedule but it looks bad ... >> >> Ric >> >> -Verba nihil operandi melius est quam absurde. >> >> >> > > ----- End forwarded message ----- > From ncoghlan at gmail.com Mon Jul 5 23:05:58 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 6 Jul 2010 07:05:58 +1000 Subject: [Python-Dev] Licensing // PSF // Motion of non-confidence In-Reply-To: References: Message-ID: On Tue, Jul 6, 2010 at 6:04 AM, Brett Cannon wrote: > I have tried to give you the benefit of the doubt, Anatoly, and have > tried to overlook your general attitude of being somewhat pushy, but > this has pushed me over the edge. If you had some questions about the > license, you should have asked the PSF or here on python-dev instead > of saying that the PSF is not doing a good job. From where I come from > you first try to calmly talk to the people who are doing something > that you have questions about before calling for their overthrowing. > > Considering essentially all of the core developers are PSF members you > have just insulted the entire development team by saying they are > doing a poor job in shepherding the project they work on in their > spare time . Good job. And by the way, some of the core developers > also used to be (that's me) or currently are on the board of > directors. Well done indeed. Thanks for taking the time to respond Brett - saves me from saying basically the same thing. Anatoly - try to give other people a little credit for not being complete idiots, OK? A lot of stuff in the world doesn't make much sense from a green field point of view, but is comprehensible once the long and involved history is taken into account. Jumping to the conclusion that people are incompetent idiots just encourages them to ignore your input (since you're clearly not listening to anyone else). As Brett noted, yes, the LICENSE file is complicated, but most people don't bother reading it themselves - they ask what FSF and OSI think of it, and get the answers "BSD style" and "GPL compatible" and are happy with that. The corporate history is such that the PSF probably doesn't have the legal rights to simplify it (we might have some scope to tidy it up, such as splitting it into two files as you suggest, but we would have to spend money to find out for sure, and don't consider that a particularly good use of contributor's funds). As far as the issue of not including the contributor agreement related licenses in the source goes, note that the contributor agreement explicitly provides the PSF with relicensing rights. If you want to dispute the legal effectiveness of that then you'll want to A) be a lawyer or at least consult one and B) take it up with Van Lindbergh, the PSF's lawyer. Normally we don't require contributor agreements for minor patches and other submissions, but given the attitude you have displayed here, I expect we'll make an exception for you (i.e. until you provide evidence of a change of heart by signing a contributor agreement, I'd consider any patches you provide to be on sufficiently legally dubious ground that we aren't in a position to accept them). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Mon Jul 5 23:37:26 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 6 Jul 2010 07:37:26 +1000 Subject: [Python-Dev] Licensing // PSF // Motion of non-confidence In-Reply-To: References: Message-ID: On Tue, Jul 6, 2010 at 7:05 AM, Nick Coghlan wrote: > Normally we don't require contributor agreements for > minor patches and other submissions, but given the attitude you have > displayed here, I expect we'll make an exception for you (i.e. until > you provide evidence of a change of heart by signing a contributor > agreement, I'd consider any patches you provide to be on sufficiently > legally dubious ground that we aren't in a position to accept them). Sorry, that crack was uncalled for. You have made positive contributions to the language in the past, and hopefully will be willing and able to do so in the future. Just try to lose the attitude that everyone else is dumber than dirt. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From hodgestar+pythondev at gmail.com Tue Jul 6 00:21:23 2010 From: hodgestar+pythondev at gmail.com (Simon Cross) Date: Tue, 6 Jul 2010 00:21:23 +0200 Subject: [Python-Dev] Licensing // PSF // Motion of non-confidence In-Reply-To: References: Message-ID: On Mon, Jul 5, 2010 at 11:05 PM, Nick Coghlan wrote: > As Brett noted, yes, the LICENSE file is complicated, but most people > don't bother reading it themselves - they ask what FSF and OSI think > of it, and get the answers "BSD style" and "GPL compatible" and are > happy with that. Presumably at least the BEOPEN.COM LICENSE AGREEMENT FOR PYTHON 2.0 can be removed from the LICENSE file because otherwise it appears that answer to "GPL compatible" must be no? Schiavo Simon From solipsis at pitrou.net Tue Jul 6 02:13:35 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 6 Jul 2010 02:13:35 +0200 Subject: [Python-Dev] Licensing // PSF // Motion of non-confidence References: Message-ID: <20100706021335.5c304fab@pitrou.net> On Tue, 6 Jul 2010 07:05:58 +1000 Nick Coghlan wrote: > > As Brett noted, yes, the LICENSE file is complicated, but most people > don't bother reading it themselves - they ask what FSF and OSI think > of it, and get the answers "BSD style" and "GPL compatible" and are > happy with that. Which is the very wrong thing to do, though. License text should be understandable by non-lawyer people; IIUC OpenBSD has a very sane policy in this respect. > The corporate history is such that the PSF probably > doesn't have the legal rights to simplify it Yes, that's probably the real problem and why it's not of much use to argue about it. You have to trust the PSF (and/or python-dev), because the license text may not be able to change very soon (if any day at all). Regards Antoine. From nyamatongwe at gmail.com Tue Jul 6 02:50:21 2010 From: nyamatongwe at gmail.com (Neil Hodgson) Date: Tue, 6 Jul 2010 10:50:21 +1000 Subject: [Python-Dev] Licensing // PSF // Motion of non-confidence In-Reply-To: References: Message-ID: anatoly techtonik: > The file consists of several licenses for multiple versions of Python. > It is an unusual mix that negatively affects understanding. A simpler license would be better. There have been moves in the past to simplify the license of Python but this would require agreement from the current rights owners including CWI and CNRI. IIRC not all of the rights owners are willing to agree to a change. Neil From steve at holdenweb.com Tue Jul 6 03:03:36 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 05 Jul 2010 21:03:36 -0400 Subject: [Python-Dev] Licensing // PSF // Motion of non-confidence In-Reply-To: References: Message-ID: Neil Hodgson wrote: > anatoly techtonik: > >> The file consists of several licenses for multiple versions of Python. >> It is an unusual mix that negatively affects understanding. > > A simpler license would be better. > > There have been moves in the past to simplify the license of Python > but this would require agreement from the current rights owners > including CWI and CNRI. IIRC not all of the rights owners are willing > to agree to a change. > That is the current position. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From stephen at xemacs.org Tue Jul 6 05:12:18 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 06 Jul 2010 12:12:18 +0900 Subject: [Python-Dev] SVN <-> HG workflow to split Python Library by Module In-Reply-To: References: <87vd8wx2ek.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87zky59uct.fsf@uwakimon.sk.tsukuba.ac.jp> Jesse Noller writes: > On Sat, Jul 3, 2010 at 7:05 AM, Dirkjan Ochtman wrote: > > On Sat, Jul 3, 2010 at 12:53, Stephen J. Turnbull wrote: > >> The point of submodules a la git is subtly different. ?It is that you > >> can mix and match *known versions* of the modules. ?So, eg, in order > >> to work on recent urllib, maybe you need a recent *but stable* email > >> but you don't want any of the work being done on removing the GIL > >> because it's somewhat unstable. > > > > This sounds like it could also be done with good branching + merging. > > > > Cheers, > > > > Dirkjan > > What Dirkjan said; since using mercurial more and more lately, local > branches are an insanely good tool and method of coordinating work I have nothing against local branches as a tool and method of coordinating work. But what Dirkjan said is insufficiently precise. It's not about the branching and merging operations (submodules are, after all, just a way of specifying certain branch/merge patterns). It's about knowing certain things about what's in the branch, and how the content is referenced (ie, with submodule-local tags, rather than project-global tags). It's about modularity over history, as well as in the static, instantaneous structure of the application. Since history needs to be modular, too, it might not work as well as I made it sound above, I admit. But there are times when such a capability is quite useful, and in the stdlib it seems likely that most of the modules will have sufficiently independent histories. Consider the problem going *backwards*. You have a current Python 3 in your workspace, and you suddenly realize that if you rewind the GIL (core) work to about March a nagging intermittent bug will go away, but email needs to be at least May 1 but no later than June 15. How do you spell that without submodules, or a preexisting branch that happens to have exactly the right stuff in it? I know it can be done, but it requires some thought and quite a bit of dexterity with branching, checking out the right versions of the right files and overlaying them on the branch you are creating without pulling in unwanted versions of other files. Now, how do you keep that branch updated with bugfixes and stuff that doesn't directly affect your current work? You'll have to keep rewinding the GIL- related and email-related files by hand. Submodules make that easy. From stephen at xemacs.org Tue Jul 6 05:38:36 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 06 Jul 2010 12:38:36 +0900 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> <4C2E7CE6.9060501@v.loewis.de> <4C2FC0EE.1060702@v.loewis.de> Message-ID: <87y6dp9t4z.fsf@uwakimon.sk.tsukuba.ac.jp> Georg Brandl writes: > I wouldn't say that. strip works well and it does so logically, > once one understands the DAG. The only thing discouraged is to strip > changesets once pushed to the public repo, but that holds as well for > getting rid of the changesets by making a new clone without them. Actually, neither method of "rewinding" a local repo is especially discouraged, if you know what you're doing. What you mustn't do is strip the changes in the public repo itself (or turn them into unreferencable garbage). Back in the bad old days of git, "push" often had such effects, but with modern versions of any of the dVCSes, public repos are allowed to refuse pushes that would garbage any changesets, and even if they're permitted, you need to use a --force option to push. There's nothing wrong with stripping your local repo(s), even if the stripped changes have already been pushed to the public repo. I would say it's unusual for most people to want to strip publicly available changesets, but after all, you just end up with an out-of-date repo, which is exactly what Mercurial is all about handling. Ie, it is *normal* to have an out-of-date repo locally when there are other active committers; it's just unusual to intentionally strip. But the consequences are the same. As with any out-of-date repo, you will need to merge before pushing. New users probably shouldn't do it to excess, because of the possibility of exposing yourself to a difficult merge. But the flip side is that new users probably want to update from the public repo before doing significant work for exactly the same reasons. From stephen at xemacs.org Tue Jul 6 05:58:26 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 06 Jul 2010 12:58:26 +0900 Subject: [Python-Dev] Licensing // PSF // Motion of non-confidence In-Reply-To: <20100706021335.5c304fab@pitrou.net> References: <20100706021335.5c304fab@pitrou.net> Message-ID: <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> Antoine Pitrou writes: > Which is the very wrong thing to do, though. License text should be > understandable by non-lawyer people; This is a common mistake, at least with respect to common-law systems. Licenses are written in a formal language intended to have precise semantics, especially in the event of a dispute going to court. What you wrote is precisely analogous to "a computer program should be understandable to non-programmer people". The fact is, in the U.S. if an ordinary person thinks they understand a license, then it's probably quite unpredictable what a court will say about attempts to enforce it. WTF-an-economist-defending-lawyers??!?!!-ly y'rs, From solipsis at pitrou.net Tue Jul 6 06:47:58 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 06 Jul 2010 06:47:58 +0200 Subject: [Python-Dev] Licensing // PSF // Motion of non-confidence In-Reply-To: <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20100706021335.5c304fab@pitrou.net> <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <1278391678.3159.12.camel@localhost.localdomain> Le mardi 06 juillet 2010 ? 12:58 +0900, Stephen J. Turnbull a ?crit : > Antoine Pitrou writes: > > > Which is the very wrong thing to do, though. License text should be > > understandable by non-lawyer people; > > This is a common mistake, at least with respect to common-law systems. > Licenses are written in a formal language intended to have precise > semantics, especially in the event of a dispute going to court. What > you wrote is precisely analogous to "a computer program should be > understandable to non-programmer people". The point of free software licenses, though (as opposed to proprietary licenses), is not mainly to go to court (to ?protect IP?, as the PSF says - quite naively in my opinion); it is to enable trust among people. Hence the requirement for being readable and understandable by the very people whom they help work together. (and besides, of course, a lawyer's opinion can never make you sure of anything wrt. court testing; lawyers very frequently disagree between themselves, and they are very careful to never provide any formal guarantee; for example, several French ?IP? lawyers have argued that free licenses have no value in French authorship right; that hasn't prevented companies from making business with the GPL and other free licenses here) From guido at python.org Tue Jul 6 07:28:28 2010 From: guido at python.org (Guido van Rossum) Date: Tue, 6 Jul 2010 07:28:28 +0200 Subject: [Python-Dev] Licensing // PSF // Motion of non-confidence In-Reply-To: <1278391678.3159.12.camel@localhost.localdomain> References: <20100706021335.5c304fab@pitrou.net> <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> <1278391678.3159.12.camel@localhost.localdomain> Message-ID: On Tue, Jul 6, 2010 at 6:47 AM, Antoine Pitrou wrote: > Le mardi 06 juillet 2010 ? 12:58 +0900, Stephen J. Turnbull a ?crit : >> Antoine Pitrou writes: >> >> ?> Which is the very wrong thing to do, though. License text should be >> ?> understandable by non-lawyer people; >> >> This is a common mistake, at least with respect to common-law systems. >> Licenses are written in a formal language intended to have precise >> semantics, especially in the event of a dispute going to court. ?What >> you wrote is precisely analogous to "a computer program should be >> understandable to non-programmer people". > > The point of free software licenses, though (as opposed to proprietary > licenses), is not mainly to go to court (to ?protect IP?, as the PSF > says - quite naively in my opinion); it is to enable trust among people. > Hence the requirement for being readable and understandable by the very > people whom they help work together. Really? That's the *last* thing where I would be looking for trust. among developers. Open source licenses do take part into a trust relationship -- however it is trust between corporate entities with too many lawyers who wouldn't trust the open source development communities otherwise. Their worries are mainly about people contributing tainted material to open source projects which then later can be used to place pressure on deep pockets (see the SCO UNIX suit for example). A secondary reasoning for some open source licenses might be to prevent others from running off with the good stuff and selling it for profit. The GPL is big on that, but it's never motivated me with Python (hence the tenuous relationship at best with the FSF and GPL software). > (and besides, of course, a lawyer's opinion can never make you sure of > anything wrt. court testing; lawyers very frequently disagree between > themselves, and they are very careful to never provide any formal > guarantee; for example, several French ?IP? lawyers have argued that > free licenses have no value in French authorship right; that hasn't > prevented companies from making business with the GPL and other free > licenses here) There are two systems. The courts and the lawyers. The lawyers are the most important -- they control the backroom deals and trust. The courts are mostly there as a back-up threat since even winning a lawsuit is incredibly expensive compared to settling. -- --Guido van Rossum (python.org/~guido) From ben+python at benfinney.id.au Tue Jul 6 08:44:05 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 06 Jul 2010 16:44:05 +1000 Subject: [Python-Dev] Licensing References: <20100706021335.5c304fab@pitrou.net> <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> <1278391678.3159.12.camel@localhost.localdomain> Message-ID: <874ogdds96.fsf_-_@benfinney.id.au> Guido van Rossum writes: > A secondary reasoning for some open source licenses might be to > prevent others from running off with the good stuff and selling it for > profit. The GPL is big on that [?] Really, it's not. Please stop spreading this canard. The GPL explicitly and deliberately grants the freedom to sell the work for profit. Every copyright holder who grants license under the terms of the GPL is explicitly saying ?you can seel this software for any price you like? . Whatever other complaints people may have against the GPL, it's simply *false* to claim what Guido did above. Please stop it. -- \ ?We cannot solve our problems with the same thinking we used | `\ when we created them.? ?Albert Einstein | _o__) | Ben Finney From nir at winpdb.org Tue Jul 6 09:10:09 2010 From: nir at winpdb.org (Nir Aides) Date: Tue, 6 Jul 2010 10:10:09 +0300 Subject: [Python-Dev] Licensing In-Reply-To: <874ogdds96.fsf_-_@benfinney.id.au> References: <20100706021335.5c304fab@pitrou.net> <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> <1278391678.3159.12.camel@localhost.localdomain> <874ogdds96.fsf_-_@benfinney.id.au> Message-ID: I take "...running off with the good stuff and selling it for profit" to mean "creating derivative work and commercializing it as proprietary code" which you can not do with GPL licensed code. Also, while the GPL does not prevent selling copies for profit it does not make it very practical either. On Tue, Jul 6, 2010 at 9:44 AM, Ben Finney > wrote: > Guido van Rossum writes: > > > A secondary reasoning for some open source licenses might be to > > prevent others from running off with the good stuff and selling it for > > profit. The GPL is big on that [?] > > Really, it's not. Please stop spreading this canard. > > The GPL explicitly and deliberately grants the freedom to sell the work > for profit. Every copyright holder who grants license under the terms of > the GPL is explicitly saying ?you can seel this software for any price > you like? . > > Whatever other complaints people may have against the GPL, it's simply > *false* to claim what Guido did above. Please stop it. > > -- > \ ?We cannot solve our problems with the same thinking we used | > `\ when we created them.? ?Albert Einstein | > _o__) | > Ben Finney > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/nir%40winpdb.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Tue Jul 6 09:22:22 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 06 Jul 2010 17:22:22 +1000 Subject: [Python-Dev] Licensing References: <20100706021335.5c304fab@pitrou.net> <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> <1278391678.3159.12.camel@localhost.localdomain> <874ogdds96.fsf_-_@benfinney.id.au> Message-ID: <87vd8tcbwx.fsf@benfinney.id.au> Nir Aides writes: > I take "...running off with the good stuff and selling it for profit" to > mean "creating derivative work and commercializing it as proprietary code" > which you can not do with GPL licensed code. It's the ?proprietary? which is the distinguishing criterion there. The ?selling? and ?commercial? is totally orthogonal to that. That's the point: selling, and commercial activity in general, is explicitly encouraged and permission granted by the GPL. Too many people speak as though it were otherwise. To those who do: Please stop. -- \ ?Following fashion and the status quo is easy. Thinking about | `\ your users' lives and creating something practical is much | _o__) harder.? ?Ryan Singer, 2008-07-09 | Ben Finney From a.badger at gmail.com Tue Jul 6 09:31:27 2010 From: a.badger at gmail.com (Toshio Kuratomi) Date: Tue, 6 Jul 2010 03:31:27 -0400 Subject: [Python-Dev] Licensing In-Reply-To: References: <20100706021335.5c304fab@pitrou.net> <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> <1278391678.3159.12.camel@localhost.localdomain> <874ogdds96.fsf_-_@benfinney.id.au> Message-ID: <20100706073127.GF16407@unaka.lan> On Tue, Jul 06, 2010 at 10:10:09AM +0300, Nir Aides wrote: > I take "...running off with the good stuff and selling it for profit" to mean > "creating derivative work and commercializing it as proprietary code" which you > can not do with GPL licensed code. Also, while the GPL does not prevent selling > copies for profit it does not make it very practical either. > Uhmmm.... http://finance.yahoo.com/q/is?s=RHT&annual It is very possible to make money with the GPL. The GPL does, as you say, prevents you from creating derivative works that are proprietary code. It does *not* prevent you from creating derivative works and commercializing it. -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From hsoft at hardcoded.net Tue Jul 6 12:01:08 2010 From: hsoft at hardcoded.net (Virgil Dupras) Date: Tue, 6 Jul 2010 12:01:08 +0200 Subject: [Python-Dev] Licensing In-Reply-To: <87vd8tcbwx.fsf@benfinney.id.au> References: <20100706021335.5c304fab@pitrou.net> <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> <1278391678.3159.12.camel@localhost.localdomain> <874ogdds96.fsf_-_@benfinney.id.au> <87vd8tcbwx.fsf@benfinney.id.au> Message-ID: On Tue, Jul 6, 2010 at 9:22 AM, Ben Finney wrote: > That's the point: selling, and commercial activity in general, is > explicitly encouraged and permission granted by the GPL. Too many people > speak as though it were otherwise. To those who do: Please stop. > Please, GPL advocates also spread their own type of FUD, claiming "free as in speech ain't the same thing as free as in beer, people!". While true, the bottom line is that Python being BSD-type enables me to make money with it that I wouldn't make if Python was GPL-type. Moreover, I don't think that GPL license allows money-making that BSD type wouldn't allow. Hence the common point of view saying "BSD-type is more commercial-friendly than GPL". I've written an article last year that, while it doesn't address this issue specifically, it touches it. http://www.hardcoded.net/articles/going_open_source.htm Virgil Dupras From jnoller at gmail.com Tue Jul 6 14:08:37 2010 From: jnoller at gmail.com (Jesse Noller) Date: Tue, 6 Jul 2010 08:08:37 -0400 Subject: [Python-Dev] Licensing In-Reply-To: References: <20100706021335.5c304fab@pitrou.net> <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> <1278391678.3159.12.camel@localhost.localdomain> <874ogdds96.fsf_-_@benfinney.id.au> <87vd8tcbwx.fsf@benfinney.id.au> Message-ID: On Tue, Jul 6, 2010 at 6:01 AM, Virgil Dupras wrote: > On Tue, Jul 6, 2010 at 9:22 AM, Ben Finney wrote: > >> That's the point: selling, and commercial activity in general, is >> explicitly encouraged and permission granted by the GPL. Too many people >> speak as though it were otherwise. To those who do: Please stop. >> > > Please, GPL advocates also spread their own type of FUD, claiming > "free as in speech ain't the same thing as free as in beer, people!". > While true, the bottom line is that Python being BSD-type enables me > to make money with it that I wouldn't make if Python was GPL-type. > Moreover, I don't think that GPL license allows money-making that BSD > type wouldn't allow. Hence the common point of view saying "BSD-type > is more commercial-friendly than GPL". > > I've written an article last year that, while it doesn't address this > issue specifically, it touches it. > > http://www.hardcoded.net/articles/going_open_source.htm > Can we please drop the GPL slap fighting? It's completely worthless here. Take it to reddit or someplace else. The Python / PSF license won't be changing anytime soon. Ben could have just have easily responded to Guido in private if he felt that strongly. jesse From steve at pearwood.info Tue Jul 6 14:09:18 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 6 Jul 2010 22:09:18 +1000 Subject: [Python-Dev] Licensing // PSF // Motion of non-confidence In-Reply-To: <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20100706021335.5c304fab@pitrou.net> <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <201007062209.18668.steve@pearwood.info> On Tue, 6 Jul 2010 01:58:26 pm Stephen J. Turnbull wrote: > Antoine Pitrou writes: > > Which is the very wrong thing to do, though. License text should > > be understandable by non-lawyer people; > > This is a common mistake, at least with respect to common-law > systems. Licenses are written in a formal language intended to have > precise semantics, especially in the event of a dispute going to > court. What you wrote is precisely analogous to "a computer program > should be understandable to non-programmer people". You've never used Apple's much-missed Hypertalk, have you? :) Given that Python has often been described as executable pseudo-code, I think it is ironic that you're implying that comprehensibility of language is a bad thing! Python is no less precise in its semantics than (say) APL. There are movements to discourage unreadable legalise in favour of simpler language that is more readable while still being precise. For example, the Canadian Bar Association supports the Plain English Movement: http://en.wikipedia.org/wiki/Plain_Language_Movement and of course excessive formality and legalise is often criticised even by lawyers for *harming* precision. (When even the judge can't work out what you mean, that's a problem.) None of this is to imply that the Python licence is guilty of such excessive legalise. But I think that, to the extent that other priorities and legal obligations permit it, we should always be be open to the idea of improving the readability and comprehensibility of "legal source code". > The fact is, in the U.S. if an ordinary person thinks they understand > a license, then it's probably quite unpredictable what a court will > say about attempts to enforce it. I'm not sure that this is a fact or just an opinion, but *my* opinion is that this is a safe bet. Most people in the industry consider that it's generally unpredictable what a court will say about licences in general (particularly the shrink-wrap variety). It's certainly true that the general public generally has no clue about licences, contracts, or legal agreements in general, but then agreements written by lawyers aren't always much better. I've been asked to sign agreements that are nonsensical, e.g. circular definitions where Clause N says to refer to Clause X, and Clause X says to refer to Clause N, or NDAs that prohibited me from doing *anything* with the "confidential information" the other party gave me, including the work they wanted me to do. Or blatantly illegal, e.g. non-compete clauses that don't have a hope in hell of surviving a legal challenge, including one that would have meant that I was agreeing to never work for any person or company in Australia who ever had with a telephone. -- Steven D'Aprano From walter at livinglogic.de Tue Jul 6 14:10:14 2010 From: walter at livinglogic.de (=?UTF-8?B?V2FsdGVyIETDtnJ3YWxk?=) Date: Tue, 06 Jul 2010 14:10:14 +0200 Subject: [Python-Dev] blocking 2.7 In-Reply-To: References: <201007031426.53880.victor.stinner@haypocalc.com> <4C302ADD.30608@netwok.org> Message-ID: <4C331D26.1040501@livinglogic.de> On 05.07.10 16:19, Nick Coghlan wrote: > On Mon, Jul 5, 2010 at 5:20 AM, Terry Reedy wrote: >> On 7/4/2010 2:31 AM, ?ric Araujo wrote: >>>> >>>> But Python tests lack coverage stats, so it is hard to say anything. >>> >>> FYI: http://coverage.livinglogic.de/ >> >> Turns out the audioop is one of the best covered modules, at 98% > > Alas, those are only the stats for the audioop test suite. audioop > itself is written in C, so the automatic coverage stats generated by > livinglogic don't provide any details. http://coverage.livinglogic.de/ *does* include coverage info for stuff written in C, see for example: http://coverage.livinglogic.de/Objects/unicodeobject.c.html However it *is* strange that test_audioop.py gets executed, but audioop.c doesn't seem to be. Servus, Walter From ben+python at benfinney.id.au Tue Jul 6 14:53:28 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 06 Jul 2010 22:53:28 +1000 Subject: [Python-Dev] Licensing References: <20100706021335.5c304fab@pitrou.net> <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> <1278391678.3159.12.camel@localhost.localdomain> <874ogdds96.fsf_-_@benfinney.id.au> <87vd8tcbwx.fsf@benfinney.id.au> Message-ID: <87ocekdb5j.fsf@benfinney.id.au> Jesse Noller writes: > The Python / PSF license won't be changing anytime soon. The existing license for Python suits me fine. > Ben could have just have easily responded to Guido in private if he > felt that strongly. No. I responded in the same forum where the falsehood was put forth, to correct that falsehood. That's done now; thanks for your attention, all. -- \ ?Timid men prefer the calm of despotism to the boisterous sea | `\ of liberty.? ?Thomas Jefferson | _o__) | Ben Finney From dickinsm at gmail.com Tue Jul 6 15:07:14 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 6 Jul 2010 14:07:14 +0100 Subject: [Python-Dev] blocking 2.7 In-Reply-To: <4C331D26.1040501@livinglogic.de> References: <201007031426.53880.victor.stinner@haypocalc.com> <4C302ADD.30608@netwok.org> <4C331D26.1040501@livinglogic.de> Message-ID: On Tue, Jul 6, 2010 at 1:10 PM, Walter D?rwald wrote: > http://coverage.livinglogic.de/ *does* include coverage info for stuff > written in C, see for example: > > ? http://coverage.livinglogic.de/Objects/unicodeobject.c.html > > However it *is* strange that test_audioop.py gets executed, but > audioop.c doesn't seem to be. It looks as though none of the extension modules (besides those that are compiled statically into the interpreter) are reporting coverage. I wonder whether the correct flags are being passed to the module build stage? Incidentally, there doesn't seem to be any of the usual 'make' output I'd associate with the module-building stage in the build log at: http://coverage.livinglogic.de/buildlog.txt For example, I'd expect to see the string 'mathmodule' somewhere in that output. Mark From stephen at xemacs.org Tue Jul 6 15:35:20 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 06 Jul 2010 22:35:20 +0900 Subject: [Python-Dev] Licensing // PSF // Motion of non-confidence In-Reply-To: <201007062209.18668.steve@pearwood.info> References: <20100706021335.5c304fab@pitrou.net> <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> <201007062209.18668.steve@pearwood.info> Message-ID: <87bpakag2v.fsf@uwakimon.sk.tsukuba.ac.jp> Steven D'Aprano writes: > On Tue, 6 Jul 2010 01:58:26 pm Stephen J. Turnbull wrote: > > Licenses are written in a formal language intended to have > > precise semantics, especially in the event of a dispute going to > > court. What you wrote is precisely analogous to "a computer program > > should be understandable to non-programmer people". > > You've never used Apple's much-missed Hypertalk, have you? :) No. I was solving quadratic programs back then, and FORTRAN was much better for that. But I think it's more relevant that my mother tried writing HyperCard stacks, and gave up. On the rare occasions she wanted her computer to do something she couldn't do with MacPaint or MacWrite, she called me. She never complained about me writing programs in BASIC, even though they were totally incomprehensible to her.... And mentioning the "Python as executable pseudo-code" thing, I think you're way overestimating what average non-programmer people can cope with. (I'd be pleased to be proved wrong, especially by the undergrads I teach!!!) As for missing it, why would I when I've got Python? From barry at python.org Tue Jul 6 16:18:40 2010 From: barry at python.org (Barry Warsaw) Date: Tue, 6 Jul 2010 10:18:40 -0400 Subject: [Python-Dev] Signs of neglect? In-Reply-To: <4C30BDCD.8010201@netwok.org> References: <4C25E20D.2040007@holdenweb.com> <4C30BDCD.8010201@netwok.org> Message-ID: <20100706101840.53532bb5@heresy> On Jul 04, 2010, at 06:58 PM, ?ric Araujo wrote: >I?d like to volunteer to maintain a tool but I?m not sure where I can >help. I?m already proposing changes to Brett for >Tools/scripts/patchcheck.py, and I have commented on Tools/i18n bugs, >but these ones are already maintained by their authors (e.g. Barry is >assigned pygettext bugs) and I?m by no means a gettext expert. It's been a while since I did much pygettext stuff. I think Martin's basically taken it over in recent years. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Tue Jul 6 16:21:33 2010 From: barry at python.org (Barry Warsaw) Date: Tue, 6 Jul 2010 10:21:33 -0400 Subject: [Python-Dev] [RELEASE] Python 2.7 released In-Reply-To: References: Message-ID: <20100706102133.7edea2f9@heresy> On Jul 04, 2010, at 11:03 AM, Benjamin Peterson wrote: >2010/7/4 Benjamin Peterson : >> On behalf of the Python development team, I'm jocund to announce the >> second release candidate of Python 2.7. > >Arg!!! This should, of course, be "final release". Congratulations Benjamin! -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From walter at livinglogic.de Tue Jul 6 16:24:50 2010 From: walter at livinglogic.de (=?UTF-8?B?V2FsdGVyIETDtnJ3YWxk?=) Date: Tue, 06 Jul 2010 16:24:50 +0200 Subject: [Python-Dev] Coverage, was: Re: blocking 2.7 In-Reply-To: References: <201007031426.53880.victor.stinner@haypocalc.com> <4C302ADD.30608@netwok.org> <4C331D26.1040501@livinglogic.de> Message-ID: <4C333CB2.1040107@livinglogic.de> On 06.07.10 15:07, Mark Dickinson wrote: > On Tue, Jul 6, 2010 at 1:10 PM, Walter D?rwald wrote: >> http://coverage.livinglogic.de/ *does* include coverage info for stuff >> written in C, see for example: >> >> http://coverage.livinglogic.de/Objects/unicodeobject.c.html >> >> However it *is* strange that test_audioop.py gets executed, but >> audioop.c doesn't seem to be. > > It looks as though none of the extension modules (besides those that > are compiled statically into the interpreter) are reporting coverage. > I wonder whether the correct flags are being passed to the module > build stage? Incidentally, there doesn't seem to be any of the usual > 'make' output I'd associate with the module-building stage in the > build log at: > > http://coverage.livinglogic.de/buildlog.txt > > For example, I'd expect to see the string 'mathmodule' somewhere in that output. True, there seems to be a problem. I'm running ./configure --enable-unicode=ucs4 --with-pydebug and then make coverage This doesn't seem to build extension modules. However as far as I understand the Makefile, "make coverage" should build extension modules: # Default target all: build_all build_all: $(BUILDPYTHON) oldsharedmods sharedmods gdbhooks coverage: @echo "Building with support for coverage checking:" $(MAKE) clean $(MAKE) all CFLAGS="$(CFLAGS) -O0 -pg -fprofile-arcs -ftest-coverage" LIBS="$(LIBS) -lgcov" # Build the shared modules sharedmods: $(BUILDPYTHON) @case $$MAKEFLAGS in \ *s*) $(RUNSHARED) CC='$(CC)' LDSHARED='$(BLDSHARED)' LDFLAGS='$(LDFLAGS)' OPT='$(OPT)' ./$(BUILDPYTHON) -E $(srcdir)/setup.py -q build;; \ *) $(RUNSHARED) CC='$(CC)' LDSHARED='$(BLDSHARED)' LDFLAGS='$(LDFLAGS)' OPT='$(OPT)' ./$(BUILDPYTHON) -E $(srcdir)/setup.py build;; \ esac I'm rerunning now with "make && make coverage" to see if this fixes anything. Servus, Walter From ronaldoussoren at mac.com Tue Jul 6 16:51:53 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Tue, 06 Jul 2010 16:51:53 +0200 Subject: [Python-Dev] thoughts on the bytes/string discussion In-Reply-To: <4C271E66.5050902@canterbury.ac.nz> References: <11597.1277401099@parc.com> <96ADD4CE-3A24-45A7-B219-2940195DC3D0@twistedmatrix.com> <4C25B319.8040804@canterbury.ac.nz> <4C271E66.5050902@canterbury.ac.nz> Message-ID: <50FFA855-6FC5-4A01-B5C1-28FB63AC452F@mac.com> On 27 Jun, 2010, at 11:48, Greg Ewing wrote: > Stefan Behnel wrote: >> Greg Ewing, 26.06.2010 09:58: >>> Would there be any sanity in having an option to compile >>> Python with UTF-8 as the internal string representation? >> It would break Py_UNICODE, because the internal size of a unicode character would no longer be fixed. > > It's not fixed anyway with the 2-char build -- some > characters are represented using a pair of surrogates. It is for practical purposes not even fixed in 4-char builds. In 4-char builds every Unicode code points corresponds to one item in a python unicode string, but a base characters with combining characters is still a sequence of characters and should IMHO almost always be treated as a single object. As an example, given s="be\N{COMBINING DIAERESIS}" s[:2] or s[2:] is almost certainly semanticly invalid. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From ldlandis at gmail.com Tue Jul 6 17:17:50 2010 From: ldlandis at gmail.com (LD 'Gus' Landis) Date: Tue, 6 Jul 2010 09:17:50 -0600 Subject: [Python-Dev] Licensing In-Reply-To: <87vd8tcbwx.fsf@benfinney.id.au> References: <20100706021335.5c304fab@pitrou.net> <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> <1278391678.3159.12.camel@localhost.localdomain> <874ogdds96.fsf_-_@benfinney.id.au> <87vd8tcbwx.fsf@benfinney.id.au> Message-ID: Yes. The BSD license on FreeBSD has allowed Apple to make MacOS X a completely proprietary product. The BSD license allows you to take and never release your mods. It has very little to do with money, IMO. On Tue, Jul 6, 2010 at 1:22 AM, Ben Finney wrote: > Nir Aides writes: > >> I take "...running off with the good stuff and selling it for profit" to >> mean "creating derivative work and commercializing it as proprietary code" >> which you can not do with GPL licensed code. > > It's the ?proprietary? which is the distinguishing criterion there. The > ?selling? and ?commercial? is totally orthogonal to that. > > That's the point: selling, and commercial activity in general, is > explicitly encouraged and permission granted by the GPL. Too many people > speak as though it were otherwise. To those who do: Please stop. > > -- > ?\ ? ? ? ?Following fashion and the status quo is easy. Thinking about | > ?`\ ? ? ? ?your users' lives and creating something practical is much | > _o__) ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?harder.? ?Ryan Singer, 2008-07-09 | > Ben Finney > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ldlandis%40gmail.com > -- --- NOTE: If it is important CALL ME - I may miss email, which I do NOT normally check on weekends nor on a regular basis during any other day. --- LD Landis - N0YRQ - de la tierra del encanto 3960 Schooner Loop, Las Cruces, NM 88012 575-448-1763 N32 21'48.28" W106 46'5.80" From van.lindberg at gmail.com Tue Jul 6 17:38:56 2010 From: van.lindberg at gmail.com (VanL) Date: Tue, 06 Jul 2010 10:38:56 -0500 Subject: [Python-Dev] Licensing // PSF // Motion of non-confidence In-Reply-To: <1278391678.3159.12.camel@localhost.localdomain> References: <20100706021335.5c304fab@pitrou.net> <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> <1278391678.3159.12.camel@localhost.localdomain> Message-ID: On 7/5/2010 11:47 PM, Antoine Pitrou wrote: > The point of free software licenses, though (as opposed to proprietary > licenses), is not mainly to go to court (to ?protect IP?, as the PSF > says - quite naively in my opinion); it is to enable trust among people. Yes, that is true. Open source licenses are social documents as much as they are legal documents. However, they need to be legally enforceable so as to have their intended social effect. From van.lindberg at gmail.com Tue Jul 6 17:42:17 2010 From: van.lindberg at gmail.com (VanL) Date: Tue, 06 Jul 2010 10:42:17 -0500 Subject: [Python-Dev] Licensing // PSF // Motion of non-confidence In-Reply-To: References: Message-ID: On 7/5/2010 8:03 PM, Steve Holden wrote: > Neil Hodgson wrote: >> There have been moves in the past to simplify the license of Python >> but this would require agreement from the current rights owners >> including CWI and CNRI. IIRC not all of the rights owners are willing >> to agree to a change. >> > That is the current position. This is a pet project of mine, but it needs round tuits that are currently in short supply. From stephen at xemacs.org Tue Jul 6 18:36:30 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 07 Jul 2010 01:36:30 +0900 Subject: [Python-Dev] Licensing In-Reply-To: References: <20100706021335.5c304fab@pitrou.net> <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> <1278391678.3159.12.camel@localhost.localdomain> <874ogdds96.fsf_-_@benfinney.id.au> <87vd8tcbwx.fsf@benfinney.id.au> Message-ID: <877hl8a7ox.fsf@uwakimon.sk.tsukuba.ac.jp> LD 'Gus' Landis writes: > Yes. The BSD license on FreeBSD has allowed Apple to > make MacOS X a completely proprietary product. That's simply not true. http://www.opensource.apple.com/release/mac-os-x-1064/. From ldlandis at gmail.com Tue Jul 6 18:51:21 2010 From: ldlandis at gmail.com (LD 'Gus' Landis) Date: Tue, 6 Jul 2010 10:51:21 -0600 Subject: [Python-Dev] Licensing In-Reply-To: <877hl8a7ox.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20100706021335.5c304fab@pitrou.net> <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> <1278391678.3159.12.camel@localhost.localdomain> <874ogdds96.fsf_-_@benfinney.id.au> <87vd8tcbwx.fsf@benfinney.id.au> <877hl8a7ox.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: I stand corrected. Thanks for the pointer Stephen! On Tue, Jul 6, 2010 at 10:36 AM, Stephen J. Turnbull wrote: > LD 'Gus' Landis writes: > ?> Yes. The BSD license on FreeBSD has allowed Apple to > ?> make MacOS X a completely proprietary product. > > That's simply not true. > http://www.opensource.apple.com/release/mac-os-x-1064/. > -- --- NOTE: If it is important CALL ME - I may miss email, which I do NOT normally check on weekends nor on a regular basis during any other day. --- LD Landis - N0YRQ - de la tierra del encanto 3960 Schooner Loop, Las Cruces, NM 88012 575-448-1763 N32 21'48.28" W106 46'5.80" From glyph at twistedmatrix.com Tue Jul 6 20:51:22 2010 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Tue, 6 Jul 2010 14:51:22 -0400 Subject: [Python-Dev] Licensing // PSF // Motion of non-confidence In-Reply-To: <201007062209.18668.steve@pearwood.info> References: <20100706021335.5c304fab@pitrou.net> <87wrt99s7x.fsf@uwakimon.sk.tsukuba.ac.jp> <201007062209.18668.steve@pearwood.info> Message-ID: <1176E7CB-BE54-463B-B877-4F987A6228C2@twistedmatrix.com> On Jul 6, 2010, at 8:09 AM, Steven D'Aprano wrote: > You've never used Apple's much-missed Hypertalk, have you? :) on mailingListMessage get the message put it into aMessage if the thread of aMessage contains license wankery put aMessage into the trash end if end mailingListMessage -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.belopolsky at gmail.com Tue Jul 6 21:59:05 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 6 Jul 2010 15:59:05 -0400 Subject: [Python-Dev] Include datetime.py in stdlib or not? Message-ID: This idea has been discussed extensively in this and other forums and I believe it is time to make a decision. The proposal is to add pure python implementation of datetime module to stdlib. The current C implementation will transparently override pure python definitions in CPython. Other python implementations will have an option of supplying their own fast implementation. This approach has already been adopted by several modules including pickle, heapq and warnings. It has even been suggested [1] that this is the direction in which the majority of CPython extension modules should be heading. This proposal has brought mostly positive feedback on the tracker [2] with only a few objections being raised. 1. Since this does not bring any new functionality and datetime module is not expected to evolve, there is no need for pure python version. 2. There are other areas of stdlib that can benefit more from pure python equivalents. 3. Reference implementations should be written by a senior CPython developer and not scraped from external projects like PyPy. Let me briefly address these objections: 1. Availability of pure python equivalents of standard library modules is very helpful for debugging python applications. This is particularly true when the stdlib module is designed to be extendable by and calls into user-supplied code. This is true in the case of datetime module which relies on 3rd-party or user-supplied code for any timezone support. The datetime module indeed saw very little development in the last 6 years. However this lack of development may itself be the result of pure python version not being available. For example, the idea to supply a concrete tzinfo object representing UTC has been brought up back in 2002. [3] An RFE [4] was created in the tracker in January, 2009 and took more than 1.5 years to implement. If you look at the history of issue5094, you will see that development slowed down considerably when C coding started. Note that for this particular feature, there was probably no need to have it implemented in C to begin with. (Most common operations involve datetime objects in the same timezone and those don't need to call timezone methods.) 2. Unlike other areas of stdlib, datetime module was originally prototyped in python and it turns out that it hardly changed between python 2.3 and 2.6 with a couple of features added in 2.7. A port to 3.x was uneventful as well. 3. The version of datetime.py [5] that I propose for inclusion is substantially the pure python prototype written by Tim Peters and others back in 2003. The PyPy changes are very few [6]. I believe the code is substantially ready for inclusion. There are a few items that need to be fixed related to how floating point arguments to timedelta are handled, as well as some clean-up of docstrings and error messages (both C and python implementations can see some improvement in this area). The biggest item in terms of development effort would be to refactor test_datetime to test both implementations. A simple solution [7] of importing test_datetime twice with and without _datetime will probably not be accepted because it is not compatible with alternative unittest runners. What do you think? Please reply here or add a comment at http://bugs.python.org/issue7989. [1] http://bugs.python.org/issue5094#msg106498 [2] http://bugs.python.org/issue7989 [3] http://www.zope.org/Members/fdrake/DateTimeWiki/SuggestedRequirements [4] http://bugs.python.org/issue5094 [5] http://svn.python.org/view/*checkout*/sandbox/branches/py3k-datetime/datetime.py [6] http://bugs.python.org/file17701/datetime-sandbox-pypy.diff [7] http://bugs.python.org/file17848/issue7989.diff From brett at python.org Tue Jul 6 22:30:00 2010 From: brett at python.org (Brett Cannon) Date: Tue, 6 Jul 2010 13:30:00 -0700 Subject: [Python-Dev] Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On Tue, Jul 6, 2010 at 12:59, Alexander Belopolsky wrote: > This idea has been discussed extensively in this and other forums and > I believe it is time to make a decision. > > The proposal is to add pure python implementation of datetime module > to stdlib. ? The current C implementation will transparently override > pure python definitions in CPython. ?Other python implementations will > have an option of supplying their own fast implementation. ?This > approach has already been adopted by several modules including pickle, > heapq and warnings. ? It has even been suggested [1] that this is the > direction in which the majority of CPython extension modules should be > heading. > > ?This proposal has brought mostly positive feedback on the tracker [2] > with only a few objections being raised. > > 1. Since this does not bring any new functionality and datetime module > is not expected to evolve, there is no need for pure python version. > 2. There are other areas of stdlib that can benefit more from pure > python equivalents. > 3. Reference implementations should be written by a senior CPython > developer and not scraped from external projects like PyPy. I should mention that PyPy has said they are quite happy to donate their datetime implementation which is what Alexander (I believe) has been working off of. Also, adding a pure Python version alleviates the need of the other VMs from having to maintain the same module separately. Making the stdlib shareable (and thus eventually breaking it out from CPython) was discussed at the language summit at PyCon 2010 and generally agreed upon, and this is a step towards making that happen. -Brett > > Let me briefly address these objections: > > 1. Availability of pure python equivalents of standard library modules > is very helpful for debugging python applications. This is > particularly true when the stdlib module is designed to be extendable > by and calls into user-supplied code. ?This is true in the case of > datetime module which relies on 3rd-party or user-supplied code for > any timezone support. > > The datetime module indeed saw very little development in the last 6 > years. ? However this lack of development may itself be the result of > pure python version not being available. ?For example, the idea to > supply a concrete tzinfo object representing UTC has been brought up > back in 2002. [3] ?An RFE [4] was created in the tracker in January, > 2009 and took more than 1.5 years to implement. ?If you look at the > history of issue5094, you will see that development slowed down > considerably when C coding started. ?Note that for this particular > feature, there was probably no need to have it implemented in C to > begin with. ?(Most common operations involve datetime objects in the > same timezone and those don't need to call timezone methods.) > > 2. Unlike other areas of stdlib, datetime module was originally > prototyped in python and it turns out that it hardly changed between > python 2.3 and 2.6 with a couple of features added in 2.7. ?A port to > 3.x was uneventful as well. > > 3. The version of datetime.py [5] that I propose for inclusion is > substantially the pure python prototype written by Tim Peters and > others back in 2003. ?The PyPy changes are very few [6]. > > I believe the code is substantially ready for inclusion. ?There are a > few items that need to be fixed related to how floating point > arguments to timedelta are handled, as well as some clean-up of > docstrings and error messages (both C and python implementations can > see some improvement in this area). ?The biggest item in terms of > development effort would be to refactor ?test_datetime to test both > implementations. ?A simple solution [7] of importing test_datetime > twice with and without _datetime will probably not be accepted because > it is not compatible with alternative unittest runners. > > What do you think? ?Please reply here or add a comment at > http://bugs.python.org/issue7989. > > [1] http://bugs.python.org/issue5094#msg106498 > [2] http://bugs.python.org/issue7989 > [3] http://www.zope.org/Members/fdrake/DateTimeWiki/SuggestedRequirements > [4] http://bugs.python.org/issue5094 > [5] http://svn.python.org/view/*checkout*/sandbox/branches/py3k-datetime/datetime.py > [6] http://bugs.python.org/file17701/datetime-sandbox-pypy.diff > [7] http://bugs.python.org/file17848/issue7989.diff > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From ncoghlan at gmail.com Tue Jul 6 22:50:59 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 7 Jul 2010 06:50:59 +1000 Subject: [Python-Dev] blocking 2.7 In-Reply-To: <4C331D26.1040501@livinglogic.de> References: <201007031426.53880.victor.stinner@haypocalc.com> <4C302ADD.30608@netwok.org> <4C331D26.1040501@livinglogic.de> Message-ID: On Tue, Jul 6, 2010 at 10:10 PM, Walter D?rwald wrote: > On 05.07.10 16:19, Nick Coghlan wrote: > http://coverage.livinglogic.de/ *does* include coverage info for stuff > written in C, see for example: > > ? http://coverage.livinglogic.de/Objects/unicodeobject.c.html Ah, I missed that. Cool. > However it *is* strange that test_audioop.py gets executed, but > audioop.c doesn't seem to be. There do seem to be a *lot* of N/A's against the C code (that's why I thought the C code wasn't included in the stats collection in the first place). Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From tjreedy at udel.edu Tue Jul 6 23:05:43 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 06 Jul 2010 17:05:43 -0400 Subject: [Python-Dev] Licensing In-Reply-To: References: Message-ID: <4C339AA7.3040204@udel.edu> I think there are a couple of potential action items that have come out of the discussion. 1. Python License If there is not already, could there be an explanatory note, something like (worded to be 'neutral': "The Python License is complicated because Python has been developed at various times under the auspices of four different organizations. Each retains ownership of the code developed or contributed during its tenure and continues to license its portion of the code under its own Python license." Perhaps add: "The PSF cannot unilaterally change this." It would be nice if a layperson summary could be added: "Overall, the Python License is similar to the MIT license." and even "Basically, you can do what you want as long as you do it at your own risk and do not claim ownership of either the code or the name Python." Such paraphrases have been posted on Python-list, though without legal standing. But I would understand if our lawyer objected that for the PSF, rather than individuals, to say the same would somehow give the paraphrase a legal standing it should not have. 2. Contibutor License I signed this some time ago, but wondered a bit about the discrepancy between this and the distribution license. I appreciate that Anatoly's question about the same has elicited an explanation that I can understand: The PSF requests that we give the PSF a clear, understandable license that allows the PSF both to distribute our contributions *and* to re-license it under the complicated license that it is forced to use for distribution. To put it another way: the contributor agreement is simple so contributors do not have to bother (as contributors) with the complications of the distribution license. Perhaps this could be clearer on the contributor license page. PS to Anatoly: I hope your questions, at least on the contributor agreement, are sufficiently well answered that you will sign it, send it in, and continue contributing. I say this as someone who did read and think about it and decide there was nothing to worry about because I would keep ownership of my words, trusted that they would appear in at least one more Python version, and otherwise did not excessively care what PSF did with them. I also say this as someone who currently would not upload a package of mine to the PyPI repository because for that I *would* care. ------------------- Comment on trust. Trust works both ways. So does distrust. Asking contributors to give written licenses in addition to the license implicit in the act of contribution is an act of distrust. It says something like "We worry that you might change you mind and sue, and a court might not immediately toss the suit." So it should not surprise if the occasional person reacts with overt hurt and distrust. -- Terry Jan Reedy From ncoghlan at gmail.com Tue Jul 6 23:18:31 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 7 Jul 2010 07:18:31 +1000 Subject: [Python-Dev] Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On Wed, Jul 7, 2010 at 5:59 AM, Alexander Belopolsky wrote: > What do you think? ?Please reply here or add a comment at > http://bugs.python.org/issue7989. (For those that haven't read the tracker discussion, it's long, but worth skimming to get a better idea of the various points of view). +1 on the general idea, but I haven't looked at the patches in order to be able to comment on the specifics (except that following any of the test_warnings, test_heapq, test_pickle, test_io, etc. styles of testing parallel implementations should be fine). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Tue Jul 6 23:27:10 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 7 Jul 2010 07:27:10 +1000 Subject: [Python-Dev] Licensing In-Reply-To: <4C339AA7.3040204@udel.edu> References: <4C339AA7.3040204@udel.edu> Message-ID: On Wed, Jul 7, 2010 at 7:05 AM, Terry Reedy wrote: > Asking contributors to give written licenses in addition to the license > implicit in the act of contribution is an act of distrust. It says something > like "We worry that you might change you mind and sue, and a court might not > immediately toss the suit." So it should not surprise if the occasional > person reacts with overt hurt and distrust. The other (IMO, more important) element to it is that it acts as an assertion that the developer actually *has* the rights to contribute the code they're contributing. So, rather than being worried about someone changing their mind about their contributions (although that's admittedly part of it), we're more concerned that contributors actually think about who owns the copyright on the code they're offering and make sure the appropriate permissions are in place. For example, if you look at some of the code that even Guido has submitted (e.g. pgen2), that's actually come in under Google's contributor agreement, rather than Guido's personal one. Presumably that was work he did on company time, so the copyright actually rests with Google rather than Guido. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From hodgestar+pythondev at gmail.com Tue Jul 6 23:32:27 2010 From: hodgestar+pythondev at gmail.com (Simon Cross) Date: Tue, 6 Jul 2010 23:32:27 +0200 Subject: [Python-Dev] Licensing In-Reply-To: <4C339AA7.3040204@udel.edu> References: <4C339AA7.3040204@udel.edu> Message-ID: On Tue, Jul 6, 2010 at 11:05 PM, Terry Reedy wrote: > 1. Python License > > If there is not already, could there be an explanatory note, something like > (worded to be 'neutral': As a sub-point, I'd like to see something short explaining how the different licenses in the LICENSE file are meat to be combined. At the moment the terms and conditions section just lists them without explanation. Schiavo Simon From mal at egenix.com Tue Jul 6 23:57:27 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 06 Jul 2010 23:57:27 +0200 Subject: [Python-Dev] Licensing In-Reply-To: <4C339AA7.3040204@udel.edu> References: <4C339AA7.3040204@udel.edu> Message-ID: <4C33A6C7.4010905@egenix.com> Terry Reedy wrote: > Comment on trust. Trust works both ways. So does distrust. > > Asking contributors to give written licenses in addition to the license > implicit in the act of contribution is an act of distrust. It says > something like "We worry that you might change you mind and sue, and a > court might not immediately toss the suit." So it should not surprise if > the occasional person reacts with overt hurt and distrust. The written contributor agreements are needed to enable the PSF to defend the IP in the Python software. They are just a legal tool, nothing more. Note that the PSF doesn't relicense the contributed code under the whole license stack. The contributed code is (currently) being relicensed under the PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2 (the top part of the stack), which is a very straight forward BSD-style license. The other licenses in the stack only apply to the code owned by the resp. parties CWI, CNRI, BeOpen and the cast of thousands (which fortunately didn't get to send in their lawyers and still had a very good time). Apart from that, the Python distribution also comes with 3rd party code under various other BSD-style licenses. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 06 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2010-07-19: EuroPython 2010, Birmingham, UK 12 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From tjreedy at udel.edu Wed Jul 7 01:18:09 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 06 Jul 2010 19:18:09 -0400 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken Message-ID: [Also posted to http://bugs.python.org/issue2986 Developed with input from Eli Bendersky, who will write patchfile(s) for whichever change option is chosen.] Summary: difflib.SeqeunceMatcher was developed, documented, and originally operated as "a flexible class for comparing pairs of sequences of any [hashable] type". An "experimental" heuristic was added in 2.3a1 to speed up its application to sequences of code lines, which are selected from an unbounded set of possibilities. As explained below, this heuristic partly to completely disables SequenceMatcher for realistic-length sequences from a small finite alphabet. The regression is easy to fix. The docs were never changed to reflect the effect of the heuristic, but should be, with whatever additional change is made. In the commit message for revision 26661, which added the heuristic, Tim Peters wrote "While I like what I've seen of the effects so far, I still consider this experimental. Please give it a try!" Several people who have tried it discovered the problem with small alphabets and posted to the tracker. Issues #1528074, #1678339. #1678345, and #4622 are now-closed duplicates of #2986. The heuristic needs revision. Open questions (discussed after the examples): what exactly to do, which versions to do it too, and who will do it. --- Some minimal difference examples: from difflib import SequenceMatcher as SM # base example print(SM(None, 'x' + 'y'*199, 'y'*199).ratio()) # should be and is 0.9975 (rounded) # make 'y' junk print(SM(lambda c:c=='y', 'x' + 'y'*199, 'y'*199).ratio()) # should be and is 0.0 # Increment b by 1 char print(SM(None, 'x' + 'y'*199, 'y'*200).ratio()) # should be .995, but now is 0.0 because y is treated as junk # Reverse a and b, which increments b print(SM(None, 'y'*199, 'x' + 'y'*199).ratio()) # should be .9975, as before, but now is 0.0 because y is junked The reason for the bug is the heuristic: if the second sequence is at least 200 items long then any item occurring more than one percent of the time in the second sequence is treated as junk. This was aimed at recurring code lines like 'else:' and 'return', but can be fatal for small alphabets where common items are necessary content. A more realistic example than the above is comparing DNA gene sequences. Without the heuristic SequenceMatcher.get_opcodes() reports an appropriate sequence of matches and edits and .ratio works as documented and expected. For 1000/2000/6000 bases, the times on a old Athlon 2800 machine are <1/2/12 seconds. Since 6000 is longer than most genes, this is a realistic and practical use. With the heuristic, everything is junk and there is only one match, ''=='' augmented by the initial prefix of matching bases. This is followed by one edit: replace the rest of the first sequence with the rest of the second sequence. A much faster way to find the first mismatch would be i = 0 while first[i] == second[i]: i+=1 The match ratio, based on the initial matching prefix only, is spuriously low. --- Questions: 1: what change should be make. Proposed fix: Disentangle the heuristic from the calculation of the internal b2j dict that maps items to indexes in the second sequence b. Only apply the heuristic (or not) afterward. Version A: Modify the heuristic to only eliminate common items when there are more than, say, 100 items (when len(b2j)> 100 where b2j is first calculated without popularity deletions). The would leave DNA, protein, and printable ascii+[\n\r\t] sequences alone. On the other hand, realistic sequences of more than 200 code lines should have at least 100 different lines, and so the heuristic should continue to be applied when it (mostly?) 'should' be. This change leaves the API unchanged and does not require a user decision. Version B: add a parameter to .__init__ to make the heuristic optional. If the default were True ('use it'), then the code would run the same as now (even when bad). With the heuristic turned off, users would be able to get the .ratio they may expect and need. On the other hand, users would have to understand the heuristic to know when and when not to use it. Version C: A more radical alternative would be to make one or more of the tuning parameters user settable, with one setting turning it off. 2. What type of issue is this, and what version get changed. I see the proposal as partial reversion of a change that sometimes causes a regression, in order to fix the regression. Such would usually be called a bugfix. Other tracker reviewers claim this issue is a feature request, not a bugfix. Either way, 3.2 gets the fix. The practical issue is whether at least 2.7(.1) should get the fix, or whether the bug should forever continue in 2.x. 3. Who will make the change. Eli will write a patch and I will check it. However, Georg Brandel assigned the issue to Tim Peters, with a request for comment, but Tim never responded. Is there an active committer who will grab the issue and do a commit review when a patch is ready? -- Terry Jan Reedy From techtonik at gmail.com Wed Jul 7 01:47:32 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Wed, 7 Jul 2010 02:47:32 +0300 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: <20100702143409.67f805a4@pitrou.net> References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> Message-ID: On Fri, Jul 2, 2010 at 3:34 PM, Antoine Pitrou wrote: >> >> > After the switch, hg.python.org/cpython will be the official repo, and >> > code.python.org/hg will probably be closed. >> >> Why this transition is not described in PEP? > > Because it's not a transition. It's a mirror. It was put in place > before the hg migration plan was accepted, IIRC. Where is this migration plan then if it is not in PEP? >> How code.python.org/hg is synchronized with Subversion? > > What does your question mean exactly? It's a mirror (well, a set of > mirrors) and is synchronized roughly every 5 minutes. Method. Software used, which parameters are set for it, how to repeat the process? >> Why it is not possible to leave code.python.org/hg as is in slave mode >> and then realtime replication is ready just switch master/slave over? > > The two sets of repositories use different conversion tools and rules. > They have nothing in common (different changeset IDs, different > metadata, different branch/clone layout). That would be nice to hear about in more detail. As I understand there is no place where it is described. I already see +1 from Fred Drake and another +1 from Steve Holden down the thread. However, Antoine Pitrou, Dirkjan Ochtman and Jesse Noller object. They afraid that contributors won't survive low-level details about Mercurial migration. I'd say there a plenty of ways isolate them and at the same time satisfy "Mercurial aficionados" either on the same page or in different places. On Fri, Jul 2, 2010 at 4:06 PM, Stephen J. Turnbull wrote: > > There is no reason at this point to suppose the transition can't be > complete by the end of summer. ?However, as always, the devil is in > the details, and one of them may be a showstopper. ?We'll just have to > see about that. The transition can be complete in a few minutes. The question is how good it will be. As there are no plan, no roadmap, no status - it is hard to judge if it is feasible at all. Ok. Given that nobody is able/willing to say anything more - I've gathered all your feedback concerning current status of Mercurial migration on this Wave - https://wave.google.com/wave/waveref/googlewave.com/w+4_fnAVHwA I hope you will find the time to enhance it with more info so not contributors proficient with Mercurial could help to speed up the transition. -- anatoly t. From jnoller at gmail.com Wed Jul 7 02:13:54 2010 From: jnoller at gmail.com (Jesse Noller) Date: Tue, 6 Jul 2010 20:13:54 -0400 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> Message-ID: On Tue, Jul 6, 2010 at 7:47 PM, anatoly techtonik wrote: > On Fri, Jul 2, 2010 at 3:34 PM, Antoine Pitrou wrote: >>> >>> > After the switch, hg.python.org/cpython will be the official repo, and >>> > code.python.org/hg will probably be closed. >>> >>> Why this transition is not described in PEP? >> >> Because it's not a transition. It's a mirror. It was put in place >> before the hg migration plan was accepted, IIRC. > > Where is this migration plan then if it is not in PEP? > >>> How code.python.org/hg is synchronized with Subversion? >> >> What does your question mean exactly? It's a mirror (well, a set of >> mirrors) and is synchronized roughly every 5 minutes. > > Method. Software used, which parameters are set for it, how to repeat > the process? > >>> Why it is not possible to leave code.python.org/hg as is in slave mode >>> and then realtime replication is ready just switch master/slave over? >> >> The two sets of repositories use different conversion tools and rules. >> They have nothing in common (different changeset IDs, different >> metadata, different branch/clone layout). > > That would be nice to hear about in more detail. As I understand there > is no place where it is described. I already see +1 from Fred Drake > and another +1 from Steve Holden down the thread. > > However, Antoine Pitrou, Dirkjan Ochtman and Jesse Noller object. They > afraid that contributors won't survive low-level details about > Mercurial migration. I'd say there a plenty of ways isolate them and > at the same time satisfy "Mercurial aficionados" either on the same > page or in different places. No, I don't need you misrepresenting anything I've said Anatoly - I said there's no need to maintain SVN alongside mercurial after we convert, and doing so is silly. I maintain that once we convert, we very happily stay converted, and drop official "other" mirrors unless other volunteers step up to maintain them. I have no problem with additional documentation should people wish to volunteer to write it. We do not work for you Anatoly. > On Fri, Jul 2, 2010 at 4:06 PM, Stephen J. Turnbull wrote: >> >> There is no reason at this point to suppose the transition can't be >> complete by the end of summer. ?However, as always, the devil is in >> the details, and one of them may be a showstopper. ?We'll just have to >> see about that. > > The transition can be complete in a few minutes. The question is how > good it will be. As there are no plan, no roadmap, no status - it is > hard to judge if it is feasible at all. No. There is no question except in your mind. We all have a rough idea of the status, modulo the PEPs being updated. It is also perfectly feasible. I would love it, and offer you a christmas card if you could drop the hyperbole and misrepresentation. > > Ok. Given that nobody is able/willing to say anything more - I've > gathered all your feedback concerning current status of Mercurial > migration on this Wave - > https://wave.google.com/wave/waveref/googlewave.com/w+4_fnAVHwA ?I > hope you will find the time to enhance it with more info so not > contributors proficient with Mercurial could help to speed up the > transition. While the summary is nice; your wave entry has nothing to do with the mercurial transition, if you want to help, please ask someone to take on an open task, or volunteer to write/accentuate the PEPs, or help with documentation for post-migration workflow. Your contributions can be effective and useful, rather than noisemaking and abrasive. The mercurial transition will occur, barring someone directly involved finding show-stopping reasons otherwise, with or without you. The decision was made some time ago, and despite your recent noisemaking, will continue on. jesse From bioinformed at gmail.com Wed Jul 7 03:47:41 2010 From: bioinformed at gmail.com (Kevin Jacobs ) Date: Tue, 6 Jul 2010 21:47:41 -0400 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: References: Message-ID: On Tue, Jul 6, 2010 at 7:18 PM, Terry Reedy wrote: > [Also posted to http://bugs.python.org/issue2986 > A much faster way to find the first mismatch would be > i = 0 > while first[i] == second[i]: > i+=1 > The match ratio, based on the initial matching prefix only, is spuriously > low. > > I don't have much experience with the Python sequence matcher, but many classical edit distance and alignment algorithms benefit from stripping any common prefix and suffix before engaging in heavy-lifting. This is trivially optimal for Hamming-like distances and easily shown to be for Levenshtein and Damerau type distances. -Kevin -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Wed Jul 7 05:54:25 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 06 Jul 2010 23:54:25 -0400 Subject: [Python-Dev] Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On 7/6/2010 3:59 PM, Alexander Belopolsky wrote: I am more interested in Brett's overall vision than this particular module. I understand that to be one of a stdlib that is separate from CPython and is indeed the standard Python library. Questions: !. Would the other distributions use a standard stdlib rather than current individual versions? If so, and if at least one used the Python version of each module, this would alleviate the concern that non-use == non-testing. (Test improvement would also help this.) 2. Would the other distributions pool their currently separate stdlib efforts to help maintain one standard stdlib. If so, this would alleviate the concern about the extra effort to maintain both a C and Python version. (Test improvement would also help this also.) 3. What version of Python would be allowed for use in the stdlib? I would like the stdlib for 3.x to be able to use 3.x code. This would be only a minor concern for CPython as long as 2.7 is maintained, but a major concern for the other implementation currently 'stuck' in 2.x only. A good 3to2 would be needed. I generally favor having Python versions of modules available. My current post on difflib.SequenceMatcher is based on experiments with an altered version. I copied difflib.py to my test directory, renamed it diff2lib.py, so I could import both versions, found and edited the appropriate method, and off I went. If difflib were in C, my post would have been based on speculation about how a fixed version would operate, rather than on data. 4. Does not ctypes make it possible to replace a method of a Python-coded class with a faster C version, with something like try: connect to methods.dll check that function xyx exists replace Someclass.xyy with ctypes wrapper except: pass For instance, the SequenceMatcher heuristic was added to speedup the matching process that I believe is encapsulated in one O(n**2) or so bottleneck method. I believe most everything else is O(n) bookkeeping. > This proposal has brought mostly positive feedback on the tracker [2] > with only a few objections being raised. > > 1. Since this does not bring any new functionality and datetime module > is not expected to evolve, there is no need for pure python version. see above > 2. There are other areas of stdlib that can benefit more from pure > python equivalents. Possibly true, but developers do what they do, and this seems mostly done. > 3. Reference implementations should be written by a senior CPython > developer and not scraped from external projects like PyPy. I did not see that im my reading of the thread. In any case, what matters is quality, not authorship. > What do you think? Please reply here or add a comment at > http://bugs.python.org/issue7989. From scanning that and the posts here, it seems like a pep or other doc on dual version modules would be a good idea. It should at least document how to code the switch from python version to the x coded version and how to test both, as discussed. -- Terry Jan Reedy From tim.peters at gmail.com Wed Jul 7 06:20:29 2010 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 7 Jul 2010 00:20:29 -0400 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: References: Message-ID: [Terry Reedy] > [Also posted to http://bugs.python.org/issue2986 > Developed with input from Eli Bendersky, who will write patchfile(s) for > whichever change option is chosen.] Thanks for paying attention to this, Terry (and Ed)! I somehow managed to miss the whole discussion over the intervening years :-( > Summary: difflib.SeqeunceMatcher was developed, documented, and originally > operated as "a flexible class for comparing pairs of sequences of any > [hashable] type". Although it can be used for that, its true intent was to produce intuitive diffs for revisions of text files (including code files) edited by humans. Where "intuitive" means approximately "less jarring than the often-odd diffs produced by algorithms working on some rigid mathematical notion of 'minimal edit distance'". Whether it's useful for more than that I can't say, because that's all I ever developed (or used) the algorithm for. > An "experimental" heuristic was added in 2.3a1 Big bad on me for that! At the time I fully intended to document that, and at least make it tunable, but life intervened and it got dropped on the floor. > to speed up its application to sequences of code lines, Yes, that was the intent. I was corresponding with a user at the time who had odd notions (well, by my standards) of how to format C code, which left him with many hundreds of lines containing only an open brace, or a close brace, or just a semicolon (etc). difflib spun its wheels frantically trying to sort this out, and the heuristic in question cut processing time from hours (in the worst cases) to seconds. Since that (text file comparison) was always the primary case for this class, it was worth doing something about. But it should not have gone in the way it did (undocumented & unfinished, as you correctly note). > which are selected from an > unbounded set of possibilities. As explained below, this heuristic partly to > completely disables SequenceMatcher for realistic-length sequences from a > small finite alphabet. Which wasn't an anticipated use case, so should not be favored. Slowing down difflib for what it was intended for is not a good idea - practicality beats purity. Ya, ya, I understand someone playing around with DNA sequences might find difflib tempting at first, but fix this and they're still going to be unhappy. There are much better (faster, "more standard") algorithms for comparing sequences drawn from tiny alphabets, and especially so for DNA matching. > The regression is easy to fix. The docs were never > changed to reflect the effect of the heuristic, but should be, with whatever > additional change is made. True - and always was. > In the commit message for revision 26661, which added the heuristic, Tim > Peters wrote "While I like what I've seen of the effects so far, I still > consider this experimental. ?Please give it a try!" Several people who have > tried it discovered the problem with small alphabets and posted to the > tracker. Issues #1528074, #1678339. #1678345, and #4622 are now-closed > duplicates of #2986. The heuristic needs revision. > > Open questions (discussed after the examples): what exactly to do, which > versions to do it too, and who will do it. > > --- > Some minimal difference examples: > > from difflib import SequenceMatcher as SM > > # base example > print(SM(None, 'x' + 'y'*199, 'y'*199).ratio()) > # should be and is 0.9975 (rounded) > > # make 'y' junk > print(SM(lambda c:c=='y', 'x' + 'y'*199, 'y'*199).ratio()) > # should be and is 0.0 > > # Increment b by 1 char > print(SM(None, 'x' + 'y'*199, 'y'*200).ratio()) > # should be .995, but now is 0.0 because y is treated as junk > > # Reverse a and b, which increments b > print(SM(None, 'y'*199, 'x' + 'y'*199).ratio()) > # should be .9975, as before, but now is 0.0 because y is junked > > The reason for the bug is the heuristic: if the second sequence is at least > 200 items long then any item occurring more than one percent of the time in > the second sequence is treated as junk. This was aimed at recurring code > lines like 'else:' and 'return', but can be fatal for small alphabets where > common items are necessary content. Indeed, it makes no sense at all for tiny alphabets. OTOH, as above, it gave factor-of-thousands speedups for intended use cases, and that's more important to me. There should certainly be a way to turn off the "auto junk" heuristic, and to tune it, but - sorry for being pragmatic ;-) - it was a valuable speed improvement for what I expect still remain difflib's overwhelmingly most common use cases. > A more realistic example than the above is comparing DNA gene sequences. Comparing DNA sequences is realistic, but using SequenceMatcher to do so is unrealistic except for a beginner just playing with the ideas. There should be a way to disable the heuristic so the beginner can have their fun, but any serious work in this area will need to use different algorithms. > Without the heuristic SequenceMatcher.get_opcodes() reports an appropriate > sequence of matches and edits and .ratio works as documented and expected. > ?For 1000/2000/6000 bases, the times on a old Athlon 2800 machine are > <1/2/12 seconds. Since 6000 is longer than most genes, this is a realistic > and practical use. Did I just call you a beginner just playing with the ideas? Could be ;-) People doing serious work in this area employ a great many other approaches specifically designed for DNA matching: http://en.wikipedia.org/wiki/Sequence_alignment and have developed dozens of software packages to implement them: http://en.wikipedia.org/wiki/Sequence_alignment_software > With the heuristic, everything is junk and there is only one match, ''=='' > augmented by the initial prefix of matching bases. This is followed by one > edit: replace the rest of the first sequence with the rest of the second > sequence. A much faster way to find the first mismatch would be > ? i = 0 > ? while first[i] == second[i]: > ? ? ?i+=1 > The match ratio, based on the initial matching prefix only, is spuriously > low. The details don't really matter to the conclusion: by definition, "junk" is ignored for purposes of finding a match. Therefore any method of automatically calling something "junk" is going to screw up _some_ use case. I agree that's unacceptable - there must be a way to turn that off. But, again, it proved valuable for SequenceMatcher's _primary_ use case, so keeping it enabled by default is the most sensible thing to do. > --- > Questions: > > 1: what change should be make. > > Proposed fix: Disentangle the heuristic from the calculation of the internal > b2j dict that maps items to indexes in the second sequence b. Only apply the > heuristic (or not) afterward. > > Version A: Modify the heuristic to only eliminate common items when there > are more than, say, 100 items (when len(b2j)> 100 where b2j is first > calculated without popularity deletions). > > The would leave DNA, protein, and printable ascii+[\n\r\t] sequences alone. > On the other hand, realistic sequences of more than 200 code lines should > have at least 100 different lines, and so the heuristic should continue to > be applied when it (mostly?) 'should' be. This change leaves the API > unchanged and does not require a user decision. It's a better heuristic, in the sense that it sidesteps a fatal problem in the "tiny alphabet" use case. I'm sure it will leave fatal problems for other (unanticipated) use cases, though. So, an improvement, but not "a solution". > Version B: add a parameter to .__init__ to make the heuristic optional. If > the default were True ('use it'), then the code would run the same as now > (even when bad). With the heuristic turned off, users would be able to get > the .ratio they may expect and need. On the other hand, users would have to > understand the heuristic to know when and when not to use it. > > Version C: A more radical alternative would be to make one or more of the > tuning parameters user settable, with one setting turning it off. I don't see that they're mutually exclusive. I expect that the best design would have both a better default heuristic _and_ a way to tune or disable it entirely. A & C above. > 2. What type of issue is this, and what version get changed. > > I see the proposal as partial reversion of a change that sometimes causes a > regression, in order to fix the regression. Such would usually be called a > bugfix. Other tracker reviewers claim this issue is a feature request, not a > bugfix. Either way, 3.2 gets the fix. It's definitely a bug in the sense that it broke some previously working code. But pull the patch, and the user I corresponded with at the start will see code comparison times leap back from seconds to hours, which effectively makes difflib useless for him, and so breaks _his_ real application (he used difflib in a commercial text editor, BTW). Both are unacceptable - but that's how things are sometimes. I favor the primary use case (text file comparison). > The practical issue is whether at least 2.7(.1) should get the fix, or whether > the bug should forever continue in 2.x. I'm afraid there's no true fix here without an API change (any heuristic for auto-identifying junk will break _some_ use case). Your #A sounds good to me for 2.7, and a combination of #A and #C sounds best for 3.x. > 3. Who will make the change. > > Eli will write a patch and I will check it. However, Georg Brandel assigned > the issue to Tim Peters, with a request for comment, but Tim never > responded. I see he asked In March of 2009 - he should have picked a month I was paying attention - or at least a year ;-) > Is there an active committer who will grab the issue and do a > commit review when a patch is ready? If Guido will lend me his time machine, I'll go back to 2002 and apply the patch then, when it should have been done :-( From guido at python.org Wed Jul 7 06:59:10 2010 From: guido at python.org (Guido van Rossum) Date: Wed, 7 Jul 2010 06:59:10 +0200 Subject: [Python-Dev] Licensing In-Reply-To: References: <4C339AA7.3040204@udel.edu> Message-ID: On Tue, Jul 6, 2010 at 11:27 PM, Nick Coghlan wrote: > For example, if you look at some of the code that even Guido has > submitted (e.g. pgen2), that's actually come in under Google's > contributor agreement, rather than Guido's personal one. Presumably > that was work he did on company time, so the copyright actually rests > with Google rather than Guido. I hope you are misremembering some details. I did that work while at Elemental Security (i.e. before I joined Google). It should have Elemental Security's contributor agreement. I developed that code initially for inclusion in Elemental's product line (as part of a parser for a domain-specific language named "Fuel" which did not get open-sourced -- probably for the better. -- --Guido van Rossum (python.org/~guido) From stefan_ml at behnel.de Wed Jul 7 08:06:36 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 07 Jul 2010 08:06:36 +0200 Subject: [Python-Dev] thoughts on the bytes/string discussion In-Reply-To: <50FFA855-6FC5-4A01-B5C1-28FB63AC452F@mac.com> References: <11597.1277401099@parc.com> <96ADD4CE-3A24-45A7-B219-2940195DC3D0@twistedmatrix.com> <4C25B319.8040804@canterbury.ac.nz> <4C271E66.5050902@canterbury.ac.nz> <50FFA855-6FC5-4A01-B5C1-28FB63AC452F@mac.com> Message-ID: Ronald Oussoren, 06.07.2010 16:51: > On 27 Jun, 2010, at 11:48, Greg Ewing wrote: > >> Stefan Behnel wrote: >>> Greg Ewing, 26.06.2010 09:58: >>>> Would there be any sanity in having an option to compile Python >>>> with UTF-8 as the internal string representation? >>> It would break Py_UNICODE, because the internal size of a unicode >>> character would no longer be fixed. >> >> It's not fixed anyway with the 2-char build -- some characters are >> represented using a pair of surrogates. > > It is for practical purposes not even fixed in 4-char builds. In 4-char > builds every Unicode code points corresponds to one item in a python > unicode string, but a base characters with combining characters is still > a sequence of characters and should IMHO almost always be treated as a > single object. As an example, given s="be\N{COMBINING DIAERESIS}" s[:2] > or s[2:] is almost certainly semanticly invalid. Sure. However, this is not a problem for the purpose of the C-API, especially for Cython (which is the angle from which I brought this up). All Cython cares about is that it mimics CPython's sematics excactly when transforming code, and a CPython runtime will ignore surrogate pairs and combining characters during iteration and indexing, and when determining the string length. So a single character unicode string can currently be safely aliased by Py_UNICODE with correct Python semantics. That would no longer be the case if the internal representation switched to UTF-8 and/or if CPython started to take surrogates and combining characters into account when considering the string length. Note that it's impossible to determine if a unicode string contains surrogate pairs because it's running on a narrow unicode build or because the user entered them into the string. But the user would likely expect the second case to treat them as separate code points, whereas the first is an implementation detail that should normally be invisible. Combining characters are a lot clearer here, as they can only be entered by users, so keeping them separate as provided is IMHO the expected behaviour. I think the main theme here is that the interpretation of code points and their transformation for user interfaces and backends is left to the user code. Py_UNICODE represents a code point in the current system, including surrogate pair 'escapes'. And that would change if the underlying encoding switched to something other than UTF-16/UCS-4. Stefan From eliben at gmail.com Wed Jul 7 09:45:40 2010 From: eliben at gmail.com (Eli Bendersky) Date: Wed, 7 Jul 2010 10:45:40 +0300 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: References: Message-ID: [snip] > Yes, that was the intent. ?I was corresponding with a user at the time > who had odd notions (well, by my standards) of how to format C code, > which left him with many hundreds of lines containing only an open > brace, or a close brace, or just a semicolon (etc). ?difflib spun its > wheels frantically trying to sort this out, and the heuristic in > question cut processing time from hours (in the worst cases) to > seconds. > > Since that (text file comparison) was always the primary case for this > class, it was worth doing something about. ?But it should not have > gone in the way it did (undocumented & unfinished, as you correctly > note). > >> which are selected from an >> unbounded set of possibilities. As explained below, this heuristic partly to >> completely disables SequenceMatcher for realistic-length sequences from a >> small finite alphabet. > > Which wasn't an anticipated use case, so should not be favored. > Slowing down difflib for what it was intended for is not a good idea - > practicality beats purity. > > Ya, ya, I understand someone playing around with DNA sequences might > find difflib tempting at first, but fix this and they're still going > to be unhappy. ?There are much better (faster, "more standard") > algorithms for comparing sequences drawn from tiny alphabets, and > especially so for DNA matching. Tim, thanks for your insights. In response to the description above, however, I would like to explain my use case, which originally got me interested in this issue with SequenceMatcher. I was not comparing DNAs, but using SequenceMatcher in my automatic testbench checker that verified the output of some logic design. I didn't want exact comparisons, so I was very happy to see difflib.SequenceMatcher in stdlib, with its useful ratio/quick_ratio functions. I was comparing the output sequence to an expected sequence with a 0.995 ratio threshold and was very happy. Until my sequence got longer than 200 elements... So this isn't DNA, and the alphabet wasn't too tiny, but on the other hand there was nothing in the module to suggest that it should be only used to comparing lines in files. On the contrary, its general-sounding name - SequenceMatcher, lulled me into the (false?) belief that I can just use it for my sequence comparison without worrying about finding better algorithms or implementing stuff like edit distance myself. Judging by the comments in other related issues, I'm far from being the only one. Therefore, I think that you should just admit that your excellent module became useful for more purposes than you originally intended it for :-) !! I completely respect your desire to keep the "intended purposes" as fast as possible, but there are solutions (some of which were presented by Terry) that can make it more useful without any harm to the performance of the intended purpose. As Terry noted, I will be very happy to submit a patch with tests for whatever decision that will be reached by pydev on this matter. Eli From merwok at netwok.org Wed Jul 7 11:12:56 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Wed, 07 Jul 2010 11:12:56 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> Message-ID: <4C344518.8050602@netwok.org> >> Because it's not a transition. It's a mirror. It was put in place >> before the hg migration plan was accepted, IIRC. > Where is this migration plan then if it is not in PEP? The ?hg migration plan? is PEP 385. It means moving from svn.python.org to hg.python.org. It is not possible to make code.python.org/hg a mirror of the future official hg repo (a quick search on the hgbook or the Mercurial wiki will tell you how changeset hashes work). The code.p.o/hg repo was a simple read-only mirror provided as a service to the community. Now that the official repo will be a new, clean Mercurial repo, the current mirror will become redundant and will probably be closed. People using that mirror to follow cpython development and/or propose patches will just redo a clone and rebase their patches. >> The two sets of repositories use different conversion tools and rules. >> They have nothing in common (different changeset IDs, different >> metadata, different branch/clone layout). > That would be nice to hear about in more detail. As I understand there > is no place where it is described. The layout of the future official repo is in PEP 385. The layout of the old mirror can be understood by looking at the Web interface (one repo per svn maintenance branch, etc.) > The transition can be complete in a few minutes. The question is how > good it will be. As there are no plan, no roadmap, no status The transition is not just a command to be run, it includes discussion and writing tools before that, and feedback and support after that. What we call transition is the whole process. People have worked and are still working on the PEP, on the repo, on the policy and on the documentation, and it is unfair not to acknowledge that work and say there is no plan. Regards From mal at egenix.com Wed Jul 7 11:13:09 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 07 Jul 2010 11:13:09 +0200 Subject: [Python-Dev] thoughts on the bytes/string discussion In-Reply-To: <50FFA855-6FC5-4A01-B5C1-28FB63AC452F@mac.com> References: <11597.1277401099@parc.com> <96ADD4CE-3A24-45A7-B219-2940195DC3D0@twistedmatrix.com> <4C25B319.8040804@canterbury.ac.nz> <4C271E66.5050902@canterbury.ac.nz> <50FFA855-6FC5-4A01-B5C1-28FB63AC452F@mac.com> Message-ID: <4C344525.2010002@egenix.com> Ronald Oussoren wrote: > > On 27 Jun, 2010, at 11:48, Greg Ewing wrote: > >> Stefan Behnel wrote: >>> Greg Ewing, 26.06.2010 09:58: >>>> Would there be any sanity in having an option to compile >>>> Python with UTF-8 as the internal string representation? >>> It would break Py_UNICODE, because the internal size of a unicode character would no longer be fixed. >> >> It's not fixed anyway with the 2-char build -- some >> characters are represented using a pair of surrogates. > > It is for practical purposes not even fixed in 4-char builds. In 4-char builds every Unicode code points corresponds to one item in a python unicode string, but a base characters with combining characters is still a sequence of characters and should IMHO almost always be treated as a single object. As an example, given s="be\N{COMBINING DIAERESIS}" s[:2] or s[2:] is almost certainly semanticly invalid. Just to clarify: Python uses code units for Unicode storage. Whether those code units map to code points or glyphs depends on the used Python build and the code points in question. See http://www.egenix.com/library/presentations/#PythonAndUnicode for more background information (esp. page 8). Note that using UTF-8 as internal storage format would not work in Python, since Python is a Unicode producer, i.e. it needs to be able to generate and work with code points that are not allowed in UTF-8, e.g. lone surrogates. Another reason not to use UTF-8 encoded code units is that slicing based on code units could easily create invalid UTF-8 which would then render the data unusable. This is a lot less likely to happen with UCS2 or UCS4. And finally: RAM is cheap and today's CPUs work better with 16- or 32-bit values than 8-bit characters. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 07 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2010-07-19: EuroPython 2010, Birmingham, UK 11 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From dirkjan at ochtman.nl Wed Jul 7 11:53:34 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Wed, 7 Jul 2010 11:53:34 +0200 Subject: [Python-Dev] Mercurial migration readiness In-Reply-To: References: <4C2DD1D7.6040109@jollybox.de> <4C2DD53E.7040107@netwok.org> <20100702143409.67f805a4@pitrou.net> Message-ID: On Wed, Jul 7, 2010 at 01:47, anatoly techtonik wrote: > That would be nice to hear about in more detail. As I understand there > is no place where it is described. I already see +1 from Fred Drake > and another +1 from Steve Holden down the thread. > > However, Antoine Pitrou, Dirkjan Ochtman and Jesse Noller object. They > afraid that contributors won't survive low-level details about > Mercurial migration. I'm happy to answer direct questions about the transition process I have in mind (and mostly written down in the PEP), or the resulting hg repository. I think the PEP has some details about what I think constitutes a good conversion. If there are things that Fred or Steve miss from that discussion, I'd be happy to add to the PEP. Cheers, Dirkjan From greg.ewing at canterbury.ac.nz Wed Jul 7 12:56:18 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 07 Jul 2010 22:56:18 +1200 Subject: [Python-Dev] thoughts on the bytes/string discussion In-Reply-To: <4C344525.2010002@egenix.com> References: <11597.1277401099@parc.com> <96ADD4CE-3A24-45A7-B219-2940195DC3D0@twistedmatrix.com> <4C25B319.8040804@canterbury.ac.nz> <4C271E66.5050902@canterbury.ac.nz> <50FFA855-6FC5-4A01-B5C1-28FB63AC452F@mac.com> <4C344525.2010002@egenix.com> Message-ID: <4C345D52.5070905@canterbury.ac.nz> M.-A. Lemburg wrote: > Note that using UTF-8 as internal storage format would not work > in Python, since Python is a Unicode producer, i.e. it needs to > be able to generate and work with code points that are not allowed > in UTF-8, e.g. lone surrogates. Well, it wouldn't strictly be UTF-8, any more than the 2-byte build is strictly UTF-16, in the sense that lone surrogates can be produced. > Another reason not to use UTF-8 encoded code units is that slicing > based on code units could easily create invalid UTF-8 which would > then render the data unusable. This is a lot less likely to happen > with UCS2 or UCS4. The use cases I had in mind for a 1-byte build are those for which the alternative would be keeping everything in bytes. Applications using a 1-byte build would need to be aware of the fact and take care to slice strings at valid places. If they were using bytes, they would have to face exactly the same issues. > And finally: RAM is cheap and today's CPUs work better with 16- or > 32-bit values than 8-bit characters. Yet some people have reported significant performance benefits for some applications from using a 2-byte build instead of a 4-byte build. I was just speculating whether a 1-byte build might be of further advantage in a few specialised cases. No matter how much RAM or processing speed you have, it's always possible to find an application that stresses the limits. -- Greg From solipsis at pitrou.net Wed Jul 7 12:56:48 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 7 Jul 2010 12:56:48 +0200 Subject: [Python-Dev] thoughts on the bytes/string discussion References: <11597.1277401099@parc.com> <96ADD4CE-3A24-45A7-B219-2940195DC3D0@twistedmatrix.com> <4C25B319.8040804@canterbury.ac.nz> <4C271E66.5050902@canterbury.ac.nz> <50FFA855-6FC5-4A01-B5C1-28FB63AC452F@mac.com> <4C344525.2010002@egenix.com> Message-ID: <20100707125648.0e38a744@pitrou.net> On Wed, 07 Jul 2010 11:13:09 +0200 "M.-A. Lemburg" wrote: > > And finally: RAM is cheap and today's CPUs work better with 16- or > 32-bit values than 8-bit characters. The latter is wrong. There is no cost in accessing bytes rather than words on modern CPUs. (actually, bytes are cheaper overall since they cost less cache) Regards Antoine. From solipsis at pitrou.net Wed Jul 7 13:08:48 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 7 Jul 2010 13:08:48 +0200 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken References: Message-ID: <20100707130848.47c0a1a3@pitrou.net> On Tue, 06 Jul 2010 19:18:09 -0400 Terry Reedy wrote: > > Version A: Modify the heuristic to only eliminate common items when > there are more than, say, 100 items (when len(b2j)> 100 where b2j is > first calculated without popularity deletions). [...] > > Version B: add a parameter to .__init__ to make the heuristic optional. [...] > > Version C: A more radical alternative would be to make one or more of > the tuning parameters user settable, with one setting turning it off. Version B would have my favour (but please make the default be True). Version A can lead to regressions (including performance regressions such as described by Tim), and version C looks far more complicated to use. Regards Antoine. From ncoghlan at gmail.com Wed Jul 7 14:48:50 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 7 Jul 2010 22:48:50 +1000 Subject: [Python-Dev] Licensing In-Reply-To: References: <4C339AA7.3040204@udel.edu> Message-ID: On Wed, Jul 7, 2010 at 2:59 PM, Guido van Rossum wrote: > On Tue, Jul 6, 2010 at 11:27 PM, Nick Coghlan wrote: >> For example, if you look at some of the code that even Guido has >> submitted (e.g. pgen2), that's actually come in under Google's >> contributor agreement, rather than Guido's personal one. Presumably >> that was work he did on company time, so the copyright actually rests >> with Google rather than Guido. > > I hope you are misremembering some details. I did that work while at > Elemental Security (i.e. before I joined Google). It should have > Elemental Security's contributor agreement. I developed that code > initially for inclusion in Elemental's product line (as part of a > parser for a domain-specific language named "Fuel" which did not get > open-sourced -- probably for the better. Whoops, I got my timeline wrong (it did seem a little off when I wrote it - I think part of my brain was trying to tell me the dates didn't match up). I must have been thinking of something else I was working on recently that had Google's name in the header, most likely the abc module. So apologies for the confusion - just s/pgen2/abc/ in my example to make it line up with my intent :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Wed Jul 7 15:28:36 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 7 Jul 2010 23:28:36 +1000 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: References: Message-ID: On Wed, Jul 7, 2010 at 9:18 AM, Terry Reedy wrote: > In the commit message for revision 26661, which added the heuristic, Tim > Peters wrote "While I like what I've seen of the effects so far, I still > consider this experimental. ?Please give it a try!" Several people who have > tried it discovered the problem with small alphabets and posted to the > tracker. Issues #1528074, #1678339. #1678345, and #4622 are now-closed > duplicates of #2986. The heuristic needs revision. Python 2.3 you say... Hmm, I've been using difflib.SequenceMatcher for years in a serial bit error rate tester (with typical message sizes ranging from tens of bytes to tens of thousands of bytes) that occasionally gives unexpected results. I'd been blaming hardware glitches (and, to be fair, all of the odd results I can recall off the top of my head were definitively traced to problems in the hardware under test), but I should probably check I'm not running afoul of this bug. And Tim, the algorithm may not be optimal as a general purpose binary diff algorithm, but it's still a hell of a lot faster than the hardware I use it to test. Compared to the equipment configuration times, the data comparison time is trivial. There's another possibility here - perhaps the heuristic should be off by default in SequenceMatcher, with a TextMatcher subclass that enables it (and Differ and HtmlDiff then inheriting from the latter)? There's currently barely anything in the SequenceMatcher documentation to indicate that it is designed primarily for comparing text rather than arbitrary sequences (the closest it gets is the reference to Ratcliff/Obserhelp gestalt pattern matching and then the link to the Ratcliff/Metzener Dr Dobb's article - and until this thread, I'd never followed the article link). Rather than reverting to Tim's undocumented vision, perhaps we should better articulate it by separating the general purpose matcher from an optimised text matcher. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From alexander.belopolsky at gmail.com Wed Jul 7 17:29:19 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 7 Jul 2010 11:29:19 -0400 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? Message-ID: On Tue, Jul 6, 2010 at 11:54 PM, Terry Reedy wrote: > On 7/6/2010 3:59 PM, Alexander Belopolsky wrote: > > I am more interested in Brett's overall vision than this particular module. > I understand that to be one of a stdlib that is separate from CPython and is > indeed the standard Python library. > I am also very much interested in the overall vision, but I would like to keep the datetime.py thread focused, so I a going to reply to broad questions under a separate subject. > Questions: > > 1. Would the other distributions use a standard stdlib rather than current > individual versions? I certainly hope they will. In the ideal word, passing test.regrtest with unmodified Lib should be *the* definition of what is called Python. I understand that there is already some work underway in this direction such as marking implementation specific tests with appropriate decorators. > > 2. Would the other distributions pool their currently separate stdlib > efforts to help maintain one standard stdlib? I believe that making stdlib and test.regrtest more friendly to alternative implementations will go long way towards this goal. It will, of course, be a decision that each project will have to make. > > 3. What version of Python would be allowed for use in the stdlib? I would > like the stdlib for 3.x to be able to use 3.x code. This would be only a > minor concern for CPython as long as 2.7 is maintained, but a major concern > for the other implementation currently 'stuck' in 2.x only. A good 3to2 > would be needed. Availability of python equivalents will hopefully help "other implementation currently 'stuck' in 2.x only" to get "unstuck" and move to 3.x. I understand that this is a somewhat sensitive issue at the moment, but I believe a decision has been made supporting new features for 2.x is outside of python-dev focus. > 4. Does not ctypes make it possible to replace a method of a Python-coded > class with a faster C version, with something like > ?try: > ? ?connect to methods.dll > ? ?check that function xyx exists > ? ?replace Someclass.xyy with ctypes wrapper > ?except: pass > For instance, the SequenceMatcher heuristic was added to speedup the > matching process that I believe is encapsulated in one O(n**2) or so > bottleneck method. I believe most everything else is O(n) bookkeeping. > The ctypes modules is very CPython centric as far as I know. For the new modules, this may be a valid way to rapidly develop accelerated versions. For modules that are already written in C, I don't see much benefit in replacing them with ctypes wrappers. > [.. datetime specific discussion skipped ..] > From scanning that and the posts here, it seems like a pep or other doc on > dual version modules would be a good idea. It should at least document how > to code the switch from python version to the x coded version and how to > test both, as discussed. > I am certainly not ready to write such PEP. I may be in a better position to contribute to it after I gain more experience with test_datetime.py. At the moment I have more questions than answers. For example, the established practice appears to be: modulename.py # Python code try: from _modulename import * except: pass This is supposed to generate a .pyc file with no python definitions in it if _modulename is available. The problem with datetime.py is that it have several helper methods like _ymd2ord() that will still stay in the module. Should an "else:" clause be added to clean these up? should these methods become class or static methods as appropriate? The established practice for testing is py_module = support.import_fresh_module('modulename', blocked=['_modulename']) c_module = support.import_fresh_module('modulename', fresh=['_modulename']) class TestDefnitions: # not a unittest.TestCase subclass def test_foo(self): self.module.foo(..) ... class C_Test(TestDefnitions, unittest.TestCase): module = c_module class Py_Test(TestDefnitions, unittest.TestCase): module = py_module For datetime.py this approach presents several problems: 1. replacing datetime with self.module.datetime everywhere can get messy quickly. 2. There are test classes defined at the test_datetime module level that subclass from datetime classes. The self.module is not available at the module level. These should probably be moved to setUp() methods and attached to test case self. 3. If #2 is resolved by moving definitions inside functions, the classes will become unpickleable and pickle tests will break. Some hackery involving injecting these classes into __main__ or module globals may be required. These challenges make datetime.py an interesting showcase for other modules, so rather than writing a PEP based on abstract ideas, I think it is better to get datetime.py integrated first and try to establish the best practices on the way. From fuzzyman at voidspace.org.uk Wed Jul 7 17:39:38 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 07 Jul 2010 16:39:38 +0100 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: <4C349FBA.9040800@voidspace.org.uk> On 07/07/2010 16:29, Alexander Belopolsky wrote: > [snip...] > >> 4. Does not ctypes make it possible to replace a method of a Python-coded >> class with a faster C version, with something like >> try: >> connect to methods.dll >> check that function xyx exists >> replace Someclass.xyy with ctypes wrapper >> except: pass >> For instance, the SequenceMatcher heuristic was added to speedup the >> matching process that I believe is encapsulated in one O(n**2) or so >> bottleneck method. I believe most everything else is O(n) bookkeeping. >> >> > The ctypes modules is very CPython centric as far as I know. For the > new modules, this may be a valid way to rapidly develop accelerated > versions. For modules that are already written in C, I don't see > much benefit in replacing them with ctypes wrappers. > > > Nope, both IronPython and PyPy have ctypes implementations and Jython is in the process of "growing" one. Using ctypes for C extensions is the most portable way of providing C extensions for Python (other than providing a pure-Python implementation of course). Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From turnbull at sk.tsukuba.ac.jp Wed Jul 7 17:35:37 2010 From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull) Date: Thu, 08 Jul 2010 00:35:37 +0900 Subject: [Python-Dev] thoughts on the bytes/string discussion In-Reply-To: <4C345D52.5070905@canterbury.ac.nz> References: <11597.1277401099@parc.com> <96ADD4CE-3A24-45A7-B219-2940195DC3D0@twistedmatrix.com> <4C25B319.8040804@canterbury.ac.nz> <4C271E66.5050902@canterbury.ac.nz> <50FFA855-6FC5-4A01-B5C1-28FB63AC452F@mac.com> <4C344525.2010002@egenix.com> <4C345D52.5070905@canterbury.ac.nz> Message-ID: <87pqyzjody.fsf@uwakimon.sk.tsukuba.ac.jp> Greg Ewing writes: > The use cases I had in mind for a 1-byte build are those for > which the alternative would be keeping everything in bytes. > Applications using a 1-byte build would need to be aware of > the fact and take care to slice strings at valid places. If > they were using bytes, they would have to face exactly the > same issues. In other words, the people who want to use bytes have no less pain, and the people who want to use characters suffer much greater pain. How can this be a win? If you live in an ASCII-only world, there are a few APIs where bytes aren't allowed, and indeed it would be a win to use those APIs on ASCII-encoded bytestrings. And I don't mean ISO-8859-1-only, either; UTF-8 is not compatible with ISO-8859-1 at the byte level. But the proposal Guido supports would address that by making those APIs polymorphic. > > And finally: RAM is cheap and today's CPUs work better with 16- or > > 32-bit values than 8-bit characters. > > Yet some people have reported significant performance benefits > for some applications from using a 2-byte build instead of a > 4-byte build. I was just speculating whether a 1-byte build > might be of further advantage in a few specialised cases. Of course it would be. But as soon as you want to do *any* I/O in text mode with non-ASCII characters, you're in real pain. What do you do if a user cut/pastes some text containing proper quotation marks or an en-dash at prompt in a terminal? So polymorphism is a far better way to optimize those special cases, as it allows a byte string in any encoding to be treated as text, not just UTF-8. From jnoller at gmail.com Wed Jul 7 17:43:34 2010 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 7 Jul 2010 11:43:34 -0400 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On Wed, Jul 7, 2010 at 11:29 AM, Alexander Belopolsky wrote: > On Tue, Jul 6, 2010 at 11:54 PM, Terry Reedy wrote: >> On 7/6/2010 3:59 PM, Alexander Belopolsky wrote: >> >> I am more interested in Brett's overall vision than this particular module. >> I understand that to be one of a stdlib that is separate from CPython and is >> indeed the standard Python library. >> > > I am also very much interested in the overall vision, but I would like > to keep the datetime.py thread focused, so I a going to reply to broad > questions under a separate subject. > >> Questions: >> >> 1. Would the other distributions use a standard stdlib rather than current >> individual versions? > > I certainly hope they will. ?In the ideal word, passing test.regrtest > with unmodified Lib should be *the* definition of what is called > Python. ?I understand that there is already some work underway in this > direction such as marking implementation specific tests with > appropriate decorators. > >> >> 2. Would the other distributions pool their currently separate stdlib >> efforts to help maintain one standard stdlib? > > I believe that making stdlib and test.regrtest more friendly to > alternative implementations will go long way towards this goal. ?It > will, of course, be a decision that each project will have to make. > >> >> 3. What version of Python would be allowed for use in the stdlib? I would >> like the stdlib for 3.x to be able to use 3.x code. This would be only a >> minor concern for CPython as long as 2.7 is maintained, but a major concern >> for the other implementation currently 'stuck' in 2.x only. A good 3to2 >> would be needed. > > Availability of python equivalents will hopefully help ?"other > implementation currently 'stuck' in 2.x only" to get "unstuck" and > move to 3.x. ? I understand that this is a somewhat sensitive issue at > the moment, but I believe a decision has been made supporting new > features for 2.x is outside of python-dev focus. [the rest snipped for now] I agree with Alexander's responses. Brett can chime in here too, and so can Frank W. or any of the other people who were involved in the conversation. Essentially, many of us agreed "one stdlib to bind them", from a canonical repository would help everyone involved. Any modules which were specific to the implementation - such as multiprocessing would either be flagged as such or not included in the shared repo (TBD). This effort has been on hold largely due to the fact we're waiting on the mercurial migration. It's not something I think any of us would want to do prior to that, and requires a fair amount of scaffolding / build tools /etc to make it a net win. Below, you will find the partially completed draft PEP (from a private mercurial repo) Brett/Frank and I had worked on (but again, paused due to mercurial/etc). Now that we're edging closer to 3.2 (this would not happen before then) and mercurial, I think we might need to find the time to finish the PEP: PEP: XXXX Title: Making the Standard Library a Separate Project Version: $Revision: 65628 $ Last-Modified: $Date: 2008-08-10 06:59:20 -0700 (Sun, 10 Aug 2008) $ Author: XXX Status: Draft Type: Process Content-Type: text/x-rst Created: 14-Aug-2009 Post-History: .. warning:: This PEP will not be submitted until the migration of CPython to Mercurial occurs. Abstract ======== XXX Rationale ========= Although the C implementation of Python (CPython) is the original and reference implementation of the Python language, there are now a number of additional implementations that are widely used and reasonably complete implementations. Among these implementations are Jython_, IronPython_, and PyPy_. At `PyCon 2009`_, representatives of multiple implementations of Python agreed that it would be a good idea to divide the Python Standard Library into two logical components, the first being a shared library that is essential for an implementation of Python to be considered a full implementation. All Python implementations would share this library on equal terms. The second library would be an implementation-specific standard library for things that are either implementation details for a specific VM or that depend on internals of each implementation (for example, if part of the implementation must be written in C for CPython or written in Java for Jython). The test suite should be similarly exposed and shared between all implementations on equal terms: one set of tests that must pass to be considered a full implementation, and one set of implementation-specific tests layered on top of the shared test suite (think garbage collection vs refcounting, etc). The same pattern should apply to documentation as well. The idea is to put CPython on a more equal footing with the other implementations, and to remove the need to have Jython, IronPython or PyPy specific cases in the CPython standard library. Criteria for Inclusion/Exclusion of Code ========================================= To be included in the shared library, a module must have a pure Python implementation. If the module also has a native implementation, the identical unit tests must pass in both the pure and native versions. The modules must not use any features that are considered implementation-dependent, and must only depend on other modules within the shared library unless specifically listed below (whose tests are included in the shared library): XXX: need to specify the subsets of the sys and os module that are required. XXX: also include modules that have no pure Python implementation but are still expected to be included (e.g. datetime) * sys * os Modules ------- XXX: maybe we shouldn't list all the modules to include but instead only the module to *exclude*; list is rather long and only listed what to remove gets the point across that most things will make the transition Modules to Exclude '''''''''''''''''' XXX Python 2.6 or 2.7? Intra-Module Objects to Exclude ''''''''''''''''''''''''''''''' XXX E.g. sys._getframe() Documentation Notation ---------------------- XXX How to document a module is CPython-specific in Sphinx Documentation ============= XXX Language docs, shared library, PEPs Code Layout =========== XXX Copyright ========= This document has been placed in the public domain. .. _Jython: http://www.jython.org./ .. _IronPython: http://www.codeplex.com/IronPython .. _PyPy: http://codespeak.net/pypy/ .. _PyCon 2009: http://us.pycon.org/2009/about/ .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: jesse From ctb at msu.edu Wed Jul 7 17:54:00 2010 From: ctb at msu.edu (C. Titus Brown) Date: Wed, 7 Jul 2010 08:54:00 -0700 Subject: [Python-Dev] query: docstring formatting in python distutils code Message-ID: <20100707155358.GB26419@idyll.org> Hi all, over on the fellowship o' the packaging mailing list, one of our GSoC students (merwok) asked about how much formatting info should go into Python stdlib docstrings. Right now the stdlib docstrings are primarily text, AFAIK; but with the switch to Sphinx for the official Python docs, should we permit ReST-general and/or Sphinx-specific markup in docstrings? Hmm, I don't actually see that the stdlib docstrings are imported into the Python documentation anywhere, so maybe the use of Sphinx isn't that relevant. But how about ReST in general? See http://sphinx.pocoo.org/markup/index.html for sphinx-specific markup constructs. thanks, --titus -- C. Titus Brown, ctb at msu.edu From anand.shashwat at gmail.com Wed Jul 7 18:06:10 2010 From: anand.shashwat at gmail.com (Shashwat Anand) Date: Wed, 7 Jul 2010 21:36:10 +0530 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: <20100707155358.GB26419@idyll.org> References: <20100707155358.GB26419@idyll.org> Message-ID: On Wed, Jul 7, 2010 at 9:24 PM, C. Titus Brown wrote: > Hi all, > > over on the fellowship o' the packaging mailing list, one of our GSoC > students > (merwok) asked about how much formatting info should go into Python stdlib > docstrings. Right now the stdlib docstrings are primarily text, AFAIK; but > with the switch to Sphinx for the official Python docs, should we permit > ReST-general and/or Sphinx-specific markup in docstrings? > > Hmm, I don't actually see that the stdlib docstrings are imported into the > Python documentation anywhere, so maybe the use of Sphinx isn't that > relevant. But how about ReST in general? > So will we be able to use .__docs__ within python interpretor, which is quite handy feature. >>> print(os.getcwd.__doc__) getcwd() -> path Return a string representing the current working directory. Also some python interpretors like bpython uses it ; a snapshot here - h ttp://cl.ly/c5bb3be4a01d9d44732f So will it be ok to break them ? > > See > > http://sphinx.pocoo.org/markup/index.html > > for sphinx-specific markup constructs. > > thanks, > --titus > -- > C. Titus Brown, ctb at msu.edu > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ctb at msu.edu Wed Jul 7 18:08:24 2010 From: ctb at msu.edu (C. Titus Brown) Date: Wed, 7 Jul 2010 09:08:24 -0700 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> Message-ID: <20100707160824.GB31746@idyll.org> On Wed, Jul 07, 2010 at 09:36:10PM +0530, Shashwat Anand wrote: > On Wed, Jul 7, 2010 at 9:24 PM, C. Titus Brown wrote: > > > Hi all, > > > > over on the fellowship o' the packaging mailing list, one of our GSoC > > students > > (merwok) asked about how much formatting info should go into Python stdlib > > docstrings. Right now the stdlib docstrings are primarily text, AFAIK; but > > with the switch to Sphinx for the official Python docs, should we permit > > ReST-general and/or Sphinx-specific markup in docstrings? > > > > Hmm, I don't actually see that the stdlib docstrings are imported into the > > Python documentation anywhere, so maybe the use of Sphinx isn't that > > relevant. But how about ReST in general? > > So will we be able to use .__docs__ within python interpretor, which is > quite handy feature. > >>> print(os.getcwd.__doc__) > getcwd() -> path > > Return a string representing the current working directory. > Also some python interpretors like bpython uses it ; a snapshot here - h > ttp://cl.ly/c5bb3be4a01d9d44732f > So will it be ok to break them ? I don't understand... Frist, you can already use help(os.getcwd) to get the same result. Second, what would we be breaking? We'd be making the straight text representation a bit more cluttered in return for adding certain kinds of meta-information into the markup. I think it's a judgement call... cheers, --titus -- C. Titus Brown, ctb at msu.edu From fuzzyman at voidspace.org.uk Wed Jul 7 18:09:40 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 07 Jul 2010 17:09:40 +0100 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> Message-ID: <4C34A6C4.2030205@voidspace.org.uk> On 07/07/2010 17:06, Shashwat Anand wrote: > > > On Wed, Jul 7, 2010 at 9:24 PM, C. Titus Brown > wrote: > > Hi all, > > over on the fellowship o' the packaging mailing list, one of our > GSoC students > (merwok) asked about how much formatting info should go into > Python stdlib > docstrings. Right now the stdlib docstrings are primarily text, > AFAIK; but > with the switch to Sphinx for the official Python docs, should we > permit > ReST-general and/or Sphinx-specific markup in docstrings? > > Hmm, I don't actually see that the stdlib docstrings are imported > into the > Python documentation anywhere, so maybe the use of Sphinx isn't that > relevant. But how about ReST in general? > > > So will we be able to use .__docs__ within python interpretor, which > is quite handy feature. > >>> print(os.getcwd.__doc__) > getcwd() -> path > > Return a string representing the current working directory. > Also some python interpretors like bpython uses it ; a snapshot here - > http://cl.ly/c5bb3be4a01d9d44732f > So will it be ok to break them ? Using ReST won't *break* these tools, but may make the output less readable. I would say that the major use of docstrings is for interactive help - so interactive readability should be *the most important* (but perhaps not only) factor when considering how to format standard library docstrings. Michael > > See > > http://sphinx.pocoo.org/markup/index.html > > for sphinx-specific markup constructs. > > thanks, > --titus > -- > C. Titus Brown, ctb at msu.edu > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ctb at msu.edu Wed Jul 7 18:12:06 2010 From: ctb at msu.edu (C. Titus Brown) Date: Wed, 7 Jul 2010 09:12:06 -0700 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: <4C34A6C4.2030205@voidspace.org.uk> References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> Message-ID: <20100707161206.GC1701@idyll.org> On Wed, Jul 07, 2010 at 05:09:40PM +0100, Michael Foord wrote: > On 07/07/2010 17:06, Shashwat Anand wrote: >> On Wed, Jul 7, 2010 at 9:24 PM, C. Titus Brown > > wrote: >> >> Hi all, >> >> over on the fellowship o' the packaging mailing list, one of our >> GSoC students >> (merwok) asked about how much formatting info should go into >> Python stdlib >> docstrings. Right now the stdlib docstrings are primarily text, >> AFAIK; but >> with the switch to Sphinx for the official Python docs, should we >> permit >> ReST-general and/or Sphinx-specific markup in docstrings? >> >> Hmm, I don't actually see that the stdlib docstrings are imported >> into the >> Python documentation anywhere, so maybe the use of Sphinx isn't that >> relevant. But how about ReST in general? >> >> >> So will we be able to use .__docs__ within python interpretor, which >> is quite handy feature. >> >>> print(os.getcwd.__doc__) >> getcwd() -> path >> >> Return a string representing the current working directory. >> Also some python interpretors like bpython uses it ; a snapshot here - >> http://cl.ly/c5bb3be4a01d9d44732f >> So will it be ok to break them ? > > Using ReST won't *break* these tools, but may make the output less > readable. > > I would say that the major use of docstrings is for interactive help - > so interactive readability should be *the most important* (but perhaps > not only) factor when considering how to format standard library > docstrings. OK. I guess docutils isn't in the stdlib (should it be?) or else we could modify 'help' to use it to prepare a straight text formatting. cheers, --titus -- C. Titus Brown, ctb at msu.edu From anand.shashwat at gmail.com Wed Jul 7 18:15:10 2010 From: anand.shashwat at gmail.com (Shashwat Anand) Date: Wed, 7 Jul 2010 21:45:10 +0530 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: <4C34A6C4.2030205@voidspace.org.uk> References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> Message-ID: On Wed, Jul 7, 2010 at 9:39 PM, Michael Foord wrote: > On 07/07/2010 17:06, Shashwat Anand wrote: > > > > On Wed, Jul 7, 2010 at 9:24 PM, C. Titus Brown wrote: > >> Hi all, >> >> over on the fellowship o' the packaging mailing list, one of our GSoC >> students >> (merwok) asked about how much formatting info should go into Python stdlib >> docstrings. Right now the stdlib docstrings are primarily text, AFAIK; >> but >> with the switch to Sphinx for the official Python docs, should we permit >> ReST-general and/or Sphinx-specific markup in docstrings? >> >> Hmm, I don't actually see that the stdlib docstrings are imported into the >> Python documentation anywhere, so maybe the use of Sphinx isn't that >> relevant. But how about ReST in general? >> > > So will we be able to use .__docs__ within python interpretor, which is > quite handy feature. > >>> print(os.getcwd.__doc__) > getcwd() -> path > > Return a string representing the current working directory. > Also some python interpretors like bpython uses it ; a snapshot here - h > ttp://cl.ly/c5bb3be4a01d9d44732f > So will it be ok to break them ? > > > Using ReST won't *break* these tools, but may make the output less > readable. > Oops. Sorry for the wrong choice of word. I meant the 'output will be less readable', text are perhaps easier to read than ReST, thats what I meant. > > I would say that the major use of docstrings is for interactive help - so > interactive readability should be *the most important* (but perhaps not > only) factor when considering how to format standard library docstrings. > > Michael > > > >> >> See >> >> http://sphinx.pocoo.org/markup/index.html >> >> for sphinx-specific markup constructs. >> >> thanks, >> --titus >> -- >> C. Titus Brown, ctb at msu.edu >> >> > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.orghttp://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > > > -- http://www.ironpythoninaction.com/http://www.voidspace.org.uk/blog > > READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.peters at gmail.com Wed Jul 7 18:44:52 2010 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 7 Jul 2010 12:44:52 -0400 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: References: Message-ID: [Nick Coghlan] > ... > Hmm, I've been using difflib.SequenceMatcher for years in a serial bit > error rate tester (with typical message sizes ranging from tens of > bytes to tens of thousands of bytes) that occasionally gives > unexpected results. I'd been blaming hardware glitches (and, to be > fair, all of the odd results I can recall off the top of my head were > definitively traced to problems in the hardware under test), but I > should probably check I'm not running afoul of this bug. That would be prudent ;-) > And Tim, the algorithm may not be optimal as a general purpose binary > diff algorithm, but it's still a hell of a lot faster than the > hardware I use it to test. Compared to the equipment configuration > times, the data comparison time is trivial. I'm all in favor of people using the class for any purpose they find useful. Just saying the overwhelmingly most common use is for comparing text files, and that's important - most important, since most widely used. > There's another possibility here - perhaps the heuristic should be off > by default in SequenceMatcher, with a TextMatcher subclass that > enables it (and Differ and HtmlDiff then inheriting from the latter)? Or, to make life easiest for the most common uses, create a subclass that _didn't_ have any notion of "junk" whatsoever. Or a new flag to turn auto-junk heuristics off. Unfortunately, no true solution exists without changing _something_ in the API, and since the behavior changed 8 years ago there's just no guessing how many uses rely on the by-now-long-current behavior. > There's currently barely anything in the SequenceMatcher documentation > to indicate that it is designed primarily for comparing text rather > than arbitrary sequences (the closest it gets is the reference to > Ratcliff/Obserhelp gestalt pattern matching and then the link to the > Ratcliff/Metzener Dr Dobb's article - and until this thread, I'd never > followed the article link). It's designed to compare sequences of hashable elements. The use cases that _drove_ the implementation were (1) viewing a file as a sequence of lines; and (2), viewing a line as a sequence of characters. Use cases always drive implementation (although rarely mentioned the docs), but striving for natural generalization sometimes pays off. I expected it would in this case, and that others have found unanticipated uses confirms that it did. Unfortunately, I screwed up by not finishing what I started 8 years ago (adding an auto-junk heuristic that was very valuable in the primary use case, but turned out to have very bad effects in some other cases). > Rather than reverting to Tim's undocumented vision, perhaps we should > better articulate it by separating the general purpose matcher from > an optimised text matcher. Having a notion of junk can improve the quality of results (in the sense of making them more intuitive to humans, which was an explicit goal of the module), and can yield enormous speedups. Is this restricted solely to comparing text files? I don't know that to be the case, but offhand doubt it's true. As always, if they exist, we won't hear from people with other use cases who never noticed the change (except perhaps to see improved speed and better results) - not until we "turn off" the heuristic on them, and then they'll view _that_ as "a bug". From g.brandl at gmx.net Wed Jul 7 19:30:49 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 07 Jul 2010 19:30:49 +0200 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: <4C34A6C4.2030205@voidspace.org.uk> References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> Message-ID: Am 07.07.2010 18:09, schrieb Michael Foord: >> Hi all, >> >> over on the fellowship o' the packaging mailing list, one of our GSoC students >> (merwok) asked about how much formatting info should go into Python stdlib >> docstrings. Right now the stdlib docstrings are primarily text, AFAIK; but >> with the switch to Sphinx for the official Python docs, should we permit >> ReST-general and/or Sphinx-specific markup in docstrings? I promised to write a PEP about that some time in the future. (Probably after 3.2 final.) >> Hmm, I don't actually see that the stdlib docstrings are imported into the >> Python documentation anywhere, so maybe the use of Sphinx isn't that >> relevant. But how about ReST in general? >> >> >> So will we be able to use .__docs__ within python interpretor, which is quite >> handy feature. >> >>> print(os.getcwd.__doc__) >> getcwd() -> path >> >> Return a string representing the current working directory. >> Also some python interpretors like bpython uses it ; a snapshot here - >> http://cl.ly/c5bb3be4a01d9d44732f >> So will it be ok to break them ? > > Using ReST won't *break* these tools, but may make the output less readable. > > I would say that the major use of docstrings is for interactive help - so > interactive readability should be *the most important* (but perhaps not only) > factor when considering how to format standard library docstrings. Agreed. However, reST doesn't need to be less readable if the specific inline markup is not used. For example, using `identifier` to refer to a function or *var* to refer to a variable (which is already done at quite a few places) is very readable IMO. Using ``code`` also isn't bad, considering that double quotes are not much different and potentially ambiguous. Overall, I think that we can make stdlib docstrings valid reST -- even if it's reST without much markup -- but valid, so that people pulling in stdlib doc- strings into Sphinx docs won't get ugly warnings. What I would *not* like to see is heavy markup and Sphinx specifics -- that would only make sense if we included the docstrings in the docs, and I don't see that coming. cheers, Georg From orsenthil at gmail.com Wed Jul 7 19:36:58 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Wed, 7 Jul 2010 23:06:58 +0530 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> Message-ID: On Wed, Jul 7, 2010 at 11:00 PM, Georg Brandl wrote: > Agreed. ?However, reST doesn't need to be less readable if the specific > inline markup is not used. ?For example, using `identifier` to refer to a > function or *var* to refer to a variable (which is already done at quite a > few places) is very readable IMO. ?Using ``code`` also isn't bad, considering > that double quotes are not much different and potentially ambiguous. What are the specific advantages that you see? Can it more useful in some cases than the other? -- Senthil From guido at python.org Wed Jul 7 19:44:19 2010 From: guido at python.org (Guido van Rossum) Date: Wed, 7 Jul 2010 19:44:19 +0200 Subject: [Python-Dev] Licensing In-Reply-To: References: <4C339AA7.3040204@udel.edu> Message-ID: On Wed, Jul 7, 2010 at 2:48 PM, Nick Coghlan wrote: > On Wed, Jul 7, 2010 at 2:59 PM, Guido van Rossum wrote: >> On Tue, Jul 6, 2010 at 11:27 PM, Nick Coghlan wrote: >>> For example, if you look at some of the code that even Guido has >>> submitted (e.g. pgen2), that's actually come in under Google's >>> contributor agreement, rather than Guido's personal one. Presumably >>> that was work he did on company time, so the copyright actually rests >>> with Google rather than Guido. >> >> I hope you are misremembering some details. I did that work while at >> Elemental Security (i.e. before I joined Google). It should have >> Elemental Security's contributor agreement. I developed that code >> initially for inclusion in Elemental's product line (as part of a >> parser for a domain-specific language named "Fuel" which did not get >> open-sourced -- probably for the better. > > Whoops, I got my timeline wrong (it did seem a little off when I wrote > it - I think part of my brain was trying to tell me the dates didn't > match up). I must have been thinking of something else I was working > on recently that had Google's name in the header, most likely the abc > module. Yeah, that, and anything I contributed *after* pgen2. > So apologies for the confusion - just s/pgen2/abc/ in my example to > make it line up with my intent :) No problem! Just setting the record straight. -- --Guido van Rossum (python.org/~guido) From eliben at gmail.com Wed Jul 7 19:44:31 2010 From: eliben at gmail.com (Eli Bendersky) Date: Wed, 7 Jul 2010 19:44:31 +0200 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: References: Message-ID: > Rather than reverting to Tim's > undocumented vision, perhaps we should better articulate it by > separating the general purpose matcher from an optimised text matcher. > For what it's worth, my benchmarking showed that modifying the heuristic to only kick in when there are more than 100 kinds of elements (Terry's option A) didn't affect the runtime of matching whatsoever, even when the heuristic *does* kick in. All it adds, really, is the overhead of a single 'if' statement. So it wouldn't be right to assume that somehow modifying the heuristic or allowing to turn it off will negatively affect performance in the special case Tim originally optimized for. Eli From merwok at netwok.org Wed Jul 7 19:53:48 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Wed, 07 Jul 2010 19:53:48 +0200 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> Message-ID: <4C34BF2C.8090600@netwok.org> > I promised to write a PEP about that some time in the future. (Probably after > 3.2 final.) Nice. It seems that projects putting Sphinxy reST in their doc are using automatic doc generation. This is however not always the best way to make good doc, as demonstrated by Python?s hand-written, very-high-quality documentation. > Agreed. However, reST doesn't need to be less readable if the specific > inline markup is not used. For example, using `identifier` to refer to a > function or *var* to refer to a variable (which is already done at quite a > few places) is very readable IMO. Using ``code`` also isn't bad, considering > that double quotes are not much different and potentially ambiguous. > > Overall, I think that we can make stdlib docstrings valid reST -- even if it's > reST without much markup -- but valid, so that people pulling in stdlib doc- > strings into Sphinx docs won't get ugly warnings. > > What I would *not* like to see is heavy markup and Sphinx specifics -- that > would only make sense if we included the docstrings in the docs, and I don't > see that coming. Clear answer, thanks! We have backported some modules in distutils2, and some docstrings already contain Sphinxy reST (e.g. :param: and :pep:), it?s good to know now that we shouldn?t put hours into that to see it reverted later. Regards From barry at python.org Wed Jul 7 20:12:17 2010 From: barry at python.org (Barry Warsaw) Date: Wed, 7 Jul 2010 14:12:17 -0400 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> Message-ID: <20100707141217.377d6a93@heresy> On Jul 07, 2010, at 07:30 PM, Georg Brandl wrote: >Overall, I think that we can make stdlib docstrings valid reST -- even >if it's reST without much markup -- but valid, so that people pulling >in stdlib doc- strings into Sphinx docs won't get ugly warnings. > >What I would *not* like to see is heavy markup and Sphinx specifics -- >that would only make sense if we included the docstrings in the docs, >and I don't see that coming. Does it make sense to add (reST-style) epydoc markup for API signatures? E.g. def create_foo(name, parent=None): """Create the named foo. The named foo must not already exist, but if optional `parent` is given, it must exist. :param name: The name of the new foo. :type name: string :param parent: The new foo's parent. If given, this must exist. :type parent: string :return: The new foo. :rtype: `Foo` :raises BadFooNameError: when `name` is illegal. :raises FooAlreadyExistsError: when a foo with `name` already exists. :raises BadParentError: when the foo's parent does not exist. """ We could then generate automatic API docs from this, a la: http://www.blender.org/documentation/248PythonDoc/ -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From robert.kern at gmail.com Wed Jul 7 20:26:50 2010 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 07 Jul 2010 14:26:50 -0400 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: <4C34BF2C.8090600@netwok.org> References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <4C34BF2C.8090600@netwok.org> Message-ID: On 7/7/10 1:53 PM, ?ric Araujo wrote: >> I promised to write a PEP about that some time in the future. (Probably after >> 3.2 final.) > > Nice. > > It seems that projects putting Sphinxy reST in their doc are using > automatic doc generation. This is however not always the best way to > make good doc, as demonstrated by Python?s hand-written, > very-high-quality documentation. This is a false dichotomy. Many of those projects using Sphinxy reST in their docstrings are using the semi-automatic[1] doc generation provided by Sphinx to construct *part* of their documentation. Namely, the reference of functions, classes and methods. A large part of Python's library reference consists of exactly this. Having a function's docstring provide the content for its entry in the library reference has the substantial DRY benefit of having exactly one source for the comprehensive documentation of that function available from both help() and the manual. As someone who uses the interactive prompt to play with things and read docstrings intensively, I would really like to see docstrings providing the same information as the manual. Of course, opinions differ about how comprehensive docstrings should be compared to the reference manual's entries. And there are technical reasons for not wanting to try to extract docstrings from code (e.g. platform-specific modules). But one should not fear that the quality of the manual would decline. [1] That's the really nice thing about Sphinx's docstring extraction features in contrast with other such tools. It doesn't generate a manual from the docstrings; it makes you explicitly reference the docstrings into the manual's text. This would fit in very naturally with Python's library reference. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From g.brandl at gmx.net Wed Jul 7 20:27:32 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 07 Jul 2010 20:27:32 +0200 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: <4C34BF2C.8090600@netwok.org> References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <4C34BF2C.8090600@netwok.org> Message-ID: Am 07.07.2010 19:53, schrieb ?ric Araujo: >> I promised to write a PEP about that some time in the future. (Probably after >> 3.2 final.) > > Nice. > > It seems that projects putting Sphinxy reST in their doc are using > automatic doc generation. This is however not always the best way to > make good doc, as demonstrated by Python?s hand-written, > very-high-quality documentation. I know, and this is what I originally intended for Sphinx. However, the calls for automatic doc generation are very loud, and it's understandable that most project can't afford writing their documentation twice. >> Agreed. However, reST doesn't need to be less readable if the specific >> inline markup is not used. For example, using `identifier` to refer to a >> function or *var* to refer to a variable (which is already done at quite a >> few places) is very readable IMO. Using ``code`` also isn't bad, considering >> that double quotes are not much different and potentially ambiguous. >> >> Overall, I think that we can make stdlib docstrings valid reST -- even if it's >> reST without much markup -- but valid, so that people pulling in stdlib doc- >> strings into Sphinx docs won't get ugly warnings. >> >> What I would *not* like to see is heavy markup and Sphinx specifics -- that >> would only make sense if we included the docstrings in the docs, and I don't >> see that coming. > > Clear answer, thanks! We have backported some modules in distutils2, and > some docstrings already contain Sphinxy reST (e.g. :param: and :pep:), > it?s good to know now that we shouldn?t put hours into that to see it > reverted later. :pep: isn't Sphinxy, put provided by docutils, and the :param foo: field lists are also valid reST, if rendered a bit awkwardly without the transforms that Sphinx does to display them in epydoc style. cheers, Georg From alexander.belopolsky at gmail.com Wed Jul 7 20:37:33 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 7 Jul 2010 14:37:33 -0400 Subject: [Python-Dev] Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On Tue, Jul 6, 2010 at 11:54 PM, Terry Reedy wrote: > On 7/6/2010 3:59 PM, Alexander Belopolsky wrote: [.. skipping more general stdlib discussion see "Python equivalents in stdlib" thread ..] >> 2. There are other areas of stdlib that can benefit more from pure >> python equivalents. > > Possibly true, but developers do what they do, and this seems mostly done. > The reason I want to do datetime module is that there are some long-standing bugs/RFEs that would require some experimentation to get it right. Such experimentation is unfeasible in C where more effort goes into thinking about integer overflow and reference counting than into actual design. Here are some of those issues: http://bugs.python.org/issue5516 = equality not symmetric for subclasses of datetime.date and datetime.datetime http://bugs.python.org/issue2736 = datetime needs an "epoch" method http://bugs.python.org/issue7584 = datetime.rfcformat() for Date and Time on the Internet http://bugs.python.org/issue1100942 = Add datetime.time.strptime and datetime.date.strptime http://bugs.python.org/issue8860 = Rounding in timedelta constructor is inconsistent with that in timedelta arithmetics http://bugs.python.org/issue1647654 = No obvious and correct way to get the time zone offset http://bugs.python.org/issue5288 = tzinfo objects with sub-minute offsets are not supported (e.g. UTC+05:53:28) http://bugs.python.org/issue1982 = Feature: extend strftime to accept milliseconds >> 3. Reference implementations should be written by a senior CPython >> developer and not scraped from external projects like PyPy. > > I did not see that in my reading of the thread. > This POV was brought up in the #python-dev IRC channel. > In any case, what matters is quality, not authorship. I completely agree and I think the sooner the python code gets into the main tree the more reviews it will get before the next release. From solipsis at pitrou.net Wed Jul 7 20:40:08 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 7 Jul 2010 20:40:08 +0200 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken References: Message-ID: <20100707204008.37f8e333@pitrou.net> On Wed, 7 Jul 2010 19:44:31 +0200 Eli Bendersky wrote: > > For what it's worth, my benchmarking showed that modifying the > heuristic to only kick in when there are more than 100 kinds of > elements (Terry's option A) didn't affect the runtime of matching > whatsoever, even when the heuristic *does* kick in. All it adds, > really, is the overhead of a single 'if' statement. So it wouldn't be > right to assume that somehow modifying the heuristic or allowing to > turn it off will negatively affect performance in the special case Tim > originally optimized for. Just because it doesn't affect performance in your tests doesn't mean it won't do so in the general case. Consider a case where Tim's junk optimization kicked in and helped improve performance a lot, but where there are still less than 100 alphabet symbols. The new heuristic will ruin this use case. That's why I'm advocating a dedicated flag instead. From barry at python.org Wed Jul 7 20:40:59 2010 From: barry at python.org (Barry Warsaw) Date: Wed, 7 Jul 2010 14:40:59 -0400 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <4C2C75C1.8020708@scottdial.com> References: <20100624115048.4fd152e3@heresy> <20100624170944.7e68ad21@heresy> <4C23D3C2.1060500@scottdial.com> <4C246E81.3020302@scottdial.com> <4C265DC6.4080600@ubuntu.com> <4C268433.30405@scottdial.com> <20100630145329.736f2aab@heresy> <4C2C75C1.8020708@scottdial.com> Message-ID: <20100707144059.07bae9f2@heresy> On Jul 01, 2010, at 07:02 AM, Scott Dial wrote: >I decided to prove to myself that it was not a significant issue to >have parallel directory structures in a .tar.bz2, and I was surprised >to find it much worse at that then I had imagined. For example, > ># cd /usr/lib/python2.6/site-packages ># tar --exclude="*.pyc" --exclude="*.pyo" \ > -cjf mercurial.tar.bz2 mercurial ># du -h mercurial.tar.bz2 >640K mercurial.tar.bz2 > ># cp -a mercurial mercurial2 ># tar --exclude="*.pyc" --exclude="*.pyo" \ > -cjf mercurial2.tar.bz2 mercurial mercurial2 ># du -h mercurial.tar.bz2 >1.3M mercurial2.tar.bz2 > >So, I was definitely wrong in saying that you do better than doubling. [...] >I appreciate all your replies. I am not sure a PEP is really needed >here, but to having had all of this discussed and explained on the >mailing list is certainly useful. I trust that yourself and the debuntu >python group will end up chasing down and taking care of any quirks >that this change might cause, so I am not worried about it. :D Getting back to this after the US holiday. Thanks for running these numbers Scott. I've opened a bug in the Python tracker and attached my latest patch: http://bugs.python.org/issue9193 The one difference from previous versions of the patch is that the .so tag is now settable via "./configure --with-so-abi-tag=foo". This would generate shared libs like _multiprocessing.foo.so. I'd like to get consensus as to whether folks feel that a PEP is needed. My own thought is that I'd rather not do a PEP specific to this change, but I would update PEP 384 with the implications on .so versioning. Please also feel free to review the patch in that issue. Thanks, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From solipsis at pitrou.net Wed Jul 7 20:42:46 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 7 Jul 2010 20:42:46 +0200 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? References: <4C349FBA.9040800@voidspace.org.uk> Message-ID: <20100707204246.2dc3d0e0@pitrou.net> On Wed, 07 Jul 2010 16:39:38 +0100 Michael Foord wrote: > On 07/07/2010 16:29, Alexander Belopolsky wrote: > > [snip...] > > > >> 4. Does not ctypes make it possible to replace a method of a Python-coded > >> class with a faster C version, with something like > >> try: > >> connect to methods.dll > >> check that function xyx exists > >> replace Someclass.xyy with ctypes wrapper > >> except: pass > >> For instance, the SequenceMatcher heuristic was added to speedup the > >> matching process that I believe is encapsulated in one O(n**2) or so > >> bottleneck method. I believe most everything else is O(n) bookkeeping. > >> > >> > > The ctypes modules is very CPython centric as far as I know. For the > > new modules, this may be a valid way to rapidly develop accelerated > > versions. For modules that are already written in C, I don't see > > much benefit in replacing them with ctypes wrappers. > > Nope, both IronPython and PyPy have ctypes implementations and Jython is > in the process of "growing" one. Using ctypes for C extensions is the > most portable way of providing C extensions for Python (other than > providing a pure-Python implementation of course). Except that ctypes doesn't help provide C extensions at all. It only helps provide wrappers around existing C libraries, which is quite a different thing. Which, in the end, makes the original suggestion meaningless. Regards Antoine. From alexander.belopolsky at gmail.com Wed Jul 7 20:46:49 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 7 Jul 2010 14:46:49 -0400 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: <20100707141217.377d6a93@heresy> References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707141217.377d6a93@heresy> Message-ID: On Wed, Jul 7, 2010 at 2:12 PM, Barry Warsaw wrote: .. > Does it make sense to add (reST-style) epydoc markup for API signatures? > E.g. > > def create_foo(name, parent=None): > ? ?"""Create the named foo. > > ? ?The named foo must not already exist, but if optional `parent` is given, > ? ?it must exist. > > ? ?:param name: The name of the new foo. > ? ?:type name: string .. -1. Repeating the function signature in the docstring only adds clutter and Java-style formal types/exceptions specifications are rarely appropriate in Python. I think marking arguments up with * as in *name*, *parent*, should be enough in most cases. From solipsis at pitrou.net Wed Jul 7 20:46:11 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 7 Jul 2010 20:46:11 +0200 Subject: [Python-Dev] query: docstring formatting in python distutils code References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707141217.377d6a93@heresy> Message-ID: <20100707204611.78153447@pitrou.net> On Wed, 7 Jul 2010 14:12:17 -0400 Barry Warsaw wrote: > On Jul 07, 2010, at 07:30 PM, Georg Brandl wrote: > > >Overall, I think that we can make stdlib docstrings valid reST -- even > >if it's reST without much markup -- but valid, so that people pulling > >in stdlib doc- strings into Sphinx docs won't get ugly warnings. > > > >What I would *not* like to see is heavy markup and Sphinx specifics -- > >that would only make sense if we included the docstrings in the docs, > >and I don't see that coming. > > Does it make sense to add (reST-style) epydoc markup for API signatures? > E.g. It really looks ugly (and annoying to decipher) when viewed in plain text. Regards Antoine. > > def create_foo(name, parent=None): > """Create the named foo. > > The named foo must not already exist, but if optional `parent` is given, > it must exist. > > :param name: The name of the new foo. > :type name: string > :param parent: The new foo's parent. If given, this must exist. > :type parent: string > :return: The new foo. > :rtype: `Foo` > :raises BadFooNameError: when `name` is illegal. > :raises FooAlreadyExistsError: when a foo with `name` already exists. > :raises BadParentError: when the foo's parent does not exist. > """ > > We could then generate automatic API docs from this, a la: > > http://www.blender.org/documentation/248PythonDoc/ > > -Barry > From alexander.belopolsky at gmail.com Wed Jul 7 20:59:17 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 7 Jul 2010 14:59:17 -0400 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: <20100707204246.2dc3d0e0@pitrou.net> References: <4C349FBA.9040800@voidspace.org.uk> <20100707204246.2dc3d0e0@pitrou.net> Message-ID: On Wed, Jul 7, 2010 at 2:42 PM, Antoine Pitrou wrote: .. > Except that ctypes doesn't help provide C extensions at all. It only > helps provide wrappers around existing C libraries, which is quite a > different thing. Yet it may allow writing an equivalent of a C extension in pure python. For example posix or time modules could be easily reimplemented that way if the libc was less platform dependent. Such reimplementation, however is unlikely to be very useful. > Which, in the end, makes the original suggestion meaningless. It is not meaningless, but would require effectively exposing pyport.h in pure python and using it to select the correct signature for a given library function. This would be a tremendous effort and is hardly justified. From fdrake at acm.org Wed Jul 7 21:11:41 2010 From: fdrake at acm.org (Fred Drake) Date: Wed, 7 Jul 2010 15:11:41 -0400 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <4C34BF2C.8090600@netwok.org> Message-ID: On Wed, Jul 7, 2010 at 2:27 PM, Georg Brandl wrote: > I know, and this is what I originally intended for Sphinx. ?However, the calls > for automatic doc generation are very loud, and it's understandable that most > project can't afford writing their documentation twice. The ability to provide extended content beyond what's provided in the docstring using the auto* constructs may make it feasible to start avoiding some of those DRY violations for Python's standard library; I'm enjoying those for another project. I hope we don't end up in a position where we can't use the auto* constructs in the Python documentation. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein From brett at python.org Wed Jul 7 21:32:01 2010 From: brett at python.org (Brett Cannon) Date: Wed, 7 Jul 2010 12:32:01 -0700 Subject: [Python-Dev] Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On Tue, Jul 6, 2010 at 20:54, Terry Reedy wrote: > On 7/6/2010 3:59 PM, Alexander Belopolsky wrote: > > I am more interested in Brett's overall vision than this particular module. > I understand that to be one of a stdlib that is separate from CPython and is > indeed the standard Python library. > > Questions: > > !. Would the other distributions use a standard stdlib rather than current > individual versions? The idea is that the stdlib just becomes a subrepo that the other VMs simply pull in to gain their version of the stdlib. > If so, and if at least one used the Python version of > each module, this would alleviate the concern that non-use == non-testing. > (Test improvement would also help this.) > > 2. Would the other distributions pool their currently separate stdlib > efforts to help maintain one standard stdlib. If so, this would alleviate > the concern about the extra effort to maintain both a C and Python version. > (Test improvement would also help this also.) That's the idea. We already have contributors from the various VMs who has commit privileges, but they all work in their own repos for convenience. My hope is that if we break the stdlib out into its own repository that people simply pull in then other VM contributors will work directly off of the stdlib repo instead of their own, magnifying the usefulness of their work. > > 3. What version of Python would be allowed for use in the stdlib? I would > like the stdlib for 3.x to be able to use 3.x code. This would be only a > minor concern for CPython as long as 2.7 is maintained, but a major concern > for the other implementation currently 'stuck' in 2.x only. A good 3to2 > would be needed. This will only affect py3k. > > I generally favor having Python versions of modules available. My current > post on difflib.SequenceMatcher is based on experiments with an altered > version. I copied difflib.py to my test directory, renamed it diff2lib.py, > so I could import both versions, found and edited the appropriate method, > and off I went. If difflib were in C, my post would have been based on > speculation about how a fixed version would operate, rather than on data. > The effect upon CPython would be the extension modules become just performance improvements, nothing more (unless they have to be in C as in the case for sqlite3). > 4. Does not ctypes make it possible to replace a method of a Python-coded > class with a faster C version, with something like > ?try: > ? ?connect to methods.dll > ? ?check that function xyx exists > ? ?replace Someclass.xyy with ctypes wrapper > ?except: pass > For instance, the SequenceMatcher heuristic was added to speedup the > matching process that I believe is encapsulated in one O(n**2) or so > bottleneck method. I believe most everything else is O(n) bookkeeping. > There is no need to go that far. All one needs to do is structure the extension code such that when the extension module is imported, it overrides key objects in the Python version. Using ctypes is just added complexity. > >> ?This proposal has brought mostly positive feedback on the tracker [2] >> with only a few objections being raised. >> >> 1. Since this does not bring any new functionality and datetime module >> is not expected to evolve, there is no need for pure python version. > > see above > >> 2. There are other areas of stdlib that can benefit more from pure >> python equivalents. > > Possibly true, but developers do what they do, and this seems mostly done. > >> 3. Reference implementations should be written by a senior CPython >> developer and not scraped from external projects like PyPy. > > I did not see that im my reading of the thread. In any case, what matters is > quality, not authorship. > >> What do you think? ?Please reply here or add a comment at >> http://bugs.python.org/issue7989. > > From scanning that and the posts here, it seems like a pep or other doc on > dual version modules would be a good idea. It should at least document how > to code the switch from python version to the x coded version and how to > test both, as discussed. Franke Wierzbicki and I started such a PEP, but we both got busy with other stuff. And since I am most likely going to be the one spearheading this on the CPython side this will most likely not move forward until I have time to get to it (which might be quite a while). -Brett > > -- > Terry Jan Reedy > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > From brett at python.org Wed Jul 7 21:45:10 2010 From: brett at python.org (Brett Cannon) Date: Wed, 7 Jul 2010 12:45:10 -0700 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On Wed, Jul 7, 2010 at 08:29, Alexander Belopolsky wrote: > On Tue, Jul 6, 2010 at 11:54 PM, Terry Reedy wrote: >> On 7/6/2010 3:59 PM, Alexander Belopolsky wrote: >> >> I am more interested in Brett's overall vision than this particular module. >> I understand that to be one of a stdlib that is separate from CPython and is >> indeed the standard Python library. >> > > I am also very much interested in the overall vision, but I would like > to keep the datetime.py thread focused, so I a going to reply to broad > questions under a separate subject. > >> Questions: >> >> 1. Would the other distributions use a standard stdlib rather than current >> individual versions? > > I certainly hope they will. ?In the ideal word, passing test.regrtest > with unmodified Lib should be *the* definition of what is called > Python. ?I understand that there is already some work underway in this > direction such as marking implementation specific tests with > appropriate decorators. > >> >> 2. Would the other distributions pool their currently separate stdlib >> efforts to help maintain one standard stdlib? > > I believe that making stdlib and test.regrtest more friendly to > alternative implementations will go long way towards this goal. ?It > will, of course, be a decision that each project will have to make. > >> >> 3. What version of Python would be allowed for use in the stdlib? I would >> like the stdlib for 3.x to be able to use 3.x code. This would be only a >> minor concern for CPython as long as 2.7 is maintained, but a major concern >> for the other implementation currently 'stuck' in 2.x only. A good 3to2 >> would be needed. > > Availability of python equivalents will hopefully help ?"other > implementation currently 'stuck' in 2.x only" to get "unstuck" and > move to 3.x. ? I understand that this is a somewhat sensitive issue at > the moment, but I believe a decision has been made supporting new > features for 2.x is outside of python-dev focus. > > >> 4. Does not ctypes make it possible to replace a method of a Python-coded >> class with a faster C version, with something like >> ?try: >> ? ?connect to methods.dll >> ? ?check that function xyx exists >> ? ?replace Someclass.xyy with ctypes wrapper >> ?except: pass >> For instance, the SequenceMatcher heuristic was added to speedup the >> matching process that I believe is encapsulated in one O(n**2) or so >> bottleneck method. I believe most everything else is O(n) bookkeeping. >> > The ctypes modules is very CPython centric as far as I know. ? For the > new modules, this may be a valid way to rapidly develop accelerated > versions. ? For modules that are already written in C, I don't see > much benefit in replacing them with ctypes wrappers. > > >> [.. datetime specific discussion skipped ..] >> From scanning that and the posts here, it seems like a pep or other doc on >> dual version modules would be a good idea. It should at least document how >> to code the switch from python version to the x coded version and how to >> test both, as discussed. >> > I am certainly not ready to write such PEP. ? I may be in a better > position to contribute to it after I gain more experience with > test_datetime.py. ? At the moment I have more questions than answers. > > For example, the established practice appears to be: > > modulename.py > > # Python code > > try: > ? ?from _modulename import * > except: > ? ?pass > > This is supposed to generate a .pyc file with no python definitions in > it if ?_modulename is available. ?The problem with datetime.py is that > it have several helper methods like _ymd2ord() that will still stay in > the module. ?Should an "else:" clause be added to clean these up? > should these methods become class or static methods as appropriate? > > The ?established practice for testing is > > py_module = support.import_fresh_module('modulename', blocked=['_modulename']) > c_module = support.import_fresh_module('modulename', fresh=['_modulename']) > > class TestDefnitions: # not a unittest.TestCase subclass > ? ? ? def test_foo(self): > ? ? ? ? ? ?self.module.foo(..) > ? ? ? ... > > class C_Test(TestDefnitions, unittest.TestCase): > ? ? ? module = c_module > > class Py_Test(TestDefnitions, unittest.TestCase): > ? ? ? module = py_module > > > For datetime.py this approach presents several problems: > > 1. replacing datetime with self.module.datetime everywhere can get > messy quickly. > 2. There are test classes defined at the test_datetime module level > that subclass from datetime classes. ?The self.module is not available > at the module level. ?These should probably be moved to setUp() > methods and attached to test case self. > 3. If #2 is resolved by moving definitions inside functions, the > classes will become unpickleable and pickle tests will break. ?Some > hackery involving injecting these classes into __main__ or module > globals may be required. So I have been thinking about this about how to possibly make this standard test scaffolding a little cleaner. I think a class decorator might do the trick. If you had all test methods take a module argument you could pass in the module that should be used to test. Then you simply rename test_* to _test_*, create test_*_(c|py), and then have those methods call their _test_* equivalents with the proper module to test. You could even make this generic by having the keyword arguments to the decorator by what the test suffix is named. The benefit of this is you don't have to define one base class and then two subclasses; you define a single test class and simply add a decorator. This addresses #1. As for #3, that I can't answer and might simply require restructuring those specific pickle tests. -Brett > > These challenges make datetime.py an interesting showcase for other > modules, so rather than writing a PEP based on abstract ideas, I think > it is better to get datetime.py integrated first and try to establish > the best practices on the way. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From brett at python.org Wed Jul 7 21:50:58 2010 From: brett at python.org (Brett Cannon) Date: Wed, 7 Jul 2010 12:50:58 -0700 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: <20100707204611.78153447@pitrou.net> References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707141217.377d6a93@heresy> <20100707204611.78153447@pitrou.net> Message-ID: On Wed, Jul 7, 2010 at 11:46, Antoine Pitrou wrote: > On Wed, 7 Jul 2010 14:12:17 -0400 > Barry Warsaw wrote: >> On Jul 07, 2010, at 07:30 PM, Georg Brandl wrote: >> >> >Overall, I think that we can make stdlib docstrings valid reST -- even >> >if it's reST without much markup -- but valid, so that people pulling >> >in stdlib doc- strings into Sphinx docs won't get ugly warnings. >> > >> >What I would *not* like to see is heavy markup and Sphinx specifics -- >> >that would only make sense if we included the docstrings in the docs, >> >and I don't see that coming. >> >> Does it make sense to add (reST-style) epydoc markup for API signatures? >> E.g. > > It really looks ugly (and annoying to decipher) when viewed in plain > text. I agree. And it is highly repetitive since the signature information is right there already. All of that info in those annotations can easily be written in paragraph form if needed and honestly would read better to my eyes. -Brett > > Regards > > Antoine. > > >> >> def create_foo(name, parent=None): >> ? ? """Create the named foo. >> >> ? ? The named foo must not already exist, but if optional `parent` is given, >> ? ? it must exist. >> >> ? ? :param name: The name of the new foo. >> ? ? :type name: string >> ? ? :param parent: The new foo's parent. ?If given, this must exist. >> ? ? :type parent: string >> ? ? :return: The new foo. >> ? ? :rtype: `Foo` >> ? ? :raises BadFooNameError: when `name` is illegal. >> ? ? :raises FooAlreadyExistsError: when a foo with `name` already exists. >> ? ? :raises BadParentError: when the foo's parent does not exist. >> ? ? """ >> >> We could then generate automatic API docs from this, a la: >> >> http://www.blender.org/documentation/248PythonDoc/ >> >> -Barry >> > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From brett at python.org Wed Jul 7 21:52:26 2010 From: brett at python.org (Brett Cannon) Date: Wed, 7 Jul 2010 12:52:26 -0700 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> Message-ID: On Wed, Jul 7, 2010 at 10:30, Georg Brandl wrote: > Am 07.07.2010 18:09, schrieb Michael Foord: >>> ? ? Hi all, >>> >>> ? ? over on the fellowship o' the packaging mailing list, one of our GSoC students >>> ? ? (merwok) asked about how much formatting info should go into Python stdlib >>> ? ? docstrings. ?Right now the stdlib docstrings are primarily text, AFAIK; but >>> ? ? with the switch to Sphinx for the official Python docs, should we permit >>> ? ? ReST-general and/or Sphinx-specific markup in docstrings? > > I promised to write a PEP about that some time in the future. ?(Probably after > 3.2 final.) For those of you who aren't aware, there actually already is a PEP on using reST in docstrings: http://python.org/dev/peps/pep-0287/ . But it could stand to be updated by Georg with current practice with our internal doc practices as 2002 was back when we were still using LaTeX. -Brett > >>> ? ? Hmm, I don't actually see that the stdlib docstrings are imported into the >>> ? ? Python documentation anywhere, so maybe the use of Sphinx isn't that >>> ? ? relevant. ?But how about ReST in general? >>> >>> >>> So will we be able to use .__docs__ within python interpretor, which is quite >>> handy feature. >>> >>> print(os.getcwd.__doc__) >>> getcwd() -> path >>> >>> Return a string representing the current working directory. >>> Also some python interpretors like bpython uses it ; a snapshot here - >>> ?http://cl.ly/c5bb3be4a01d9d44732f >>> So will it be ok to break them ? >> >> Using ReST won't *break* these tools, but may make the output less readable. >> >> I would say that the major use of docstrings is for interactive help - so >> interactive readability should be *the most important* (but perhaps not only) >> factor when considering how to format standard library docstrings. > > Agreed. ?However, reST doesn't need to be less readable if the specific > inline markup is not used. ?For example, using `identifier` to refer to a > function or *var* to refer to a variable (which is already done at quite a > few places) is very readable IMO. ?Using ``code`` also isn't bad, considering > that double quotes are not much different and potentially ambiguous. > > Overall, I think that we can make stdlib docstrings valid reST -- even if it's > reST without much markup -- but valid, so that people pulling in stdlib doc- > strings into Sphinx docs won't get ugly warnings. > > What I would *not* like to see is heavy markup and Sphinx specifics -- that > would only make sense if we included the docstrings in the docs, and I don't > see that coming. > > cheers, > Georg > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From tseaver at palladion.com Wed Jul 7 22:11:29 2010 From: tseaver at palladion.com (Tres Seaver) Date: Wed, 07 Jul 2010 16:11:29 -0400 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: <20100707204008.37f8e333@pitrou.net> References: <20100707204008.37f8e333@pitrou.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Antoine Pitrou wrote: > On Wed, 7 Jul 2010 19:44:31 +0200 > Eli Bendersky wrote: >> For what it's worth, my benchmarking showed that modifying the >> heuristic to only kick in when there are more than 100 kinds of >> elements (Terry's option A) didn't affect the runtime of matching >> whatsoever, even when the heuristic *does* kick in. All it adds, >> really, is the overhead of a single 'if' statement. So it wouldn't be >> right to assume that somehow modifying the heuristic or allowing to >> turn it off will negatively affect performance in the special case Tim >> originally optimized for. > > Just because it doesn't affect performance in your tests doesn't mean it > won't do so in the general case. Consider a case where Tim's junk > optimization kicked in and helped improve performance a lot, but where > there are still less than 100 alphabet symbols. The new heuristic will > ruin this use case. That would describe pretty much every C program ever written, for instance, and nearly as high a percentage of all Python modules / scripts ever written: the 'string.printable' set, less formfeed and vertical tab, is 98 characters long. > That's why I'm advocating a dedicated flag instead. +1. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkw032wACgkQ+gerLs4ltQ5xigCfVLhTzFX733cZAO2Jv6JZQm0i HoIAmQEnTyxa2oLAuE22M7FZHUS00xDu =WYt2 -----END PGP SIGNATURE----- From alexander.belopolsky at gmail.com Wed Jul 7 22:16:03 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 7 Jul 2010 16:16:03 -0400 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On Wed, Jul 7, 2010 at 3:45 PM, Brett Cannon wrote: > On Wed, Jul 7, 2010 at 08:29, Alexander Belopolsky > wrote: .. >> For datetime.py this approach presents several problems: >> >> 1. replacing datetime with self.module.datetime everywhere can get >> messy quickly. >> 2. There are test classes defined at the test_datetime module level >> that subclass from datetime classes. ?The self.module is not available >> at the module level. ?These should probably be moved to setUp() >> methods and attached to test case self. >> 3. If #2 is resolved by moving definitions inside functions, the >> classes will become unpickleable and pickle tests will break. ?Some >> hackery involving injecting these classes into __main__ or module >> globals may be required. > > So I have been thinking about this about how to possibly make this > standard test scaffolding a little cleaner. I think a class decorator > might do the trick. If you had all test methods take a module argument > you could pass in the module that should be used to test. Then you > simply rename test_* to _test_*, create test_*_(c|py), and then have > those methods call their _test_* equivalents with the proper module to > test. You could even make this generic by having the keyword arguments > to the decorator by what the test suffix is named. > Hmm, I've been playing with the idea of using a metaclass to do essentially the same, but a class decorator may be a simpler solution. I still don't see how this address #1, though. In the ideal world, I would like not to touch the body of test_* methods. These methods, however are written assuming from datetime import date, time, datetime, tzinfo, etc at the top of test_datetime.py. Even if the decorator will call _test_* with six additional arguments named date, time, datetime, tzinfo, etc, it will not work because by the time decorator (or even metaclass machinery) gets to operate, these names are already resolved as globals. > The benefit of this is you don't have to define one base class and > then two subclasses; you define a single test class and simply add a > decorator. I like this. > This addresses #1. Except it does not. :-( > As for #3, that I can't answer and might > simply require restructuring those specific pickle tests. What about #2? From brett at python.org Wed Jul 7 22:33:50 2010 From: brett at python.org (Brett Cannon) Date: Wed, 7 Jul 2010 13:33:50 -0700 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On Wed, Jul 7, 2010 at 13:16, Alexander Belopolsky wrote: > On Wed, Jul 7, 2010 at 3:45 PM, Brett Cannon wrote: >> On Wed, Jul 7, 2010 at 08:29, Alexander Belopolsky >> wrote: > .. >>> For datetime.py this approach presents several problems: >>> >>> 1. replacing datetime with self.module.datetime everywhere can get >>> messy quickly. >>> 2. There are test classes defined at the test_datetime module level >>> that subclass from datetime classes. ?The self.module is not available >>> at the module level. ?These should probably be moved to setUp() >>> methods and attached to test case self. >>> 3. If #2 is resolved by moving definitions inside functions, the >>> classes will become unpickleable and pickle tests will break. ?Some >>> hackery involving injecting these classes into __main__ or module >>> globals may be required. >> >> So I have been thinking about this about how to possibly make this >> standard test scaffolding a little cleaner. I think a class decorator >> might do the trick. If you had all test methods take a module argument >> you could pass in the module that should be used to test. Then you >> simply rename test_* to _test_*, create test_*_(c|py), and then have >> those methods call their _test_* equivalents with the proper module to >> test. You could even make this generic by having the keyword arguments >> to the decorator by what the test suffix is named. >> > > Hmm, I've been playing with the idea of using a metaclass to do > essentially the same, but a class decorator may be a simpler solution. > ?I still don't see how this address #1, though. ?In the ideal world, > I would like not to touch the body of test_* methods. ?These methods, > however are written assuming from datetime import date, time, > datetime, tzinfo, etc at the top of test_datetime.py. ?Even if the > decorator will call _test_* with six additional arguments named date, > time, datetime, tzinfo, etc, it will not work because by the time > decorator (or even metaclass machinery) gets to operate, these names > are already resolved as globals. Well, I personally would call that bad form to import those classes explicitly, but that's just me. You will simply need to make them work off of the module object. There is nothing wrong with "cleaning up" the tests as part of your work; the tests code should not be enshrined as perfect. > >> The benefit of this is you don't have to define one base class and >> then two subclasses; you define a single test class and simply add a >> decorator. > > I like this. > >> This addresses #1. > > Except it does not. :-( > >> As for #3, that I can't answer and might >> simply require restructuring those specific pickle tests. > > What about #2? > Either define two different subclasses or write a function that returns the class using the superclass that you want. From alexander.belopolsky at gmail.com Wed Jul 7 22:53:23 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 7 Jul 2010 16:53:23 -0400 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On Wed, Jul 7, 2010 at 4:33 PM, Brett Cannon wrote: >>>> 2. There are test classes defined at the test_datetime module level >>>> that subclass from datetime classes. ?The self.module is not available >>>> at the module level. ?These should probably be moved to setUp() >>>> methods and attached to test case self. .. >> What about #2? > Either define two different subclasses or write a function that > returns the class using the superclass that you want. > Selecting one of two globally defined different subclasses will be ugly in parameterized tests. An in the other approach, the class definitions will have to be moved away from the module level and inside a scope where module variable is present. Yes, it looks like some refactoring is unavoidable. From g.brandl at gmx.net Wed Jul 7 22:54:58 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 07 Jul 2010 22:54:58 +0200 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: <20100707141217.377d6a93@heresy> References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707141217.377d6a93@heresy> Message-ID: Am 07.07.2010 20:12, schrieb Barry Warsaw: > On Jul 07, 2010, at 07:30 PM, Georg Brandl wrote: > >>Overall, I think that we can make stdlib docstrings valid reST -- even >>if it's reST without much markup -- but valid, so that people pulling >>in stdlib doc- strings into Sphinx docs won't get ugly warnings. >> >>What I would *not* like to see is heavy markup and Sphinx specifics -- >>that would only make sense if we included the docstrings in the docs, >>and I don't see that coming. > > Does it make sense to add (reST-style) epydoc markup for API signatures? > E.g. > > def create_foo(name, parent=None): > """Create the named foo. > > The named foo must not already exist, but if optional `parent` is given, > it must exist. > > :param name: The name of the new foo. > :type name: string > :param parent: The new foo's parent. If given, this must exist. > :type parent: string > :return: The new foo. > :rtype: `Foo` > :raises BadFooNameError: when `name` is illegal. > :raises FooAlreadyExistsError: when a foo with `name` already exists. > :raises BadParentError: when the foo's parent does not exist. > """ > > We could then generate automatic API docs from this, a la: > > http://www.blender.org/documentation/248PythonDoc/ Yes, but: do we want this? Georg From mg at lazybytes.net Wed Jul 7 22:58:47 2010 From: mg at lazybytes.net (Martin Geisler) Date: Wed, 07 Jul 2010 22:58:47 +0200 Subject: [Python-Dev] query: docstring formatting in python distutils code References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707161206.GC1701@idyll.org> Message-ID: <87pqyzt3eg.fsf@hbox.dyndns.org> "C. Titus Brown" writes: > I guess docutils isn't in the stdlib (should it be?) or else we could > modify 'help' to use it to prepare a straight text formatting. We're using light-weight ReST markup in the Mercurial help texts and transform it into straight text upon display in the terminal. We want no external dependencies for Mercurial, so I wrote a "mini ReST" parser in about 400 lines of code. It cheats a lot and can only handle simple constructs... but maybe it would be interesting for Python's help? You find it here: http://selenic.com/hg/file/tip/mercurial/minirst.py Its test and the corresponding output shows the markup it can parse: http://selenic.com/hg/file/tip/tests/test-minirst.py http://selenic.com/hg/file/tip/tests/test-minirst.py.out It would of course be much nicer to have Docutils in the standard library. I'm not a Docutils developer, but to me it seems that Docutils is now a very stable and widely used package, so it would IMHO make sense to include it. -- Martin Geisler Mercurial links: http://mercurial.ch/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From g.brandl at gmx.net Wed Jul 7 22:58:00 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 07 Jul 2010 22:58:00 +0200 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <4C34BF2C.8090600@netwok.org> Message-ID: Am 07.07.2010 21:11, schrieb Fred Drake: > On Wed, Jul 7, 2010 at 2:27 PM, Georg Brandl wrote: >> I know, and this is what I originally intended for Sphinx. However, the calls >> for automatic doc generation are very loud, and it's understandable that most >> project can't afford writing their documentation twice. > > The ability to provide extended content beyond what's provided in the > docstring using the auto* constructs may make it feasible to start > avoiding some of those DRY violations for Python's standard library; > I'm enjoying those for another project. > > I hope we don't end up in a position where we can't use the auto* > constructs in the Python documentation. Let's say we were okay with giving up single-source docs, one potential problem is that autodoc needs to import the modules in question, which can become a problem, on one hand for platform-specific modules, on the other because the Python building the docs is not necessarily the Python that is documented. Georg From g.brandl at gmx.net Wed Jul 7 23:04:33 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 07 Jul 2010 23:04:33 +0200 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <20100707144059.07bae9f2@heresy> References: <20100624115048.4fd152e3@heresy> <20100624170944.7e68ad21@heresy> <4C23D3C2.1060500@scottdial.com> <4C246E81.3020302@scottdial.com> <4C265DC6.4080600@ubuntu.com> <4C268433.30405@scottdial.com> <20100630145329.736f2aab@heresy> <4C2C75C1.8020708@scottdial.com> <20100707144059.07bae9f2@heresy> Message-ID: Am 07.07.2010 20:40, schrieb Barry Warsaw: > Getting back to this after the US holiday. Thanks for running these numbers > Scott. I've opened a bug in the Python tracker and attached my latest patch: > > http://bugs.python.org/issue9193 > > The one difference from previous versions of the patch is that the .so tag is > now settable via "./configure --with-so-abi-tag=foo". This would generate > shared libs like _multiprocessing.foo.so. > > I'd like to get consensus as to whether folks feel that a PEP is needed. My > own thought is that I'd rather not do a PEP specific to this change, but I > would update PEP 384 with the implications on .so versioning. Please also > feel free to review the patch in that issue. I can see where this is going... writing it into PEP 384 would automatically get the change accepted? From g.brandl at gmx.net Wed Jul 7 23:00:30 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 07 Jul 2010 23:00:30 +0200 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> Message-ID: Am 07.07.2010 21:52, schrieb Brett Cannon: > On Wed, Jul 7, 2010 at 10:30, Georg Brandl wrote: >> Am 07.07.2010 18:09, schrieb Michael Foord: >>>> Hi all, >>>> >>>> over on the fellowship o' the packaging mailing list, one of our GSoC students >>>> (merwok) asked about how much formatting info should go into Python stdlib >>>> docstrings. Right now the stdlib docstrings are primarily text, AFAIK; but >>>> with the switch to Sphinx for the official Python docs, should we permit >>>> ReST-general and/or Sphinx-specific markup in docstrings? >> >> I promised to write a PEP about that some time in the future. (Probably after >> 3.2 final.) > > For those of you who aren't aware, there actually already is a PEP on > using reST in docstrings: http://python.org/dev/peps/pep-0287/ . > > But it could stand to be updated by Georg with current practice with > our internal doc practices as 2002 was back when we were still using > LaTeX. Thanks for the reference, Brett. I do not intend to do my work in PEP 287 (though it probably could be given a few updates, have to look in detail), but rather refer to it in the new one, since my proposed PEP was less about actual syntax of reST docstrings than about their use in the standard library. As far as I could see, most recommendations from PEP 287 would apply to them. Georg From solipsis at pitrou.net Wed Jul 7 23:22:34 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 7 Jul 2010 23:22:34 +0200 Subject: [Python-Dev] query: docstring formatting in python distutils code References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707161206.GC1701@idyll.org> <87pqyzt3eg.fsf@hbox.dyndns.org> Message-ID: <20100707232234.767669d9@pitrou.net> On Wed, 07 Jul 2010 22:58:47 +0200 Martin Geisler wrote: > "C. Titus Brown" writes: > > > I guess docutils isn't in the stdlib (should it be?) or else we could > > modify 'help' to use it to prepare a straight text formatting. > > We're using light-weight ReST markup in the Mercurial help texts and > transform it into straight text upon display in the terminal. > > We want no external dependencies for Mercurial, so I wrote a "mini ReST" > parser in about 400 lines of code. It cheats a lot and can only handle > simple constructs... but maybe it would be interesting for Python's > help? You find it here: > > http://selenic.com/hg/file/tip/mercurial/minirst.py Given that Mercurial is GPL, this is probably of no use to us, unfortunately. Regards Antoine. From brett at python.org Wed Jul 7 23:36:25 2010 From: brett at python.org (Brett Cannon) Date: Wed, 7 Jul 2010 14:36:25 -0700 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On Wed, Jul 7, 2010 at 13:53, Alexander Belopolsky wrote: > On Wed, Jul 7, 2010 at 4:33 PM, Brett Cannon wrote: > >>>>> 2. There are test classes defined at the test_datetime module level >>>>> that subclass from datetime classes. ?The self.module is not available >>>>> at the module level. ?These should probably be moved to setUp() >>>>> methods and attached to test case self. > .. >>> What about #2? > >> Either define two different subclasses or write a function that >> returns the class using the superclass that you want. >> > > Selecting one of two globally defined different subclasses will be > ugly in parameterized tests. Didn't say it was a pretty solution. =) > ?An in the other approach, the class > definitions will have to be moved away from the module level and > inside a scope where module variable is present. Yep, which is not a big deal. > ?Yes, it looks like > some refactoring is unavoidable. > =) From fuzzyman at voidspace.org.uk Wed Jul 7 23:54:51 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 07 Jul 2010 22:54:51 +0100 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707141217.377d6a93@heresy> Message-ID: <4C34F7AB.8000608@voidspace.org.uk> On 07/07/2010 21:54, Georg Brandl wrote: > Am 07.07.2010 20:12, schrieb Barry Warsaw: > >> On Jul 07, 2010, at 07:30 PM, Georg Brandl wrote: >> >> >>> Overall, I think that we can make stdlib docstrings valid reST -- even >>> if it's reST without much markup -- but valid, so that people pulling >>> in stdlib doc- strings into Sphinx docs won't get ugly warnings. >>> >>> What I would *not* like to see is heavy markup and Sphinx specifics -- >>> that would only make sense if we included the docstrings in the docs, >>> and I don't see that coming. >>> >> Does it make sense to add (reST-style) epydoc markup for API signatures? >> E.g. >> >> def create_foo(name, parent=None): >> """Create the named foo. >> >> The named foo must not already exist, but if optional `parent` is given, >> it must exist. >> >> :param name: The name of the new foo. >> :type name: string >> :param parent: The new foo's parent. If given, this must exist. >> :type parent: string >> :return: The new foo. >> :rtype: `Foo` >> :raises BadFooNameError: when `name` is illegal. >> :raises FooAlreadyExistsError: when a foo with `name` already exists. >> :raises BadParentError: when the foo's parent does not exist. >> """ >> >> We could then generate automatic API docs from this, a la: >> >> http://www.blender.org/documentation/248PythonDoc/ >> > Yes, but: do we want this? > > -1 :-) I find those epydoc style framed API docs very hard to read and navigate. On the other hand autogenerated API docs *can* be good looking and usable. Michael > Georg > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From john.arbash.meinel at gmail.com Wed Jul 7 23:56:23 2010 From: john.arbash.meinel at gmail.com (John Arbash Meinel) Date: Wed, 07 Jul 2010 16:56:23 -0500 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <4C2C75C1.8020708@scottdial.com> References: <20100624115048.4fd152e3@heresy> <20100624170944.7e68ad21@heresy> <4C23D3C2.1060500@scottdial.com> <4C246E81.3020302@scottdial.com> <4C265DC6.4080600@ubuntu.com> <4C268433.30405@scottdial.com> <20100630145329.736f2aab@heresy> <4C2C75C1.8020708@scottdial.com> Message-ID: <4C34F807.8030101@gmail.com> Scott Dial wrote: > On 6/30/2010 2:53 PM, Barry Warsaw wrote: >> It might be amazing, but it's still a significant overhead. As I've >> described, multiply that by all the py files in all the distro packages >> containing Python source code, and then still try to fit it on a CDROM. > > I decided to prove to myself that it was not a significant issue to have > parallel directory structures in a .tar.bz2, and I was surprised to find > it much worse at that then I had imagined. For example, > > # cd /usr/lib/python2.6/site-packages > # tar --exclude="*.pyc" --exclude="*.pyo" \ > -cjf mercurial.tar.bz2 mercurial > # du -h mercurial.tar.bz2 > 640K mercurial.tar.bz2 > > # cp -a mercurial mercurial2 > # tar --exclude="*.pyc" --exclude="*.pyo" \ > -cjf mercurial2.tar.bz2 mercurial mercurial2 > # du -h mercurial.tar.bz2 > 1.3M mercurial2.tar.bz2 > I believe the standard (and largest) block size for .bz2 is 900kB, and I *think* that is uncompressed. Though I know that bz2 can chain, since it can compress all NULL bytes extremely well (multiple GB down to kB, IIRC). There was a question as to whether LZMA would do better here, I'm using 7zip, but .xz should perform similarly. $ du -sh mercurial* 2.6M mercurial 2.6M mercurial2 366K mercurial.tar.bz2 734K mercurial2.tar.bz2 303K mercurial.7z 310K mercurial2.7z So LZMA with the 'normal' compression has a big enough window to find almost all of the redundancy, and 310kB is certainly a very small increase over the 303kB. And clearly bz2 does not, since 734kB is actually slightly more than 2x 366kB. John =:-> From fuzzyman at voidspace.org.uk Wed Jul 7 23:56:56 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 07 Jul 2010 22:56:56 +0100 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: <4C34F828.3080100@voidspace.org.uk> On 07/07/2010 21:33, Brett Cannon wrote: > On Wed, Jul 7, 2010 at 13:16, Alexander Belopolsky > wrote: > >> On Wed, Jul 7, 2010 at 3:45 PM, Brett Cannon wrote: >> >>> On Wed, Jul 7, 2010 at 08:29, Alexander Belopolsky >>> wrote: >>> >> .. >> >>>> For datetime.py this approach presents several problems: >>>> >>>> 1. replacing datetime with self.module.datetime everywhere can get >>>> messy quickly. >>>> 2. There are test classes defined at the test_datetime module level >>>> that subclass from datetime classes. The self.module is not available >>>> at the module level. These should probably be moved to setUp() >>>> methods and attached to test case self. >>>> 3. If #2 is resolved by moving definitions inside functions, the >>>> classes will become unpickleable and pickle tests will break. Some >>>> hackery involving injecting these classes into __main__ or module >>>> globals may be required. >>>> >>> So I have been thinking about this about how to possibly make this >>> standard test scaffolding a little cleaner. I think a class decorator >>> might do the trick. If you had all test methods take a module argument >>> you could pass in the module that should be used to test. Then you >>> simply rename test_* to _test_*, create test_*_(c|py), and then have >>> those methods call their _test_* equivalents with the proper module to >>> test. You could even make this generic by having the keyword arguments >>> to the decorator by what the test suffix is named. >>> >>> >> Hmm, I've been playing with the idea of using a metaclass to do >> essentially the same, but a class decorator may be a simpler solution. >> I still don't see how this address #1, though. In the ideal world, >> I would like not to touch the body of test_* methods. These methods, >> however are written assuming from datetime import date, time, >> datetime, tzinfo, etc at the top of test_datetime.py. Even if the >> decorator will call _test_* with six additional arguments named date, >> time, datetime, tzinfo, etc, it will not work because by the time >> decorator (or even metaclass machinery) gets to operate, these names >> are already resolved as globals. >> > Well, I personally would call that bad form to import those classes > explicitly, but that's just me. You will simply need to make them work > off of the module object. There is nothing wrong with "cleaning up" > the tests as part of your work; the tests code should not be enshrined > as perfect. > > Yep - each test should take the module under test (either in C or Python) as the parameter and used classes / functions as attributes off the module object. Using a class decorator to duplicate each _test_ into two test_* methods sounds like a good approach. Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From alexander.belopolsky at gmail.com Thu Jul 8 00:08:47 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 7 Jul 2010 18:08:47 -0400 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: <4C34F828.3080100@voidspace.org.uk> References: <4C34F828.3080100@voidspace.org.uk> Message-ID: On Wed, Jul 7, 2010 at 5:56 PM, Michael Foord wrote: .. >> Well, I personally would call that bad form to import those classes >> explicitly, but that's just me. You will simply need to make them work >> off of the module object. There is nothing wrong with "cleaning up" >> the tests as part of your work; the tests code should not be enshrined >> as perfect. >> >> > > Yep - each test should take the module under test (either in C or Python) as > the parameter and used classes / functions as attributes off the module > object. > This is somewhat uncharted territory. So far test_* methods had no parameters except self and module was attached to the TestCase subclass. It would be accessed inside test_* methods as self.module. I think changing test_* methods' signature is too much of a price to pay for saving self. prefix. I will still have to touch every date, time, datetime, etc symbols throughout the test file. From fdrake at acm.org Thu Jul 8 00:08:45 2010 From: fdrake at acm.org (Fred Drake) Date: Wed, 7 Jul 2010 18:08:45 -0400 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <4C34BF2C.8090600@netwok.org> Message-ID: On Wed, Jul 7, 2010 at 4:58 PM, Georg Brandl wrote: > Let's say we were okay with giving up single-source docs, It's not clear that this is a goal. > one potential > problem is that autodoc needs to import the modules in question, which > can become a problem, on one hand for platform-specific modules, on the > other because the Python building the docs is not necessarily the Python > that is documented. This is an excellent point. I'm less worried about the platform-specific issues, since we can decide that those in particular can't use the auto* support, but the need to build docs for different versions/implementations of Python is an interesting use case. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein From ncoghlan at gmail.com Thu Jul 8 00:09:13 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 8 Jul 2010 08:09:13 +1000 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <20100707144059.07bae9f2@heresy> References: <20100624115048.4fd152e3@heresy> <20100624170944.7e68ad21@heresy> <4C23D3C2.1060500@scottdial.com> <4C246E81.3020302@scottdial.com> <4C265DC6.4080600@ubuntu.com> <4C268433.30405@scottdial.com> <20100630145329.736f2aab@heresy> <4C2C75C1.8020708@scottdial.com> <20100707144059.07bae9f2@heresy> Message-ID: On Thu, Jul 8, 2010 at 4:40 AM, Barry Warsaw wrote: > I'd like to get consensus as to whether folks feel that a PEP is needed. ?My > own thought is that I'd rather not do a PEP specific to this change, but I > would update PEP 384 with the implications on .so versioning. ?Please also > feel free to review the patch in that issue. I suspect you could write a new PEP faster than you could convince those suggesting the change needs a PEP (including me) that one isn't necessary. Presumably you were going to do a summary email for the mailing list anyway - just tidy up the formatting a bit and check it in as a PEP instead :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From tjreedy at udel.edu Thu Jul 8 00:17:33 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 07 Jul 2010 18:17:33 -0400 Subject: [Python-Dev] Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On 7/7/2010 3:32 PM, Brett Cannon wrote: > That's the idea. We already have contributors from the various VMs who > has commit privileges, but they all work in their own repos for > convenience. My hope is that if we break the stdlib out into its own > repository that people simply pull in then other VM contributors will > work directly off of the stdlib repo instead of their own, magnifying > the usefulness of their work. I was wondering if you had more than 'hope', but thinking about it now, I think it premature to ask for commitments. Once a Python3 stdlib hg subrepository is set up and running, the logic of joining in should be obvious -- or not. I am now seeing that a more complete common Python-level test suite is also important. Being able to move Python code, that only uses the stdlibk,between implementations and have it just work would be good for all of them. >> 3. What version of Python would be allowed for use in the stdlib? I would >> like the stdlib for 3.x to be able to use 3.x code. This would be only a >> minor concern for CPython as long as 2.7 is maintained, but a major concern >> for the other implementation currently 'stuck' in 2.x only. A good 3to2 >> would be needed. > > This will only affect py3k. Good. The Python3 stdlib should gradually become modern Python3 code. (An example archaism -- the use in difflib of dicts with arbitrary values used as sets -- which I plan to fix.) >> I generally favor having Python versions of modules available. My current >> post on difflib.SequenceMatcher is based on experiments with an altered >> version. I copied difflib.py to my test directory, renamed it diff2lib.py, >> so I could import both versions, found and edited the appropriate method, >> and off I went. If difflib were in C, my post would have been based on >> speculation about how a fixed version would operate, rather than on data. >> > > The effect upon CPython would be the extension modules become just > performance improvements, nothing more (unless they have to be in C as > in the case for sqlite3). As pre- and jit compilation improve, the need for hand-coded C will go down. For instance, annotate (in a branch, not trunk) and compile with Cython. >> 4. Does not ctypes make it possible to replace a method of a Python-coded >> class with a faster C version, with something like >> try: >> connect to methods.dll >> check that function xyx exists >> replace Someclass.xyy with ctypes wrapper >> except: pass >> For instance, the SequenceMatcher heuristic was added to speedup the >> matching process that I believe is encapsulated in one O(n**2) or so >> bottleneck method. I believe most everything else is O(n) bookkeeping. > There is no need to go that far. All one needs to do is structure the > extension code such that when the extension module is imported, it > overrides key objects in the Python version. Is it possible to replace a python-coded function in a python-coded class with a C-coded function? I had the impression from the issue discussion that one would have to recode the entire class, even if only a single method really needed it. > Using ctypes is just added complexity. Only to be used if easier than extra C coding. -- Terry Jan Reedy From ncoghlan at gmail.com Thu Jul 8 00:27:56 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 8 Jul 2010 08:27:56 +1000 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On Thu, Jul 8, 2010 at 7:36 AM, Brett Cannon wrote: >> Selecting one of two globally defined different subclasses will be >> ugly in parameterized tests. > > Didn't say it was a pretty solution. =) > >> ?An in the other approach, the class >> definitions will have to be moved away from the module level and >> inside a scope where module variable is present. > > Yep, which is not a big deal. > >> ?Yes, it looks like >> some refactoring is unavoidable. If you want to run the same module twice with different instances of an imported module (or any other parameterised globals), creative use of run_module() can provide module level scoping without completely restructuring your tests. 1. Move the current tests aside into a new file that isn't automatically invoked by regrtest (e.g. _test_datetime_inner.py). 2. In that code, remove any imports from datetime (instead, assume datetime will be injected into the module's namespace)* 3. In test_datetime.py itself, use runpy.run_module() to import the renamed module twice, once with the Python version of datetime in init_globals and once with the C version. *How the removals work: "import datetime" is dropped entirely "from datetime import x, y, x" becomes "x, y, z = datetime.x, datetime.y, datetime.z" There would be additional things to do to make the attribution of the test results clearer in order to make this effective in practice though. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Thu Jul 8 00:31:22 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 8 Jul 2010 08:31:22 +1000 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: <4C34F828.3080100@voidspace.org.uk> References: <4C34F828.3080100@voidspace.org.uk> Message-ID: On Thu, Jul 8, 2010 at 7:56 AM, Michael Foord wrote: > Using a class decorator to duplicate each _test_ into two test_* methods > sounds ?like a good approach. Note that parameterised methods have a similar problem to parameterised modules - unittest results are reported in terms of "testmodule.testclass.testfunction", so proper attribution of results in the test output will require additional work. The separate subclasses approach doesn't share this issue, since it changes the value of the second item in accordance with the module under test. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From tjreedy at udel.edu Thu Jul 8 00:32:19 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 07 Jul 2010 18:32:19 -0400 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: References: <20100707204008.37f8e333@pitrou.net> Message-ID: On 7/7/2010 4:11 PM, Tres Seaver wrote: > Antoine Pitrou wrote: >> On Wed, 7 Jul 2010 19:44:31 +0200 >> Eli Bendersky wrote: >>> For what it's worth, my benchmarking showed that modifying the >>> heuristic to only kick in when there are more than 100 kinds of >>> elements (Terry's option A) didn't affect the runtime of matching >>> whatsoever, even when the heuristic *does* kick in. All it adds, >>> really, is the overhead of a single 'if' statement. So it wouldn't be >>> right to assume that somehow modifying the heuristic or allowing to >>> turn it off will negatively affect performance in the special case Tim >>> originally optimized for. >> >> Just because it doesn't affect performance in your tests doesn't mean it >> won't do so in the general case. Consider a case where Tim's junk >> optimization kicked in and helped improve performance a lot, but where >> there are still less than 100 alphabet symbols. The new heuristic will >> ruin this use case. > > That would describe pretty much every C program ever written, for > instance, and nearly as high a percentage of all Python modules / > scripts ever written: the 'string.printable' set, less formfeed and > vertical tab, is 98 characters long. In the primary use case, programs are compared by line, not characters, and there are more than 100 different lines in any sensible program of at least 200 lines. -- Terry Jan Reedy From brett at python.org Thu Jul 8 00:37:04 2010 From: brett at python.org (Brett Cannon) Date: Wed, 7 Jul 2010 15:37:04 -0700 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: <4C34F828.3080100@voidspace.org.uk> Message-ID: On Wed, Jul 7, 2010 at 15:31, Nick Coghlan wrote: > On Thu, Jul 8, 2010 at 7:56 AM, Michael Foord wrote: >> Using a class decorator to duplicate each _test_ into two test_* methods >> sounds ?like a good approach. > > Note that parameterised methods have a similar problem to > parameterised modules - unittest results are reported in terms of > "testmodule.testclass.testfunction", so proper attribution of results > in the test output will require additional work. The separate > subclasses approach doesn't share this issue, since it changes the > value of the second item in accordance with the module under test. This is why a new method would need to be created with a special suffix to delineate what module the test was called with. So instead of testclass specifying what module was used, it would be testfunction. I guess it becomes a question of what boilerplate you prefer. One nice benefit of the class decorator that I can think of is it could handle the import trickery for you so you wouldn't even need to worry about that issue. This could also allow the decorator to not bother running the tests twice if the extension helper was not available. -Brett > > Cheers, > Nick. > > -- > Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia > From alexander.belopolsky at gmail.com Thu Jul 8 00:41:21 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 7 Jul 2010 18:41:21 -0400 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On Wed, Jul 7, 2010 at 6:27 PM, Nick Coghlan wrote: .. > If you want to run the same module twice with different instances of > an imported module (or any other parameterised globals), creative use > of run_module() can provide module level scoping without completely > restructuring your tests. > This is what the current patch at http://bugs.python.org/file17848/issue7989.diff does, but at expense of not exposing testcases to unittest correctly. > 1. Move the current tests aside into a new file that isn't > automatically invoked by regrtest (e.g. _test_datetime_inner.py). Yes, I already have datetimetester.py. > 2. In that code, remove any imports from datetime (instead, assume > datetime will be injected into the module's namespace)* Hmm. That will make datetimetester not importable. > 3. In test_datetime.py itself, use runpy.run_module() to import the > renamed module twice, once with the Python version of datetime in > init_globals and once with the C version. > > *How the removals work: > "import datetime" is dropped entirely > "from datetime import x, y, x" becomes "x, y, z = datetime.x, > datetime.y, datetime.z" > I'll try that. > There would be additional things to do to make the attribution of the > test results clearer in order to make this effective in practice > though. Thanks. I would really like to make it work first and improve later. I hope this will do the trick. From benjamin at python.org Thu Jul 8 01:13:33 2010 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 7 Jul 2010 18:13:33 -0500 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: <4C34F828.3080100@voidspace.org.uk> Message-ID: 2010/7/7 Nick Coghlan : > On Thu, Jul 8, 2010 at 7:56 AM, Michael Foord wrote: >> Using a class decorator to duplicate each _test_ into two test_* methods >> sounds ?like a good approach. > > Note that parameterised methods have a similar problem to > parameterised modules - unittest results are reported in terms of > "testmodule.testclass.testfunction", so proper attribution of results > in the test output will require additional work. The separate > subclasses approach doesn't share this issue, since it changes the > value of the second item in accordance with the module under test. A good parameterized implementation, though, gives the repr() of the parameters in failure output. -- Regards, Benjamin From doko at ubuntu.com Thu Jul 8 01:47:12 2010 From: doko at ubuntu.com (Matthias Klose) Date: Thu, 08 Jul 2010 01:47:12 +0200 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <20100707144059.07bae9f2@heresy> References: <20100624115048.4fd152e3@heresy> <20100624170944.7e68ad21@heresy> <4C23D3C2.1060500@scottdial.com> <4C246E81.3020302@scottdial.com> <4C265DC6.4080600@ubuntu.com> <4C268433.30405@scottdial.com> <20100630145329.736f2aab@heresy> <4C2C75C1.8020708@scottdial.com> <20100707144059.07bae9f2@heresy> Message-ID: <4C351200.2080900@ubuntu.com> On 07.07.2010 20:40, Barry Warsaw wrote: > Getting back to this after the US holiday. Thanks for running these numbers > Scott. I've opened a bug in the Python tracker and attached my latest patch: > > http://bugs.python.org/issue9193 > > The one difference from previous versions of the patch is that the .so tag is > now settable via "./configure --with-so-abi-tag=foo". This would generate > shared libs like _multiprocessing.foo.so. - imo, it's wrong to lookup _multiprocessing.so first, before looking up _multiprocessing.foo.so (at least for the use case to put the extensions for multiple python versions into one directory). - why is the flexibility of specifying the "foo" needed? The naming for the __pycache__ files is fixed, why have it configurable for extensions? Matthias From stephen at xemacs.org Thu Jul 8 01:46:23 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 08 Jul 2010 08:46:23 +0900 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: <20100707232234.767669d9@pitrou.net> References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707161206.GC1701@idyll.org> <87pqyzt3eg.fsf@hbox.dyndns.org> <20100707232234.767669d9@pitrou.net> Message-ID: <87k4p6kg8g.fsf@uwakimon.sk.tsukuba.ac.jp> Antoine Pitrou writes: > > http://selenic.com/hg/file/tip/mercurial/minirst.py > > Given that Mercurial is GPL, this is probably of no use to us, > unfortunately. Given that Martin apparently is the only or main author, I don't see a problem as long as he's willing. Martin? From benjamin at python.org Thu Jul 8 02:12:47 2010 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 7 Jul 2010 19:12:47 -0500 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: <87k4p6kg8g.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707161206.GC1701@idyll.org> <87pqyzt3eg.fsf@hbox.dyndns.org> <20100707232234.767669d9@pitrou.net> <87k4p6kg8g.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: 2010/7/7 Stephen J. Turnbull : > Antoine Pitrou writes: > > ?> > ? http://selenic.com/hg/file/tip/mercurial/minirst.py > ?> > ?> Given that Mercurial is GPL, this is probably of no use to us, > ?> unfortunately. > > Given that Martin apparently is the only or main author, I don't see a > problem as long as he's willing. And he hasn't assigned the copyright away. -- Regards, Benjamin From tjreedy at udel.edu Thu Jul 8 03:04:17 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 07 Jul 2010 21:04:17 -0400 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: References: Message-ID: I had considered the possibility of option A for 2.7 and A & C for 3.2. But see below. Since posting, I did an experiment with a 700 char paragraph of text (the summary from the post) compared to an 'edited' version. I did the comparision with and without the current heuristic. I did not notice much time difference (a couple of seconds either way) and the edit list was essentially the same. The heuristic lowered the reported match ratio from .96 to .88, which would be bad when one wanted the unaltered value. I do not know which, if any, chars other than 'e' were junked as that info currently is not exposed. I propose below that it should be. I intentionally did not list as options D. Keep the status quo that is buggy for certain uses. Good things often have more uses than the inventor intended or expected. They should not be prevented. E. Completely remove the heuristic, which would restore 'buggy' performance for other uses. One of the closed issues was E, rejected for that reason. --- I also did not list one of my original ideas, but after the discussion it is looking better to me. It is based on the following statement of the current heuristic: "Disregard as junk common items that occur in more that 1% of the positions in second sequence b, as long as b is long enough so that duplicates cannot be called common." Tim, I do not know if you remember why you choose 200 as the cutoff, but the above assumes that the following in not just a coincidence: (2 > 199*.01) == True (2 > 200*.01) == False In other words, 200 is the smallest length for b that prevents the junking of duplicates. F. Generalize the heuristic by replacing '1' with 'k', where k can be None (no heuristic) or 1-99. If not None, replace 200 by 200/k to minimally avoid junking of duplicates. If len(b) >= 200/k, then item counts should be greater than (len(b)*k)//100, which need only be calculated once. Implementation: Add a new parameter named 'common' or 'threshold' or whatever that defaults to 1. After computing b2j without the heuristic, if 'common' is not None, prune b2j as described above. My thinking here is that a user either knows or can easily find out the length of b and the size of the intented or actual alphabet of b (the latter is len(set(b)). So the user can conditionally call SequenceMatcher with 'common' set to None or an int as appropriate, perhaps after some experimentation. So the threshold is the only tuning parameter actually needed, and it also allows the heuristic to be turned off. The reason I did not list this before is the problem with 2.7.1. F, unlike option A, overtly changes the api, and some would object to that for 2.7 even though is really is a bugfix. However, option F will not not break code while the covert change of option A could break code. So this may be the best option for a bad situation. It is a minimal change that gives the user complete flexibility. In other words, I see three options for 2.7.1+: D. keep the current sometimes buggy behavior A. covertly change the heuristic to mostly fix it but possibly break some uses. F. add a parameter, with a no-change default, that allows the user to select a change if and when wanted. Another advantage of F is that then 2.7 and 3.2 would get the same change. -- Other changes that apply regardless of the heuristic/api change: Update the code to use sets (newer than difflib) instead of dicts with values set to 1. Directly expose the set of 'common' items as an additional attribute of SequenceMatcher instances. Such instance attributes are currently undocumented, so adding one can hardly be a problem. Add documention thereof. Being able to see the effect of the heuristic when it is not turned off might help people decide whether or not to use it, or how to tune the threshold for smallish alphabets where 1 is too small. -- Terry Jan Reedy From tjreedy at udel.edu Thu Jul 8 03:45:30 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 07 Jul 2010 21:45:30 -0400 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: <20100707204246.2dc3d0e0@pitrou.net> References: <4C349FBA.9040800@voidspace.org.uk> <20100707204246.2dc3d0e0@pitrou.net> Message-ID: On 7/7/2010 2:42 PM, Antoine Pitrou wrote: I wrote >> 4. Does not ctypes make it possible to replace a method of a Python-coded >> class with a faster C version, with something like >> try: >> connect to methods.dll methods.dll to be written >> check that function xyx exists >> replace Someclass.xyy with ctypes wrapper >> except: pass >> For instance, the SequenceMatcher heuristic was added to speedup the >> matching process that I believe is encapsulated in one O(n**2) or so >> bottleneck method. I believe most everything else is O(n) bookkeeping. > Except that ctypes doesn't help provide C extensions at all. It only > helps provide wrappers around existing C libraries, which is quite a > different thing. > Which, in the end, makes the original suggestion meaningless. To you, so let me restate it. It would be easier for many people to only rewrite, for instance, difflib.SequenceMatcher.get_longest_matching in C than to rewrite the whole SequenceMatcher class, let alone the whole difflib module. I got the impression from the datetime issue tracker discussion that it is not possible to replace a single method of a Python-coded class with a C version. I got this from statement that seems to say that having parallel Python and C versions is a nuisance because one must replace large chunks of Python, at least a class if not the whole module. If that impression is wrong, and I hope it is, the suggestion is unnecessary. If it is right, then replacing the Python-coded function with a Python-coded wrapper for a function in a miscellaneous shared library might be both possible and useful. But again, if the premise is wrong, skip the conclusion. -- Terry Jan Reedy From fuzzyman at voidspace.org.uk Thu Jul 8 03:57:10 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 08 Jul 2010 02:57:10 +0100 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: <4C349FBA.9040800@voidspace.org.uk> <20100707204246.2dc3d0e0@pitrou.net> Message-ID: <4C353076.5010008@voidspace.org.uk> On 08/07/2010 02:45, Terry Reedy wrote: > On 7/7/2010 2:42 PM, Antoine Pitrou wrote: > > I wrote >>> 4. Does not ctypes make it possible to replace a method of a >>> Python-coded >>> class with a faster C version, with something like >>> try: >>> connect to methods.dll > > methods.dll to be written > >>> check that function xyx exists >>> replace Someclass.xyy with ctypes wrapper >>> except: pass >>> For instance, the SequenceMatcher heuristic was added to speedup the >>> matching process that I believe is encapsulated in one O(n**2) or so >>> bottleneck method. I believe most everything else is O(n) bookkeeping. > >> Except that ctypes doesn't help provide C extensions at all. It only >> helps provide wrappers around existing C libraries, which is quite a >> different thing. >> Which, in the end, makes the original suggestion meaningless. > > To you, so let me restate it. It would be easier for many people to > only rewrite, for instance, > difflib.SequenceMatcher.get_longest_matching in C than to rewrite the > whole SequenceMatcher class, let alone the whole difflib module. > > I got the impression from the datetime issue tracker discussion that > it is not possible to replace a single method of a Python-coded class > with a C version. I got this from statement that seems to say that > having parallel Python and C versions is a nuisance because one must > replace large chunks of Python, at least a class if not the whole > module. If that impression is wrong, and I hope it is, the suggestion > is unnecessary. > > If it is right, then replacing the Python-coded function with a > Python-coded wrapper for a function in a miscellaneous shared library > might be both possible and useful. But again, if the premise is wrong, > skip the conclusion. > Would it be possible to provide a single method in C by providing a C base class with a single method and have the full implementation inherit from the C base class if it is available or otherwise a pure Python base class? Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From robertc at robertcollins.net Thu Jul 8 03:58:22 2010 From: robertc at robertcollins.net (Robert Collins) Date: Thu, 8 Jul 2010 11:58:22 +1000 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: <4C34F828.3080100@voidspace.org.uk> Message-ID: On pypi - testscenarios; Its been discussed on TIP before. Its a 'run a function to parameterise some tests' API, it changes the id() of the test to include the parameters, and it can be hooked in via load_tests quite trivially. Cheers, Rob From tjreedy at udel.edu Thu Jul 8 04:16:20 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 07 Jul 2010 22:16:20 -0400 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On 7/7/2010 11:43 AM, Jesse Noller wrote: > The idea is to put CPython on a more equal footing with the other > implementations, I would reverse this to "The idea is to put the other implementations on a more equal footing with CPython." The subtle difference is the implication of whether the idea is to pull CPython down (the former) or raise the others up (the latter) ;-). -- Terry Jan Reedy From stephen at xemacs.org Thu Jul 8 04:58:14 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 08 Jul 2010 11:58:14 +0900 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707161206.GC1701@idyll.org> <87pqyzt3eg.fsf@hbox.dyndns.org> <20100707232234.767669d9@pitrou.net> <87k4p6kg8g.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87fwzuk7cp.fsf@uwakimon.sk.tsukuba.ac.jp> Benjamin Peterson writes: > 2010/7/7 Stephen J. Turnbull : > > Antoine Pitrou writes: > > > > ?> > ? http://selenic.com/hg/file/tip/mercurial/minirst.py > > ?> > > ?> Given that Mercurial is GPL, this is probably of no use to us, > > ?> unfortunately. > > > > Given that Martin apparently is the only or main author, I don't see a > > problem as long as he's willing. > > And he hasn't assigned the copyright away. (Or that the assignment has an automatic author-use-ok clause like the standard FSF assignment does, etc.) Just ask Martin, there are too many possibilities here to worry about. If maybe we want it, and he is willing to contribute the parts he wrote to Python under Python's license, then we can worry about whether we really want it and about how much any required hoop-jumping will cost. From g.brandl at gmx.net Thu Jul 8 09:14:53 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 08 Jul 2010 09:14:53 +0200 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: References: <20100624115048.4fd152e3@heresy> <20100624170944.7e68ad21@heresy> <4C23D3C2.1060500@scottdial.com> <4C246E81.3020302@scottdial.com> <4C265DC6.4080600@ubuntu.com> <4C268433.30405@scottdial.com> <20100630145329.736f2aab@heresy> <4C2C75C1.8020708@scottdial.com> <20100707144059.07bae9f2@heresy> Message-ID: Am 07.07.2010 23:04, schrieb Georg Brandl: > Am 07.07.2010 20:40, schrieb Barry Warsaw: > >> Getting back to this after the US holiday. Thanks for running these numbers >> Scott. I've opened a bug in the Python tracker and attached my latest patch: >> >> http://bugs.python.org/issue9193 >> >> The one difference from previous versions of the patch is that the .so tag is >> now settable via "./configure --with-so-abi-tag=foo". This would generate >> shared libs like _multiprocessing.foo.so. >> >> I'd like to get consensus as to whether folks feel that a PEP is needed. My >> own thought is that I'd rather not do a PEP specific to this change, but I >> would update PEP 384 with the implications on .so versioning. Please also >> feel free to review the patch in that issue. > > I can see where this is going... writing it into PEP 384 would automatically get > the change accepted? I hit "Send" prematurely. I wanted to add that I'd be okay with this change, be it in a new PEP or an old one. Georg From mg at aragost.com Thu Jul 8 09:32:34 2010 From: mg at aragost.com (Martin Geisler) Date: Thu, 08 Jul 2010 09:32:34 +0200 Subject: [Python-Dev] query: docstring formatting in python distutils code References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707161206.GC1701@idyll.org> <87pqyzt3eg.fsf@hbox.dyndns.org> <20100707232234.767669d9@pitrou.net> <87k4p6kg8g.fsf@uwakimon.sk.tsukuba.ac.jp> <87fwzuk7cp.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <8739vuctt9.fsf@aragost.com> "Stephen J. Turnbull" writes: > Benjamin Peterson writes: > > 2010/7/7 Stephen J. Turnbull : > > > Antoine Pitrou writes: > > > > > > ?> > ? http://selenic.com/hg/file/tip/mercurial/minirst.py > > > ?> > > > ?> Given that Mercurial is GPL, this is probably of no use to us, > > > ?> unfortunately. I must admit that reading this felt strange somehow... that a piece of open source code should be useless. But I understand what you mean :) > > > Given that Martin apparently is the only or main author, I don't > > > see a problem as long as he's willing. > > > > And he hasn't assigned the copyright away. > > (Or that the assignment has an automatic author-use-ok clause like the > standard FSF assignment does, etc.) We don't assign copyright in Mercurial, so this should be no problem. This also meant that we had to contact about 300 guys when changing from GPLv2 to GPLv2+. > Just ask Martin, there are too many possibilities here to worry about. > If maybe we want it, and he is willing to contribute the parts he > wrote to Python under Python's license, then we can worry about > whether we really want it and about how much any required hoop-jumping > will cost. I would be happy to relicense it under the Python license. -- Martin Geisler aragost Trifork: Professional Mercurial support http://aragost.com/mercurial/ From solipsis at pitrou.net Thu Jul 8 11:15:30 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 8 Jul 2010 11:15:30 +0200 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken References: Message-ID: <20100708111530.1f5e9f9e@pitrou.net> On Wed, 07 Jul 2010 21:04:17 -0400 Terry Reedy wrote: > > In other words, I see three options for 2.7.1+: [...] I don't think 2.7 should get any change at all here. Only 3.2 should be modified. As Tim said, difflib works ok for its intended use (regular text diffs). Making it work for other uses is a new feature, not a bugfix. Regards Antoine. From solipsis at pitrou.net Thu Jul 8 11:24:33 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 8 Jul 2010 11:24:33 +0200 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? References: <4C349FBA.9040800@voidspace.org.uk> <20100707204246.2dc3d0e0@pitrou.net> Message-ID: <20100708112433.74fcc2c8@pitrou.net> On Wed, 07 Jul 2010 21:45:30 -0400 Terry Reedy wrote: > > > Except that ctypes doesn't help provide C extensions at all. It only > > helps provide wrappers around existing C libraries, which is quite a > > different thing. > > Which, in the end, makes the original suggestion meaningless. > > To you, so let me restate it. It would be easier for many people to only > rewrite, for instance, difflib.SequenceMatcher.get_longest_matching in > C than to rewrite the whole SequenceMatcher class, let alone the whole > difflib module. And you still haven't understood my point. ctypes doesn't allow you to write any C code, only to interface with existing C code. So, regardless of whether get_longest_matching() is a function or method, it would have to be written in C manually, and that would certainly be in an extension module. (admittedly, you can instead make a pure C library with such a function and then wrap it with ctypes, but I don't see the point: you still have to write most C code yourself) > I got the impression from the datetime issue tracker discussion that it > is not possible to replace a single method of a Python-coded class with > a C version. And that's a wrong impression. Inheritance allows you to do that (see Michael's answer). Besides, you can also code that method as a helper function. It is not difficult to graft a function from a module into another module. Antoine. From steve at holdenweb.com Thu Jul 8 12:41:17 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 08 Jul 2010 06:41:17 -0400 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: <8739vuctt9.fsf@aragost.com> References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707161206.GC1701@idyll.org> <87pqyzt3eg.fsf@hbox.dyndns.org> <20100707232234.767669d9@pitrou.net> <87k4p6kg8g.fsf@uwakimon.sk.tsukuba.ac.jp> <87fwzuk7cp.fsf@uwakimon.sk.tsukuba.ac.jp> <8739vuctt9.fsf@aragost.com> Message-ID: <4C35AB4D.90305@holdenweb.com> Martin Geisler wrote: > "Stephen J. Turnbull" writes: > >> Benjamin Peterson writes: >> > 2010/7/7 Stephen J. Turnbull : >> > > Antoine Pitrou writes: >> > > >> > > > > http://selenic.com/hg/file/tip/mercurial/minirst.py >> > > > >> > > > Given that Mercurial is GPL, this is probably of no use to us, >> > > > unfortunately. > > I must admit that reading this felt strange somehow... that a piece of > open source code should be useless. But I understand what you mean :) > >> > > Given that Martin apparently is the only or main author, I don't >> > > see a problem as long as he's willing. >> > >> > And he hasn't assigned the copyright away. >> >> (Or that the assignment has an automatic author-use-ok clause like the >> standard FSF assignment does, etc.) > > We don't assign copyright in Mercurial, so this should be no problem. > This also meant that we had to contact about 300 guys when changing from > GPLv2 to GPLv2+. > >> Just ask Martin, there are too many possibilities here to worry about. >> If maybe we want it, and he is willing to contribute the parts he >> wrote to Python under Python's license, then we can worry about >> whether we really want it and about how much any required hoop-jumping >> will cost. > > I would be happy to relicense it under the Python license. > I believe the ideal outcome, if it is possible, is for you to sign a contributor agreement. This will license your material to the PSF in such a way that we can release it under whatever license we deem necessary. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Thu Jul 8 12:41:17 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 08 Jul 2010 06:41:17 -0400 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: <8739vuctt9.fsf@aragost.com> References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707161206.GC1701@idyll.org> <87pqyzt3eg.fsf@hbox.dyndns.org> <20100707232234.767669d9@pitrou.net> <87k4p6kg8g.fsf@uwakimon.sk.tsukuba.ac.jp> <87fwzuk7cp.fsf@uwakimon.sk.tsukuba.ac.jp> <8739vuctt9.fsf@aragost.com> Message-ID: <4C35AB4D.90305@holdenweb.com> Martin Geisler wrote: > "Stephen J. Turnbull" writes: > >> Benjamin Peterson writes: >> > 2010/7/7 Stephen J. Turnbull : >> > > Antoine Pitrou writes: >> > > >> > > > > http://selenic.com/hg/file/tip/mercurial/minirst.py >> > > > >> > > > Given that Mercurial is GPL, this is probably of no use to us, >> > > > unfortunately. > > I must admit that reading this felt strange somehow... that a piece of > open source code should be useless. But I understand what you mean :) > >> > > Given that Martin apparently is the only or main author, I don't >> > > see a problem as long as he's willing. >> > >> > And he hasn't assigned the copyright away. >> >> (Or that the assignment has an automatic author-use-ok clause like the >> standard FSF assignment does, etc.) > > We don't assign copyright in Mercurial, so this should be no problem. > This also meant that we had to contact about 300 guys when changing from > GPLv2 to GPLv2+. > >> Just ask Martin, there are too many possibilities here to worry about. >> If maybe we want it, and he is willing to contribute the parts he >> wrote to Python under Python's license, then we can worry about >> whether we really want it and about how much any required hoop-jumping >> will cost. > > I would be happy to relicense it under the Python license. > I believe the ideal outcome, if it is possible, is for you to sign a contributor agreement. This will license your material to the PSF in such a way that we can release it under whatever license we deem necessary. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From tim.peters at gmail.com Thu Jul 8 15:39:08 2010 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 8 Jul 2010 09:39:08 -0400 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: <20100708111530.1f5e9f9e@pitrou.net> References: <20100708111530.1f5e9f9e@pitrou.net> Message-ID: [Antoine Pitrou] > I don't think 2.7 should get any change at all here. Only 3.2 should be > modified. As Tim said, difflib works ok for its intended use (regular > text diffs). That was the use case that drove the implementation, but it's going too far to say that was the only "intended" case. I believe (but can't prove) that remains the most common use (& overwhelmingly so), but it was indeed _intended_ to work for any sequences of hashable elements. And it always did, and it still does, in the sense that it computes a diff that transforms the first sequence into the second sequence. The problem is that I introduced a heuristic speedup with the primary use case in mind that turned out to vastly damage the _quality_ of the results for some other uses (a correct diff isn't necessarily a useful diff - for example, "delete the entire sequence you started with, then insert the entire new sequence" is a correct diff for any pair of input sequences, but not a useful diff for most purposes). > Making it work for other uses is a new feature, not a bugfix. Definitely not a new feature. These other cases used to deliver much better diffs, before I introduced the heuristic in question. People with these other cases are asking for a way to get the results they used to get - and we know that's so because a few figured out they get what they want just by (in effect) reverting the checkin (made about 8 years ago) that _introduced_ the heuristic. So they're looking for a way to restore older behavior, not to introduce new behavior. Of course this is obscured by that the change happened so long ago that I bet most of them don't know at first that it _was_ the old behavior. From ncoghlan at gmail.com Thu Jul 8 16:59:02 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 9 Jul 2010 00:59:02 +1000 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: <4C34F828.3080100@voidspace.org.uk> Message-ID: On Thu, Jul 8, 2010 at 9:13 AM, Benjamin Peterson wrote: > 2010/7/7 Nick Coghlan : >> On Thu, Jul 8, 2010 at 7:56 AM, Michael Foord wrote: >>> Using a class decorator to duplicate each _test_ into two test_* methods >>> sounds ?like a good approach. >> >> Note that parameterised methods have a similar problem to >> parameterised modules - unittest results are reported in terms of >> "testmodule.testclass.testfunction", so proper attribution of results >> in the test output will require additional work. The separate >> subclasses approach doesn't share this issue, since it changes the >> value of the second item in accordance with the module under test. > > A good parameterized implementation, though, gives the repr() of the > parameters in failure output. That would qualify as "additional work" if your tests aren't already set up that way (and doesn't cover the case of unexpected exceptions in a test, where the test method doesn't get to say much about the way the error is reported). I realised during the day that my suggested approach was more complicated than is actually necessary - once the existing tests have been moved to a separate module, *that test module* can itself be imported twice, once with the python version of the module to be tested and once with the C version. You can then do some hackery to distinguish the test classes without having to modify the test code itself (note, the below code should work in theory, but isn't actually tested): ============= py_module_tests = support.import_fresh_module('moduletester', fresh=['modulename'], blocked=['_modulename']) c_module_tests = support.import_fresh_module('moduletester', fresh=['modulename', '_modulename']) test_modules = [py_module_tests, c_module_tests] suffixes = ["_Py", "_C"] for module, suffix in zip(test_modules, suffixes): for obj in module.itervalues(): if isinstance(obj, unittest,TestCase): obj.__name__ += suffix setattr(module, obj.__name__, obj) def test_main(): for module in test_modules: module.test_main() ============= Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Thu Jul 8 17:06:29 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 9 Jul 2010 01:06:29 +1000 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: <4C34F828.3080100@voidspace.org.uk> Message-ID: On Fri, Jul 9, 2010 at 12:59 AM, Nick Coghlan wrote: > ? ?for obj in module.itervalues(): > ? ? ? ?if isinstance(obj, unittest,TestCase): Hmm, isn't there a never-quite-made-it-into-the-Zen line about "syntax shall not look like grit on Tim's monitor"? (s/,/./ in that second line) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From mg at aragost.com Thu Jul 8 17:44:54 2010 From: mg at aragost.com (Martin Geisler) Date: Thu, 08 Jul 2010 17:44:54 +0200 Subject: [Python-Dev] query: docstring formatting in python distutils code References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707161206.GC1701@idyll.org> <87pqyzt3eg.fsf@hbox.dyndns.org> <20100707232234.767669d9@pitrou.net> <87k4p6kg8g.fsf@uwakimon.sk.tsukuba.ac.jp> <87fwzuk7cp.fsf@uwakimon.sk.tsukuba.ac.jp> <8739vuctt9.fsf@aragost.com> <4C35AB4D.90305@holdenweb.com> Message-ID: <87ocei9dvt.fsf@aragost.com> Steve Holden writes: > Martin Geisler wrote: >> "Stephen J. Turnbull" writes: >> >>> Just ask Martin, there are too many possibilities here to worry >>> about. If maybe we want it, and he is willing to contribute the >>> parts he wrote to Python under Python's license, then we can worry >>> about whether we really want it and about how much any required >>> hoop-jumping will cost. >> >> I would be happy to relicense it under the Python license. > > I believe the ideal outcome, if it is possible, is for you to sign a > contributor agreement. This will license your material to the PSF in > such a way that we can release it under whatever license we deem > necessary. Sure, I'll be happy to sign a contributor agreement if you guys think it worthwhile to use my little parser and formatter. -- Martin Geisler aragost Trifork -- Professional Mercurial support http://aragost.com/mercurial/ From jnoller at gmail.com Thu Jul 8 19:46:33 2010 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 8 Jul 2010 13:46:33 -0400 Subject: [Python-Dev] Call for Applications - PSF Sponsored Sprints Message-ID: [Sending this to Python-Dev, as it might very well be of interest as the sprints are focused on Python "core" tasks] The PSF is happy to open our first call for applications for sprint funding! Have you ever had a group of people together to hack towards a common goal? You've hosted a sprint! Have you ever wanted to get a group of like minded Pythonistas together to hack for a day? You're going to want to hold a sprint! Whether you call them Sprints, Hackfests, Hack-a-thons, or any other name, they're a great way to hang out with like-minded developers and work on common code. Sprints are an unbeatable way to build friendships and contacts that will last for years to come, and they're a great way to learn about something new if you're just starting out. The Python Software Foundation has set aside funds to be distributed to world-wide sprint efforts. We're anticipating 2-3 events per month focused on covering topics to help the entire community: - Python Core bug triage and patch submission (on-boarding new contributors) - Python Core documentation (including process documentation) improvements - Porting libraries/applications to Python 3 - Python website/wiki content improvements - PyPI packaging hosting site improvements - Contribution to other "core" projects, such as packaging related issues. If you are interested in holding a sprint on any of the topics above and you're looking for some money to help out with sprint costs, we can help (up to a max of $250 USD). Prepare an application including the following information: - Date and Location: Where will the event be? What day and time? - Organizers: Who are the event organizers and sprint coach? Is the sprint being run by a Python user group? - Attendees: How many participants do you expect? - Goal: What is the focus and goal of the sprint? - Budget: How much funding you are requesting, and what will you use it for? - Applications should be sent to: sprints at python.org with the subject "Sprint Funding Application - " We encourage anyone - even those who have never held, or been to a sprint - to consider holding one. We will help you as much as we can with welcome packets, advertising, and hooking you up with required resources - anything to make it possible. As part of being approved, the you will need to agree to deliver a report (hopefully, with pictures!) of the sprint to the Sprint Committee, so we can post it on the sprint blog and site: http://www.pythonsprints.com If you have any questions or need more information, contact us by email at sprints at python.org. More information is up on our blog: http://pythonsprints.com/2010/07/8/call-applications-now-open/ From brett at python.org Thu Jul 8 21:09:38 2010 From: brett at python.org (Brett Cannon) Date: Thu, 8 Jul 2010 12:09:38 -0700 Subject: [Python-Dev] Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On Wed, Jul 7, 2010 at 15:17, Terry Reedy wrote: > On 7/7/2010 3:32 PM, Brett Cannon wrote: > >> That's the idea. We already have contributors from the various VMs who >> has commit privileges, but they all work in their own repos for >> convenience. My hope is that if we break the stdlib out into its own >> repository that people simply pull in then other VM contributors will >> work directly off of the stdlib repo instead of their own, magnifying >> the usefulness of their work. > > I was wondering if you had more than 'hope', but thinking about it now, I > think it premature to ask for commitments. Once a Python3 stdlib hg > subrepository is set up and running, the logic of joining in should be > obvious -- or not. I can say that all the VM representatives have all said they like the idea. -Brett > > I am now seeing that a more complete common Python-level test suite is also > important. Being able to move Python code, that only uses the > stdlibk,between implementations and have it just work would be good for all > of them. > >>> 3. What version of Python would be allowed for use in the stdlib? I would >>> like the stdlib for 3.x to be able to use 3.x code. This would be only a >>> minor concern for CPython as long as 2.7 is maintained, but a major >>> concern >>> for the other implementation currently 'stuck' in 2.x only. A good 3to2 >>> would be needed. >> >> This will only affect py3k. > > Good. The Python3 stdlib should gradually become modern Python3 code. (An > example archaism -- the use in difflib of dicts with arbitrary values used > as sets -- which I plan to fix.) > >>> I generally favor having Python versions of modules available. My current >>> post on difflib.SequenceMatcher is based on experiments with an altered >>> version. I copied difflib.py to my test directory, renamed it >>> diff2lib.py, >>> so I could import both versions, found and edited the appropriate method, >>> and off I went. If difflib were in C, my post would have been based on >>> speculation about how a fixed version would operate, rather than on data. >>> >> >> The effect upon CPython would be the extension modules become just >> performance improvements, nothing more (unless they have to be in C as >> in the case for sqlite3). > > As pre- and jit compilation improve, the need for hand-coded C will go down. > For instance, annotate (in a branch, not trunk) and compile with Cython. > >>> 4. Does not ctypes make it possible to replace a method of a Python-coded >>> class with a faster C version, with something like >>> ?try: >>> ? ?connect to methods.dll >>> ? ?check that function xyx exists >>> ? ?replace Someclass.xyy with ctypes wrapper >>> ?except: pass >>> For instance, the SequenceMatcher heuristic was added to speedup the >>> matching process that I believe is encapsulated in one O(n**2) or so >>> bottleneck method. I believe most everything else is O(n) bookkeeping. > >> There is no need to go that far. All one needs to do is structure the >> extension code such that when the extension module is imported, it >> overrides key objects in the Python version. > > Is it possible to replace a python-coded function in a python-coded class > with a C-coded function? I had the impression from the issue discussion that > one would have to recode the entire class, even if only a single method > really needed it. > >> Using ctypes is just added complexity. > > Only to be used if easier than extra C coding. > > -- > Terry Jan Reedy > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > From brett at python.org Thu Jul 8 21:18:17 2010 From: brett at python.org (Brett Cannon) Date: Thu, 8 Jul 2010 12:18:17 -0700 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: <4C34F828.3080100@voidspace.org.uk> Message-ID: On Thu, Jul 8, 2010 at 07:59, Nick Coghlan wrote: > On Thu, Jul 8, 2010 at 9:13 AM, Benjamin Peterson wrote: >> 2010/7/7 Nick Coghlan : >>> On Thu, Jul 8, 2010 at 7:56 AM, Michael Foord wrote: >>>> Using a class decorator to duplicate each _test_ into two test_* methods >>>> sounds ?like a good approach. >>> >>> Note that parameterised methods have a similar problem to >>> parameterised modules - unittest results are reported in terms of >>> "testmodule.testclass.testfunction", so proper attribution of results >>> in the test output will require additional work. The separate >>> subclasses approach doesn't share this issue, since it changes the >>> value of the second item in accordance with the module under test. >> >> A good parameterized implementation, though, gives the repr() of the >> parameters in failure output. > > That would qualify as "additional work" if your tests aren't already > set up that way (and doesn't cover the case of unexpected exceptions > in a test, where the test method doesn't get to say much about the way > the error is reported). > > I realised during the day that my suggested approach was more > complicated than is actually necessary - once the existing tests have > been moved to a separate module, *that test module* can itself be > imported twice, once with the python version of the module to be > tested and once with the C version. You can then do some hackery to > distinguish the test classes without having to modify the test code > itself (note, the below code should work in theory, but isn't actually > tested): > > ============= > py_module_tests = support.import_fresh_module('moduletester', > fresh=['modulename'], blocked=['_modulename']) > c_module_tests = support.import_fresh_module('moduletester', > fresh=['modulename', '_modulename']) > > test_modules = [py_module_tests, c_module_tests] > suffixes = ["_Py", "_C"] > > for module, suffix in zip(test_modules, suffixes): > ? ?for obj in module.itervalues(): > ? ? ? ?if isinstance(obj, unittest,TestCase): > ? ? ? ? ? ?obj.__name__ += suffix > ? ? ? ? ? ?setattr(module, obj.__name__, obj) > > def test_main(): > ? ?for module in test_modules: > ? ? ? ?module.test_main() > ============= Very cool solution (assuming it works =) ! One issue I see with this is deciding how to organize tests that are specific to one version of a module compared to another. For instance, test_warnings has some tests specific to _warnings because of the hoops it has to jump through in order to get overriding showwarnings and friends to work. I guess I could try to make them generic enough that they don't require a specific module. Otherwise I would insert the module-specific tests into test_warnings to have that module also call gnostic_test_warnings to run the universal tests. From alexander.belopolsky at gmail.com Thu Jul 8 21:24:37 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 8 Jul 2010 15:24:37 -0400 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: <4C34F828.3080100@voidspace.org.uk> Message-ID: On Thu, Jul 8, 2010 at 10:59 AM, Nick Coghlan wrote: .. > I realised during the day that my suggested approach was more > complicated than is actually necessary - once the existing tests have > been moved to a separate module, *that test module* can itself be > imported twice, once with the python version of the module to be > tested and once with the C version. You can then do some hackery to > distinguish the test classes without having to modify the test code > itself (note, the below code should work in theory, but isn't actually > tested): > > ============= > py_module_tests = support.import_fresh_module('moduletester', > fresh=['modulename'], blocked=['_modulename']) > c_module_tests = support.import_fresh_module('moduletester', > fresh=['modulename', '_modulename']) > > test_modules = [py_module_tests, c_module_tests] > suffixes = ["_Py", "_C"] > > for module, suffix in zip(test_modules, suffixes): > ? ?for obj in module.itervalues(): > ? ? ? ?if isinstance(obj, unittest,TestCase): > ? ? ? ? ? ?obj.__name__ += suffix > ? ? ? ? ? ?setattr(module, obj.__name__, obj) > > def test_main(): > ? ?for module in test_modules: > ? ? ? ?module.test_main() > ============= Yes, this is definitely an improvement over my current datetime patch [1]_, but it still requires a custom test_main() and does not make the test cases discoverable by alternative unittest runners. I think that can be fixed by injecting imported TestCase subclasses into the main test module globals. I'll try to implement that for datetime. Thanks, Nick - great idea! .. [1] http://bugs.python.org/file17848/issue7989.diff From solipsis at pitrou.net Thu Jul 8 21:29:29 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 8 Jul 2010 21:29:29 +0200 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? References: <4C34F828.3080100@voidspace.org.uk> Message-ID: <20100708212929.2542e0bd@pitrou.net> On Fri, 9 Jul 2010 00:59:02 +1000 Nick Coghlan wrote: > py_module_tests = support.import_fresh_module('moduletester', > fresh=['modulename'], blocked=['_modulename']) > c_module_tests = support.import_fresh_module('moduletester', > fresh=['modulename', '_modulename']) I don't really like the proliferation of module test helpers, it only makes things confusing and forces you to switch between more files in your editor. By contrast, the subclassing solution is simple, explicit and obvious. (I also wonder what problem this subthread is trying to solve at all. Just my 2 eurocents) Regards Antoine. From python at mrabarnett.plus.com Thu Jul 8 21:52:44 2010 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 08 Jul 2010 20:52:44 +0100 Subject: [Python-Dev] New regex module for 3.2? Message-ID: <4C362C8C.80707@mrabarnett.plus.com> Hi all, I re-implemented the re module, adding new features and speed improvements. It's available at: http://pypi.python.org/pypi/regex under the name "regex" so that it can be tried alongside "re". I'd be interested in any comments or feedback. How does it compare with "re" in terms of speed on real-world data? The benchmarks suggest it should be faster, or at worst comparable. How much interest would there be in putting it in Python 3.2? From alexander.belopolsky at gmail.com Thu Jul 8 22:20:13 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 8 Jul 2010 16:20:13 -0400 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: <20100708212929.2542e0bd@pitrou.net> References: <4C34F828.3080100@voidspace.org.uk> <20100708212929.2542e0bd@pitrou.net> Message-ID: On Thu, Jul 8, 2010 at 3:29 PM, Antoine Pitrou wrote: > On Fri, 9 Jul 2010 00:59:02 +1000 > Nick Coghlan wrote: .. > I don't really like the proliferation of module test helpers, it only > makes things confusing and forces you to switch between more files in > your editor. By contrast, the subclassing solution is simple, explicit > and obvious. > And would require a lot of tedious and error prone work to retrofit existing tests. Since we don't have meta regression tests, there is no obvious way to assure that retrofitting does not change the tests. Note that test_pickle uses both the subclassing solution *and* a helper pickletester module because this neatly separates maulti-implementation machinery from the actual test definitions. > (I also wonder what problem this subthread is trying to solve at all. The problem is to find a simple solution that will allow running existing unit tests written for a C extension on both the original extension and the added pure python equivalent. When the existing tests were developed over many years and have 100+ test cases, this is not as easy task as it would be if you wrote your tests from scratch. From g.brandl at gmx.net Thu Jul 8 23:21:45 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 08 Jul 2010 23:21:45 +0200 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: <87ocei9dvt.fsf@aragost.com> References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707161206.GC1701@idyll.org> <87pqyzt3eg.fsf@hbox.dyndns.org> <20100707232234.767669d9@pitrou.net> <87k4p6kg8g.fsf@uwakimon.sk.tsukuba.ac.jp> <87fwzuk7cp.fsf@uwakimon.sk.tsukuba.ac.jp> <8739vuctt9.fsf@aragost.com> <4C35AB4D.90305@holdenweb.com> <87ocei9dvt.fsf@aragost.com> Message-ID: Am 08.07.2010 17:44, schrieb Martin Geisler: > Steve Holden writes: > >> Martin Geisler wrote: >>> "Stephen J. Turnbull" writes: >>> >>>> Just ask Martin, there are too many possibilities here to worry >>>> about. If maybe we want it, and he is willing to contribute the >>>> parts he wrote to Python under Python's license, then we can worry >>>> about whether we really want it and about how much any required >>>> hoop-jumping will cost. >>> >>> I would be happy to relicense it under the Python license. >> >> I believe the ideal outcome, if it is possible, is for you to sign a >> contributor agreement. This will license your material to the PSF in >> such a way that we can release it under whatever license we deem >> necessary. > > Sure, I'll be happy to sign a contributor agreement if you guys think it > worthwhile to use my little parser and formatter. Problem is, in the case of help() we have no way of knowing whether the given __doc__ string is supposed to be (mini)reST. Of course, reverting to showing the plain content on parsing errors is one possibility, but I can still imagine instances where something is successfully interpreted as reST, but intended to be read and understood verbatim by the author. It's different for Hg, of course, there you can just decide that help texts have to be reST. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ncoghlan at gmail.com Thu Jul 8 23:34:19 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 9 Jul 2010 07:34:19 +1000 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: <4C362C8C.80707@mrabarnett.plus.com> References: <4C362C8C.80707@mrabarnett.plus.com> Message-ID: On Fri, Jul 9, 2010 at 5:52 AM, MRAB wrote: > Hi all, > > I re-implemented the re module, adding new features and speed > improvements. It's available at: > > ? ?http://pypi.python.org/pypi/regex > > under the name "regex" so that it can be tried alongside "re". > > I'd be interested in any comments or feedback. How does it compare with > "re" in terms of speed on real-world data? The benchmarks suggest it > should be faster, or at worst comparable. > > How much interest would there be in putting it in Python 3.2? The list of fixed bugs/new features is certainly impressive. How does Python's test suite go if you drop it in place of the current "re" module? (Ditto for test suites of major applications and frameworks like Django, etc). Off the top of my head, I would say that this won't have enough time to bake properly for inclusion in 3.2, but if the potential benefits and intended backwards compatibility are borne out by real world usage and the code fares well under review then it may be a contender for 3.3. If the backwards compatibility isn't quite there (and can't be improved), then adding it under a name other than "re" wouldn't be impossible, but it would be a harder sell. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From fdrake at acm.org Thu Jul 8 23:41:52 2010 From: fdrake at acm.org (Fred Drake) Date: Thu, 8 Jul 2010 17:41:52 -0400 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707161206.GC1701@idyll.org> <87pqyzt3eg.fsf@hbox.dyndns.org> <20100707232234.767669d9@pitrou.net> <87k4p6kg8g.fsf@uwakimon.sk.tsukuba.ac.jp> <87fwzuk7cp.fsf@uwakimon.sk.tsukuba.ac.jp> <8739vuctt9.fsf@aragost.com> <4C35AB4D.90305@holdenweb.com> <87ocei9dvt.fsf@aragost.com> Message-ID: On Thu, Jul 8, 2010 at 5:21 PM, Georg Brandl wrote: > Problem is, in the case of help() we have no way of knowing whether the > given __doc__ string is supposed to be (mini)reST. ?Of course, reverting > to showing the plain content on parsing errors is one possibility, but > I can still imagine instances where something is successfully interpreted > as reST, but intended to be read and understood verbatim by the author. The docstring processing PEP provides for this: http://www.python.org/dev/peps/pep-0258/#id42 -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein From alexander.belopolsky at gmail.com Thu Jul 8 23:42:59 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 8 Jul 2010 17:42:59 -0400 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707161206.GC1701@idyll.org> <87pqyzt3eg.fsf@hbox.dyndns.org> <20100707232234.767669d9@pitrou.net> <87k4p6kg8g.fsf@uwakimon.sk.tsukuba.ac.jp> <87fwzuk7cp.fsf@uwakimon.sk.tsukuba.ac.jp> <8739vuctt9.fsf@aragost.com> <4C35AB4D.90305@holdenweb.com> <87ocei9dvt.fsf@aragost.com> Message-ID: On Thu, Jul 8, 2010 at 5:21 PM, Georg Brandl wrote: .. > Problem is, in the case of help() we have no way of knowing whether the > given __doc__ string is supposed to be (mini)reST. I am against mark-up in doc-strings, but this problem can be easily solved by placing a magic character at __doc__[0] to indicate that the rest is (mini)reST. The magic character should be chosen to be inconspicuous and unlikely to appear at the start of a plain-text docstting. For example, any type closing braces, ), }. ] will do, or any end of sentence punctuation such as . or !. From ncoghlan at gmail.com Thu Jul 8 23:46:48 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 9 Jul 2010 07:46:48 +1000 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: <4C34F828.3080100@voidspace.org.uk> Message-ID: On Fri, Jul 9, 2010 at 5:24 AM, Alexander Belopolsky wrote: > Yes, this is definitely an improvement over my current datetime patch > [1]_, but it still requires a custom test_main() and does not make the > test cases discoverable by alternative unittest runners. ?I think that > can be fixed by injecting imported TestCase subclasses into the main > test module globals. So include something along the lines of "globals()[obj.__name__] = obj" in the name hacking loop to make the test classes more discoverable? Good idea. Including a comment in the main test module along the lines of your reply to Antoine would be good, too (i.e. this is acknowledged as being something of a hack to make sure we don't break the datetime tests when updating them to be applied to both the existing C module and the new pure Python equivalent). As Antoine says, using explicit subclasses is a *much* cleaner way of doing this kind of thing when the tests are being written from scratch to test multiple implementations within a single interpreter. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From benjamin at python.org Thu Jul 8 23:48:45 2010 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 8 Jul 2010 16:48:45 -0500 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: <4C362C8C.80707@mrabarnett.plus.com> References: <4C362C8C.80707@mrabarnett.plus.com> Message-ID: 2010/7/8 MRAB : > Hi all, > > I re-implemented the re module, adding new features and speed > improvements. It's available at: > > ? ?http://pypi.python.org/pypi/regex > > under the name "regex" so that it can be tried alongside "re". > > I'd be interested in any comments or feedback. How does it compare with > "re" in terms of speed on real-world data? The benchmarks suggest it > should be faster, or at worst comparable. > > How much interest would there be in putting it in Python 3.2? It would really be nice if you explained incrementally everything you changed, so it could better be evaluated. -- Regards, Benjamin From solipsis at pitrou.net Thu Jul 8 23:53:45 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 8 Jul 2010 23:53:45 +0200 Subject: [Python-Dev] New regex module for 3.2? References: <4C362C8C.80707@mrabarnett.plus.com> Message-ID: <20100708235345.7a856263@pitrou.net> On Thu, 08 Jul 2010 20:52:44 +0100 MRAB wrote: > > I'd be interested in any comments or feedback. How does it compare with > "re" in terms of speed on real-world data? The benchmarks suggest it > should be faster, or at worst comparable. Can you publish these benchmarks somewhere? (or send them here) > How much interest would there be in putting it in Python 3.2? I think there's certainly interest (especially given that the original re module doesn't really have an expert and active maintainer). Since it's a very big change (and a rather annoying to undo one), though, it must really not add any maintenance problems, and ideally you should promise to maintain it at least for a couple of years. Bonus points if the internals are sufficiently documented, too. Thanks Antoine. From python at mrabarnett.plus.com Thu Jul 8 23:54:36 2010 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 08 Jul 2010 22:54:36 +0100 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> Message-ID: <4C36491C.2050300@mrabarnett.plus.com> Nick Coghlan wrote: > On Fri, Jul 9, 2010 at 5:52 AM, MRAB wrote: >> Hi all, >> >> I re-implemented the re module, adding new features and speed >> improvements. It's available at: >> >> http://pypi.python.org/pypi/regex >> >> under the name "regex" so that it can be tried alongside "re". >> >> I'd be interested in any comments or feedback. How does it compare with >> "re" in terms of speed on real-world data? The benchmarks suggest it >> should be faster, or at worst comparable. >> >> How much interest would there be in putting it in Python 3.2? > > The list of fixed bugs/new features is certainly impressive. How does > Python's test suite go if you drop it in place of the current "re" > module? (Ditto for test suites of major applications and frameworks > like Django, etc). > > Off the top of my head, I would say that this won't have enough time > to bake properly for inclusion in 3.2, but if the potential benefits > and intended backwards compatibility are borne out by real world usage > and the code fares well under review then it may be a contender for > 3.3. If the backwards compatibility isn't quite there (and can't be > improved), then adding it under a name other than "re" wouldn't be > impossible, but it would be a harder sell. > You should be able to replace: import re with: import regex as re and still have everything work the same, ie it's backwards compatible with re. From fdrake at acm.org Fri Jul 9 00:01:20 2010 From: fdrake at acm.org (Fred Drake) Date: Thu, 8 Jul 2010 18:01:20 -0400 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707161206.GC1701@idyll.org> <87pqyzt3eg.fsf@hbox.dyndns.org> <20100707232234.767669d9@pitrou.net> <87k4p6kg8g.fsf@uwakimon.sk.tsukuba.ac.jp> <87fwzuk7cp.fsf@uwakimon.sk.tsukuba.ac.jp> <8739vuctt9.fsf@aragost.com> <4C35AB4D.90305@holdenweb.com> <87ocei9dvt.fsf@aragost.com> Message-ID: On Thu, Jul 8, 2010 at 5:42 PM, Alexander Belopolsky wrote: > I am against mark-up in doc-strings, but this problem can be easily > solved by placing a magic character at __doc__[0] to indicate that the > rest is ?(mini)reST. Or __docformat__ can be set appropriately. See: http://www.python.org/dev/peps/pep-0258/#id42 -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein From ncoghlan at gmail.com Fri Jul 9 00:13:01 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 9 Jul 2010 08:13:01 +1000 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: <4C36491C.2050300@mrabarnett.plus.com> References: <4C362C8C.80707@mrabarnett.plus.com> <4C36491C.2050300@mrabarnett.plus.com> Message-ID: On Fri, Jul 9, 2010 at 7:54 AM, MRAB wrote: > You should be able to replace: > > ? ?import re > > with: > > ? ?import regex as re > > and still have everything work the same, ie it's backwards compatible > with re. That's not what I'm asking. I'm asking what happens if you take an existing Python installation's re module, move it aside, and drop regex in its place as "re.py". Doing that and then running Python's own test suite as well as the test suites of major Python applications and frameworks like Twisted, Zope and Django would provide solid evidence that the new version really *is* backwards compatible, rather than that it is *meant* to be backwards compatible. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From steve at holdenweb.com Fri Jul 9 01:14:11 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 08 Jul 2010 19:14:11 -0400 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707161206.GC1701@idyll.org> <87pqyzt3eg.fsf@hbox.dyndns.org> <20100707232234.767669d9@pitrou.net> <87k4p6kg8g.fsf@uwakimon.sk.tsukuba.ac.jp> <87fwzuk7cp.fsf@uwakimon.sk.tsukuba.ac.jp> <8739vuctt9.fsf@aragost.com> <4C35AB4D.90305@holdenweb.com> <87ocei9dvt.fsf@aragost.com> Message-ID: Fred Drake wrote: > On Thu, Jul 8, 2010 at 5:42 PM, Alexander Belopolsky > wrote: >> I am against mark-up in doc-strings, but this problem can be easily >> solved by placing a magic character at __doc__[0] to indicate that the >> rest is (mini)reST. > > Or __docformat__ can be set appropriately. See: > > http://www.python.org/dev/peps/pep-0258/#id42 > That is _so_ Python 2 ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From victor.stinner at haypocalc.com Fri Jul 9 02:11:35 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Fri, 9 Jul 2010 02:11:35 +0200 Subject: [Python-Dev] Full unicode support for the import machinery Message-ID: <201007090211.35828.victor.stinner@haypocalc.com> Hi, I'm trying to fix Python to support undecodable bytes in the Python path since some months ago. My first try was really huge and sometimes ugly. When it was possible, I extracted some short and simple patches and applied them to py3k (sometimes with an issue, sometimes directly in the svn). When it was no more possible to split the big patch, I restarted the work from scratch. The main change from my previous try is that I changed import.c to use unicode strings instead of byte strings. With the surrogate hack (PEP 383), unicode is a superset of bytes and so it is "forward compatible". I just created a branch called "import_unicode" (based on py3k) including all my patches. It's still a work in progress. It is possible to start Python installed in an undecodable path (eg. directory with an non-ASCII character with C locale for Linux), which is an huge progress, but some tests are still failing. The last biggest problem is that code object filenames are not reencoded after that the file system encoding is changed (but sys.path and sys.modules filenames are reencoded). I think that I will register all code objects into a list to be able to reencode their filename attribute (and then drop the list). I created an svn branch because I think that it's easier to review short commits than one unique huge patch. The branch also helps me to share the branch between different computers, and allow other people to review the commits (and/or contribute!). Some people will maybe understand better my work with the "whole picture" :-) -- There are at least 4 issues related to this work: #3080: Full unicode import system #4352: imp.find_module() fails with a UnicodeDecodeError when called with non-ASCII search paths #8611: Python3 doesn't support locale different than utf8 and an non-ASCII path (POSIX) #8988: import + coding = failure (3.1.2/win32) -- Some examples of previous issues related to my secret goal (patch import machinery): #8391: os.execvpe() doesn't support surrogates in env #8393: subprocess: support undecodable current working directory on POSIX OS #8412: os.system() doesn't support surrogates nor bytes #8485: Don't accept bytearray as filenames, or simplify the API #?8514: Add fsencode() functions to os module #8610: Python3/POSIX: errors if file system encoding is None (-> create initfsencoding() in pythonrun.c) #8715: Create PyUnicode_EncodeFSDefault() function ... -- Victor Stinner http://www.haypocalc.com/ From python at mrabarnett.plus.com Fri Jul 9 02:35:14 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 09 Jul 2010 01:35:14 +0100 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C36491C.2050300@mrabarnett.plus.com> Message-ID: <4C366EC2.3000203@mrabarnett.plus.com> Nick Coghlan wrote: > On Fri, Jul 9, 2010 at 7:54 AM, MRAB wrote: >> You should be able to replace: >> >> import re >> >> with: >> >> import regex as re >> >> and still have everything work the same, ie it's backwards compatible >> with re. > > That's not what I'm asking. I'm asking what happens if you take an > existing Python installation's re module, move it aside, and drop > regex in its place as "re.py". > > Doing that and then running Python's own test suite as well as the > test suites of major Python applications and frameworks like Twisted, > Zope and Django would provide solid evidence that the new version > really *is* backwards compatible, rather than that it is *meant* to be > backwards compatible. > I had to recompile the .pyd to change its internal name from "regex" to "re", but apart from that it passed Python's own test suite except for where I expected it to fail: 1. Some of the inline flags are scoped; for example, putting "(?i)" at the end of a regex will now have no effect because it's no longer a global, all-or-nothing, flag. 2. The .sub method will treat unmatched groups in an expansion as empty strings. The re module raises an exception in such cases, which means that users currently need a workaround. From alexander.belopolsky at gmail.com Fri Jul 9 03:44:48 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 8 Jul 2010 21:44:48 -0400 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: <4C34F828.3080100@voidspace.org.uk> Message-ID: On Thu, Jul 8, 2010 at 5:46 PM, Nick Coghlan wrote: .. > So include something along the lines of "globals()[obj.__name__] = > obj" in the name hacking loop to make the test classes more > discoverable? Good idea. > As often happens, a good idea turns quite ugly when facing real world realities. I've uploaded a new patch at http://bugs.python.org/issue7989 and here is what I had to do to make this work for datetime: ========== import unittest import sys; sys.modules['_pickle'] = None from test.support import import_fresh_module, run_unittest TESTS = 'test.datetimetester' pure_tests = import_fresh_module(TESTS, fresh=['datetime', '_strptime', 'time'], blocked=['_datetime']) fast_tests = import_fresh_module(TESTS, fresh=['datetime', '_datetime', '_strptime', 'time']) test_modules = [pure_tests, fast_tests] test_suffixes = ["_Pure", "_Fast"] globs = globals() for module, suffix in zip(test_modules, test_suffixes): for name, cls in module.__dict__.items(): if isinstance(cls, type) and issubclass(cls, unittest.TestCase): name += suffix cls.__name__ = name globs[name] = cls def setUp(self, module=module, setup=cls.setUp): self._save_sys_modules = sys.modules.copy() sys.modules[TESTS] = module sys.modules['datetime'] = module.datetime_module sys.modules['_strptime'] = module.datetime_module._strptime setup(self) def tearDown(self, teardown=cls.tearDown): teardown(self) sys.modules = self._save_sys_modules cls.setUp = setUp cls.tearDown = tearDown def test_main(): run_unittest(__name__) ========= and it still requires that '_pickle' is disabled to pass pickle tests. From stephen at xemacs.org Fri Jul 9 03:39:17 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 09 Jul 2010 10:39:17 +0900 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707161206.GC1701@idyll.org> <87pqyzt3eg.fsf@hbox.dyndns.org> <20100707232234.767669d9@pitrou.net> <87k4p6kg8g.fsf@uwakimon.sk.tsukuba.ac.jp> <87fwzuk7cp.fsf@uwakimon.sk.tsukuba.ac.jp> <8739vuctt9.fsf@aragost.com> <4C35AB4D.90305@holdenweb.com> <87ocei9dvt.fsf@aragost.com> Message-ID: <8739vtjuwq.fsf@uwakimon.sk.tsukuba.ac.jp> Steve Holden writes: > That is _so_ Python 2 ;-) High praise! From orsenthil at gmail.com Fri Jul 9 05:52:32 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Fri, 9 Jul 2010 09:22:32 +0530 Subject: [Python-Dev] Can ftp url start with file:// ? Message-ID: <20100709035232.GA6427@remy> Strictly not a Python question, but I wanted to know from the experience of others in this list. Is this is valid ftp url? # file://ftp.example.com/blah.txt (an ftp URL) My answer is no. When we have the scheme specifically mentioned as file:// it is no point in considering it as ftp url (which should start with ftp://). If I go ahead with this assumption and fix a bug in stdlib, I am introducing a regression because at the moment the above is considered a ftp url. -- Senthil A real diplomat is one who can cut his neighbor's throat without having his neighbour notice it. -- Trygve Lie From steve at pearwood.info Fri Jul 9 06:23:40 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 9 Jul 2010 14:23:40 +1000 Subject: [Python-Dev] Can ftp url start with file:// ? In-Reply-To: <20100709035232.GA6427@remy> References: <20100709035232.GA6427@remy> Message-ID: <201007091423.41200.steve@pearwood.info> On Fri, 9 Jul 2010 01:52:32 pm Senthil Kumaran wrote: > Strictly not a Python question, but I wanted to know from the > experience of others in this list. > > Is this is valid ftp url? > > # file://ftp.example.com/blah.txt (an ftp URL) > > My answer is no. When we have the scheme specifically mentioned as > file:// it is no point in considering it as ftp url (which should > start with ftp://). I agree. Just because the host is *called* ftp doesn't mean you should use the ftp protocol to get the file. http://en.wikipedia.org/wiki/File_URI_scheme > If I go ahead with this assumption and fix a bug in stdlib, I am > introducing a regression because at the moment the above is > considered a ftp url. Do you have a url for the bug report? -- Steven D'Aprano From orsenthil at gmail.com Fri Jul 9 06:38:04 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Fri, 9 Jul 2010 10:08:04 +0530 Subject: [Python-Dev] Can ftp url start with file:// ? In-Reply-To: <201007091423.41200.steve@pearwood.info> References: <20100709035232.GA6427@remy> <201007091423.41200.steve@pearwood.info> Message-ID: <20100709043804.GA9768@remy> On Fri, Jul 09, 2010 at 02:23:40PM +1000, Steven D'Aprano wrote: > > Is this is valid ftp url? > > > > # file://ftp.example.com/blah.txt (an ftp URL) > > > > My answer is no. When we have the scheme specifically mentioned as > > file:// it is no point in considering it as ftp url (which should > > start with ftp://). > > I agree. Just because the host is *called* ftp doesn't mean you should > use the ftp protocol to get the file. It was not just for the host being called ftp.example.com It was for a pattern that file:/// is local file (correct) and file://localhost/somepath is again local file (correct again) but file://anyhost.domain/file.txt is actually ftp (pretty weird). > Do you have a url for the bug report? http://bugs.python.org/issue8801 Don't go into the suggestion in the report, but just notice that file url lead to an ftp error exception. -- Senthil From g.brandl at gmx.net Fri Jul 9 08:51:30 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 09 Jul 2010 08:51:30 +0200 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707161206.GC1701@idyll.org> <87pqyzt3eg.fsf@hbox.dyndns.org> <20100707232234.767669d9@pitrou.net> <87k4p6kg8g.fsf@uwakimon.sk.tsukuba.ac.jp> <87fwzuk7cp.fsf@uwakimon.sk.tsukuba.ac.jp> <8739vuctt9.fsf@aragost.com> <4C35AB4D.90305@holdenweb.com> <87ocei9dvt.fsf@aragost.com> Message-ID: Am 09.07.2010 00:01, schrieb Fred Drake: > On Thu, Jul 8, 2010 at 5:42 PM, Alexander Belopolsky > wrote: >> I am against mark-up in doc-strings, but this problem can be easily >> solved by placing a magic character at __doc__[0] to indicate that the >> rest is (mini)reST. Ugh. :) > Or __docformat__ can be set appropriately. See: > > http://www.python.org/dev/peps/pep-0258/#id42 Yes, but[tm] it is not always easy to find the correct module to look for __docformat__ when given an object. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Fri Jul 9 08:54:31 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 09 Jul 2010 08:54:31 +0200 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: <4C366EC2.3000203@mrabarnett.plus.com> References: <4C362C8C.80707@mrabarnett.plus.com> <4C36491C.2050300@mrabarnett.plus.com> <4C366EC2.3000203@mrabarnett.plus.com> Message-ID: Am 09.07.2010 02:35, schrieb MRAB: >> That's not what I'm asking. I'm asking what happens if you take an >> existing Python installation's re module, move it aside, and drop >> regex in its place as "re.py". >> >> Doing that and then running Python's own test suite as well as the >> test suites of major Python applications and frameworks like Twisted, >> Zope and Django would provide solid evidence that the new version >> really *is* backwards compatible, rather than that it is *meant* to be >> backwards compatible. >> > I had to recompile the .pyd to change its internal name from "regex" to > "re", but apart from that it passed Python's own test suite except for > where I expected it to fail: > > 1. Some of the inline flags are scoped; for example, putting "(?i)" at > the end of a regex will now have no effect because it's no longer a > global, all-or-nothing, flag. That is problematic. I've often seen people put these flags at the end of a regex, probably for readability purposes. IMHO it would be better to limit flag scoping to the explicit (?flags-flags: ) groups. > 2. The .sub method will treat unmatched groups in an expansion as empty > strings. The re module raises an exception in such cases, which means > that users currently need a workaround. That is a change for the better, I would say. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From kristjan at ccpgames.com Fri Jul 9 10:38:53 2010 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Fri, 9 Jul 2010 08:38:53 +0000 Subject: [Python-Dev] Py_Buffer and pep3118 Message-ID: <930F189C8A437347B80DF2C156F7EC7F0AAC03D4B8@exchis.ccp.ad.local> Hello there. I?m in the process of upgrading code to use python 2.7 and the new Py_Buffer objects. I have, however, come across an inconsistency in the documentation, both in the pep and the python docs, with the actual implementation. In addition to the different function signatures mentioned in issue 8946, the Py_Buffer member 'obj' is undocumented. The inline comment in object.h mentiones that it is a "borrowed reference" but this appears inconsistent with the implementation of PyBuffer_Release(). In short, what the documentation fails to mention (and the pep) is whether posessing a locked Py_Buffer structure also constitutes holding a reference to the exporting object? I think it does, but is this guaranteed to be so? Also, is the PyBuffer_Release() guaranteed to be "safe" to call multiple times (as the current implementation is)? Kristj?n -------------- next part -------------- An HTML attachment was scrubbed... URL: From mg at aragost.com Fri Jul 9 13:40:05 2010 From: mg at aragost.com (Martin Geisler) Date: Fri, 09 Jul 2010 13:40:05 +0200 Subject: [Python-Dev] query: docstring formatting in python distutils code References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707161206.GC1701@idyll.org> <87pqyzt3eg.fsf@hbox.dyndns.org> <20100707232234.767669d9@pitrou.net> <87k4p6kg8g.fsf@uwakimon.sk.tsukuba.ac.jp> <87fwzuk7cp.fsf@uwakimon.sk.tsukuba.ac.jp> <8739vuctt9.fsf@aragost.com> <4C35AB4D.90305@holdenweb.com> <87ocei9dvt.fsf@aragost.com> Message-ID: <87mxu0zxwq.fsf@aragost.com> Georg Brandl writes: > Am 08.07.2010 17:44, schrieb Martin Geisler: >> Steve Holden writes: >> >>> Martin Geisler wrote: >>>> "Stephen J. Turnbull" writes: >>>> >>>>> Just ask Martin, there are too many possibilities here to worry >>>>> about. If maybe we want it, and he is willing to contribute the >>>>> parts he wrote to Python under Python's license, then we can worry >>>>> about whether we really want it and about how much any required >>>>> hoop-jumping will cost. >>>> >>>> I would be happy to relicense it under the Python license. >>> >>> I believe the ideal outcome, if it is possible, is for you to sign a >>> contributor agreement. This will license your material to the PSF in >>> such a way that we can release it under whatever license we deem >>> necessary. >> >> Sure, I'll be happy to sign a contributor agreement if you guys think it >> worthwhile to use my little parser and formatter. > > Problem is, in the case of help() we have no way of knowing whether the > given __doc__ string is supposed to be (mini)reST. Of course, reverting > to showing the plain content on parsing errors is one possibility, but > I can still imagine instances where something is successfully interpreted > as reST, but intended to be read and understood verbatim by the author. The minirst module is actually designed to never fail: the text is first divided into paragraphs, and the paragraphs are then looked at one by one to see if they look like something more specific like a list. > It's different for Hg, of course, there you can just decide that help > texts have to be reST. Right, good point. At first I figured that you could also "just" convert the docstrings in Python, but since the help builtin is used on lots of code outside of Python this isn't really enough. -- Martin Geisler aragost Trifork -- Professional Mercurial support http://aragost.com/mercurial/ From fdrake at acm.org Fri Jul 9 14:35:11 2010 From: fdrake at acm.org (Fred Drake) Date: Fri, 9 Jul 2010 08:35:11 -0400 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707161206.GC1701@idyll.org> <87pqyzt3eg.fsf@hbox.dyndns.org> <20100707232234.767669d9@pitrou.net> <87k4p6kg8g.fsf@uwakimon.sk.tsukuba.ac.jp> <87fwzuk7cp.fsf@uwakimon.sk.tsukuba.ac.jp> <8739vuctt9.fsf@aragost.com> <4C35AB4D.90305@holdenweb.com> <87ocei9dvt.fsf@aragost.com> Message-ID: On Fri, Jul 9, 2010 at 2:51 AM, Georg Brandl wrote: > Yes, but[tm] it is not always easy to find the correct module to look for > __docformat__ when given an object. True. That PEP was written before decorators were common, in particular. That's changed the landscape in substantial ways. It may be that at this point, there's no good way to deal with formatting in docstrings at anything other than the project level, where policies for the code can be set if desired. Ideally, the format of a docstring would be captured along with the docstring itself. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein From kristjan at ccpgames.com Fri Jul 9 14:55:59 2010 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Fri, 9 Jul 2010 12:55:59 +0000 Subject: [Python-Dev] Py_Buffer and pep3118 In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F0AAC03D4B8@exchis.ccp.ad.local> References: <930F189C8A437347B80DF2C156F7EC7F0AAC03D4B8@exchis.ccp.ad.local> Message-ID: <930F189C8A437347B80DF2C156F7EC7F0AAC03D542@exchis.ccp.ad.local> In addition, the PyBufferProcs documentation has not bee updated to reflect the new buffer interface. From: python-dev-bounces+kristjan=ccpgames.com at python.org [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Kristj?n Valur J?nsson Sent: 9. j?l? 2010 08:39 To: Python-Dev Subject: [Python-Dev] Py_Buffer and pep3118 Hello there. I?m in the process of upgrading code to use python 2.7 and the new Py_Buffer objects. I have, however, come across an inconsistency in the documentation, both in the pep and the python docs, with the actual implementation. In addition to the different function signatures mentioned in issue 8946, the Py_Buffer member 'obj' is undocumented. The inline comment in object.h mentiones that it is a "borrowed reference" but this appears inconsistent with the implementation of PyBuffer_Release(). In short, what the documentation fails to mention (and the pep) is whether posessing a locked Py_Buffer structure also constitutes holding a reference to the exporting object? I think it does, but is this guaranteed to be so? Also, is the PyBuffer_Release() guaranteed to be "safe" to call multiple times (as the current implementation is)? Kristj?n -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Jul 9 15:01:56 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 9 Jul 2010 23:01:56 +1000 Subject: [Python-Dev] Full unicode support for the import machinery In-Reply-To: <201007090211.35828.victor.stinner@haypocalc.com> References: <201007090211.35828.victor.stinner@haypocalc.com> Message-ID: On Fri, Jul 9, 2010 at 10:11 AM, Victor Stinner wrote: > I created an svn branch because I think that it's easier to review short > commits than one unique huge patch. The branch also helps me to share the > branch between different computers, and allow other people to review the > commits (and/or contribute!). Thanks for doing that, it does indeed make it much easier to follow your train of thought. The overall approach looks sane, and while I haven't done a line-by-line review of every patch on the branch, the ones I did examine in detail all looked correct. I'll try to keep up as you make more changes. You're a brave soul, venturing into that there-is-no-Unicode-there-is-only-ASCII maze, but you've already made substantial improvements. The addition of new more Unicode friendly C APIs for errors and warnings should be of general use outside this work as well (but given where you're up to, I don't advocate trying to cherry pick them off the branch). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From solipsis at pitrou.net Fri Jul 9 15:17:36 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 9 Jul 2010 15:17:36 +0200 Subject: [Python-Dev] Py_Buffer and pep3118 References: <930F189C8A437347B80DF2C156F7EC7F0AAC03D4B8@exchis.ccp.ad.local> Message-ID: <20100709151736.184754d7@pitrou.net> On Fri, 9 Jul 2010 08:38:53 +0000 Kristj?n Valur J?nsson wrote: > > In short, what the documentation fails to mention (and the pep) is whether posessing a > locked Py_Buffer structure also constitutes holding a reference to the exporting object? It does. > I think it does, but is this guaranteed to be so? Now that's a good question. This was changed so that every producer of tp_buffer doesn't have to do it by itself. In any case, acquiring the buffer means it will be valid until it is released, which is what you should primarily care about. > Also, is the PyBuffer_Release() guaranteed to be "safe" to call > multiple times (as the current implementation is)? It's not, but we could make it so. Regards Antoine. From barry at python.org Fri Jul 9 15:28:15 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 9 Jul 2010 09:28:15 -0400 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707141217.377d6a93@heresy> <20100707204611.78153447@pitrou.net> Message-ID: <20100709092815.57bce276@heresy> On Jul 07, 2010, at 12:50 PM, Brett Cannon wrote: >On Wed, Jul 7, 2010 at 11:46, Antoine Pitrou >wrote: >> On Wed, 7 Jul 2010 14:12:17 -0400 >> Barry Warsaw wrote: >>> On Jul 07, 2010, at 07:30 PM, Georg Brandl wrote: >>> >>> >Overall, I think that we can make stdlib docstrings valid reST -- >>> >even if it's reST without much markup -- but valid, so that people >>> >pulling in stdlib doc- strings into Sphinx docs won't get ugly >>> >warnings. >>> > >>> >What I would *not* like to see is heavy markup and Sphinx >>> >specifics -- that would only make sense if we included the >>> >docstrings in the docs, and I don't see that coming. >>> >>> Does it make sense to add (reST-style) epydoc markup for API >>> signatures? E.g. >> >> It really looks ugly (and annoying to decipher) when viewed in plain >> text. > >I agree. And it is highly repetitive since the signature information >is right there already. All of that info in those annotations can >easily be written in paragraph form if needed and honestly would read >better to my eyes. I actually find it easier to glean the signature details from a regularized docstring than from prose. Especially for autogenerated API documentation, the formal specification lends a consistency to the output that prose doesn't often provide. IME, there isn't much (unnecessary) repeating yourself. Either way, we need to be diligent in accurately describing the signature and semantics of our APIs. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From techtonik at gmail.com Fri Jul 9 16:06:59 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Fri, 9 Jul 2010 17:06:59 +0300 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: <4C362C8C.80707@mrabarnett.plus.com> References: <4C362C8C.80707@mrabarnett.plus.com> Message-ID: On Thu, Jul 8, 2010 at 10:52 PM, MRAB wrote: > Hi all, > > I re-implemented the re module, adding new features and speed > improvements. It's available at: > > ? ?http://pypi.python.org/pypi/regex > > under the name "regex" so that it can be tried alongside "re". > > I'd be interested in any comments or feedback. How does it compare with > "re" in terms of speed on real-world data? The benchmarks suggest it > should be faster, or at worst comparable. And where are the benchmarks? In particular it would be interesting to see it compared both to re from stdlib and re2 from http://code.google.com/p/re2/ -- anatoly t. From dickinsm at gmail.com Fri Jul 9 16:40:13 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 9 Jul 2010 15:40:13 +0100 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists. Message-ID: While looking at a parser module issue (http://bugs.python.org/issue9154) I noticed that Python's grammar doesn't permit trailing commas after keyword-only args. That is, def f(a, b,): pass is valid syntax, while def f(*, a, b,): pass is not. I was just curious whether the latter was deliberate or an oversight. And if an oversight, is it worth fixing after the moratorium expires? (See also http://bugs.python.org/issue2009.) Mark From jyasskin at gmail.com Fri Jul 9 17:59:43 2010 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Fri, 9 Jul 2010 08:59:43 -0700 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> Message-ID: On Fri, Jul 9, 2010 at 7:06 AM, anatoly techtonik wrote: > On Thu, Jul 8, 2010 at 10:52 PM, MRAB wrote: >> Hi all, >> >> I re-implemented the re module, adding new features and speed >> improvements. It's available at: >> >> ? ?http://pypi.python.org/pypi/regex >> >> under the name "regex" so that it can be tried alongside "re". >> >> I'd be interested in any comments or feedback. How does it compare with >> "re" in terms of speed on real-world data? The benchmarks suggest it >> should be faster, or at worst comparable. > > And where are the benchmarks? > In particular it would be interesting to see it compared both to re > from stdlib and re2 from ?http://code.google.com/p/re2/ While the re2 comparison might be interesting from an abstract standpoint, it intentionally supports a different regex language from Python so that it can run faster and use less memory. Since re2 can never replace Python's re module, it doesn't make sense to hold MRAB's new module to that standard. From status at bugs.python.org Fri Jul 9 18:08:06 2010 From: status at bugs.python.org (Python tracker) Date: Fri, 9 Jul 2010 18:08:06 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20100709160806.C420B78139@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2010-07-02 - 2010-07-09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2813 open (+52) / 18217 closed (+15) / 21030 total (+67) Open issues with patches: 1133 Average duration of open issues: 706 days. Median duration of open issues: 503 days. Open Issues Breakdown open 2779 (+51) languishing 14 ( +0) pending 19 ( +1) Issues Created Or Reopened (75) _______________________________ backport 3.0-style \u/\U processing in raw strings when unicod 2010-07-07 http://bugs.python.org/issue2570 reopened alexandre.vassalotti patch audioop: check that length is a multiple of the size 2010-07-04 CLOSED http://bugs.python.org/issue7673 reopened pitrou patch sys.getwindowsversion as PyStructSequence 2010-07-08 CLOSED http://bugs.python.org/issue7766 reopened brian.curtin patch, patch, needs review dis module documentation gives no indication of the dangers of 2010-07-05 http://bugs.python.org/issue7829 reopened tjreedy needs review `make patchcheck` should check the whitespace of .c/.h files 2010-07-05 http://bugs.python.org/issue8912 reopened belopolsky patch, needs review Year range in timetuple 2010-07-05 http://bugs.python.org/issue9005 reopened belopolsky patch Separate compilation of time and datetime modules 2010-07-04 http://bugs.python.org/issue9012 reopened doko patch str.capitalize() should not lower the rest of the string 2010-07-04 CLOSED http://bugs.python.org/issue9091 reopened tjreedy patch test_coercion fails in refleak runs 2010-07-02 CLOSED http://bugs.python.org/issue9145 created pitrou Segfault in hashlib in OpenSSL FIPS mode using non-FIPS-compli 2010-07-02 http://bugs.python.org/issue9146 created dmalcolm patch dis.show_code() variant that accepts source strings (and retur 2010-07-03 http://bugs.python.org/issue9147 created ncoghlan os.execve puts process to background on windows 2010-07-03 http://bugs.python.org/issue9148 created techtonik colorsys.py function rgb_to_hls 2010-07-03 CLOSED http://bugs.python.org/issue9149 created David.Hood IDLE should not save trailing whitespace after strip trailing 2010-07-03 http://bugs.python.org/issue9150 created rhprogrammer Demo/classes/Dates.py does not work in 3.x 2010-07-03 CLOSED http://bugs.python.org/issue9151 created belopolsky patch, easy Dead code in datetime module 2010-07-04 CLOSED http://bugs.python.org/issue9152 created belopolsky patch, easy Run tests and demos as part of the test suite 2010-07-04 http://bugs.python.org/issue9153 created merwok Parser module doesn't understand function annotations. 2010-07-04 http://bugs.python.org/issue9154 created mark.dickinson patch Reserve COMPARE_OP for rich comparisons 2010-07-04 http://bugs.python.org/issue9155 created serprex patch socket._fileobject: read raises AttributeError when closed in 2010-07-05 http://bugs.python.org/issue9156 created panzi Allow inspection of used decorators on a function 2010-07-05 http://bugs.python.org/issue9157 created mitar PyArg_ParseTuple y* documentation is incorrect 2010-07-05 CLOSED http://bugs.python.org/issue9158 created terrence patch asyncore.file_wrapper does not register its fd to socket_map 2010-07-05 CLOSED http://bugs.python.org/issue9159 created kimhyunkang Documentation link of 3.2 not working at docs.python.org 2010-07-05 CLOSED http://bugs.python.org/issue9160 created eli.bendersky add_option in optparse no longer accepts unicode string 2010-07-05 http://bugs.python.org/issue9161 created cmcqueen1975 License for multiprocessing files 2010-07-05 http://bugs.python.org/issue9162 created FladischerMichael easy test_gdb fails 2010-07-05 CLOSED http://bugs.python.org/issue9163 created Michael.Blume 2.7 sysconfig should handle arch duplicates while building uni 2010-07-05 http://bugs.python.org/issue9164 created jyrkiwahlstedt Add math.isfinite() 2010-07-05 http://bugs.python.org/issue9165 created benjamin.peterson patch minor misstatement in What's New in 2.7 2010-07-05 CLOSED http://bugs.python.org/issue9166 created stutzbach patch argv double encoding on OSX 2010-07-05 http://bugs.python.org/issue9167 created piro patch setuid in smtp.py sheds privileges before binding port 2010-07-05 http://bugs.python.org/issue9168 created alfmel patch Deprecate use of more than 3 positional arguments in timedelta 2010-07-05 http://bugs.python.org/issue9169 created belopolsky zipfile cannot read AES encrypted files 2010-07-05 http://bugs.python.org/issue9170 created aymill sysconfig module does not support -m option 2010-07-05 http://bugs.python.org/issue9171 created giampaolo.rodola easy zipfile.extractall always raises an OSError after successfully 2010-07-05 http://bugs.python.org/issue9172 created aymill logger statement not guarded in shutil._make_tarball 2010-07-06 http://bugs.python.org/issue9173 created paulsmith patch unittest assertEqual error output sub-optimal for single strin 2010-07-06 http://bugs.python.org/issue9174 created r.david.murray patch ctypes doesn't build on hp-ux 2010-07-06 http://bugs.python.org/issue9175 created ronaldoussoren module termios doesn't build on HP-UX 2010-07-06 http://bugs.python.org/issue9176 created ronaldoussoren ssl.read/write on closed socket raises AttributeError 2010-07-06 http://bugs.python.org/issue9177 created cbay Compile problem on HP-UX 11 2010-07-06 CLOSED http://bugs.python.org/issue9178 created ronaldoussoren Lookback with group references incorrect (two issues?) 2010-07-06 http://bugs.python.org/issue9179 created acooke Memory Error 2010-07-06 http://bugs.python.org/issue9180 created freakcycle Solaris extension building does not work with 64 bit python 2010-07-06 http://bugs.python.org/issue9181 created redcomet document ???--??? as a way to distinguish option w/ narg='+' f 2010-07-06 http://bugs.python.org/issue9182 reopened merwok easy Intern UTC timezone 2010-07-06 http://bugs.python.org/issue9183 created belopolsky patch, easy open() doc: default value for buffering is -1, not None 2010-07-06 http://bugs.python.org/issue9184 created haypo patch os.getcwd causes infinite loop on solaris 2010-07-07 http://bugs.python.org/issue9185 created csernazs math.log1p(-1.0) produces OverflowError; should be ValueError 2010-07-07 CLOSED http://bugs.python.org/issue9186 created mark.dickinson patch Can't find file "vcvars64.bat" 2010-07-07 CLOSED http://bugs.python.org/issue9187 created Lloyd.Sheng test_gdb fails for UCS2 builds with UCS2 gdb 2010-07-07 http://bugs.python.org/issue9188 created pitrou patch Improve CFLAGS handling 2010-07-07 http://bugs.python.org/issue9189 created jyasskin Undefined behaviour in _PyFloat_Pack4 2010-07-07 http://bugs.python.org/issue9190 created mark.dickinson winreg.c:Reg2Py() may leak memory (in unusual circumstances) 2010-07-07 http://bugs.python.org/issue9191 created stutzbach Add a a way to determine float format 2010-07-07 http://bugs.python.org/issue9192 created benjamin.peterson Versioned .so files 2010-07-07 http://bugs.python.org/issue9193 created barry patch winreg:fixupMultiSZ should check that P < Q in the inner loop 2010-07-08 http://bugs.python.org/issue9194 created stutzbach Link in docs from "String Formatting Operations" to "Template 2010-07-08 http://bugs.python.org/issue9195 created cmcqueen1975 Improve docs for string interpolation "%s" re Unicode strings 2010-07-08 http://bugs.python.org/issue9196 created cmcqueen1975 subprocess module causing crash 2010-07-08 http://bugs.python.org/issue9197 created mpalluat Should repr() print unicode characters outside the BMP? 2010-07-08 http://bugs.python.org/issue9198 created amaury.forgeotdarc patch distutil upload command crashes when displaying server respons 2010-07-08 http://bugs.python.org/issue9199 created pje str.isprintable() is always False for large code points 2010-07-08 http://bugs.python.org/issue9200 created amaury.forgeotdarc IDLE: raises Exception TclError in a special case 2010-07-08 http://bugs.python.org/issue9201 created doko Update platform.win32_ver() to account for change to #8413 2010-07-08 CLOSED http://bugs.python.org/issue9202 created brian.curtin patch, needs review Use computed gotos by default 2010-07-08 http://bugs.python.org/issue9203 created pitrou The documentation of PyType_Type in py3k mentions types.TypeTy 2010-07-08 http://bugs.python.org/issue9204 created durban Parent process hanging in multiprocessing if children terminat 2010-07-08 http://bugs.python.org/issue9205 created gdb patch repr(os.stat(..)) is broken by the structseq change 2010-07-08 CLOSED http://bugs.python.org/issue9206 created belopolsky patch multiprocessing occasionally spits out exception during shutdo 2010-07-08 http://bugs.python.org/issue9207 created gdb SMTPHandler in the logging module does not handle unicode stri 2010-07-08 http://bugs.python.org/issue9208 created norbidur pstats module crashes on trailing backslash 2010-07-09 http://bugs.python.org/issue9209 created isandler remove --with-wctype-functions configure option 2010-07-09 http://bugs.python.org/issue9210 created amaury.forgeotdarc patch Cannot close socket after ssl handshake failed 2010-07-09 CLOSED http://bugs.python.org/issue9211 created metachris Issues Now Closed (59) ______________________ Broken pipes should be handled better in 2.x 940 days http://bugs.python.org/issue1596 tjreedy Backport of PEP 3102 "keyword-only arguments" to 2.6 912 days http://bugs.python.org/issue1745 alexandre.vassalotti patch email.MIMEText.MIMEText.as_string incorrectly folding long sub 888 days http://bugs.python.org/issue1974 nicdumz test_al converted to unittest 884 days http://bugs.python.org/issue2022 benjamin.peterson patch test_cd converted to unittest 884 days http://bugs.python.org/issue2023 benjamin.peterson patch test_gl.py converted to unittest 884 days http://bugs.python.org/issue2024 benjamin.peterson patch test_sunaudiodev.py converted to unittest 883 days http://bugs.python.org/issue2044 benjamin.peterson patch Python should expose a pthread_cond_timedwait API for threadin 748 days http://bugs.python.org/issue3155 pitrou Cleanup of tp_basicsize inheritance 732 days http://bugs.python.org/issue3268 benjamin.peterson patch Fork + shelve causes shelve corruption and backtrace 566 days http://bugs.python.org/issue4679 belopolsky Error in atexit._run_exitfuncs [...] Exception expected for va 527 days http://bugs.python.org/issue5089 mark.dickinson urlencode does not handle "bytes" and could easily handle alte 480 days http://bugs.python.org/issue5468 orsenthil patch Implement a way to change the python process name 458 days http://bugs.python.org/issue5672 piro patch What should happen to signals when the main thread dies? 402 days http://bugs.python.org/issue6162 tjreedy Enhance dis.dis to autocompile codestrings 357 days http://bugs.python.org/issue6507 merwok patch, easy Do we have something like os.pid_exists()? 350 days http://bugs.python.org/issue6554 tjreedy interrupts during long writes cause connection corruption with 346 days http://bugs.python.org/issue6587 pitrou otparse: processing variable number of short option arguments 346 days http://bugs.python.org/issue6592 tjreedy audioop: check that length is a multiple of the size 0 days http://bugs.python.org/issue7673 mark.dickinson patch sys.getwindowsversion as PyStructSequence 0 days http://bugs.python.org/issue7766 brian.curtin patch, patch, needs review Fnmatch cache is never cleared during usage 156 days http://bugs.python.org/issue7846 r.david.murray patch, needs review MemoryError While Executing Python Code 146 days http://bugs.python.org/issue7899 mark.dickinson String interpolation doesn't work with sys.version_info 84 days http://bugs.python.org/issue8413 belopolsky patch Let cmd.cmd.intro be unicode friendly 76 days http://bugs.python.org/issue8470 r.david.murray itertools.filterfalse() function missing 75 days http://bugs.python.org/issue8472 georg.brandl test_gdb, gdb/libpython.py: Unable to read information on pyth 78 days http://bugs.python.org/issue8482 dmalcolm test_gdb can fail with compiler opts 66 days http://bugs.python.org/issue8605 pitrou patch 2to3 fails with assertion failure on "from itertools import *" 31 days http://bugs.python.org/issue8892 benjamin.peterson patch PyBuffer_Release signature in 3.1 documentation is incorrect 30 days http://bugs.python.org/issue8946 krisvale Provide parseable repr to datetime.timezone 21 days http://bugs.python.org/issue9000 belopolsky patch, easy 2.7: eval hangs on AIX 16 days http://bugs.python.org/issue9020 pitrou patch At least some Tools utilities are still Python 2 9 days http://bugs.python.org/issue9083 merwok str.capitalize() should not lower the rest of the string 1 days http://bugs.python.org/issue9091 orsenthil patch Make python-m pickletools do something useful 9 days http://bugs.python.org/issue9094 belopolsky patch, easy importlib doesn't handle valid marshalled data with invalid se 7 days http://bugs.python.org/issue9096 brett.cannon remove numbers from 3-.. level entries in docs toc 5 days http://bugs.python.org/issue9106 tjreedy Parser module fails to validate class decorators correctly. 4 days http://bugs.python.org/issue9128 mark.dickinson patch Parser module fails on short relative imports 4 days http://bugs.python.org/issue9130 mark.dickinson patch RuntimeError when profiling Decimal 8 days http://bugs.python.org/issue9136 mark.dickinson patch Add examples for str.format() 1 days http://bugs.python.org/issue9139 ezio.melotti patch Import error for multiprocessing in 2.7rc2 on Windows 1 days http://bugs.python.org/issue9144 jnoller patch test_coercion fails in refleak runs 2 days http://bugs.python.org/issue9145 flox colorsys.py function rgb_to_hls 2 days http://bugs.python.org/issue9149 David.Hood Demo/classes/Dates.py does not work in 3.x 1 days http://bugs.python.org/issue9151 merwok patch, easy Dead code in datetime module 1 days http://bugs.python.org/issue9152 belopolsky patch, easy PyArg_ParseTuple y* documentation is incorrect 1 days http://bugs.python.org/issue9158 terrence patch asyncore.file_wrapper does not register its fd to socket_map 0 days http://bugs.python.org/issue9159 kimhyunkang Documentation link of 3.2 not working at docs.python.org 0 days http://bugs.python.org/issue9160 georg.brandl test_gdb fails 3 days http://bugs.python.org/issue9163 dmalcolm minor misstatement in What's New in 2.7 0 days http://bugs.python.org/issue9166 georg.brandl patch Compile problem on HP-UX 11 0 days http://bugs.python.org/issue9178 loewis math.log1p(-1.0) produces OverflowError; should be ValueError 0 days http://bugs.python.org/issue9186 mark.dickinson patch Can't find file "vcvars64.bat" 0 days http://bugs.python.org/issue9187 skrah Update platform.win32_ver() to account for change to #8413 0 days http://bugs.python.org/issue9202 brian.curtin patch, needs review repr(os.stat(..)) is broken by the structseq change 0 days http://bugs.python.org/issue9206 benjamin.peterson patch Cannot close socket after ssl handshake failed 0 days http://bugs.python.org/issue9211 metachris logging handlers raise exception on level 2367 days http://bugs.python.org/issue876421 Mickey.Killianey socket_htons does not work under AIX 64-bit 2299 days http://bugs.python.org/issue921868 skrah easy Add an updating load function in pickle 2230 days http://bugs.python.org/issue960821 belopolsky Top Issues Most Discussed (10) ______________________________ 20 Regexp 2.7 (modifications to current re 2.2.2) 815 days open http://bugs.python.org/issue2636 19 Use Py_UCS4 instead of Py_UNICODE in unicodectype.c 523 days open http://bugs.python.org/issue5127 18 Add pure Python implementation of datetime module to CPython 137 days open http://bugs.python.org/issue7989 16 multiprocessing occasionally spits out exception during shutdow 1 days open http://bugs.python.org/issue9207 13 Reserve COMPARE_OP for rich comparisons 5 days open http://bugs.python.org/issue9155 11 Should repr() print unicode characters outside the BMP? 1 days open http://bugs.python.org/issue9198 11 String interpolation doesn't work with sys.version_info 84 days closed http://bugs.python.org/issue8413 10 Fnmatch cache is never cleared during usage 156 days closed http://bugs.python.org/issue7846 9 difflib.SequenceMatcher not matching long sequences 773 days open http://bugs.python.org/issue2986 8 2.7 sysconfig should handle arch duplicates while building univ 4 days open http://bugs.python.org/issue9164 From eric at trueblade.com Fri Jul 9 18:36:01 2010 From: eric at trueblade.com (Eric Smith) Date: Fri, 09 Jul 2010 12:36:01 -0400 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists. In-Reply-To: References: Message-ID: <4C374FF1.9010600@trueblade.com> On 7/9/10 10:40 AM, Mark Dickinson wrote: > While looking at a parser module issue > (http://bugs.python.org/issue9154) I noticed that Python's grammar > doesn't permit trailing commas after keyword-only args. That is, > > def f(a, b,): pass > > is valid syntax, while > > def f(*, a, b,): pass > > is not. I was just curious whether the latter was deliberate or an > oversight. And if an oversight, is it worth fixing after the > moratorium expires? (See also http://bugs.python.org/issue2009.) I use trailing commas all the time in argument lists. I don't use keyword-only args much, but I will when I switch to 3.x. I'd like to see this fixed. I'd argue it's a bug fix, but that's me. -- Eric. From janssen at parc.com Fri Jul 9 18:41:49 2010 From: janssen at parc.com (Bill Janssen) Date: Fri, 9 Jul 2010 09:41:49 PDT Subject: [Python-Dev] Can ftp url start with file:// ? In-Reply-To: <20100709043804.GA9768@remy> References: <20100709035232.GA6427@remy> <201007091423.41200.steve@pearwood.info> <20100709043804.GA9768@remy> Message-ID: <43622.1278693709@parc.com> Senthil Kumaran wrote: > On Fri, Jul 09, 2010 at 02:23:40PM +1000, Steven D'Aprano wrote: > > > Is this is valid ftp url? > > > > > > # file://ftp.example.com/blah.txt (an ftp URL) > > > > > > My answer is no. When we have the scheme specifically mentioned as > > > file:// it is no point in considering it as ftp url (which should > > > start with ftp://). > > > > I agree. Just because the host is *called* ftp doesn't mean you should > > use the ftp protocol to get the file. > > It was not just for the host being called ftp.example.com > > It was for a pattern that file:/// is local file (correct) and > file://localhost/somepath is again local file (correct again) but > file://anyhost.domain/file.txt is actually ftp (pretty weird). RFC 1738 explicitly says that "file:///" is pretty much useless for anything except host=localhost: ``The file URL scheme is unusual in that it does not specify an Internet protocol or access method for such files; as such, its utility in network protocols between hosts is limited.'' So, FTP is *not* the "default protocol". On the other hand, if actually begins with "ftp.", it's a pretty good guess that FTP will work. Similarly, if actually begins with "www.", it's a pretty good guess that HTTP will work. This seems to me like a practicality-vs.-purity consideration. Bill From python at mrabarnett.plus.com Fri Jul 9 19:28:08 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 09 Jul 2010 18:28:08 +0100 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> Message-ID: <4C375C28.80404@mrabarnett.plus.com> anatoly techtonik wrote: > On Thu, Jul 8, 2010 at 10:52 PM, MRAB wrote: >> Hi all, >> >> I re-implemented the re module, adding new features and speed >> improvements. It's available at: >> >> http://pypi.python.org/pypi/regex >> >> under the name "regex" so that it can be tried alongside "re". >> >> I'd be interested in any comments or feedback. How does it compare with >> "re" in terms of speed on real-world data? The benchmarks suggest it >> should be faster, or at worst comparable. > > And where are the benchmarks? > In particular it would be interesting to see it compared both to re > from stdlib and re2 from http://code.google.com/p/re2/ > The benchmarks bm_regex_effbot.py and bm_regex_v8.py both perform multiple runs of the tests multiple times, giving just the total times for each set. Here are the averages: Python26 BENCHMARK re regex ratio bm_regex_effbot 0.135secs 0.083secs 1.63 bm_regex_v8 0.153secs 0.085secs 1.80 Python31 BENCHMARK re regex ratio bm_regex_effbot 0.138secs 0.083secs 1.66 bm_regex_v8 0.170secs 0.091secs 1.87 From tlesher at gmail.com Fri Jul 9 19:31:34 2010 From: tlesher at gmail.com (Tim Lesher) Date: Fri, 9 Jul 2010 13:31:34 -0400 Subject: [Python-Dev] Can ftp url start with file:// ? In-Reply-To: <43622.1278693709@parc.com> References: <20100709035232.GA6427@remy> <201007091423.41200.steve@pearwood.info> <20100709043804.GA9768@remy> <43622.1278693709@parc.com> Message-ID: On Fri, Jul 9, 2010 at 12:41, Bill Janssen wrote: > So, FTP is *not* the "default protocol". On the other hand, if > actually begins with "ftp.", it's a pretty good guess that FTP will > work. Actually, FTP *is* the default protocol for most URLs with hostnames in urllib.py. urllib.open_file() delegates to open_ftp() if there's a any host other than the exact string "localhost", and open_local_file() otherwise. >>> import urllib >>> f =urllib.urlopen('file:///foo.txt') >>> f =urllib.urlopen('file://localhost/foo.txt') >>> f = urllib.urlopen('file://www.google.com/') Traceback (most recent call last): File "", line 1, in File "c:\python25\lib\urllib.py", line 82, in urlopen return opener.open(url) File "c:\python25\lib\urllib.py", line 190, in open return getattr(self, name)(url) File "c:\python25\lib\urllib.py", line 457, in open_file return self.open_ftp(url) File "c:\python25\lib\urllib.py", line 538, in open_ftp ftpwrapper(user, passwd, host, port, dirs) File "c:\python25\lib\urllib.py", line 844, in __init__ self.init() File "c:\python25\lib\urllib.py", line 850, in init self.ftp.connect(self.host, self.port) File "c:\python25\lib\ftplib.py", line 129, in connect raise socket.error, msg IOError: [Errno ftp error] (10060, 'Operation timed out') >>> f =urllib.urlopen('file://127.0.0.1/foo.txt') Traceback (most recent call last): File "", line 1, in File "c:\python25\lib\urllib.py", line 82, in urlopen return opener.open(url) File "c:\python25\lib\urllib.py", line 190, in open return getattr(self, name)(url) File "c:\python25\lib\urllib.py", line 457, in open_file return self.open_ftp(url) File "c:\python25\lib\urllib.py", line 538, in open_ftp ftpwrapper(user, passwd, host, port, dirs) File "c:\python25\lib\urllib.py", line 844, in __init__ self.init() File "c:\python25\lib\urllib.py", line 850, in init self.ftp.connect(self.host, self.port) File "c:\python25\lib\ftplib.py", line 129, in connect raise socket.error, msg IOError: [Errno ftp error] (10061, 'Connection refused') -- Tim Lesher -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.brandl at gmx.net Fri Jul 9 19:34:05 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 09 Jul 2010 19:34:05 +0200 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists. In-Reply-To: <4C374FF1.9010600@trueblade.com> References: <4C374FF1.9010600@trueblade.com> Message-ID: Am 09.07.2010 18:36, schrieb Eric Smith: > On 7/9/10 10:40 AM, Mark Dickinson wrote: >> While looking at a parser module issue >> (http://bugs.python.org/issue9154) I noticed that Python's grammar >> doesn't permit trailing commas after keyword-only args. That is, >> >> def f(a, b,): pass >> >> is valid syntax, while >> >> def f(*, a, b,): pass >> >> is not. I was just curious whether the latter was deliberate or an >> oversight. And if an oversight, is it worth fixing after the >> moratorium expires? (See also http://bugs.python.org/issue2009.) > > I use trailing commas all the time in argument lists. I don't use > keyword-only args much, but I will when I switch to 3.x. I'd like to see > this fixed. I'd argue it's a bug fix, but that's me. +1 Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From tlesher at gmail.com Fri Jul 9 19:42:26 2010 From: tlesher at gmail.com (Tim Lesher) Date: Fri, 9 Jul 2010 13:42:26 -0400 Subject: [Python-Dev] Can ftp url start with file:// ? In-Reply-To: References: <20100709035232.GA6427@remy> <201007091423.41200.steve@pearwood.info> <20100709043804.GA9768@remy> <43622.1278693709@parc.com> Message-ID: To be clear, Python 2.x's urllib.urlopen() has this issue; 3.1's urllib.request.urlopen() rejects non-local hosts in a file URL. -- Tim Lesher -------------- next part -------------- An HTML attachment was scrubbed... URL: From janssen at parc.com Fri Jul 9 20:04:11 2010 From: janssen at parc.com (Bill Janssen) Date: Fri, 9 Jul 2010 11:04:11 PDT Subject: [Python-Dev] Can ftp url start with file:// ? In-Reply-To: References: <20100709035232.GA6427@remy> <201007091423.41200.steve@pearwood.info> <20100709043804.GA9768@remy> <43622.1278693709@parc.com> Message-ID: <37151.1278698651@parc.com> Tim Lesher wrote: > On Fri, Jul 9, 2010 at 12:41, Bill Janssen wrote: > > > So, FTP is *not* the "default protocol". On the other hand, if > > actually begins with "ftp.", it's a pretty good guess that FTP will > > work. > > > Actually, FTP *is* the default protocol for most URLs with hostnames in > urllib.py. > > urllib.open_file() delegates to open_ftp() if there's a any host other than > the exact string "localhost", and open_local_file() otherwise. I just meant to point out that it's not specified that way by any RFC. And, while FTP might have been a good default choice last century when urllib.py was originally written, times have changed. I'd suggest that HTTP is a better (more likely to succeed) default choice in this century. If we want to perpetuate these guessing heuristics, I'd suggest using FTP if the hostname starts with "ftp.", and HTTP if the hostname starts with "www.", and raise an error otherwise. Bill > >>> import urllib > >>> f =urllib.urlopen('file:///foo.txt') > >>> f =urllib.urlopen('file://localhost/foo.txt') > >>> f = urllib.urlopen('file://www.google.com/') > Traceback (most recent call last): > File "", line 1, in > File "c:\python25\lib\urllib.py", line 82, in urlopen > return opener.open(url) > File "c:\python25\lib\urllib.py", line 190, in open > return getattr(self, name)(url) > File "c:\python25\lib\urllib.py", line 457, in open_file > return self.open_ftp(url) > File "c:\python25\lib\urllib.py", line 538, in open_ftp > ftpwrapper(user, passwd, host, port, dirs) > File "c:\python25\lib\urllib.py", line 844, in __init__ > self.init() > File "c:\python25\lib\urllib.py", line 850, in init > self.ftp.connect(self.host, self.port) > File "c:\python25\lib\ftplib.py", line 129, in connect > raise socket.error, msg > IOError: [Errno ftp error] (10060, 'Operation timed out') > >>> f =urllib.urlopen('file://127.0.0.1/foo.txt') > Traceback (most recent call last): > File "", line 1, in > File "c:\python25\lib\urllib.py", line 82, in urlopen > return opener.open(url) > File "c:\python25\lib\urllib.py", line 190, in open > return getattr(self, name)(url) > File "c:\python25\lib\urllib.py", line 457, in open_file > return self.open_ftp(url) > File "c:\python25\lib\urllib.py", line 538, in open_ftp > ftpwrapper(user, passwd, host, port, dirs) > File "c:\python25\lib\urllib.py", line 844, in __init__ > self.init() > File "c:\python25\lib\urllib.py", line 850, in init > self.ftp.connect(self.host, self.port) > File "c:\python25\lib\ftplib.py", line 129, in connect > raise socket.error, msg > IOError: [Errno ftp error] (10061, 'Connection refused') > -- > Tim Lesher > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com From p.f.moore at gmail.com Fri Jul 9 20:23:05 2010 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 9 Jul 2010 19:23:05 +0100 Subject: [Python-Dev] Can ftp url start with file:// ? In-Reply-To: <37151.1278698651@parc.com> References: <20100709035232.GA6427@remy> <201007091423.41200.steve@pearwood.info> <20100709043804.GA9768@remy> <43622.1278693709@parc.com> <37151.1278698651@parc.com> Message-ID: On 9 July 2010 19:04, Bill Janssen wrote: > If we want to perpetuate these guessing heuristics, I'd suggest using > FTP if the hostname starts with "ftp.", and HTTP if the hostname starts > with "www.", and raise an error otherwise. >From what Tim says, it sounds like guessing is only in 2.x. Removing the guessing clearly isn't a bug fix, so there's nothing to do in the 2.x line (as there won't be a 2.8). 3.1 already has the stricter (and in my view correct) behaviour, and I see no reason to introduce the 2.x behaviour into Python 3. So I'd say there's nothing to do. Paul. From fdrake at acm.org Fri Jul 9 20:32:04 2010 From: fdrake at acm.org (Fred Drake) Date: Fri, 9 Jul 2010 14:32:04 -0400 Subject: [Python-Dev] Can ftp url start with file:// ? In-Reply-To: <37151.1278698651@parc.com> References: <20100709035232.GA6427@remy> <201007091423.41200.steve@pearwood.info> <20100709043804.GA9768@remy> <43622.1278693709@parc.com> <37151.1278698651@parc.com> Message-ID: On Fri, Jul 9, 2010 at 2:04 PM, Bill Janssen wrote: >?I'd suggest that > HTTP is a better (more likely to succeed) default choice in this century. FTP access also more often reflected the actual file hierarchy of the machine, so trying that path as a system path is more likely to work that I'd expect to see for HTTP. Really, I'd expect any non-local file: URLs to be kicked back to the application, and let it decide to re-write if it wants to. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein From steve at holdenweb.com Fri Jul 9 20:33:59 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 09 Jul 2010 14:33:59 -0400 Subject: [Python-Dev] Can ftp url start with file:// ? In-Reply-To: References: <20100709035232.GA6427@remy> <201007091423.41200.steve@pearwood.info> <20100709043804.GA9768@remy> <43622.1278693709@parc.com> <37151.1278698651@parc.com> Message-ID: Paul Moore wrote: > On 9 July 2010 19:04, Bill Janssen wrote: >> If we want to perpetuate these guessing heuristics, I'd suggest using >> FTP if the hostname starts with "ftp.", and HTTP if the hostname starts >> with "www.", and raise an error otherwise. > >>From what Tim says, it sounds like guessing is only in 2.x. Removing > the guessing clearly isn't a bug fix, so there's nothing to do in the > 2.x line (as there won't be a 2.8). 3.1 already has the stricter (and > in my view correct) behaviour, and I see no reason to introduce the > 2.x behaviour into Python 3. > > So I'd say there's nothing to do. > Paul. This /could/ become a bugfix in 2.7.x if people thought it was a sufficiently egregious bug to need fixing. Given that the matter is only now coming to light it's probably bets to let sleeping dogs lie. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From collinwinter at google.com Fri Jul 9 20:58:47 2010 From: collinwinter at google.com (Collin Winter) Date: Fri, 9 Jul 2010 11:58:47 -0700 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: <4C375C28.80404@mrabarnett.plus.com> References: <4C362C8C.80707@mrabarnett.plus.com> <4C375C28.80404@mrabarnett.plus.com> Message-ID: On Fri, Jul 9, 2010 at 10:28 AM, MRAB wrote: > anatoly techtonik wrote: >> >> On Thu, Jul 8, 2010 at 10:52 PM, MRAB wrote: >>> >>> Hi all, >>> >>> I re-implemented the re module, adding new features and speed >>> improvements. It's available at: >>> >>> ? http://pypi.python.org/pypi/regex >>> >>> under the name "regex" so that it can be tried alongside "re". >>> >>> I'd be interested in any comments or feedback. How does it compare with >>> "re" in terms of speed on real-world data? The benchmarks suggest it >>> should be faster, or at worst comparable. >> >> And where are the benchmarks? >> In particular it would be interesting to see it compared both to re >> from stdlib and re2 from ?http://code.google.com/p/re2/ >> > The benchmarks bm_regex_effbot.py and bm_regex_v8.py both perform > multiple runs of the tests multiple times, giving just the total times > for each set. Here are the averages: > > Python26 > BENCHMARK ? ? ? ?re ? ? ? ? regex ? ? ?ratio > bm_regex_effbot ?0.135secs ?0.083secs ?1.63 > bm_regex_v8 ? ? ?0.153secs ?0.085secs ?1.80 > > > Python31 > BENCHMARK ? ? ? ?re ? ? ? ? regex ? ? ?ratio > bm_regex_effbot ?0.138secs ?0.083secs ?1.66 > bm_regex_v8 ? ? ?0.170secs ?0.091secs ?1.87 Out of curiosity, what are the results for the bm_regex_compile benchmark? Collin From brett at python.org Fri Jul 9 20:55:03 2010 From: brett at python.org (Brett Cannon) Date: Fri, 9 Jul 2010 11:55:03 -0700 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: <20100709092815.57bce276@heresy> References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707141217.377d6a93@heresy> <20100707204611.78153447@pitrou.net> <20100709092815.57bce276@heresy> Message-ID: On Fri, Jul 9, 2010 at 06:28, Barry Warsaw wrote: > On Jul 07, 2010, at 12:50 PM, Brett Cannon wrote: > >>On Wed, Jul 7, 2010 at 11:46, Antoine Pitrou >>wrote: >>> On Wed, 7 Jul 2010 14:12:17 -0400 >>> Barry Warsaw wrote: >>>> On Jul 07, 2010, at 07:30 PM, Georg Brandl wrote: >>>> >>>> >Overall, I think that we can make stdlib docstrings valid reST -- >>>> >even if it's reST without much markup -- but valid, so that people >>>> >pulling in stdlib doc- strings into Sphinx docs won't get ugly >>>> >warnings. >>>> > >>>> >What I would *not* like to see is heavy markup and Sphinx >>>> >specifics -- that would only make sense if we included the >>>> >docstrings in the docs, and I don't see that coming. >>>> >>>> Does it make sense to add (reST-style) epydoc markup for API >>>> signatures? E.g. >>> >>> It really looks ugly (and annoying to decipher) when viewed in plain >>> text. >> >>I agree. And it is highly repetitive since the signature information >>is right there already. All of that info in those annotations can >>easily be written in paragraph form if needed and honestly would read >>better to my eyes. > > I actually find it easier to glean the signature details from a regularized > docstring than from prose. ?Especially for autogenerated API documentation, > the formal specification lends a consistency to the output that prose doesn't > often provide. ?IME, there isn't much (unnecessary) repeating yourself. The key point there is "autogenerated API documentation", which Python does not do. > > Either way, we need to be diligent in accurately describing the signature and > semantics of our APIs. Of course. I think this is going to be something our crazy FLUFL likes but the kids don't. =) -Brett > > -Barry > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > > From barry at python.org Fri Jul 9 21:23:50 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 9 Jul 2010 15:23:50 -0400 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <20100707141217.377d6a93@heresy> <20100707204611.78153447@pitrou.net> <20100709092815.57bce276@heresy> Message-ID: <20100709152350.11b6c1d2@heresy> On Jul 09, 2010, at 11:55 AM, Brett Cannon wrote: >I think this is going to be something our crazy FLUFL likes but the >kids don't. =) Don't worry. If you're lucky, you'll get old too. Your eyes will go bad and your mind will think more about tapioca. By then you might even remember that the FLUFL was right after all. :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From tjreedy at udel.edu Fri Jul 9 21:31:33 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 09 Jul 2010 15:31:33 -0400 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists. In-Reply-To: References: Message-ID: On 7/9/2010 10:40 AM, Mark Dickinson wrote: > While looking at a parser module issue > (http://bugs.python.org/issue9154) I noticed that Python's grammar > doesn't permit trailing commas after keyword-only args. That is, > > def f(a, b,): pass > > is valid syntax, while > > def f(*, a, b,): pass > > is not. I was just curious whether the latter was deliberate or an > oversight. And if an oversight, is it worth fixing after the > moratorium expires? (See also http://bugs.python.org/issue2009.) The way I read the grammar in the function definition section of the manual, a trailing comma is only allowed after a 'defparameter' that does not come after a *. In this sense, the above is not a bug. However, while the grammar in the call doc does not allow trailing commas, the following note says "A trailing comma may be present after the positional and keyword arguments but does not affect the semantics." This has the combined effect >>> def f(*,a): pass >>> f(a=1) >>> f(a=1,) >>> def f(*,a,): pass SyntaxError: invalid syntax This violates the important principle that allowed def and call arg sequences should match to the extent sensible and possible. In this sense, the SyntaxError is a bug. So I would fix this now for 3.2 and notify the other implementors. -- Terry Jan Reedy From python at mrabarnett.plus.com Fri Jul 9 21:35:16 2010 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 09 Jul 2010 20:35:16 +0100 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C375C28.80404@mrabarnett.plus.com> Message-ID: <4C3779F4.3090808@mrabarnett.plus.com> Collin Winter wrote: > On Fri, Jul 9, 2010 at 10:28 AM, MRAB wrote: >> anatoly techtonik wrote: >>> On Thu, Jul 8, 2010 at 10:52 PM, MRAB wrote: >>>> Hi all, >>>> >>>> I re-implemented the re module, adding new features and speed >>>> improvements. It's available at: >>>> >>>> http://pypi.python.org/pypi/regex >>>> >>>> under the name "regex" so that it can be tried alongside "re". >>>> >>>> I'd be interested in any comments or feedback. How does it compare with >>>> "re" in terms of speed on real-world data? The benchmarks suggest it >>>> should be faster, or at worst comparable. >>> And where are the benchmarks? >>> In particular it would be interesting to see it compared both to re >>> from stdlib and re2 from http://code.google.com/p/re2/ >>> >> The benchmarks bm_regex_effbot.py and bm_regex_v8.py both perform >> multiple runs of the tests multiple times, giving just the total times >> for each set. Here are the averages: >> >> Python26 >> BENCHMARK re regex ratio >> bm_regex_effbot 0.135secs 0.083secs 1.63 >> bm_regex_v8 0.153secs 0.085secs 1.80 >> >> >> Python31 >> BENCHMARK re regex ratio >> bm_regex_effbot 0.138secs 0.083secs 1.66 >> bm_regex_v8 0.170secs 0.091secs 1.87 > > Out of curiosity, what are the results for the bm_regex_compile benchmark? > I concentrated my efforts on the matching speed because regexes tend to be compiled only once, and are cached anyway, so I don't think it's as important. The results are: Python26 BENCHMARK re regex ratio bm_regex_compile 0.897secs 2.792secs 0.32 Python31 BENCHMARK re regex ratio bm_regex_compile 0.902secs 2.731secs 0.33 If anyone can demonstrate that it'll have a significant impact in practice then I will, of course, look into it further. From dinov at microsoft.com Fri Jul 9 21:37:30 2010 From: dinov at microsoft.com (Dino Viehland) Date: Fri, 9 Jul 2010 19:37:30 +0000 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists. In-Reply-To: References: Message-ID: <1A472770E042064698CB5ADC83A12ACD48AF0F9D@TK5EX14MBXC118.redmond.corp.microsoft.com> Terry wrote: > This violates the important principle that allowed def and call arg > sequences should match to the extent sensible and possible. In this > sense, the SyntaxError is a bug. So I would fix this now for 3.2 and > notify the other implementors. +1 on fixing it - trailing commas are awesome. I'm always annoyed in C# where I frequently can't use them. This seems like a bug fix level change that should be easy for the other implementations to fix. From fdrake at acm.org Fri Jul 9 22:14:06 2010 From: fdrake at acm.org (Fred Drake) Date: Fri, 9 Jul 2010 16:14:06 -0400 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: <4C3779F4.3090808@mrabarnett.plus.com> References: <4C362C8C.80707@mrabarnett.plus.com> <4C375C28.80404@mrabarnett.plus.com> <4C3779F4.3090808@mrabarnett.plus.com> Message-ID: On Fri, Jul 9, 2010 at 3:35 PM, MRAB wrote: > I concentrated my efforts on the matching speed because regexes tend to > be compiled only once, and are cached anyway, so I don't think it's as > important. I think most here will agree with that, but it might be good to keep in mind that the sre implementation has already made that trade-off once. I don't remember what the compilation slowdown was at the time, but I'd be surprised if Google can't find it, given sufficient fu. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein From dickinsm at gmail.com Fri Jul 9 22:26:21 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 9 Jul 2010 21:26:21 +0100 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists. In-Reply-To: <1A472770E042064698CB5ADC83A12ACD48AF0F9D@TK5EX14MBXC118.redmond.corp.microsoft.com> References: <1A472770E042064698CB5ADC83A12ACD48AF0F9D@TK5EX14MBXC118.redmond.corp.microsoft.com> Message-ID: On Fri, Jul 9, 2010 at 8:37 PM, Dino Viehland wrote: > Terry wrote: >> This violates the important principle that allowed def and call arg >> sequences should match to the extent sensible and possible. In this >> sense, the SyntaxError is a bug. So I would fix this now for 3.2 and >> notify the other implementors. > > +1 on fixing it - trailing commas are awesome. ?I'm always annoyed in > C# where I frequently can't use them. ?This seems like a bug fix level > change that should be easy for the other implementations to fix. Thanks for all the feedback. If the grammar is changed to allow "def f(*, a,): pass", that still leaves some more open questions: which of the following should be valid? (1) def f(*args,): pass (2) def f(**kwargs,): pass (3) def f(*,): pass Just for the sake of simplicity it would seem to make sense allow all these, even if there's no obvious immediate use; for me, it keeps the mental footprint of the language small---I don't have to remember when the comma is or isn't allowed. Note that (1) and (2) aren't valid (and never have been, as far as I know) in Python 2.x, though. (Neither is (3), of course, since keyword-only arguments are 3.x only.) (3) is a bit subtle: "def f(*): pass" is actually allowed by the grammar, but produces a SyntaxError later on, when the parse tree is converted to AST: >>> def f(*): pass ... File "", line 1 SyntaxError: named arguments must follow bare * So it probably doesn't matter much whether (3) is permitted by the grammar or not. -- Mark From g.brandl at gmx.net Fri Jul 9 22:42:13 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 09 Jul 2010 22:42:13 +0200 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists. In-Reply-To: References: <1A472770E042064698CB5ADC83A12ACD48AF0F9D@TK5EX14MBXC118.redmond.corp.microsoft.com> Message-ID: Am 09.07.2010 22:26, schrieb Mark Dickinson: > On Fri, Jul 9, 2010 at 8:37 PM, Dino Viehland wrote: >> Terry wrote: >>> This violates the important principle that allowed def and call arg >>> sequences should match to the extent sensible and possible. In this >>> sense, the SyntaxError is a bug. So I would fix this now for 3.2 and >>> notify the other implementors. >> >> +1 on fixing it - trailing commas are awesome. I'm always annoyed in >> C# where I frequently can't use them. This seems like a bug fix level >> change that should be easy for the other implementations to fix. > > Thanks for all the feedback. > > If the grammar is changed to allow "def f(*, a,): pass", that still > leaves some more open questions: which of the following should be > valid? > > (1) def f(*args,): pass > (2) def f(**kwargs,): pass > (3) def f(*,): pass IMO all of them (though as you mention, (3) doesn't matter.) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From bj.wahl at oracle.com Fri Jul 9 22:43:03 2010 From: bj.wahl at oracle.com (William Wahl) Date: Fri, 9 Jul 2010 13:43:03 -0700 Subject: [Python-Dev] New regex module for 3.2? Message-ID: H as long as we aren't the ones writing the check:) BJ ----------Original Message---------- From: Fred Drake Sent: Fri, July 09, 2010 1:16 PM To: MRAB Cc: Python-Dev Subject: Re: [Python-Dev] New regex module for 3.2? On Fri, Jul 9, 2010 at 3:35 PM, MRAB wrote: > I concentrated my efforts on the matching speed because regexes tend to > be compiled only once, and are cached anyway, so I don't think it's as > important. I think most here will agree with that, but it might be good to keep in mind that the sre implementation has already made that trade-off once. I don't remember what the compilation slowdown was at the time, but I'd be surprised if Google can't find it, given sufficient fu. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/bj.wahl%40oracle.com From eric at trueblade.com Fri Jul 9 23:30:35 2010 From: eric at trueblade.com (Eric Smith) Date: Fri, 09 Jul 2010 17:30:35 -0400 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists. In-Reply-To: References: <1A472770E042064698CB5ADC83A12ACD48AF0F9D@TK5EX14MBXC118.redmond.corp.microsoft.com> Message-ID: <4C3794FB.9030803@trueblade.com> On 7/9/2010 4:42 PM, Georg Brandl wrote: > Am 09.07.2010 22:26, schrieb Mark Dickinson: >> On Fri, Jul 9, 2010 at 8:37 PM, Dino Viehland wrote: >>> Terry wrote: >>>> This violates the important principle that allowed def and call arg >>>> sequences should match to the extent sensible and possible. In this >>>> sense, the SyntaxError is a bug. So I would fix this now for 3.2 and >>>> notify the other implementors. >>> >>> +1 on fixing it - trailing commas are awesome. I'm always annoyed in >>> C# where I frequently can't use them. This seems like a bug fix level >>> change that should be easy for the other implementations to fix. >> >> Thanks for all the feedback. >> >> If the grammar is changed to allow "def f(*, a,): pass", that still >> leaves some more open questions: which of the following should be >> valid? >> >> (1) def f(*args,): pass >> (2) def f(**kwargs,): pass >> (3) def f(*,): pass > > IMO all of them (though as you mention, (3) doesn't matter.) Agreed. It's one less thing to explain: "You can have a trailing comma in an argument list, unless there's a bare star, and in that case you can't have a comma; and also if ...". Ick. Eric. From barry at python.org Fri Jul 9 23:49:15 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 9 Jul 2010 17:49:15 -0400 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: References: <20100624115048.4fd152e3@heresy> <20100624170944.7e68ad21@heresy> <4C23D3C2.1060500@scottdial.com> <4C246E81.3020302@scottdial.com> <4C265DC6.4080600@ubuntu.com> <4C268433.30405@scottdial.com> <20100630145329.736f2aab@heresy> <4C2C75C1.8020708@scottdial.com> <20100707144059.07bae9f2@heresy> Message-ID: <20100709174915.08d2933c@heresy> On Jul 08, 2010, at 09:14 AM, Georg Brandl wrote: >Am 07.07.2010 23:04, schrieb Georg Brandl: >> I can see where this is going... writing it into PEP 384 would >> automatically get the change accepted? I'm definitely not trying to get it in subversively. :) >I hit "Send" prematurely. I wanted to add that I'd be okay with this >change, be it in a new PEP or an old one. Cool. I'll take Nick up on the suggestion to summarize the thread via a new PEP. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From tjreedy at udel.edu Fri Jul 9 23:59:58 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 09 Jul 2010 17:59:58 -0400 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists. In-Reply-To: References: <1A472770E042064698CB5ADC83A12ACD48AF0F9D@TK5EX14MBXC118.redmond.corp.microsoft.com> Message-ID: On 7/9/2010 4:26 PM, Mark Dickinson wrote: > On Fri, Jul 9, 2010 at 8:37 PM, Dino Viehland wrote: >> Terry wrote: >>> This violates the important principle that allowed def and call arg >>> sequences should match to the extent sensible and possible. In this >>> sense, the SyntaxError is a bug. So I would fix this now for 3.2 and >>> notify the other implementors. >> >> +1 on fixing it - trailing commas are awesome. I'm always annoyed in >> C# where I frequently can't use them. This seems like a bug fix level >> change that should be easy for the other implementations to fix. > > Thanks for all the feedback. > > If the grammar is changed to allow "def f(*, a,): pass", that still > leaves some more open questions: which of the following should be > valid? > > (1) def f(*args,): pass > (2) def f(**kwargs,): pass > (3) def f(*,): pass I would follow the same principle of making def and call match. Currently >>> def f(*args,): pass SyntaxError: invalid syntax >>> def f(*args): pass >>> f(*(1,2,3),) SyntaxError: invalid syntax Giving the call doc "A trailing comma may be present after the positional and keyword arguments but does not affect the semantics.", I was not really expecting the second syntax error. Same behavior for f(**k,). So if you allow the definition, allow the call too. > Just for the sake of simplicity it would seem to make sense allow all > these, even if there's no obvious immediate use; for me, it keeps the > mental footprint of the language small---I don't have to remember when > the comma is or isn't allowed. Agreed. -- Terry Jan Reedy From barry at python.org Fri Jul 9 23:55:04 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 9 Jul 2010 17:55:04 -0400 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <4C351200.2080900@ubuntu.com> References: <20100624115048.4fd152e3@heresy> <20100624170944.7e68ad21@heresy> <4C23D3C2.1060500@scottdial.com> <4C246E81.3020302@scottdial.com> <4C265DC6.4080600@ubuntu.com> <4C268433.30405@scottdial.com> <20100630145329.736f2aab@heresy> <4C2C75C1.8020708@scottdial.com> <20100707144059.07bae9f2@heresy> <4C351200.2080900@ubuntu.com> Message-ID: <20100709175504.28b12e4c@heresy> On Jul 08, 2010, at 01:47 AM, Matthias Klose wrote: >On 07.07.2010 20:40, Barry Warsaw wrote: >> Getting back to this after the US holiday. Thanks for running these >> numbers Scott. I've opened a bug in the Python tracker and attached >> my latest patch: >> >> http://bugs.python.org/issue9193 >> >> The one difference from previous versions of the patch is that >> the .so tag is now settable via "./configure >> --with-so-abi-tag=foo". This would generate shared libs like >> _multiprocessing.foo.so. > > - imo, it's wrong to lookup _multiprocessing.so first, before looking > up _multiprocessing.foo.so (at least for the use case to put the > extensions for multiple python versions into one directory). Good point. > - why is the flexibility of specifying the "foo" needed? The > naming for the __pycache__ files is fixed, why have it configurable > for extensions? The 'foo' part in the shared library name is equivalent to the part in __pycache__/baz..pyc, not specifically the __pycache__ part. Specifying the is necessary because extension modules built for Python 3.2 will not be compatible with extension modules built for Python 3.3 (in the absence of PEP 384), but they will live in the same directory. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From ncoghlan at gmail.com Sat Jul 10 02:22:08 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 10 Jul 2010 10:22:08 +1000 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists. In-Reply-To: References: Message-ID: On Sat, Jul 10, 2010 at 5:31 AM, Terry Reedy wrote: > This violates the important principle that allowed def and call arg > sequences should match to the extent sensible and possible. In this sense, > the SyntaxError is a bug. So I would fix this now for 3.2 and notify the > other implementors. +1 for fixing it from me, unless any of the other implementations object. @Mark: my comment on the tracker issue had an implied "...unless you really want to" on the end :) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sat Jul 10 10:04:32 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 10 Jul 2010 18:04:32 +1000 Subject: [Python-Dev] Switching dev docs URL to Py3k? Message-ID: With the trunk closed to new development, should we switch http://docs.python.org/dev/ to show the docs built from the Py3k branch? (The in-development Py3k docs are currently at dev/py3k/) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From solipsis at pitrou.net Sat Jul 10 10:27:44 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 10 Jul 2010 10:27:44 +0200 Subject: [Python-Dev] Switching dev docs URL to Py3k? References: Message-ID: <20100710102744.648f86e7@pitrou.net> On Sat, 10 Jul 2010 18:04:32 +1000 Nick Coghlan wrote: > With the trunk closed to new development, should we switch > http://docs.python.org/dev/ to show the docs built from the Py3k > branch? Well, that page already says ?Python v3.2a0 documentation? to me. cheers Antoine. From ncoghlan at gmail.com Sat Jul 10 10:32:10 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 10 Jul 2010 18:32:10 +1000 Subject: [Python-Dev] Switching dev docs URL to Py3k? In-Reply-To: <20100710102744.648f86e7@pitrou.net> References: <20100710102744.648f86e7@pitrou.net> Message-ID: On Sat, Jul 10, 2010 at 6:27 PM, Antoine Pitrou wrote: > On Sat, 10 Jul 2010 18:04:32 +1000 > Nick Coghlan wrote: > >> With the trunk closed to new development, should we switch >> http://docs.python.org/dev/ to show the docs built from the Py3k >> branch? > > Well, that page already says ?Python v3.2a0 documentation? to me. Oops. I guess I should have *looked* before posting that question. Sorry for the noise. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From g.brandl at gmx.net Sat Jul 10 12:13:30 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 10 Jul 2010 12:13:30 +0200 Subject: [Python-Dev] Switching dev docs URL to Py3k? In-Reply-To: References: <20100710102744.648f86e7@pitrou.net> Message-ID: Am 10.07.2010 10:32, schrieb Nick Coghlan: > On Sat, Jul 10, 2010 at 6:27 PM, Antoine Pitrou wrote: >> On Sat, 10 Jul 2010 18:04:32 +1000 >> Nick Coghlan wrote: >> >>> With the trunk closed to new development, should we switch >>> http://docs.python.org/dev/ to show the docs built from the Py3k >>> branch? >> >> Well, that page already says ?Python v3.2a0 documentation? to me. > > Oops. I guess I should have *looked* before posting that question. > Sorry for the noise. Also, for "backwards compatibility", /dev/py3k still exists and is a symlink to /dev. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From rrr at ronadam.com Sat Jul 10 18:06:24 2010 From: rrr at ronadam.com (Ron Adam) Date: Sat, 10 Jul 2010 11:06:24 -0500 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> Message-ID: On 07/07/2010 12:30 PM, Georg Brandl wrote: > Am 07.07.2010 18:09, schrieb Michael Foord: > >> I would say that the major use of docstrings is for interactive help - so >> interactive readability should be *the most important* (but perhaps not only) >> factor when considering how to format standard library docstrings. > > Agreed. However, reST doesn't need to be less readable if the specific > inline markup is not used. For example, using `identifier` to refer to a > function or *var* to refer to a variable (which is already done at quite a > few places) is very readable IMO. Using ``code`` also isn't bad, considering > that double quotes are not much different and potentially ambiguous. > > Overall, I think that we can make stdlib docstrings valid reST -- even if it's > reST without much markup -- but valid, so that people pulling in stdlib doc- > strings into Sphinx docs won't get ugly warnings. > > What I would *not* like to see is heavy markup and Sphinx specifics -- that > would only make sense if we included the docstrings in the docs, and I don't > see that coming. I also agree that interactive help should be the most important factor when writing doc strings for the standard library. Are there any plans to change how pydoc's GUI mode works? Could it use a minimal set of reST to improve it's display? The patch I submitted (*which is waiting to be reviewed) extends the GUI mode so it can show the same info that is available from the help() function. http://bugs.python.org/issue2001 I think the only issues with this patch are what new functions and classes should be part of the public api. * BTW... The bug trackers main links to items with patches, and needing review, doesn't pick up feature requests. Ron From taleinat at gmail.com Sun Jul 11 01:05:22 2010 From: taleinat at gmail.com (Tal Einat) Date: Sun, 11 Jul 2010 02:05:22 +0300 Subject: [Python-Dev] Removing IDLE from the standard library Message-ID: Hello, I would like to propose removing IDLE from the standard library. I have been using IDLE since 2002 and have been doing my best to help maintain and further develop IDLE since 2005. In recent years IDLE has received negligible interest and attention from the Python community. During this time IDLE has slowly gone downhill. The documentation and tutorials grow increasingly out of date. Cross-platform support has degraded with the increasing popularity of OSX and 64-bit platforms. Bugs take months, and sometimes more than a year, to be solved. Features that have since become common-place, such as having a non-intrusive search box instead of a dialog, are obviously and painfully lacking, making IDLE feel clumsy and out-dated. For these reasons, I think it would be fitting to remove IDLE from the standard library. IDLE is no longer recommended to beginners, IMO rightfully so, and this was the main reason for its inclusion in the standard library. Furthermore, if there is little or no interest in developing and maintaining IDLE, it should be removed to avoid having buggy and badly supported software in the standard library. - Tal Einat -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Sun Jul 11 03:33:28 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 10 Jul 2010 21:33:28 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: On 7/10/2010 7:05 PM, Tal Einat wrote: > Hello, > > I would like to propose removing IDLE from the standard library. -1 I use it daily. On Windows, it works better in many ways than the awful interactive command window, which I almost never use. I would rather the latter be replaced. > I have been using IDLE since 2002 and have been doing my best to help > maintain and further develop IDLE since 2005. > > In recent years IDLE has received negligible interest and attention from > the Python community. During this time IDLE has slowly gone downhill. I would say that it has not gone uphill. > The documentation and tutorials grow increasingly out of date. > Cross-platform support has degraded with the increasing popularity of > OSX and 64-bit platforms. Does it not work with the 64bit Windows build? > Bugs take months, and sometimes more than a > year, to be solved. The problem here, it seems to me, is that all issues are autoassigned to an inactive person (KBK) who does not really accept them except once a year or so. I do not know whether all other commiter are unwilling to commit IDLE issues, no matter how obvious and trivial, or if they all think they 'belong' to KBK. If and when I get a development setup, learn how to apply patches, and get commit privileges, I would want to be able to work on IDLE issues. > Features that have since become common-place, such > as having a non-intrusive search box instead of a dialog, are obviously > and painfully lacking, making IDLE feel clumsy and out-dated. I do not know what you mean here, so the 'lack' is completely unobvious and non-painful to me. The IDLE search/replace box strikes as being as good as that I have seen with other Windows software. > For these reasons, I think it would be fitting to remove IDLE from the > standard library. IDLE is no longer recommended to beginners, IMO > rightfully so, and this was the main reason for its inclusion in the > standard library. Is there a superiour replacement that you would recommend to be packaged with the Windows distribution? It would have to have a shell replacement also. > Furthermore, if there is little or no interest in > developing and maintaining IDLE, it should be removed to avoid having > buggy and badly supported software in the standard library. For my day to day use of the shell and editor, there are no serious bugs. -- Terry Jan Reedy From miki.tebeka at gmail.com Sun Jul 11 03:51:04 2010 From: miki.tebeka at gmail.com (Miki Tebeka) Date: Sat, 10 Jul 2010 18:51:04 -0700 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: Hello Tal, > I would like to propose removing IDLE from the standard library. -1. One of the biggest "selling points" for me when switching to python was the "out of the box" working IDE with REPL, syntax highliting and a debugger. The only other candidate I think of to replace IDLE might be IPython. However for novice users who are not used to command line it might be too intimidating. There are my others IDEs out there, some better some worse. However IMO to have one bundled with Python is highly important. > Cross-platform support has degraded with the increasing popularity of OSX and 64-bit > platforms. I use IDLE on Ubuntu 64bit and before that on OS X 64 bit, never had a problem. Can you give some examples on what do you mean by "cross-platform support"? All the best, -- Miki From orsenthil at gmail.com Sun Jul 11 05:56:58 2010 From: orsenthil at gmail.com (Senthil Kumaran) Date: Sun, 11 Jul 2010 09:26:58 +0530 Subject: [Python-Dev] Can ftp url start with file:// ? In-Reply-To: References: <20100709035232.GA6427@remy> <201007091423.41200.steve@pearwood.info> <20100709043804.GA9768@remy> <43622.1278693709@parc.com> <37151.1278698651@parc.com> Message-ID: <20100711035657.GA15730@remy> On Fri, Jul 09, 2010 at 02:32:04PM -0400, Fred Drake wrote: > FTP access also more often reflected the actual file hierarchy of the > machine, so trying that path as a system path is more likely to work > that I'd expect to see for HTTP. I see the reason. But I doubt if this is a reliable approach. Also when the scheme begins with file:// it should not be confused with ftp, so I think, that portion of code in urllib which work that way should be removed. The issue8801 was fixed in a different way so that no regression is introduced. For 3.2 release, shall we do way with relying on ftp for file:// scheme ? Currently for a url like file://somehost.domain//filesys/file.txt the flow control considers it a ftp url! The expected behaviour might be throw an exception saying that file:// url wont make any sense if it not localhost or absolute path. What if 'somehost.domain' is actually hostname of the machine? Should file:// be allowed in that case or is it the reason to rely on ftp? But it still does not make much sense to use ftp, because there is no guarantee that ftp service is running in that machine. -- Senthil The whole world is a scab. The point is to pick it constructively. -- Peter Beard From ggpolo at gmail.com Sun Jul 11 06:23:38 2010 From: ggpolo at gmail.com (Guilherme Polo) Date: Sun, 11 Jul 2010 01:23:38 -0300 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: 2010/7/10 Miki Tebeka : > Hello Tal, > >> I would like to propose removing IDLE from the standard library. > -1. > One of the biggest "selling points" for me when switching to python was the > "out of the box" working IDE with REPL, syntax highliting and a debugger. > The only other candidate I think of to replace IDLE might be IPython. However > for novice users who are not used to command line it might be too intimidating. > > There are my others IDEs out there, some better some worse. However IMO > to have one bundled with Python is highly important. > >> Cross-platform support has degraded with the increasing popularity of OSX and 64-bit >> platforms. > I use IDLE on Ubuntu 64bit and before that on OS X 64 bit, never had a > problem. Can you give > some examples on what do you mean by "cross-platform support"? > By "never had a problem" do you mean using some of the latest versions ? Here, running "idle" from a mac terminal and trying to type: print "hi" crashes when entering the quotation mark. I'm mostly sure this has been fixed on versions newer than 2.6.1 (but I hope you agree with me that shouldn't happen with a version distributed on macosx), so my another example is in the form of a question: how functional is the current IDLE debugger when running on a Mac ? -- -- Guilherme H. Polo Goncalves From raymond.hettinger at gmail.com Sun Jul 11 06:29:52 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Sat, 10 Jul 2010 21:29:52 -0700 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: On Jul 10, 2010, at 9:23 PM, Guilherme Polo wrote: > 2010/7/10 Miki Tebeka : >> Hello Tal, >> >>> I would like to propose removing IDLE from the standard library. >> -1. -1 from me too. IDLE is the tool I almost always used to introduce people to Python. FWIW, I've run in on a Mac and Windows without problems. Raymond From apt.shansen at gmail.com Sun Jul 11 08:01:16 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Sat, 10 Jul 2010 23:01:16 -0700 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: On Sat, Jul 10, 2010 at 9:23 PM, Guilherme Polo wrote: > By "never had a problem" do you mean using some of the latest versions > ? Here, running "idle" from a mac terminal and trying to type: print > "hi" crashes when entering the quotation mark. Huh? Works fine for me. Python 2.6.1, OSX 10.6.3, intel. >From the lurking crowd-- Please don't consider removing IDLE until there is a compelling replacement ready. It's better to have a limited IDE that works everywhere (even if in a limited fashion-- people are free to try out one of the many excellent full-featured Python IDE's out there after they advance to that point) then not. --Stephen -------------- next part -------------- An HTML attachment was scrubbed... URL: From ronaldoussoren at mac.com Sun Jul 11 08:20:57 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sun, 11 Jul 2010 08:20:57 +0200 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: On 11 Jul, 2010, at 6:23, Guilherme Polo wrote: > 2010/7/10 Miki Tebeka : >> Hello Tal, >> >>> I would like to propose removing IDLE from the standard library. >> -1. >> One of the biggest "selling points" for me when switching to python was the >> "out of the box" working IDE with REPL, syntax highliting and a debugger. >> The only other candidate I think of to replace IDLE might be IPython. However >> for novice users who are not used to command line it might be too intimidating. >> >> There are my others IDEs out there, some better some worse. However IMO >> to have one bundled with Python is highly important. >> >>> Cross-platform support has degraded with the increasing popularity of OSX and 64-bit >>> platforms. >> I use IDLE on Ubuntu 64bit and before that on OS X 64 bit, never had a >> problem. Can you give >> some examples on what do you mean by "cross-platform support"? >> > > By "never had a problem" do you mean using some of the latest versions > ? Here, running "idle" from a mac terminal and trying to type: print > "hi" crashes when entering the quotation mark. I'm mostly sure this > has been fixed on versions newer than 2.6.1 (but I hope you agree with > me that shouldn't happen with a version distributed on macosx), so my > another example is in the form of a question: how functional is the > current IDLE debugger when running on a Mac ? Apple basicly ships whatever is available at a cutoff point in their release cycle, without much if any involvment of the python-dev community. Have you tested this behaviour with a newer release of IDLE (the current 2.6.x release and the 2.7 release)? Does the IDLE application work for you (in "/Applications/Python X.Y" if you installed using a python.org binary installer)? And most importantly: have you filed bugs about things that don't work for you? If you don't file bugs there is little chance that issues get fixed unless some core developer happens to run into the same issue while having time to work on it. W.r.t. your last question: I don't know, I don't use IDLE or its debugger myself. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From ronaldoussoren at mac.com Sun Jul 11 08:22:45 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sun, 11 Jul 2010 08:22:45 +0200 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: <20F50EEB-2336-41D9-B530-EC3A4EE7A46E@mac.com> On 11 Jul, 2010, at 1:05, Tal Einat wrote: > Hello, > > I would like to propose removing IDLE from the standard library. > > I have been using IDLE since 2002 and have been doing my best to help maintain and further develop IDLE since 2005. > > In recent years IDLE has received negligible interest and attention from the Python community. During this time IDLE has slowly gone downhill. The documentation and tutorials grow increasingly out of date. Cross-platform support has degraded with the increasing popularity of OSX and 64-bit platforms. Bugs take months, and sometimes more than a year, to be solved. Features that have since become common-place, such as having a non-intrusive search box instead of a dialog, are obviously and painfully lacking, making IDLE feel clumsy and out-dated. Are their patches that fixes these problems? If not, are their at least issues on python's tracker? > > For these reasons, I think it would be fitting to remove IDLE from the standard library. IDLE is no longer recommended to beginners, IMO rightfully so, and this was the main reason for its inclusion in the standard library. Furthermore, if there is little or no interest in developing and maintaining IDLE, it should be removed to avoid having buggy and badly supported software in the standard library. I'm -1 on that. Several books, including fairly recent ones, use IDLE as the IDE for running examples. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From list at qtrac.plus.com Sun Jul 11 09:30:02 2010 From: list at qtrac.plus.com (Mark Summerfield) Date: Sun, 11 Jul 2010 08:30:02 +0100 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <20F50EEB-2336-41D9-B530-EC3A4EE7A46E@mac.com> References: <20F50EEB-2336-41D9-B530-EC3A4EE7A46E@mac.com> Message-ID: <201007110830.02802.list@qtrac.plus.com> On 2010-07-11, Ronald Oussoren wrote: > On 11 Jul, 2010, at 1:05, Tal Einat wrote: > > Hello, > > > > I would like to propose removing IDLE from the standard library. -1 > > I have been using IDLE since 2002 and have been doing my best to help > > maintain and further develop IDLE since 2005. [snip] > I'm -1 on that. Several books, including fairly recent ones, use IDLE as > the IDE for running examples. > > Ronald Thanks for mentioning that! My book "Programming in Python 3 (second edition)" introduces IDLE in Chapter 1 as follows: "As the screenshot in Figure 1.1 shows, IDLE has a rather retro look that harks back to the days of Motif on Unix and Windows 95. This is because it uses the Tk-based Tkinter GUI library (covered in Chapter 15) rather than one of the more powerful modern GUI libraries such as PyGtk, PyQt, or wxPython. The reasons for the use of Tkinter are a mixture of history, liberal license conditions, and the fact that Tkinter is much smaller than the other GUI libraries. On the plus side, IDLE comes as standard with Python and is very simple to learn and use." I personally really dislike Tcl/Tk. Nonetheless I invariably prefer to use IDLE than the raw command line for experimenting with Python and also for doing small one off custom jobs, so I end up using IDLE most days. I use IDLE on Linux & Windows (both 32 bit) with no problems. (My usage is purely of the interactive shell, I never use IDLE for editing.) -- Mark Summerfield, Qtrac Ltd, www.qtrac.eu C++, Python, Qt, PyQt - training and consultancy "Programming in Python 3" - ISBN 0321680561 http://www.qtrac.eu/py3book.html From taleinat at gmail.com Sun Jul 11 10:57:18 2010 From: taleinat at gmail.com (Tal Einat) Date: Sun, 11 Jul 2010 11:57:18 +0300 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: > I would like to propose removing IDLE from the standard library. > > I have been using IDLE since 2002 and have been doing my best to help > maintain and further develop IDLE since 2005. I'm surprised by the amount of interest this has raised already. To answer a few questions that were raised: In recent years I have worked up many patches, both bugfixes and new features and improvements. Getting any attention to these was non-trivial, and getting patches accepted (or an explanation why they are rejected in some cases) almost always took many months, sometimes years, and some are still unresolved. It has been very frustrating. When I ran into bugs I fixed them and submitted a patch. I have also done so for quite a few bugs reported by others. However, there are currently several bugs in the tracker which nobody is taking any notice of. IIRC most of the recent bugs are related to OSX or 64-bit Windows. To those who mention that IDLE is "okay" or "not going uphill", my grandfather would say "if you aren't running forwards, you are falling behind." You should know how IDLE looks to programmers seeing it for the first time -- IDLE's quirky and old-fashioned looks and interface are a major turnoff for new users. As a result I have stopped recommending it to coworkers, despite personally liking IDLE, instead recommending the basic command-line or IPython for interactive work, and any other IDE or text editor for development. I too prefer IDLE to the basic command line, and think that something like IDLE is well-suited for learning/teaching Python. I also think an interpreter with a nice GUI can be far superior to a text-only interpreter. However, I've mostly lost hope for IDLE, and am currently hoping that something else takes its place. The fact is that for many years little effort has gone into developing and maintaining IDLE, and I believe being tucked in a corner of the Python codebase is a major reason for this. I really don't see why IDLE has to be part of the standard library, what's wrong with IDLE being an externally maintained application? Yes, IDLE still works (mostly), but us few who continue to use it could do so even if it weren't part of the standard library. - Tal Einat From ronaldoussoren at mac.com Sun Jul 11 11:03:42 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sun, 11 Jul 2010 11:03:42 +0200 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: <7EF8C99A-4163-461A-922B-1FF3003D89E2@mac.com> On 11 Jul, 2010, at 10:57, Tal Einat wrote: > > When I ran into bugs I fixed them and submitted a patch. I have also > done so for quite a few bugs reported by others. However, there are > currently several bugs in the tracker which nobody is taking any > notice of. IIRC most of the recent bugs are related to OSX or 64-bit > Windows. The OSX issues al seem to be related to general Tk or Tkinter bugs on OSX. I know to little about Tk and Tkinter to seriously work on those. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From solipsis at pitrou.net Sun Jul 11 11:12:40 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 11 Jul 2010 11:12:40 +0200 Subject: [Python-Dev] Removing IDLE from the standard library References: Message-ID: <20100711111240.1b6f43ba@pitrou.net> On Sat, 10 Jul 2010 21:33:28 -0400 Terry Reedy wrote: > > The problem here, it seems to me, is that all issues are autoassigned to > an inactive person (KBK) who does not really accept them except once a > year or so. I do not know whether all other commiter are unwilling to > commit IDLE issues, no matter how obvious and trivial, or if they all > think they 'belong' to KBK. Well, how can you expect people to commit patches for something they don't know or use? (and, admittedly, something many of us don't care about) Regards Antoine. From techtonik at gmail.com Sun Jul 11 11:19:38 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Sun, 11 Jul 2010 12:19:38 +0300 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> Message-ID: On Fri, Jul 9, 2010 at 6:59 PM, Jeffrey Yasskin wrote: > > While the re2 comparison might be interesting from an abstract > standpoint, it intentionally supports a different regex language from > Python so that it can run faster and use less memory. Since re2 can > never replace Python's re module, it doesn't make sense to hold MRAB's > new module to that standard. re2 comparison is interesting from the point of if it should be included in stdlib. -- anatoly t. From taleinat at gmail.com Sun Jul 11 11:38:11 2010 From: taleinat at gmail.com (Tal Einat) Date: Sun, 11 Jul 2010 12:38:11 +0300 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <7EF8C99A-4163-461A-922B-1FF3003D89E2@mac.com> References: <7EF8C99A-4163-461A-922B-1FF3003D89E2@mac.com> Message-ID: On Sun, Jul 11, 2010 at 12:03 PM, Ronald Oussoren wrote: > > On 11 Jul, 2010, at 10:57, Tal Einat wrote: >> >> When I ran into bugs I fixed them and submitted a patch. I have also >> done so for quite a few bugs reported by others. However, there are >> currently several bugs in the tracker which nobody is taking any >> notice of. IIRC most of the recent bugs are related to OSX or 64-bit >> Windows. > > The OSX issues al seem to be related to general Tk or Tkinter bugs > on OSX. I know to little about Tk and Tkinter to seriously work on > those. > > Ronald Perhaps, but the point is that these bugs remain. Certainly this isn't because just you, out of the entire Python development community, know little about Tk and Tkinter. Using Tkinter is a major reason that maintaining and further developing IDLE is difficult. For example, it took me many hours just to get a working Tkinter scrolled frame widget, having had to write it from scratch and struggle with the under-documented Canvas widget. Another example is that integration of the new ttk (a.k.a. Tile) widget set, which supports native look&feel on various platforms and adds modern widgets, has still not been integrated despite being available in Tk for years and despite considerable effort being invested into it. - Tal Einat From eric at trueblade.com Sun Jul 11 13:37:22 2010 From: eric at trueblade.com (Eric Smith) Date: Sun, 11 Jul 2010 07:37:22 -0400 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> Message-ID: <4C39ACF2.6060108@trueblade.com> On 7/11/2010 5:19 AM, anatoly techtonik wrote: > On Fri, Jul 9, 2010 at 6:59 PM, Jeffrey Yasskin wrote: >> >> While the re2 comparison might be interesting from an abstract >> standpoint, it intentionally supports a different regex language from >> Python so that it can run faster and use less memory. Since re2 can >> never replace Python's re module, it doesn't make sense to hold MRAB's >> new module to that standard. > > re2 comparison is interesting from the point of if it should be > included in stdlib. Is "it" re2 or regex? I don't see having 2 regular expression engines in the stdlib. Eric. From ncoghlan at gmail.com Sun Jul 11 14:19:10 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 11 Jul 2010 22:19:10 +1000 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> Message-ID: On Sun, Jul 11, 2010 at 7:19 PM, anatoly techtonik wrote: > On Fri, Jul 9, 2010 at 6:59 PM, Jeffrey Yasskin wrote: >> >> While the re2 comparison might be interesting from an abstract >> standpoint, it intentionally supports a different regex language from >> Python so that it can run faster and use less memory. Since re2 can >> never replace Python's re module, it doesn't make sense to hold MRAB's >> new module to that standard. > > re2 comparison is interesting from the point of if it should be > included in stdlib. No it isn't. re2 is a wrapper for a third party engine that surrenders some functionality in the pursuit of better bounds on memory and CPU usage. It is not a drop-in replacement for re and cannot be by design: "The one significant exception is that RE2 drops support for backreferences and generalized zero-width assertions, because they cannot be implemented efficiently." There is no reason to have two distinct regex engines in the standard library - if someone knows enough to realise they need the performance assurances of re2, they're also likely to be able to find the Python wrappers for it. regex is potentially interesting for the standard library as it *is* intended to be a drop-in replacement for re that trades longer compilation times (typically once per application) for faster match times (potentially many times per application). The performance of re2 has nothing to do with the comparison between the current re module and MRAB's regex module. Regards, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From dave at dabeaz.com Sun Jul 11 14:16:18 2010 From: dave at dabeaz.com (David Beazley) Date: Sun, 11 Jul 2010 07:16:18 -0500 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> > I would like to propose removing IDLE from the standard library. > -1000. From the Python training department, I would like to say that this would be a horrible idea. Having taught numerous on-site training courses for Python, the one thing that I've learned is that you never know what you're going to get when you show up to teach a course. More often than not, you're thrown into some training room with a bunch of PCs, operated by someone who doesn't know anything about Python at all, and who had a hard enough time installing it in the first place. The fact that IDLE comes with Python means that even in such situations, as long as Python has been installed somewhere, there's going to be at least one halfway-reasonable environment for working with it (although I suppose there are some people who would still prefer to use the Windows command shell and Notepad). For what it's worth, I think IDLE works fine as a development environment, despite the fact that it has some flaky bits. The most annoying issue that I encounter in classes is people starting IDLE by right-clicking on files. This starts up IDLE without its subprocess and causes all sorts of bizarre problems related to the environment (e.g., restarting, module imports, etc.). Other than that, it's fine. Cheers, Dave From reid.kleckner at gmail.com Sun Jul 11 15:23:13 2010 From: reid.kleckner at gmail.com (Reid Kleckner) Date: Sun, 11 Jul 2010 13:23:13 +0000 Subject: [Python-Dev] Threading bug review + commit privs Message-ID: Hey all, I'm porting some fixes for threading.py that we applied to unladen-swallow: http://bugs.python.org/issue6643 We ran into these bizarre race conditions involving fork + threads while running the test suite with a background JIT compilation thread. I really wish we weren't trying to support forking from a child thread, but it's already in the test suite. I've "solved" the problem by throwing away radioactive locks that may have been held across a fork.* If I could get a reviewer to look at this, I would be very grateful, since reviewing threading patches is somewhat tricky. =/ I'm also expecting to be doing more work merging unladen-swallow into the py3k-jit branch, so I was wondering if I could get commit privileges for that. Thanks, Reid * In general I wouldn't think this is safe, but we already do it for _active_limbo_lock in threading.py. One of the problems I've encountered is that on OS X, releasing locks held by other threads across a fork results in a crash. Furthermore, when locks are deallocated, the destructor does a non-blocking acquire and then release, which I would think would crash. However, we get lucky here, because any thread that holds a lock across a fork usually holds a reference to it on the stack. Therefore the lock is leaked and the destructor never run. Moral: fork + threads is *crazy*, avoid it if you can. From guido at python.org Sun Jul 11 15:52:02 2010 From: guido at python.org (Guido van Rossum) Date: Sun, 11 Jul 2010 15:52:02 +0200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> Message-ID: On Sun, Jul 11, 2010 at 2:16 PM, David Beazley wrote: > >> I would like to propose removing IDLE from the standard library. >> > > -1000. ? From the Python training department, I would like to say that this would be a horrible idea. ? Having taught numerous on-site training courses for Python, the one thing that I've learned is that you never know what you're going to get when you show up to teach a course. ? More often than not, you're thrown into some training room with a bunch of PCs, operated by someone who doesn't know anything about Python at all, and who had a hard enough time installing it in the first place. ? ?The fact that IDLE comes with Python means that even in such situations, as long as Python has been installed somewhere, there's going to be at least one halfway-reasonable environment for working with it (although I suppose there are some people who would still prefer to use the Windows command shell and Notepad). > > For what it's worth, I think IDLE works fine as a development environment, despite the fact that it has some flaky bits. ? ?The most annoying issue that I encounter in classes is people starting IDLE by right-clicking on files. ?This starts up IDLE without its subprocess and causes all sorts of bizarre problems related to the environment (e.g., restarting, module imports, etc.). ?Other than that, it's fine. > > Cheers, > Dave Right. IDLE fits a niche. It's never going to be the world's best Python development environment but it isn't the worst either, and it's worth keeping around. There clearly are *some* folks who care enough about IDLE to submit bug reports and fixes. How about we empower these people by giving at least one of them commit privileges? IDLE development has often been done by people who aren't otherwise contributing to the core, and we surely should trust those folks with commit privileges. -- --Guido van Rossum (python.org/~guido) From steve at holdenweb.com Sun Jul 11 16:01:07 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 11 Jul 2010 10:01:07 -0400 Subject: [Python-Dev] Can ftp url start with file:// ? In-Reply-To: <20100711035657.GA15730@remy> References: <20100709035232.GA6427@remy> <201007091423.41200.steve@pearwood.info> <20100709043804.GA9768@remy> <43622.1278693709@parc.com> <37151.1278698651@parc.com> <20100711035657.GA15730@remy> Message-ID: Senthil Kumaran wrote: > On Fri, Jul 09, 2010 at 02:32:04PM -0400, Fred Drake wrote: >> FTP access also more often reflected the actual file hierarchy of the >> machine, so trying that path as a system path is more likely to work >> that I'd expect to see for HTTP. > > I see the reason. But I doubt if this is a reliable approach. Also > when the scheme begins with file:// it should not be confused with > ftp, so I think, that portion of code in urllib which work that way > should be removed. > > The issue8801 was fixed in a different way so that no regression is > introduced. > > For 3.2 release, shall we do way with relying on ftp for file:// > scheme ? Currently for a url like > file://somehost.domain//filesys/file.txt the flow control considers it > a ftp url! > > The expected behaviour might be throw an exception saying that file:// > url wont make any sense if it not localhost or absolute path. > > What if 'somehost.domain' is actually hostname of the machine? Should > file:// be allowed in that case or is it the reason to rely on ftp? > But it still does not make much sense to use ftp, because there is no > guarantee that ftp service is running in that machine. > My own inclination would be to regard the current treatment of file: as a bug (albeit one not worth fixing on 2.x). RFC 1630 specification lists the "file" scheme as being for "local file access", and RFC 1738 says they are for "host-specific file names" and points out that "The file URL scheme is unusual in that it does not specify an Internet protocol or access method for such files; as such, its utility in network protocols between hosts is limited." Presumably a hostname in such a URI would require that some host-specific protocol be used (but this should be an access protocol like SMB or NFS, not a transfer protocol like FTP). It seems pretty clear that randomly interpreting particular host names to imply a specific remote-access protocol like FTP is bogus. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From breamoreboy at yahoo.co.uk Sun Jul 11 16:08:42 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 11 Jul 2010 15:08:42 +0100 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> Message-ID: On 11-7-2010 14:52, Guido van Rossum wrote: > On Sun, Jul 11, 2010 at 2:16 PM, David Beazley wrote: >> >>> I would like to propose removing IDLE from the standard library. >>> >> >> -1000. From the Python training department, I would like to say that this would be a horrible idea. Having taught numerous on-site training courses for Python, the one thing that I've learned is that you never know what you're going to get when you show up to teach a course. More often than not, you're thrown into some training room with a bunch of PCs, operated by someone who doesn't know anything about Python at all, and who had a hard enough time installing it in the first place. The fact that IDLE comes with Python means that even in such situations, as long as Python has been installed somewhere, there's going to be at least one halfway-reasonable environment for working with it (although I suppose there are some people who would still prefer to use the Windows command shell and Notepad). >> >> For what it's worth, I think IDLE works fine as a development environment, despite the fact that it has some flaky bits. The most annoying issue that I encounter in classes is people starting IDLE by right-clicking on files. This starts up IDLE without its subprocess and causes all sorts of bizarre problems related to the environment (e.g., restarting, module imports, etc.). Other than that, it's fine. >> >> Cheers, >> Dave > > Right. IDLE fits a niche. It's never going to be the world's best > Python development environment but it isn't the worst either, and it's > worth keeping around. > > There clearly are *some* folks who care enough about IDLE to submit > bug reports and fixes. How about we empower these people by giving at > least one of them commit privileges? IDLE development has often been > done by people who aren't otherwise contributing to the core, and we > surely should trust those folks with commit privileges. > Can I take a really big liberty and volunteer Terry Reedy for the job. Kindest regards. Mark Lawrence. From kw at codebykevin.com Sun Jul 11 15:24:54 2010 From: kw at codebykevin.com (Kevin Walzer) Date: Sun, 11 Jul 2010 09:24:54 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <7EF8C99A-4163-461A-922B-1FF3003D89E2@mac.com> References: <7EF8C99A-4163-461A-922B-1FF3003D89E2@mac.com> Message-ID: <4C39C626.40400@codebykevin.com> On 7/11/10 5:03 AM, Ronald Oussoren wrote: > > > The OSX issues al seem to be related to general Tk or Tkinter bugs > on OSX. I know to little about Tk and Tkinter to seriously work on > those. Ronald, How about http://bugs.python.org/issue6075? I first submitted that patch in May '09, and updated it in October '09 on request from you, and it's just sat there since... --Kevin -- Kevin Walzer Code by Kevin http://www.codebykevin.com From taleinat at gmail.com Sun Jul 11 16:22:28 2010 From: taleinat at gmail.com (Tal Einat) Date: Sun, 11 Jul 2010 17:22:28 +0300 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: On Sun, Jul 11, 2010 at 11:57 AM, Tal Einat wrote: >> I would like to propose removing IDLE from the standard library. >> >> I have been using IDLE since 2002 and have been doing my best to help >> maintain and further develop IDLE since 2005. > > I'm surprised by the amount of interest this has raised already. To > answer a few questions that were raised: > > To those who mention that IDLE is "okay" or "not going uphill", my > grandfather would say "if you aren't running forwards, you are falling > behind." You should know how IDLE looks to programmers seeing it for > the first time -- IDLE's quirky and old-fashioned looks and interface > are a major turnoff for new users. As a result I have stopped > recommending it to coworkers, despite personally liking IDLE, instead > recommending the basic command-line or IPython for interactive work, > and any other IDE or text editor for development. > > I too prefer IDLE to the basic command line, and think that something > like IDLE is well-suited for learning/teaching Python. I also think an > interpreter with a nice GUI can be far superior to a text-only > interpreter. However, I've mostly lost hope for IDLE, and am currently > hoping that something else takes its place. > > The fact is that for many years little effort has gone into developing > and maintaining IDLE, and I believe being tucked in a corner of the > Python codebase is a major reason for this. I really don't see why > IDLE has to be part of the standard library, what's wrong with IDLE > being an externally maintained application? > > Yes, IDLE still works (mostly), but us few who continue to use it > could do so even if it weren't part of the standard library. Most of the responses up to this point have been strongly against my proposal. The main reason given is that it is nice to have a graphical IDE supported out-of-the-box with almost any Python installation. This is especially important for novice programmers and in teaching environments. I understand this sentiment, but I think that supplying a quirky IDE with many caveats, lacking documentation, some bugs and a partially working debugger ends up causing more confusion than good. Initially (five years ago!) I tried to overcome these issues by improving IDLE, solving problems and adding a few key features. Without going into details, suffice to say that IDLE hasn't improved much since 2005 despite my efforts. For example, see http://bugs.python.org/issue1529142, where it took nearly 3 years to fix a major issue from the moment I posted the first workaround. For another example, see http://bugs.python.org/issue3068, where I posted a patch for an extension configuration dialog over two years ago, and it hasn't received as much as a sneeze in response. Although several people say that they think having IDLE in the stdlib is important, the fact is that IDLE is considered quite unimportant by most of the Python community. Having IDLE in the stdlib may be convenient for a few people, but most never use it and don't care about it. I think that in its current state, IDLE may still be helpful for learning Python, but it is more likely to drive away users who run into its various quirks and problems. And for experienced Python developers, very few actually use IDLE, and those who do could easily install it if it weren't part of the stdlib. - Tal Einat From taleinat at gmail.com Sun Jul 11 16:31:03 2010 From: taleinat at gmail.com (Tal Einat) Date: Sun, 11 Jul 2010 17:31:03 +0300 Subject: [Python-Dev] Removing IDLE from the standard library Message-ID: Guido van Rossum wrote: >David Beazley wrote: >> >>> I would like to propose removing IDLE from the standard library. >>> >> >> -1000. From the Python training department, I would like to say that this would be a horrible idea. > >Right. IDLE fits a niche. It's never going to be the world's best >Python development environment but it isn't the worst either, and it's >worth keeping around. > >There clearly are *some* folks who care enough about IDLE to submit >bug reports and fixes. How about we empower these people by giving at >least one of them commit privileges? IDLE development has often been >done by people who aren't otherwise contributing to the core, and we >surely should trust those folks with commit privileges. This would certainly be a great improvement on the current situation. However, I still think IDLE is not currently in a state that it should be suggested for use by beginners. - Tal Einat From jcea at jcea.es Sun Jul 11 17:11:43 2010 From: jcea at jcea.es (Jesus Cea) Date: Sun, 11 Jul 2010 17:11:43 +0200 Subject: [Python-Dev] Getting an optional parameter instead of creating a socket internally In-Reply-To: <20100413020319.2779.2069488812.divmod.xquotient.401@localhost.localdomain> References: <4BC395DA.3030702@jcea.es> <4BC3AA8C.70109@jcea.es> <20100413020319.2779.2069488812.divmod.xquotient.401@localhost.localdomain> Message-ID: <4C39DF2F.5000309@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 13/04/10 04:03, exarkun at twistedmatrix.com wrote: > On 12 Apr, 11:19 pm, jcea at jcea.es wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On 04/13/2010 12:47 AM, Antoine Pitrou wrote: >>> Jesus Cea jcea.es> writes: >>>> >>>> PS: "socket.setdefaulttimeout()" is not enough, because it could >>>> shutdown a perfectly functional connection, just because it was idle >>>> for >>>> too long. >>> >>> The socket timeout doesn't shutdown anything. It just puts a limit on >>> how much >>> time recv() and send() can block. Then it's up to you to detect >>> whether the >>> server is still alive (for example by pinging it through whatever >>> means the >>> application protocol gives you). >> >> A regular standard library (let say, poplib) would abort, after getting >> the timeout exception. >>>> 4. Modify client libraries to accept a new optional socket-like object >>>> as an optional parameter. This would allow things like transparent >>>> compression or encryption, or to replace the socket connection by >>>> anything else (read/write to shared memory or database, for example). >>> >>> This could be useful too. >> >> I have been thinking about this for years. Do you actually think this >> could be formally proposed?. > > Every once in a while I make a little bit more progress on the PEP I'm > working on for this. If you want to talk more about this, you can find > me in #python-dev or #twisted on freenode. > > Jean-Paul Jean-Paul, I would like to have this for 3.2. How is the PEP going?. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ . _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQCVAwUBTDnfL5lgi5GaxT1NAQJOIAP+IAARGsWGeReG21Mc70AxT9e82TqrPY65 053GpfnqDW/poCHdHKv5NeDPso02tDeJvZ53cB23ximQKM9qg1j9XzXP/5AJcjke eVJaS9K8K6/Z1o97iDZb3Evkt7q2Dn7VG4QjJn6cy9lh841HDRFn/+HIuQLgoMyh stvK53cj7n4= =JrAg -----END PGP SIGNATURE----- From hausburn at gmail.com Sun Jul 11 17:37:29 2010 From: hausburn at gmail.com (Van) Date: Sun, 11 Jul 2010 11:37:29 -0400 Subject: [Python-Dev] I use both IDLE and my console Message-ID: Hello all, Buggy or not it's quite handy and as I said I use both when programming in python. To discontinue it is just not a good idea. Hausburn -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Sun Jul 11 18:28:31 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 11 Jul 2010 18:28:31 +0200 Subject: [Python-Dev] commit privs References: Message-ID: <20100711182831.56462244@pitrou.net> On Sun, 11 Jul 2010 13:23:13 +0000 Reid Kleckner wrote: > > I'm also expecting to be doing more work merging unladen-swallow into > the py3k-jit branch, so I was wondering if I could get commit > privileges for that. It sounds good to me. Also, thanks for your threading patches! Regards Antoine. From g.brandl at gmx.net Sun Jul 11 18:54:38 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 11 Jul 2010 18:54:38 +0200 Subject: [Python-Dev] Threading bug review + commit privs In-Reply-To: References: Message-ID: Am 11.07.2010 15:23, schrieb Reid Kleckner: > I'm also expecting to be doing more work merging unladen-swallow into > the py3k-jit branch, so I was wondering if I could get commit > privileges for that. Sure -- send your public key to me, and I'll set you up. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From janssen at parc.com Sun Jul 11 19:35:16 2010 From: janssen at parc.com (Bill Janssen) Date: Sun, 11 Jul 2010 10:35:16 PDT Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: <9773.1278869716@parc.com> Tal Einat wrote: > Although several people say that they think having IDLE in the stdlib > is important, the fact is that IDLE is considered quite unimportant by > most of the Python community. Having IDLE in the stdlib may be > convenient for a few people, but most never use it and don't care > about it. I think that in its current state, IDLE may still be helpful > for learning Python, but it is more likely to drive away users who run > into its various quirks and problems. And for experienced Python > developers, very few actually use IDLE, and those who do could easily > install it if it weren't part of the stdlib. I agree with you on this, Tal. On OS X, this is particularly aggravating, as the Apple-supplied Python doesn't seem to include a working version, and installing MacPython leads to other problems (see, for instance, the thread at http://groups.google.com/group/nltk-users/browse_thread/thread/e14b647243ca5b66). For David and other teachers, there are plenty of alternative IDEs, outlined at http://wiki.python.org/moin/IntegratedDevelopmentEnvironments. Bill From fdrake at acm.org Sun Jul 11 19:35:21 2010 From: fdrake at acm.org (Fred Drake) Date: Sun, 11 Jul 2010 13:35:21 -0400 Subject: [Python-Dev] Can ftp url start with file:// ? In-Reply-To: <20100711035657.GA15730@remy> References: <20100709035232.GA6427@remy> <201007091423.41200.steve@pearwood.info> <20100709043804.GA9768@remy> <43622.1278693709@parc.com> <37151.1278698651@parc.com> <20100711035657.GA15730@remy> Message-ID: On Sat, Jul 10, 2010 at 11:56 PM, Senthil Kumaran wrote: > I see the reason. But I doubt if this is a reliable approach. ?Also > when the scheme begins with file:// it should not be confused with > ftp, so I think, that portion of code in urllib which work that way > should be removed. I'm not trying to defend the current behavior of defaulting to FTP as a good thing; it's definitely surprising. I am trying to rationalize it so I can be sure I understand why it might have been done to start with. My own preference is to kick out any non-local references with an exception that can be detected (possibly derived from ValueError) so that applications that want rewrite (hopefully with user agreement!) can do so. (Checking for non-local file: URLs on input might be better, of course.) -Fred -- Fred L. Drake From brett at python.org Sun Jul 11 20:37:24 2010 From: brett at python.org (Brett Cannon) Date: Sun, 11 Jul 2010 11:37:24 -0700 Subject: [Python-Dev] commit privs In-Reply-To: <20100711182831.56462244@pitrou.net> References: <20100711182831.56462244@pitrou.net> Message-ID: On Sun, Jul 11, 2010 at 09:28, Antoine Pitrou wrote: > On Sun, 11 Jul 2010 13:23:13 +0000 > Reid Kleckner wrote: >> >> I'm also expecting to be doing more work merging unladen-swallow into >> the py3k-jit branch, so I was wondering if I could get commit >> privileges for that. > > It sounds good to me. Also, thanks for your threading patches! +1 from me as well. From martin at v.loewis.de Sun Jul 11 20:37:57 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 11 Jul 2010 20:37:57 +0200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: <4C3A0F85.1090404@v.loewis.de> > Initially (five years ago!) I tried to overcome these issues by > improving IDLE, solving problems and adding a few key features. > Without going into details, suffice to say that IDLE hasn't improved > much since 2005 despite my efforts. For example, see > http://bugs.python.org/issue1529142, where it took nearly 3 years to > fix a major issue from the moment I posted the first workaround. For > another example, see http://bugs.python.org/issue3068, where I posted > a patch for an extension configuration dialog over two years ago, and > it hasn't received as much as a sneeze in response. I can understand that this is frustrating, but please understand that this is not specific to your patches, or to IDLE. Many other patches on bugs.python.org remain unreviewed for many years. That's because many of the issues are really tricky, and there are very few people who both have the time and the expertise to evaluate them. FWIW, I don't consider a few months as a "long" time for a patch review. At the moment, I'm personally able to perhaps review one issue per week (sometimes less); at this rate, it'll take several years until I get to everything. Regards, Martin From martin at v.loewis.de Sun Jul 11 20:40:58 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 11 Jul 2010 20:40:58 +0200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> Message-ID: <4C3A103A.1070504@v.loewis.de> >> There clearly are *some* folks who care enough about IDLE to submit >> bug reports and fixes. How about we empower these people by giving at >> least one of them commit privileges? IDLE development has often been >> done by people who aren't otherwise contributing to the core, and we >> surely should trust those folks with commit privileges. >> > > Can I take a really big liberty and volunteer Terry Reedy for the job. It doesn't work that way. You can't volunteer somebody else (*). If Terry would volunteer himself, he'd get commit access in no time. Regards, Martin (*) so when you assign bugs to me, it probably means that they get less attention, not more. From glyph at twistedmatrix.com Sun Jul 11 20:56:18 2010 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Sun, 11 Jul 2010 14:56:18 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: <217B30F6-06FE-4237-90C2-540CD6BB6A28@twistedmatrix.com> On Jul 11, 2010, at 10:22 AM, Tal Einat wrote: > Most of the responses up to this point have been strongly against my > proposal. The main reason given is that it is nice to have a graphical > IDE supported out-of-the-box with almost any Python installation. This > is especially important for novice programmers and in teaching > environments. I understand this sentiment, but I think that supplying > a quirky IDE with many caveats, lacking documentation, some bugs and a > partially working debugger ends up causing more confusion than good. The people who are actually *in* those environments seem to disagree with you :). I think you underestimate the difficulty of getting software installed and overestimate the demands of new Python users and students. While I don't ever use IDLE if there's an alternative available, I have been very grateful many times for its presence in environments where it was a struggle even to say "install Python". A workable editor and graphical shell is important, whatever its flaws. (And I think you exaggerate IDLE's flaws just a bit.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From glyph at twistedmatrix.com Sun Jul 11 20:59:14 2010 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Sun, 11 Jul 2010 14:59:14 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <4C3A0F85.1090404@v.loewis.de> References: <4C3A0F85.1090404@v.loewis.de> Message-ID: On Jul 11, 2010, at 2:37 PM, Martin v. L?wis wrote: >> Initially (five years ago!) I tried to overcome these issues by >> improving IDLE, solving problems and adding a few key features. >> Without going into details, suffice to say that IDLE hasn't improved >> much since 2005 despite my efforts. For example, see >> http://bugs.python.org/issue1529142, where it took nearly 3 years to >> fix a major issue from the moment I posted the first workaround. For >> another example, see http://bugs.python.org/issue3068, where I posted >> a patch for an extension configuration dialog over two years ago, and >> it hasn't received as much as a sneeze in response. > > I can understand that this is frustrating, but please understand that > this is not specific to your patches, or to IDLE. Many other patches on > bugs.python.org remain unreviewed for many years. That's because many of > the issues are really tricky, and there are very few people who both > have the time and the expertise to evaluate them. This problem seems to me to be the root cause here. Guido proposes to give someone interested in IDLE commit access, and hopefully that will help in this particular area. But, as I recall, at the last language summit there was quite a bit of discussion about how to address the broader issue of patches falling into a black hole. Is anybody working on it? (This seems to me like an area where a judicious application of PSF funds might help; if every single bug were actively triaged and responded to, even if it weren't reviewed, and patch contributors were directed to take specific steps to elicit a response or a review, the fact that patch reviews take a while might not be so bad.) > FWIW, I don't consider a few months as a "long" time for a patch review. It may not be a long time compared to other patch reviews, but it is a very long time for a volunteer to wait for something, especially if that "something" is "any indication that the python developers care that this patch was submitted at all". There seems to be at least one thread a month on this list from a disgruntled community member complaining (directly or indirectly) about this delay. I think that makes it a big problem. > At the moment, I'm personally able to perhaps review one issue per week > (sometimes less); at this rate, it'll take several years until I get > to everything. I guess it depends what you mean by "everything", but given that the open bug count is actually increasing at a significant rate, I would say that you can never possibly get to "everything". -------------- next part -------------- An HTML attachment was scrubbed... URL: From janssen at parc.com Sun Jul 11 21:00:17 2010 From: janssen at parc.com (Bill Janssen) Date: Sun, 11 Jul 2010 12:00:17 PDT Subject: [Python-Dev] Can ftp url start with file:// ? In-Reply-To: References: <20100709035232.GA6427@remy> <201007091423.41200.steve@pearwood.info> <20100709043804.GA9768@remy> <43622.1278693709@parc.com> <37151.1278698651@parc.com> <20100711035657.GA15730@remy> Message-ID: <10926.1278874817@parc.com> Fred Drake wrote: > On Sat, Jul 10, 2010 at 11:56 PM, Senthil Kumaran wro= > te: > > I see the reason. But I doubt if this is a reliable approach. =A0Also > > when the scheme begins with file:// it should not be confused with > > ftp, so I think, that portion of code in urllib which work that way > > should be removed. > > I'm not trying to defend the current behavior of defaulting to FTP as > a good thing; it's definitely surprising. I am trying to rationalize > it so I can be sure I understand why it might have been done to start > with. > > My own preference is to kick out any non-local references with an > exception that can be detected (possibly derived from ValueError) so > that applications that want rewrite (hopefully with user agreement!) > can do so. (Checking for non-local file: URLs on input might be > better, of course.) Sounds like a good plan to me, too. Bill From exarkun at twistedmatrix.com Sun Jul 11 21:06:52 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Sun, 11 Jul 2010 19:06:52 -0000 Subject: [Python-Dev] Getting an optional parameter instead of creating a socket internally In-Reply-To: <4C39DF2F.5000309@jcea.es> References: <4BC395DA.3030702@jcea.es> <4BC3AA8C.70109@jcea.es> <20100413020319.2779.2069488812.divmod.xquotient.401@localhost.localdomain> <4C39DF2F.5000309@jcea.es> Message-ID: <20100711190652.1937.530899525.divmod.xquotient.5@localhost.localdomain> On 03:11 pm, jcea at jcea.es wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >On 13/04/10 04:03, exarkun at twistedmatrix.com wrote: >>On 12 Apr, 11:19 pm, jcea at jcea.es wrote: >>>-----BEGIN PGP SIGNED MESSAGE----- >>>Hash: SHA1 >>> >>>On 04/13/2010 12:47 AM, Antoine Pitrou wrote: >>>>Jesus Cea jcea.es> writes: >>>>> >>>>>PS: "socket.setdefaulttimeout()" is not enough, because it could >>>>>shutdown a perfectly functional connection, just because it was >>>>>idle >>>>>for >>>>>too long. >>>> >>>>The socket timeout doesn't shutdown anything. It just puts a limit >>>>on >>>>how much >>>>time recv() and send() can block. Then it's up to you to detect >>>>whether the >>>>server is still alive (for example by pinging it through whatever >>>>means the >>>>application protocol gives you). >>> >>>A regular standard library (let say, poplib) would abort, after >>>getting >>>the timeout exception. >>>>>4. Modify client libraries to accept a new optional socket-like >>>>>object >>>>>as an optional parameter. This would allow things like transparent >>>>>compression or encryption, or to replace the socket connection by >>>>>anything else (read/write to shared memory or database, for >>>>>example). >>>> >>>>This could be useful too. >>> >>>I have been thinking about this for years. Do you actually think this >>>could be formally proposed?. >> >>Every once in a while I make a little bit more progress on the PEP I'm >>working on for this. If you want to talk more about this, you can >>find >>me in #python-dev or #twisted on freenode. >> >>Jean-Paul > >Jean-Paul, I would like to have this for 3.2. How is the PEP going?. It's still little more than an outline. You can see it here: http://twistedmatrix.com/trac/wiki/ProtocolPEP And if you're interested in helping, we can figure out a way to do that (you can have edit permission on the wiki or we can move the document elsewhere, whatever). Jean-Paul From martin at v.loewis.de Sun Jul 11 21:19:20 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 11 Jul 2010 21:19:20 +0200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> Message-ID: <4C3A1938.4020304@v.loewis.de> > (This seems to me like an area where a judicious application of PSF > funds might help; if every single bug were actively triaged and > responded to, even if it weren't reviewed, and patch contributors were > directed to take specific steps to elicit a response or a review, the > fact that patch reviews take a while might not be so bad.) I'm skeptical. The current triage process often results in assigning bugs to somebody, with the entirely unfounded expectation that somebody then will act on it - unfortunately, both the submitter and the triager seem to have that expectation. Unfortunately, it's often not clear what the submitter wants: does she want to help, or want to get help? For a bug report, I often post a message "can you provide a patch?", but sometimes, it isn't that clear. >> At the moment, I'm personally able to perhaps review one issue per week >> (sometimes less); at this rate, it'll take several years until I get >> to everything. > > I guess it depends what you mean by "everything", but given that the > open bug count is actually /increasing/ at a significant rate, I would > say that you can never possibly get to "everything". Right, I was trying to put things positively :-) Regards, Martin From solipsis at pitrou.net Sun Jul 11 21:38:01 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 11 Jul 2010 21:38:01 +0200 Subject: [Python-Dev] Removing IDLE from the standard library References: <4C3A0F85.1090404@v.loewis.de> Message-ID: <20100711213801.4d391849@pitrou.net> On Sun, 11 Jul 2010 14:59:14 -0400 Glyph Lefkowitz wrote: > > Guido proposes to give someone interested in IDLE commit access, and hopefully that will help in > this particular area. But, as I recall, at the last language summit there was quite a bit of > discussion about how to address the broader issue of patches falling into a black hole. Is > anybody working on it? I think the best way to "work on it" is to work on having more core developers, possibly with a more diverse range of interests. > (This seems to me like an area where a judicious application of PSF funds might help; if every > single bug were actively triaged and responded to, even if it weren't reviewed, and patch > contributors were directed to take specific steps to elicit a response or a review, the fact that > patch reviews take a while might not be so bad.) The operative word being "judicious". It is not obvious who should get funded, and for what tasks. Some specific issues (like email in 3.x) are large enough that they can be the sole focus of a fund grant. But I'm not sure triaging can apply. (besides, I am wary of making the job of interacting with our users and contributors a paid position, rather than a collective task.) > > FWIW, I don't consider a few months as a "long" time for a patch review. > > It may not be a long time compared to other patch reviews, but it is a very long time for a > volunteer to wait for something, especially if that "something" is "any indication that the > python developers care that this patch was submitted at all". Agreed. Also, the later the response arrives, the likelier it is to be along the lines of ?the patch doesn't apply cleanly anymore, can you write another one??. Regards Antoine. From amk at amk.ca Sun Jul 11 21:47:30 2010 From: amk at amk.ca (A.M. Kuchling) Date: Sun, 11 Jul 2010 15:47:30 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: <20100711194730.GA16734@andrew-kuchlings-macbook.local> On Sun, Jul 11, 2010 at 05:22:28PM +0300, Tal Einat wrote: > Initially (five years ago!) I tried to overcome these issues by > improving IDLE, solving problems and adding a few key features. > Without going into details, suffice to say that IDLE hasn't improved > much since 2005 despite my efforts. For example, see Is it maybe time for another idle-fork project to develop IDLE outside the stdlib for a while, and then the forked version could be swallowed wholesale by the Python 3.2 release? --amk From rdmurray at bitdance.com Sun Jul 11 22:30:54 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Sun, 11 Jul 2010 16:30:54 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> Message-ID: <20100711203054.0BF1C2182FB@kimball.webabinitio.net> On Sun, 11 Jul 2010 14:59:14 -0400, Glyph Lefkowitz wrote: > On Jul 11, 2010, at 2:37 PM, Martin v. L=F6wis wrote: > > I can understand that this is frustrating, but please understand that > > this is not specific to your patches, or to IDLE. Many other patches on > > bugs.python.org remain unreviewed for many years. That's because many of > > the issues are really tricky, and there are very few people who both > > have the time and the expertise to evaluate them. > > This problem seems to me to be the root cause here. > > Guido proposes to give someone interested in IDLE commit access, and > hopefully that will help in this particular area. But, as I recall, at > the last language summit there was quite a bit of discussion about how > to address the broader issue of patches falling into a black hole. Is > anybody working on it? As Martin indicated, the biggest single problem is people hours, and the only way to address that is to get more people involved. Jesse has started the sprint sponsorship committee. Presumably at least some reviewed and committed core patches will come out of that, as well as hopefully raising the general activity level. Jesse's effort is already bearing fruit in that I think many more people are thinking about holding sprints than has been true in the past. ("Oh, you mean *I* could do that? Cool.") I and the other triage people have gotten some new triage people involved. We've also gotten some new committers. Ezio Melotti presented a talk on core development at the Italian Pycon, and will present it again at EuroPython. Brian Curtin did a presentation on bug fixing for the Chicago users group and has turned his presentation into documentation for the Sprint committee. Dan Buch will be giving a talk on Python development at PyOhio, and Catherin Devlin has set up other activities at aimed at introducing people to core development (her "teach me" session, and I'll be leading the core sprint after the con). Hopefully all of these activities will put some more people on track to helping out with issue review, patch development, and, eventually, becoming committers. So yes, things are being done. Anyone who wants to help out or has idea is, of course, welcome :) > (This seems to me like an area where a judicious application of PSF > funds might help; if every single bug were actively triaged and > responded to, even if it weren't reviewed, and patch contributors were > directed to take specific steps to elicit a response or a review, the > fact that patch reviews take a while might not be so bad.) I scanned the commit log, and it looks to me like somewhere around 30 people have been active during the past month. That's not too bad, but each of us has specific areas of interest and limited time, and so bugs outside of those interest areas are more likely to get dropped on the floor. So, this is indeed an area where improvement is theoretically possible, but I'm not sure how we get from here to there. As you say, one option is for the PSF to fund people to do it somehow. (I'd be happy to be one of those people for some number of hours a week, by the way, but I doubt that the PSF budget is going to stretch to that kind of ongoing commitment.) But...if we had *enough* people volunteering, it would indeed be theoretically possible to consciously spread out the load so that issues get responded to in a timely fashion with constructive feedback. I'm not sure how we would structure this, but if someone steps forward to be organizer/driver, I bet we could come up with something. (Get lots of people to *sign up* for a one hour slot of triage work per week?) I don't think we have enough active volunteers now, but perhaps we can get there. It would also be great if every committer could find time to look at one bug *outside* of their main interest area for every N hours they spend on their interest area. (I try to do this, with varying degrees of success depending on the week.) -- R. David Murray www.bitdance.com From brett at python.org Sun Jul 11 23:16:14 2010 From: brett at python.org (Brett Cannon) Date: Sun, 11 Jul 2010 14:16:14 -0700 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <20100711203054.0BF1C2182FB@kimball.webabinitio.net> References: <4C3A0F85.1090404@v.loewis.de> <20100711203054.0BF1C2182FB@kimball.webabinitio.net> Message-ID: On Sun, Jul 11, 2010 at 13:30, R. David Murray wrote: > On Sun, 11 Jul 2010 14:59:14 -0400, Glyph Lefkowitz wrote: >> On Jul 11, 2010, at 2:37 PM, Martin v. L=F6wis wrote: >> > I can understand that this is frustrating, but please understand that >> > this is not specific to your patches, or to IDLE. Many other patches ?on >> > bugs.python.org remain unreviewed for many years. That's because many of >> > the issues are really tricky, and there are very few people who both >> > have the time and the expertise to evaluate them. >> >> This problem seems to me to be the root cause here. >> >> Guido proposes to give someone interested in IDLE commit access, and >> hopefully that will help in this particular area. ?But, as I recall, at >> the last language summit there was quite a bit of discussion about how >> to address the broader issue of patches falling into a black hole. ?Is >> anybody working on it? > > As Martin indicated, the biggest single problem is people hours, and > the only way to address that is to get more people involved. > > Jesse has started the sprint sponsorship committee. ?Presumably at > least some reviewed and committed core patches will come out of that, > as well as hopefully raising the general activity level. ?Jesse's effort > is already bearing fruit in that I think many more people are thinking > about holding sprints than has been true in the past. ?("Oh, you mean *I* > could do that? ?Cool.") > > I and the other triage people have gotten some new triage people > involved. ?We've also gotten some new committers. > > Ezio Melotti presented a talk on core development at the Italian > Pycon, and will present it again at EuroPython. ?Brian Curtin did > a presentation on bug fixing for the Chicago users group and has > turned his presentation into documentation for the Sprint committee. > > Dan Buch will be giving a talk on Python development at PyOhio, and > Catherin Devlin has set up other activities at aimed at introducing > people to core development (her "teach me" session, and I'll be leading > the core sprint after the con). > > Hopefully all of these activities will put some more people on track > to helping out with issue review, patch development, and, eventually, > becoming committers. > > So yes, things are being done. > > Anyone who wants to help out or has idea is, of course, welcome :) > >> (This seems to me like an area where a judicious application of PSF >> funds might help; if every single bug were actively triaged and >> responded to, even if it weren't reviewed, and patch contributors were >> directed to take specific steps to elicit a response or a review, the >> fact that patch reviews take a while might not be so bad.) > > I scanned the commit log, and it looks to me like somewhere around 30 > people have been active during the past month. ?That's not too bad, > but each of us has specific areas of interest and limited time, and so > bugs outside of those interest areas are more likely to get dropped on > the floor. > > So, this is indeed an area where improvement is theoretically possible, > but I'm not sure how we get from here to there. ?As you say, one option is > for the PSF to fund people to do it somehow. ?(I'd be happy to be one of > those people for some number of hours a week, by the way, but I doubt that > the PSF budget is going to stretch to that kind of ongoing commitment.) > I have a grant in to work on Python full-time for 2-3 months with one of the focus points being improving the developer docs. -Brett > But...if we had *enough* people volunteering, it would indeed be > theoretically possible to consciously spread out the load so that > issues get responded to in a timely fashion with constructive feedback. > I'm not sure how we would structure this, but if someone steps forward to > be organizer/driver, I bet we could come up with something. ?(Get lots > of people to *sign up* for a one hour slot of triage work per week?) > I don't think we have enough active volunteers now, but perhaps we > can get there. > > It would also be great if every committer could find time to look at > one bug *outside* of their main interest area for every N hours > they spend on their interest area. ?(I try to do this, with varying > degrees of success depending on the week.) > > -- > R. David Murray ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?www.bitdance.com > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From fuzzyman at voidspace.org.uk Sun Jul 11 23:16:43 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 11 Jul 2010 22:16:43 +0100 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <20100711194730.GA16734@andrew-kuchlings-macbook.local> References: <20100711194730.GA16734@andrew-kuchlings-macbook.local> Message-ID: <4C3A34BB.9060005@voidspace.org.uk> On 11/07/2010 20:47, A.M. Kuchling wrote: > On Sun, Jul 11, 2010 at 05:22:28PM +0300, Tal Einat wrote: > >> Initially (five years ago!) I tried to overcome these issues by >> improving IDLE, solving problems and adding a few key features. >> Without going into details, suffice to say that IDLE hasn't improved >> much since 2005 despite my efforts. For example, see >> > Is it maybe time for another idle-fork project to develop IDLE outside > the stdlib for a while, and then the forked version could be swallowed > wholesale by the Python 3.2 release? > Whilst I would *really* like to see bugfixes and minor improvements in functionality incorporated back into IDLE (who wouldn't?) I think it is important that IDLE remains focused on its nice of being a "simple" or "beginners" IDE, without trying to become a full blown IDE that is even harder for us to maintain. Speaking of which, the IDLE.app that comes with Python 2.7 for Mac OS X isn't working for me. Anyone else seeing that? All the best, Michael Foord > --amk > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From g.brandl at gmx.net Sun Jul 11 23:33:14 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 11 Jul 2010 23:33:14 +0200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> Message-ID: Am 11.07.2010 20:59, schrieb Glyph Lefkowitz: > > On Jul 11, 2010, at 2:37 PM, Martin v. L?wis wrote: > >>> Initially (five years ago!) I tried to overcome these issues by >>> improving IDLE, solving problems and adding a few key features. >>> Without going into details, suffice to say that IDLE hasn't improved >>> much since 2005 despite my efforts. For example, see >>> http://bugs.python.org/issue1529142, where it took nearly 3 years to >>> fix a major issue from the moment I posted the first workaround. For >>> another example, see http://bugs.python.org/issue3068, where I posted >>> a patch for an extension configuration dialog over two years ago, and >>> it hasn't received as much as a sneeze in response. >> >> I can understand that this is frustrating, but please understand that >> this is not specific to your patches, or to IDLE. Many other patches on >> bugs.python.org remain unreviewed for many years. >> That's because many of >> the issues are really tricky, and there are very few people who both >> have the time and the expertise to evaluate them. > > This problem seems to me to be the root cause here. > > Guido proposes to give someone interested in IDLE commit access, and hopefully > that will help in this particular area. But, as I recall, at the last language > summit there was quite a bit of discussion about how to address the broader > issue of patches falling into a black hole. Is anybody working on it? > > (This seems to me like an area where a judicious application of PSF funds might > help; if every single bug were actively triaged and responded to, even if it > weren't reviewed, and patch contributors were directed to take specific steps to > elicit a response or a review, the fact that patch reviews take a while might > not be so bad.) Honestly, how would you feel as a committer to have scores of issues assigned to you -- as a consequence of speedy triage -- knowing that you have to invest potentially hours of volunteer time into them, while the person doing the triaging is done with the bug in a few minutes and paid for it? I'd feel a little bit duped. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ncoghlan at gmail.com Sun Jul 11 23:52:59 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 12 Jul 2010 07:52:59 +1000 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <20100711203054.0BF1C2182FB@kimball.webabinitio.net> References: <4C3A0F85.1090404@v.loewis.de> <20100711203054.0BF1C2182FB@kimball.webabinitio.net> Message-ID: On Mon, Jul 12, 2010 at 6:30 AM, R. David Murray wrote: > So, this is indeed an area where improvement is theoretically possible, > but I'm not sure how we get from here to there. ?As you say, one option is > for the PSF to fund people to do it somehow. ?(I'd be happy to be one of > those people for some number of hours a week, by the way, but I doubt that > the PSF budget is going to stretch to that kind of ongoing commitment.) One problem that can arise is that when the active maintainer for a particular area gets caught up in other things (e.g. KBK for Idle in this particular case - added directly to cc list), then the commit rate for that area drastically slows down. Other committers have become accustomed to deferring to the judgement of the active maintainer, so if that judgement isn't forthcoming, then things don't go in. As a community of volunteers, obviously the amount of time we can devote to Python varies based on other commitments. Perhaps we need to encourage active maintainers to be more proactive about identifying folks that regularly submit good patches that may be candidates for commit access in that area. Otherwise, when an active maintainer's time is taken up by other things there's an understandable reluctance to tread on an active maintainer's toes by bypassing them and giving the task to someone else. Not only that, but the current maintainer is usually in the best position to judge the quality of submitted patches, as they're the most familiar with the code in question. While there are many areas of the standard library that don't even have *one* designated maintainer at this point in time, our goal should probably be to try to reach the point of having at least two maintainers for each area so that one person getting busy doesn't halt progress. Regards, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From basherwo at ncsu.edu Sun Jul 11 23:58:32 2010 From: basherwo at ncsu.edu (Bruce Sherwood) Date: Sun, 11 Jul 2010 15:58:32 -0600 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> Message-ID: Perhaps there are two separable issues. Many of us see it as extremely important that some IDLE be part of the standard Python distribution ("batteries included"), for the reasons that several people have given. However, there is merit to the suggestion to have an active separate development, with the understanding that periodically this separate development is a candidate for inclusion in the standard distribution, replacing whatever IDLE had been there. In the 2009 Google Summer of Code I was the mentor for a Brazilian student, Guilherme Polo, who completed and extended important improvements to IDLE made during the previous year by David Scherer. Given the somewhat official nature of this work, I assumed that these needed improvements would make it into the standard distribution, but as far as I know that hasn't happened. It would seem that if even this "sponsored" project didn't impact the standard Python distribution, something is broken in the procedures, and probably what is needed is, as Guido says, that someone be given the authority to get improvements to IDLE into the standard distribution. Making a significant change to the update procedures is clearly needed. Even if this needed change is made, there is also merit to Tai's suggestion of creating a separate project, to encourage developers like him to work together to improve IDLE, without having as a first priority to worry about getting it into the standard distribution, but with the clear understanding that this is the place to go for improvements to migrate into the standard distribution. Bruce Sherwood P.S. I'll add that IDLE has been EXTREMELY important for a large population of relatively casual users of Python, the thousands of science and engineering university students enrolled in the "Matter & Interactions" intro physics curriculum developed by Ruth Chabay and me ( matterandinteractions.org). A major feature of this curriculum is a serious introduction to computational modeling, in which students write short Python programs to model physical systems. Computational modeling is now central to all of science and engineering but alas has not made its way into undergraduate curricula in an institutionalized way. A big difficulty is that students come to college knowledgeable about all aspects of computers save one: programming. So the programming environment has to be exceptionally easy to learn and use. Python itself has the necessary properties, and Python+Visual (called VPython, vpython.org) lets the students focus on the physics while VPython generates real-time navigable 3D animations of the computational models, as a side effect of the computational code. IDLE has proved to be the right editing tool for this population, as essentially nothing needs to be learned, and there is near-instantaneous edit/run transitions which encourage rapid testing. In a physics course we have to focus on strict minimalism as far as the programming is concerned. We teach a bare minimum of needed concepts; for example, we introduce while loops but not for loops. We cannot afford to teach about a professional IDE; IDLE is fine and has worked well for us. (We're currently bundling with VPython the Scherer/Polo version of IDLE, which for reasons of clarity we're calling VIDLE.) A final personal note is that while I use Eclipse for working on the Visual module, which is written in C++, I find VIDLE a delightful environment for developing Python programs for physics education. -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Mon Jul 12 00:10:39 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 12 Jul 2010 00:10:39 +0200 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> Message-ID: <4C3A415F.6050709@v.loewis.de> > In the 2009 Google Summer of Code I was the mentor for a Brazilian > student, Guilherme Polo, who completed and extended important > improvements to IDLE made during the previous year by David Scherer. > Given the somewhat official nature of this work, I assumed that these > needed improvements would make it into the standard distribution, but as > far as I know that hasn't happened. It would seem that if even this > "sponsored" project didn't impact the standard Python distribution, > something is broken in the procedures, and probably what is needed is, > as Guido says, that someone be given the authority to get improvements > to IDLE into the standard distribution. Making a significant change to > the update procedures is clearly needed. I don't think so; instead, the perception of authority needs to be adjusted (in the specific case). Guilherme could have committed these changes, but, for whatever reason, decided not to. Nor did his direct mentor (i.e. you) tell him to commit the changes, and neither did I. > Even if this needed change is made, there is also merit to Tai's > suggestion of creating a separate project, to encourage developers like > him to work together to improve IDLE, without having as a first priority > to worry about getting it into the standard distribution, but with the > clear understanding that this is the place to go for improvements to > migrate into the standard distribution. Again, Guilherme could commit his changes any time. Regards, Martin From taleinat at gmail.com Mon Jul 12 00:18:22 2010 From: taleinat at gmail.com (Tal Einat) Date: Mon, 12 Jul 2010 01:18:22 +0300 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <4C3A0F85.1090404@v.loewis.de> References: <4C3A0F85.1090404@v.loewis.de> Message-ID: Martin v. L?wis wrote: >> Initially (five years ago!) I tried to overcome these issues by >> improving IDLE, solving problems and adding a few key features. >> Without going into details, suffice to say that IDLE hasn't improved >> much since 2005 despite my efforts. For example, see >> http://bugs.python.org/issue1529142, where it took nearly 3 years to >> fix a major issue from the moment I posted the first workaround. For >> another example, see http://bugs.python.org/issue3068, where I posted >> a patch for an extension configuration dialog over two years ago, and >> it hasn't received as much as a sneeze in response. > > I can understand that this is frustrating, ... I am not writing this to vent my frustration, and have purposely avoided making that the issue. > ... but please understand that > this is not specific to your patches, or to IDLE. Many other patches on > bugs.python.org remain unreviewed for many years. That's because many of > the issues are really tricky, and there are very few people who both > have the time and the expertise to evaluate them. I am aware of the situation with regard to issue reviews, but I think with IDLE there is more going on. In other parts of the Python codebase, a workaround for a major usability issue wouldn't normally have taken nearly three years to resolve after a working patch was submitted. And a working, well-tested patch wouldn't normally have sat ignored for over two years. Well, perhaps these things happen occasionally, but with IDLE this is the norm. > FWIW, I don't consider a few months as a "long" time for a patch review. > At the moment, I'm personally able to perhaps review one issue per week > (sometimes less); at this rate, it'll take several years until I get > to everything. I'm not talking about a few months, I'm talking about at least six months in most cases, years in many cases, as in the examples I mentioned. - Tal Einat From martin at v.loewis.de Mon Jul 12 00:28:58 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 12 Jul 2010 00:28:58 +0200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> Message-ID: <4C3A45AA.3050501@v.loewis.de> > I am aware of the situation with regard to issue reviews, but I think > with IDLE there is more going on. In other parts of the Python > codebase, a workaround for a major usability issue wouldn't normally > have taken nearly three years to resolve after a working patch was > submitted. Oh sure it does. Just look at all the cross-compilation bug reports and patches that get submitted. > And a working, well-tested patch wouldn't normally have sat > ignored for over two years. Well, perhaps these things happen > occasionally, but with IDLE this is the norm. Unfortunately, they happen more often than you think. Regards, Martin From ggpolo at gmail.com Mon Jul 12 00:29:26 2010 From: ggpolo at gmail.com (Guilherme Polo) Date: Sun, 11 Jul 2010 19:29:26 -0300 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <4C3A415F.6050709@v.loewis.de> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> Message-ID: 2010/7/11 "Martin v. L?wis" : >> In the 2009 Google Summer of Code I was the mentor for a Brazilian >> student, Guilherme Polo, who completed and extended important >> improvements to IDLE made during the previous year by David Scherer. >> Given the somewhat official nature of this work, I assumed that these >> needed improvements would make it into the standard distribution, but as >> far as I know that hasn't happened. It would seem that if even this >> "sponsored" project didn't impact the standard Python distribution, >> something is broken in the procedures, and probably what is needed is, >> as Guido says, that someone be given the authority to get improvements >> to IDLE into the standard distribution. Making a significant change to >> the update procedures is clearly needed. > > I don't think so; instead, the perception of authority needs to be > adjusted (in the specific case). Guilherme could have committed these > changes, but, for whatever reason, decided not to. Nor did his direct > mentor (i.e. you) tell him to commit the changes, and neither did I. > >> Even if this needed change is made, there is also merit to Tai's >> suggestion of creating a separate project, to encourage developers like >> him to work together to improve IDLE, without having as a first priority >> to worry about getting it into the standard distribution, but with the >> clear understanding that this is the place to go for improvements to >> migrate into the standard distribution. > > Again, Guilherme could commit his changes any time. > > Regards, > Martin I think Martin has always supported me in some way and I really appreciate that. But, maybe because I won commit privileges solely based on GSoC work, I felt other developers wouldn't approve my commits without previous discussion and that is the major reason for not committing most of my patches. -- -- Guilherme H. Polo Goncalves From taleinat at gmail.com Mon Jul 12 00:31:20 2010 From: taleinat at gmail.com (Tal Einat) Date: Mon, 12 Jul 2010 01:31:20 +0300 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <217B30F6-06FE-4237-90C2-540CD6BB6A28@twistedmatrix.com> References: <217B30F6-06FE-4237-90C2-540CD6BB6A28@twistedmatrix.com> Message-ID: Glyph Lefkowitz wrote: > > On Jul 11, 2010, at 10:22 AM, Tal Einat wrote: > > The people who are actually *in* those environments seem to disagree with > you :). ?I think you underestimate the difficulty of getting software > installed and overestimate the demands of new Python users and students. > While I don't ever use IDLE if there's an alternative available, I have been > very grateful many times for its presence in environments where it was a > struggle even to say "install Python". ?A workable editor and graphical > shell is important, whatever its flaws. ?(And I think you exaggerate IDLE's > flaws just a bit.) I would like to note that I am one of those in those environments. I have used IDLE to teach Python to new users, and their opinion of IDLE (and Python as a consequence) wasn't great, precisely because of the bugs and quirks. Recently I have stopped recommending IDLE for beginners and have found that this avoids quite a few snags, which is significant. I have also been in environments where installing anything was problematic and having IDLE available out-of-the-box was supposed to be a clear win. I certainly used it, but all of my coworkers (experienced Pythonistas who have worked with IDLE before) ended up preferring the basic interpreter and text editors such as vim, despite my advocacy of IDLE, because they tired of IDLE's snags (e.g. the inability to run several instances in parallel). My point is that I don't think I am exaggerating IDLE's flaws. I'm not saying that it is no longer usable or useful, but I am saying that its current state is not "okay". - Tal Einat From martin at v.loewis.de Mon Jul 12 00:36:33 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 12 Jul 2010 00:36:33 +0200 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> Message-ID: <4C3A4771.5090501@v.loewis.de> > I think Martin has always supported me in some way and I really > appreciate that. But, maybe because I won commit privileges solely > based on GSoC work, I felt other developers wouldn't approve my > commits without previous discussion and that is the major reason for > not committing most of my patches. I'm not blaming you for that; I can fully understand how you feel and how things came about. I was just refuting the claim that something was fundamentally wrong, when, among us three, there was just somebody failing to say "go ahead" (and I'd like to stress that *either* of us three could have said that). So: go ahead. I won't have the time to review your changes in detail, and nobody else will, so just put them in, and we wait for people to complain (in which case I'd hope that you would be around to respond quickly - as you did in the past). If there is also Tk patches involved, please do have them reviewed, though. Regards, Martin From rrr at ronadam.com Mon Jul 12 00:38:02 2010 From: rrr at ronadam.com (Ron Adam) Date: Sun, 11 Jul 2010 17:38:02 -0500 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: On 07/10/2010 06:05 PM, Tal Einat wrote: > Hello, > > I would like to propose removing IDLE from the standard library. > > I have been using IDLE since 2002 and have been doing my best to help > maintain and further develop IDLE since 2005. > > In recent years IDLE has received negligible interest and attention from > the Python community. During this time IDLE has slowly gone downhill. > The documentation and tutorials grow increasingly out of date. > Cross-platform support has degraded with the increasing popularity of > OSX and 64-bit platforms. Bugs take months, and sometimes more than a > year, to be solved. Features that have since become common-place, such > as having a non-intrusive search box instead of a dialog, are obviously > and painfully lacking, making IDLE feel clumsy and out-dated. > > For these reasons, I think it would be fitting to remove IDLE from the > standard library. IDLE is no longer recommended to beginners, IMO > rightfully so, and this was the main reason for its inclusion in the > standard library. Furthermore, if there is little or no interest in > developing and maintaining IDLE, it should be removed to avoid having > buggy and badly supported software in the standard library. There might be another alternative. Both idle and pydoc are applications (are there others?) that are in the standard library. As such, they or parts of them, are possibly importable to other projects. That restricts changes because a committer needs to consider the chances that a change may break something else. I suggest they be moved out of the lib directory, but still be included with python. (Possibly in the tools directory.) That removes some of the backward compatibility restrictions or at least makes it clear there isn't a need for backward compatibility. Would a change of this sort help make things any easier? (Note: idle isn't in the lib directory on Ubuntu.) Ron From taleinat at gmail.com Mon Jul 12 00:39:19 2010 From: taleinat at gmail.com (Tal Einat) Date: Mon, 12 Jul 2010 01:39:19 +0300 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> Message-ID: Guilherme Polo wrote: > 2010/7/11 "Martin v. L?wis" : >>> In the 2009 Google Summer of Code I was the mentor for a Brazilian >>> student, Guilherme Polo, who completed and extended important >>> improvements to IDLE made during the previous year by David Scherer. >>> Given the somewhat official nature of this work, I assumed that these >>> needed improvements would make it into the standard distribution, but as >>> far as I know that hasn't happened. It would seem that if even this >>> "sponsored" project didn't impact the standard Python distribution, >>> something is broken in the procedures, and probably what is needed is, >>> as Guido says, that someone be given the authority to get improvements >>> to IDLE into the standard distribution. Making a significant change to >>> the update procedures is clearly needed. >> >> I don't think so; instead, the perception of authority needs to be >> adjusted (in the specific case). Guilherme could have committed these >> changes, but, for whatever reason, decided not to. Nor did his direct >> mentor (i.e. you) tell him to commit the changes, and neither did I. >> >>> Even if this needed change is made, there is also merit to Tai's >>> suggestion of creating a separate project, to encourage developers like >>> him to work together to improve IDLE, without having as a first priority >>> to worry about getting it into the standard distribution, but with the >>> clear understanding that this is the place to go for improvements to >>> migrate into the standard distribution. >> >> Again, Guilherme could commit his changes any time. >> >> Regards, >> Martin > > I think Martin has always supported me in some way and I really > appreciate that. But, maybe because I won commit privileges solely > based on GSoC work, I felt other developers wouldn't approve my > commits without previous discussion and that is the major reason for > not committing most of my patches. FWIW this is why I started IDLE-Spoon (well, continued Noam Raphael's project of the same name, in a sense). The idea was to have a fork of IDLE with new features which need to be tried out by "beta testers" to iron out all of the glitches before making it into the main version, like IDLE-fork back in the beginning of the decade. However I have utterly failed in promoting this project and getting "beta testers" on board, at least partially due to the lack of interest from the community (and admittedly my lack of PR skills). - Tal Einat From martin at v.loewis.de Mon Jul 12 00:41:50 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 12 Jul 2010 00:41:50 +0200 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <217B30F6-06FE-4237-90C2-540CD6BB6A28@twistedmatrix.com> Message-ID: <4C3A48AE.9010707@v.loewis.de> > My point is that I don't think I am exaggerating IDLE's flaws. I'm not > saying that it is no longer usable or useful, but I am saying that its > current state is not "okay". So can you produce a list of patches that you think can be accepted as-is? Preferably, make to lists: bug fixes, and new features. The bug fixes could be either for 2.x or 3.x; the new features would preferably be for just 3.x. Regards, Martin From martin at v.loewis.de Mon Jul 12 00:44:14 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 12 Jul 2010 00:44:14 +0200 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> Message-ID: <4C3A493E.8090308@v.loewis.de> > FWIW this is why I started IDLE-Spoon (well, continued Noam Raphael's > project of the same name, in a sense). The idea was to have a fork of > IDLE with new features which need to be tried out by "beta testers" to > iron out all of the glitches before making it into the main version, > like IDLE-fork back in the beginning of the decade. However I have > utterly failed in promoting this project and getting "beta testers" on > board, at least partially due to the lack of interest from the > community (and admittedly my lack of PR skills). I think such a thing must inherently fail - precisely for these reasons. It's much better to release proposed new features along with Python, and wait for feedback. Users won't start trying things out until after the release. This is a general problem, and lead Barry Warsaw to believe that "release candidates" are an utter waste of time. Regards, Martin From breamoreboy at yahoo.co.uk Mon Jul 12 00:51:35 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 11 Jul 2010 23:51:35 +0100 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> Message-ID: On 11/07/2010 19:59, Glyph Lefkowitz wrote: > > On Jul 11, 2010, at 2:37 PM, Martin v. L?wis wrote: > >>> Initially (five years ago!) I tried to overcome these issues by >>> improving IDLE, solving problems and adding a few key features. >>> Without going into details, suffice to say that IDLE hasn't improved >>> much since 2005 despite my efforts. For example, see >>> http://bugs.python.org/issue1529142, where it took nearly 3 years to >>> fix a major issue from the moment I posted the first workaround. For >>> another example, see http://bugs.python.org/issue3068, where I posted >>> a patch for an extension configuration dialog over two years ago, and >>> it hasn't received as much as a sneeze in response. >> >> I can understand that this is frustrating, but please understand that >> this is not specific to your patches, or to IDLE. Many other patches on >> bugs.python.org remain unreviewed for many years. That's because many of >> the issues are really tricky, and there are very few people who both >> have the time and the expertise to evaluate them. > > This problem seems to me to be the root cause here. > > Guido proposes to give someone interested in IDLE commit access, and hopefully that will help in this particular area. But, as I recall, at the last language summit there was quite a bit of discussion about how to address the broader issue of patches falling into a black hole. Is anybody working on it? > > (This seems to me like an area where a judicious application of PSF funds might help; if every single bug were actively triaged and responded to, even if it weren't reviewed, and patch contributors were directed to take specific steps to elicit a response or a review, the fact that patch reviews take a while might not be so bad.) > >> FWIW, I don't consider a few months as a "long" time for a patch review. > > It may not be a long time compared to other patch reviews, but it is a very long time for a volunteer to wait for something, especially if that "something" is "any indication that the python developers care that this patch was submitted at all". > > There seems to be at least one thread a month on this list from a disgruntled community member complaining (directly or indirectly) about this delay. I think that makes it a big problem. > >> At the moment, I'm personally able to perhaps review one issue per week >> (sometimes less); at this rate, it'll take several years until I get >> to everything. > > > I guess it depends what you mean by "everything", but given that the open bug count is actually increasing at a significant rate, I would say that you can never possibly get to "everything". > > > > > > _______________________________________________ > IDLE-dev mailing list > IDLE-dev at python.org > http://mail.python.org/mailman/listinfo/idle-dev I have been attempting to fill this hole and have been faced with animosity from people who "hang out" on the python-dev IRC channel. I thought it was a complete and utter waste of space, so I don't intend going back. I would like things fixed, not a cosy little "who's round is it next" mentality from the triage team. IMHO if they spent more time doing things, and less time talking crap via IRC, things might get done. And before anyone says anything, I have been a former MBCS and CEng and only gave up cos I couldn't afford the annual fees cos of my health. Kindest regards. Mark Lawrence From breamoreboy at yahoo.co.uk Mon Jul 12 01:03:52 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 12 Jul 2010 00:03:52 +0100 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> Message-ID: On 11/07/2010 23:18, Tal Einat wrote: > Martin v. L?wis wrote: >>> Initially (five years ago!) I tried to overcome these issues by >>> improving IDLE, solving problems and adding a few key features. >>> Without going into details, suffice to say that IDLE hasn't improved >>> much since 2005 despite my efforts. For example, see >>> http://bugs.python.org/issue1529142, where it took nearly 3 years to >>> fix a major issue from the moment I posted the first workaround. For >>> another example, see http://bugs.python.org/issue3068, where I posted >>> a patch for an extension configuration dialog over two years ago, and >>> it hasn't received as much as a sneeze in response. >> >> I can understand that this is frustrating, ... > > I am not writing this to vent my frustration, and have purposely > avoided making that the issue. > >> ... but please understand that >> this is not specific to your patches, or to IDLE. Many other patches on >> bugs.python.org remain unreviewed for many years. That's because many of >> the issues are really tricky, and there are very few people who both >> have the time and the expertise to evaluate them. > > I am aware of the situation with regard to issue reviews, but I think > with IDLE there is more going on. In other parts of the Python > codebase, a workaround for a major usability issue wouldn't normally > have taken nearly three years to resolve after a working patch was > submitted. And a working, well-tested patch wouldn't normally have sat > ignored for over two years. Well, perhaps these things happen > occasionally, but with IDLE this is the norm. > >> FWIW, I don't consider a few months as a "long" time for a patch review. >> At the moment, I'm personally able to perhaps review one issue per week >> (sometimes less); at this rate, it'll take several years until I get >> to everything. > > I'm not talking about a few months, I'm talking about at least six > months in most cases, years in many cases, as in the examples I > mentioned. > > - Tal Einat I can understand your frustration, but in response to an appeal from Terry Reedy some weeks back on c.l.py I've done a substantial amount of work in the last couple of weeks to clear outstanding issues, sadly IDLE just sits in the pile. Ow, but hang on a minute, I've already volunteered TJR to take this on, I believe he's up for it, I'll support him up to the hilt, so why the hell can't we get on with it? Or would the triage team as it stands object cos they'll be put out of a job? :) Kindest regards. Mark Lawrence. From benjamin at python.org Mon Jul 12 01:08:20 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 11 Jul 2010 18:08:20 -0500 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> Message-ID: 2010/7/11 Mark Lawrence : > > I have been attempting to fill this hole and have been faced with animosity > from people who "hang out" on the python-dev IRC channel. ?I thought it was > a complete and utter waste of space, so I don't intend going back. ?I would > like things fixed, not a cosy little "who's round is it next" mentality from > the triage team. ?IMHO if they spent more time doing things, and less time > talking crap via IRC, things might get done. ?And before anyone says > anything, I have been a former MBCS and CEng and only gave up cos I couldn't > afford the annual fees cos of my health. What exactly is the "who's round is it next" mentality? -- Regards, Benjamin From breamoreboy at yahoo.co.uk Mon Jul 12 01:13:21 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 12 Jul 2010 00:13:21 +0100 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <4C3A103A.1070504@v.loewis.de> References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> <4C3A103A.1070504@v.loewis.de> Message-ID: On 11/07/2010 19:40, "Martin v. L?wis" wrote: >>> There clearly are *some* folks who care enough about IDLE to submit >>> bug reports and fixes. How about we empower these people by giving at >>> least one of them commit privileges? IDLE development has often been >>> done by people who aren't otherwise contributing to the core, and we >>> surely should trust those folks with commit privileges. >>> >> >> Can I take a really big liberty and volunteer Terry Reedy for the job. > > It doesn't work that way. You can't volunteer somebody else (*). > > If Terry would volunteer himself, he'd get commit access in no time. > > Regards, > Martin > > (*) so when you assign bugs to me, it probably means that they get less > attention, not more. Martin, Thanks for your response. IIRC Terry Reedy has already volunteered to do this, if I'm incorrect I'll apologise right now to both of you. As for assigning bugs, I've been told to use the maintainer.rst list, so either the list is wrong, or I've had finger problems. If it's the latter I again say sorry. Kindest regards. Mark Lawrence From benjamin at python.org Mon Jul 12 01:14:38 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 11 Jul 2010 18:14:38 -0500 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> Message-ID: 2010/7/11 Mark Lawrence : > I can understand your frustration, but in response to an appeal from Terry > Reedy some weeks back on c.l.py I've done a substantial amount of work in > the last couple of weeks to clear outstanding issues, sadly IDLE just sits > in the pile. ?Ow, but hang on a minute, ?I've already volunteered TJR to > take this on, I believe he's up for it, I'll support him up to the hilt, so > why the hell can't we get on with it? ?Or would the triage team as it stands > object cos they'll be put out of a job? :) And as Martin has already noted, only Terry can volunteer himself. -- Regards, Benjamin From steve at pearwood.info Mon Jul 12 01:37:16 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 12 Jul 2010 09:37:16 +1000 Subject: [Python-Dev] Can ftp url start with file:// ? In-Reply-To: References: <20100709035232.GA6427@remy> <20100711035657.GA15730@remy> Message-ID: <201007120937.17220.steve@pearwood.info> On Mon, 12 Jul 2010 12:01:07 am Steve Holden wrote: > My own inclination would be to regard the current treatment of file: > as a bug (albeit one not worth fixing on 2.x). ... > Presumably a > hostname in such a URI would require that some host-specific protocol > be used (but this should be an access protocol like SMB or NFS, not a > transfer protocol like FTP). KDE uses SMB for non-local hostnames in file URIs. > It seems pretty clear that randomly interpreting particular host > names to imply a specific remote-access protocol like FTP is bogus. Agreed. The right behaviour is to raise an exception and let the caller deal with it, or provide a means to register an alternative. -- Steven D'Aprano From steve at pearwood.info Mon Jul 12 01:42:40 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 12 Jul 2010 09:42:40 +1000 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: <4C39ACF2.6060108@trueblade.com> References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> Message-ID: <201007120942.40778.steve@pearwood.info> On Sun, 11 Jul 2010 09:37:22 pm Eric Smith wrote: > > re2 comparison is interesting from the point of if it should be > > included in stdlib. > > Is "it" re2 or regex? I don't see having 2 regular expression engines > in the stdlib. There's precedence though... the old regex engine and the new re engine were side-by-side for many years before regex was deprecated and finally removed in 2.5. Hypothetically, re2 could similarly be added to the standard library while re is deprecated. -- Steven D'Aprano From glyph at twistedmatrix.com Mon Jul 12 01:43:06 2010 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Sun, 11 Jul 2010 19:43:06 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <4C3A1938.4020304@v.loewis.de> References: <4C3A0F85.1090404@v.loewis.de> <4C3A1938.4020304@v.loewis.de> Message-ID: <1E32216D-8C3A-4BD8-8E5A-127E98B863AB@twistedmatrix.com> On Jul 11, 2010, at 3:19 PM, Martin v. L?wis wrote: > Unfortunately, it's often not clear what the submitter wants: does she > want to help, or want to get help? For a bug report, I often post a > message "can you provide a patch?", but sometimes, it isn't that clear. Perhaps this is the one area where the biggest advance could be made: a clarification of the workflow. My experience with Python issues which have been "triaged" is that everyone who triages tickets has a slightly different idea of who is responsible for the ticket and what they're supposed to do next at every point in the process. Triage, as described on , emphasizes making sure "that all fields in the issue tracker are properly set", rather than on communicating with the contributor or reporter. On Twisted, we try to encourage triagers to focus on communicating the workflow ramifications of what a particular contributor has done. We try to provide a response to the bug reporter or patch submitter that says "thanks, but in order to move this along, you need to go through the following steps" and sometimes even attach a link to the workflow document pointing out exactly where in the process the ticket is now stuck. (At least, that's what we're trying to do.) This involves a lot of repeating ourselves in ticket comments, but it's well worth it (and as more of the repetition moves into citing links to documents that have been written to describe aspects of the workflow, it's less onerous). describes what the steps are, but it's in a sort of procedural passive voice that doesn't say who is responsible for doing reviews or how to get a list of patches which need to be reviewed or what exactly a third-party non-core-committer reviewer should do to remove the 'Patch review' keyword. and meander around a bit, but a while ago we re-worked them so that each section has a specific audience (authors, reviewers, or external patch submitters) and that helped readers understand what they're intended to do. Plus, is a useful resource for core developers with only a little bit of free time to do a review. (I'm just offering some suggestions based on what I think has worked, not to hold Twisted up as a paragon of a perfect streamlined process. We still have folks complain about stuck patches, these documents are _far_ from perfect, and there are still some varying opinions about how certain workflow problems should be dealt with and differences in quality of review. Plus, we have far fewer patches to deal with than Python. Nevertheless, the situation used to be worse for us, and these measures seem to have helped.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From glyph at twistedmatrix.com Mon Jul 12 01:46:16 2010 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Sun, 11 Jul 2010 19:46:16 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> Message-ID: <5896CB82-8A76-4D64-B665-EB16DC2C4A9A@twistedmatrix.com> On Jul 11, 2010, at 5:33 PM, Georg Brandl wrote: > Honestly, how would you feel as a committer to have scores of issues assigned > to you -- as a consequence of speedy triage -- knowing that you have to invest > potentially hours of volunteer time into them, while the person doing the > triaging is done with the bug in a few minutes and paid for it? I'd feel a > little bit duped. That doesn't strike me as a particularly useful type of triage. The most useful type of triage in this case would be the kind where the bug gets re-assigned to the *original contributor*, not a core committer, with a message clearly saying "thanks! but we will not do anything further with this ticket until *you* do XYZ." This may result in some tickets getting left by wayside, but at least it will be clear that they have been left by the wayside, and whose responsibility they really are. Even so, I would certainly feel better having scores of issues assigned to me than I would feel having scores of issues that are just hanging out in limbo forever. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Mon Jul 12 01:53:39 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 11 Jul 2010 19:53:39 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: <4C3A5983.3010002@holdenweb.com> Stephen Hansen wrote: > On Sat, Jul 10, 2010 at 9:23 PM, Guilherme Polo > wrote: > > By "never had a problem" do you mean using some of the latest versions > ? Here, running "idle" from a mac terminal and trying to type: print > "hi" crashes when entering the quotation mark. > > > Huh? Works fine for me. Python 2.6.1, OSX 10.6.3, intel. > One of the good things about the python-dev community is its commitment to test-driven development. If you are prepared to define "fine" as 'successfully runs \'print "hello"\'' then I guess we should be perfectly happy about IDLE. > From the lurking crowd-- Please don't consider removing IDLE until there > is a compelling replacement ready. It's better to have a limited IDE > that works everywhere (even if in a limited fashion-- people are free to > try out one of the many excellent full-featured Python IDE's out there > after they advance to that point) then not. > 1: I refuse to see why we need a "compelling replacement" for a piece of software whose performance might be actively deterring people from taking up the language. ["Have you thought about Python?" "Yeah, but I tried it {meaning "I downloaded some random Python release and tried IDLE, which by modern standards appears completely lame"} and it sucked". If this is our standard for "compelling" then it appears the command-line interpreter is the competition. 2: Correct me if I am wrong, but isn't the implied promise of including something in the stdlib that there will be active maintainers? If that's the case then we need to either recruit more active maintainers than the package currently has or look at dropping it. 3: If IDLE *is* going to be dropped from the stdlib it should be deprecated first just like anything else. 4: It's all very well chastising "the development community" because IDLE issues get assigned to kbk and nothing happens about them, but it's not like kbk is receiving any kind of rewards for working on these tickets, and precious little indication that anyone else is prepared to roll up their sleeves and ask to take over the tickets [apologies to anyone who has actually done this and been rebuffed] to get them fixed quicker. 5: Decide for yourself whether I am one of "the lurking crowd". I teach Python classes for a living (among other things) and invest quite a bit of time in the Python community one way or the other. I am not a Mac user, but another instructor I have employed is, and he has discussed with Mac users exactly how deficient the IDLE environment is when compared with standard Mac utilities. 6: When I give students a free choice of the development environment, they often choose IDLE "because it comes with Python". This usually results in a certain amount of discussion and comparison with tools like Wing, PythonWin and so on. Which in itself isn't a bad thing, but IDLE complares so badly with the other products that I sometimes feel Python suffers by association. IDLE has simplicity on its side, but every way it interacts with the user appears to be non-standard for most platforms. It needs some maintenance, but I don't see where that's going to come from. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From martin at v.loewis.de Mon Jul 12 01:56:19 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 12 Jul 2010 01:56:19 +0200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> <4C3A103A.1070504@v.loewis.de> Message-ID: <4C3A5A23.2070903@v.loewis.de> > IIRC Terry Reedy has already volunteered to do this Hmm. I don't recall that happening. > As for assigning bugs, I've been told to use the maintainer.rst list, so > either the list is wrong, or I've had finger problems. If it's the > latter I again say sorry. I see. What copy have you been using specifically? I think I need to remove myself from these lists. Regards, Martin From breamoreboy at yahoo.co.uk Mon Jul 12 02:07:54 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 12 Jul 2010 01:07:54 +0100 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <4C3A5A23.2070903@v.loewis.de> References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> <4C3A103A.1070504@v.loewis.de> <4C3A5A23.2070903@v.loewis.de> Message-ID: On 12/07/2010 00:56, "Martin v. L?wis" wrote: > >> IIRC Terry Reedy has already volunteered to do this Apologies to Terry if this is incorrect, but I believe this to be the case. > > Hmm. I don't recall that happening. > >> As for assigning bugs, I've been told to use the maintainer.rst list, so >> either the list is wrong, or I've had finger problems. If it's the >> latter I again say sorry. > > I see. What copy have you been using specifically? I think I need to > remove myself from these lists. > > Regards, > Martin Hi Martin, Again thanks for the response. I've been working from this:- http://svn.python.org/view/*checkout*/python/branches/py3k/Misc/maintainers.rst It strikes me as being so sadly outdated that it's getting less than useless, or I assume that it's the same old case of not enough volunteers? Why did I bother. :) Kindest regards. Mark Lawrence. From jnoller at gmail.com Mon Jul 12 02:21:06 2010 From: jnoller at gmail.com (Jesse Noller) Date: Sun, 11 Jul 2010 20:21:06 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> <4C3A103A.1070504@v.loewis.de> Message-ID: On Sun, Jul 11, 2010 at 7:13 PM, Mark Lawrence wrote: > > Martin, > > Thanks for your response. > > IIRC Terry Reedy has already volunteered to do this, if I'm incorrect I'll > apologise right now to both of you. > > As for assigning bugs, I've been told to use the maintainer.rst list, so > either the list is wrong, or I've had finger problems. ?If it's the latter I > again say sorry. > > Kindest regards. > > Mark Lawrence Hey Mark - I've noticed some of your cleanup/triaging in my bug queue at least (multiprocessing) and I thought I'd thank you - I didn't know who you were! jesse From debatem1 at gmail.com Mon Jul 12 02:22:26 2010 From: debatem1 at gmail.com (geremy condra) Date: Sun, 11 Jul 2010 20:22:26 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <20100711213801.4d391849@pitrou.net> References: <4C3A0F85.1090404@v.loewis.de> <20100711213801.4d391849@pitrou.net> Message-ID: On Sun, Jul 11, 2010 at 3:38 PM, Antoine Pitrou wrote: > On Sun, 11 Jul 2010 14:59:14 -0400 > Glyph Lefkowitz wrote: >> >> Guido proposes to give someone interested in IDLE commit access, and hopefully that will help in > this particular area. ?But, as I recall, at the last language summit there was quite a bit of >> discussion about how to address the broader issue of patches falling into a black hole. ?Is >> anybody working on it? > > I think the best way to "work on it" is to work on having more core > developers, possibly with a more diverse range of interests. > >> (This seems to me like an area where a judicious application of PSF funds might help; if every >> single bug were actively triaged and responded to, even if it weren't reviewed, and patch >> contributors were directed to take specific steps to elicit a response or a review, the fact that >> patch reviews take a while might not be so bad.) > > The operative word being "judicious". It is not obvious who should get > funded, and for what tasks. > Some specific issues (like email in 3.x) are large enough that they can > be the sole focus of a fund grant. But I'm not sure triaging can apply. I'm mulling over starting a monthly triage sprint under the auspices of Jesse Noeller's PSF sponsored sprints in the hopes of making this a little more fun. I'd appreciate comments on the idea. Geremy Condra From debatem1 at gmail.com Mon Jul 12 02:23:08 2010 From: debatem1 at gmail.com (geremy condra) Date: Sun, 11 Jul 2010 20:23:08 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> <20100711213801.4d391849@pitrou.net> Message-ID: On Sun, Jul 11, 2010 at 8:22 PM, geremy condra wrote: > On Sun, Jul 11, 2010 at 3:38 PM, Antoine Pitrou wrote: >> On Sun, 11 Jul 2010 14:59:14 -0400 >> Glyph Lefkowitz wrote: >>> >>> Guido proposes to give someone interested in IDLE commit access, and hopefully that will help in > this particular area. ?But, as I recall, at the last language summit there was quite a bit of >>> discussion about how to address the broader issue of patches falling into a black hole. ?Is >>> anybody working on it? >> >> I think the best way to "work on it" is to work on having more core >> developers, possibly with a more diverse range of interests. >> >>> (This seems to me like an area where a judicious application of PSF funds might help; if every >>> single bug were actively triaged and responded to, even if it weren't reviewed, and patch >>> contributors were directed to take specific steps to elicit a response or a review, the fact that >>> patch reviews take a while might not be so bad.) >> >> The operative word being "judicious". It is not obvious who should get >> funded, and for what tasks. >> Some specific issues (like email in 3.x) are large enough that they can >> be the sole focus of a fund grant. But I'm not sure triaging can apply. > > I'm mulling over starting a monthly triage sprint under the auspices of > Jesse Noeller's PSF sponsored sprints in the hopes of making this a > little more fun. I'd appreciate comments on the idea. > > Geremy Condra > Apologies, Jesse, I thought your name had an extra 'e' in it. Geremy Condra From jnoller at gmail.com Mon Jul 12 02:29:20 2010 From: jnoller at gmail.com (Jesse Noller) Date: Sun, 11 Jul 2010 20:29:20 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> <4C3A103A.1070504@v.loewis.de> <4C3A5A23.2070903@v.loewis.de> Message-ID: On Sun, Jul 11, 2010 at 8:07 PM, Mark Lawrence wrote: >> >> I see. What copy have you been using specifically? I think I need to >> remove myself from these lists. >> >> Regards, >> Martin > > Hi Martin, > > Again thanks for the response. > > I've been working from this:- > http://svn.python.org/view/*checkout*/python/branches/py3k/Misc/maintainers.rst > > It strikes me as being so sadly outdated that it's getting less than > useless, or I assume that it's the same old case of not enough volunteers? > ?Why did I bother. :) > > Kindest regards. > > Mark Lawrence. That file is too young to be out of date, and like I said, I've found the help useful Mark, so I wouldn't throw your hands up. That file should probably be updated/refreshed as people want, the idea behind it was to build a table of experts or people who voluntarily sign up to be the more active maintainers for a domain or standard library module. I maintain the idea is a good one - even if Martin wants off due to lack of time, it's important for people who are just coming in, such as Mark, to have some idea of where or *who* to talk to about something specific. We can obviously see the frightening number of gaps we have for "specific" maintainers. jesse From debatem1 at gmail.com Mon Jul 12 02:49:29 2010 From: debatem1 at gmail.com (geremy condra) Date: Sun, 11 Jul 2010 20:49:29 -0400 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: <201007120942.40778.steve@pearwood.info> References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> Message-ID: On Sun, Jul 11, 2010 at 7:42 PM, Steven D'Aprano wrote: > On Sun, 11 Jul 2010 09:37:22 pm Eric Smith wrote: >> > re2 comparison is interesting from the point of if it should be >> > included in stdlib. >> >> Is "it" re2 or regex? I don't see having 2 regular expression engines >> in the stdlib. > > There's precedence though... the old regex engine and the new re engine > were side-by-side for many years before regex was deprecated and > finally removed in 2.5. Hypothetically, re2 could similarly be added to > the standard library while re is deprecated. While I realize I'm not the target audience for this, there are a lot of things I'd like to see more in the stdlib than a second re engine. Geremy Condra From jnoller at gmail.com Mon Jul 12 02:50:02 2010 From: jnoller at gmail.com (Jesse Noller) Date: Sun, 11 Jul 2010 20:50:02 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> <20100711213801.4d391849@pitrou.net> Message-ID: On Sun, Jul 11, 2010 at 8:22 PM, geremy condra wrote: > On Sun, Jul 11, 2010 at 3:38 PM, Antoine Pitrou wrote: >> On Sun, 11 Jul 2010 14:59:14 -0400 >> Glyph Lefkowitz wrote: >>> >>> Guido proposes to give someone interested in IDLE commit access, and hopefully that will help in > this particular area. ?But, as I recall, at the last language summit there was quite a bit of >>> discussion about how to address the broader issue of patches falling into a black hole. ?Is >>> anybody working on it? >> >> I think the best way to "work on it" is to work on having more core >> developers, possibly with a more diverse range of interests. >> >>> (This seems to me like an area where a judicious application of PSF funds might help; if every >>> single bug were actively triaged and responded to, even if it weren't reviewed, and patch >>> contributors were directed to take specific steps to elicit a response or a review, the fact that >>> patch reviews take a while might not be so bad.) >> >> The operative word being "judicious". It is not obvious who should get >> funded, and for what tasks. >> Some specific issues (like email in 3.x) are large enough that they can >> be the sole focus of a fund grant. But I'm not sure triaging can apply. > > I'm mulling over starting a monthly triage sprint under the auspices of > Jesse Noeller's PSF sponsored sprints in the hopes of making this a > little more fun. I'd appreciate comments on the idea. > > Geremy Condra Well, I'd like to get the sprint "how to" docs in shape, and then yeah - we can totally do this. The sprints focuses were designed to help with this pain (as others have pointed out) as well as other pain points we've seen. Also note, hosting a "virtual sprint" or something like that, which the sponsored sprint group simply helps advertise and promote can help with this as well. It's important though, when looking at triaging to keep in mind: Moving the bug around (reassigning, etc) is of minimal use in a fair number of cases - now, making sure it has a reproducible test case (and can be reproduced), making sure the broken platforms are enumerated, that patches have tests and docs (and are based on the current trunk/whatever) and so on are much more valuable in the long run. Just some food for thought - and something to keep in mind if and when we get to document more of this/etc. jesse From jnoller at gmail.com Mon Jul 12 03:13:09 2010 From: jnoller at gmail.com (Jesse Noller) Date: Sun, 11 Jul 2010 21:13:09 -0400 Subject: [Python-Dev] PEP 3148 ready for pronouncement [ACCEPTED] Message-ID: On Sat, May 22, 2010 at 11:38 AM, Guido van Rossum wrote: [snip] > Great points Jesse! Since I really don't have the time or expertise to > make a judgment on this PEP, I hereby appoint you chair of the > approval process for this PEP. That basically means that when you > think it's ready to be approved, you say so, and it's a done deal. The > remaining feedback cycle is up to you now -- it sounds like you're > ready for closure, which sounds good to me (again, without having read > the PEP or tried to write something using the proposed code). You can > do it however you like: you can declare it approved now, or read it > over once more yourself and suggest some final changes, or set a > period (e.g. 48 hours) during which final comments have to be > received, which you then will judge by merit or by your whim, or you > can flip a coin or say a prayer... (I've tried most of those myself in > the past and haven't done too badly if I say so myself. :-) You're the > boss now. I know you will do the right thing for this PEP. > > -- > --Guido van Rossum (python.org/~guido) > So, after some cool down - and the last rounds of discussion which triggered some jiggery-pokery on Brian's part, I'm accepting PEP 3148 "futures - execute computations asynchronously". I feel that it's a good addition, and a good start for something bigger down the road. Brian - you'll need to provide someone such as Martin or Georg your public key for ssh access into SVN, and you'll need developer access to the bug tracker. jesse From benjamin at python.org Mon Jul 12 04:07:30 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 11 Jul 2010 21:07:30 -0500 Subject: [Python-Dev] PEP 3148 ready for pronouncement [ACCEPTED] In-Reply-To: References: Message-ID: 2010/7/11 Jesse Noller : > On Sat, May 22, 2010 at 11:38 AM, Guido van Rossum wrote: > [snip] >> Great points Jesse! Since I really don't have the time or expertise to >> make a judgment on this PEP, I hereby appoint you chair of the >> approval process for this PEP. That basically means that when you >> think it's ready to be approved, you say so, and it's a done deal. The >> remaining feedback cycle is up to you now -- it sounds like you're >> ready for closure, which sounds good to me (again, without having read >> the PEP or tried to write something using the proposed code). You can >> do it however you like: you can declare it approved now, or read it >> over once more yourself and suggest some final changes, or set a >> period (e.g. 48 hours) during which final comments have to be >> received, which you then will judge by merit or by your whim, or you >> can flip a coin or say a prayer... (I've tried most of those myself in >> the past and haven't done too badly if I say so myself. :-) You're the >> boss now. I know you will do the right thing for this PEP. >> >> -- >> --Guido van Rossum (python.org/~guido) >> > > So, after some cool down - and the last rounds of discussion which > triggered some jiggery-pokery on Brian's part, I'm accepting PEP 3148 > "futures - execute computations asynchronously". I feel that it's a > good addition, and a good start for something bigger down the road. > > Brian - you'll need to provide someone such as Martin or Georg your > public key for ssh access into SVN, and you'll need developer access > to the bug tracker. Oh, don't worry. He's already been hacking happily on Windows stuff. -- Regards, Benjamin From jnoller at gmail.com Mon Jul 12 04:11:08 2010 From: jnoller at gmail.com (Jesse Noller) Date: Sun, 11 Jul 2010 22:11:08 -0400 Subject: [Python-Dev] PEP 3148 ready for pronouncement [ACCEPTED] In-Reply-To: References: Message-ID: On Sun, Jul 11, 2010 at 10:07 PM, Benjamin Peterson wrote: > 2010/7/11 Jesse Noller : >> On Sat, May 22, 2010 at 11:38 AM, Guido van Rossum wrote: >> [snip] >>> Great points Jesse! Since I really don't have the time or expertise to >>> make a judgment on this PEP, I hereby appoint you chair of the >>> approval process for this PEP. That basically means that when you >>> think it's ready to be approved, you say so, and it's a done deal. The >>> remaining feedback cycle is up to you now -- it sounds like you're >>> ready for closure, which sounds good to me (again, without having read >>> the PEP or tried to write something using the proposed code). You can >>> do it however you like: you can declare it approved now, or read it >>> over once more yourself and suggest some final changes, or set a >>> period (e.g. 48 hours) during which final comments have to be >>> received, which you then will judge by merit or by your whim, or you >>> can flip a coin or say a prayer... (I've tried most of those myself in >>> the past and haven't done too badly if I say so myself. :-) You're the >>> boss now. I know you will do the right thing for this PEP. >>> >>> -- >>> --Guido van Rossum (python.org/~guido) >>> >> >> So, after some cool down - and the last rounds of discussion which >> triggered some jiggery-pokery on Brian's part, I'm accepting PEP 3148 >> "futures - execute computations asynchronously". I feel that it's a >> good addition, and a good start for something bigger down the road. >> >> Brian - you'll need to provide someone such as Martin or Georg your >> public key for ssh access into SVN, and you'll need developer access >> to the bug tracker. > > Oh, don't worry. He's already been hacking happily on Windows stuff. > > > -- > Regards, > Benjamin > Wrong Brian - that's Brian Curtin, this is Brian Quinlan - I double checked the committer's list (http://www.python.org/dev/committers). We now have two Brians. I say we name them PresentBrian and FutureBrian. jesse From benjamin at python.org Mon Jul 12 04:12:34 2010 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 11 Jul 2010 21:12:34 -0500 Subject: [Python-Dev] PEP 3148 ready for pronouncement [ACCEPTED] In-Reply-To: References: Message-ID: 2010/7/11 Jesse Noller : > On Sun, Jul 11, 2010 at 10:07 PM, Benjamin Peterson wrote: >> 2010/7/11 Jesse Noller : >>> On Sat, May 22, 2010 at 11:38 AM, Guido van Rossum wrote: >>> [snip] >>>> Great points Jesse! Since I really don't have the time or expertise to >>>> make a judgment on this PEP, I hereby appoint you chair of the >>>> approval process for this PEP. That basically means that when you >>>> think it's ready to be approved, you say so, and it's a done deal. The >>>> remaining feedback cycle is up to you now -- it sounds like you're >>>> ready for closure, which sounds good to me (again, without having read >>>> the PEP or tried to write something using the proposed code). You can >>>> do it however you like: you can declare it approved now, or read it >>>> over once more yourself and suggest some final changes, or set a >>>> period (e.g. 48 hours) during which final comments have to be >>>> received, which you then will judge by merit or by your whim, or you >>>> can flip a coin or say a prayer... (I've tried most of those myself in >>>> the past and haven't done too badly if I say so myself. :-) You're the >>>> boss now. I know you will do the right thing for this PEP. >>>> >>>> -- >>>> --Guido van Rossum (python.org/~guido) >>>> >>> >>> So, after some cool down - and the last rounds of discussion which >>> triggered some jiggery-pokery on Brian's part, I'm accepting PEP 3148 >>> "futures - execute computations asynchronously". I feel that it's a >>> good addition, and a good start for something bigger down the road. >>> >>> Brian - you'll need to provide someone such as Martin or Georg your >>> public key for ssh access into SVN, and you'll need developer access >>> to the bug tracker. >> >> Oh, don't worry. He's already been hacking happily on Windows stuff. >> >> >> -- >> Regards, >> Benjamin >> > > Wrong Brian - that's Brian Curtin, this is Brian Quinlan - I double > checked the committer's list (http://www.python.org/dev/committers). > > We now have two Brians. I say we name them PresentBrian and FutureBrian. My apologies, Brians! -- Regards, Benjamin From tim.peters at gmail.com Mon Jul 12 05:02:48 2010 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 11 Jul 2010 23:02:48 -0400 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: References: Message-ID: [Terry Reedy] > I had considered the possibility of option A for 2.7 and A & C for 3.2. But > see below. > > Since posting, I did an experiment with a 700 char paragraph of text (the > summary from the post) compared to an 'edited' version. I did the > comparision with and without the current heuristic. I did not notice much > time difference (a couple of seconds either way) and the edit list was > essentially the same. When the sequences contain M (in seq1) and N (in seq2) instances of a given element, the inner matching loop goes around M*N times processing them. Unless the element is identified as "junk" and/or "popular", in which case the outer loop finds the M instances (in seq1), one at a time, and the inner loop is skipped for each (because b2j does not contain the junk/popular element as a key, so the inner loop ends up iterating over the empty "nothing" list). Of course the speed difference is minor unless M*N is large. For example, when M and N are both on the order of thousands, M*N is on the order of millions. Such cases motivated the original "auto junk" heuristic (large "before" and "after" text files with thousands of repetitions of boilerplate lines - I suspect at least some of this input was computer-generated). > The heuristic lowered the reported match ratio from .96 to .88, which > would be bad when one wanted the unaltered value. Except that the ratio is defined as being twice the number of matches divided by the sum of the lengths of the sequences, so has no interpretation independent of exactly how matching was performed. Junk can have a very large effect on this. IOW, the ratio is not a property of the sequences, it emerges from all the details of how matches were found. BTW, it's not clear whether ratio() computes a _useful_ value in the presence of junk, however that may be defined. Perhaps it would be better to, e.g., not use the lengths of the matches or of the input sequences directly, but to adjust all of them by pretending junk elements didn't exist. Or not - it's unclear. For example, if the sequences are ['j1', 'a', 'j2'] and ['j3', 'a', 'j4'] where j1, j2, j3, and j4 are considered to be junk, what _is_ a useful ratio? As is, it returns 0.333 (1 match, so returns 2.0 * 1 / (3 + 3) = 1/3). But if we didn't count junk elements at all, the "correct" ratio would be 1.0 (2.0 * 1 match, out of 2 total non-junk elements in the input sequences). I suspect nobody cares ;-) > I do not know which, if any, chars other than 'e' were junked as that info > currently is not exposed. I propose below that it should be. An auto-junk-detection gimmick certainly should allow finding out what it decided junk was. > I intentionally did not list as options > > D. Keep the status quo that is buggy for certain uses. > > Good things often have more uses than the inventor intended or expected. > They should not be prevented. > > E. Completely remove the heuristic, which would restore 'buggy' performance > for other uses. > > One of the closed issues was E, rejected for that reason. > > --- > I also did not list one of my original ideas, but after the discussion it is > looking better to me. It is based on the following statement of the current > heuristic: > > "Disregard as junk common items that occur in more that 1% of the positions > in second sequence b, as long as b is long enough so that duplicates cannot > be called common." I'm not sure what "so that duplicates cannot be called common" means. Being common shouldn't, on its own, disqualify something from being junk. The essence of (a better notion of) junk probably has much more to do with "lumpiness" in the distribution of element frequencies. In the DNA example, I bet no one of {A, G, C, T} appears "way more" frequently than any other, so it's crazy to call any of them "junk". But, e.g., if we were trying to compare simple melodies, a "silence" token would probably be very frequent compared to any individual note. > Tim, I do not know if you remember why you choose 200 as the cutoff, I was afraid the auto-junk heuristic would end up doing silly things on short sequences, and 200 "isn't short" - LOL. Sheesh. > but the above assumes that the following in not just a coincidence: > > (2 > 199*.01) == True > (2 > 200*.01) == False > > In other words, 200 is the smallest length for b that prevents the junking > of duplicates. Well, the rule was also "> 1%", and floating arithmetic isn't used for that part ... it's here: if n >= 200 and len(indices) * 100 > n: where n is the length of the sequence. len(indices) * 100 > n is mathematically (not necessarily numerically) equivalent to len(indices) > n / 100, or, IOW, the thing appears in more than 1% of the positions. It certainly was the intent that nothing would be called junk unless it appeared at least twice, so the "n >= 200" clause ensures that 1% of n is at least 2. > F. Generalize the heuristic by replacing '1' with 'k', where k can be None > (no heuristic) or 1-99. If not None, replace 200 by 200/k to minimally avoid > junking of duplicates. If len(b) >= 200/k, then item counts should be > greater than (len(b)*k)//100, which need only be calculated once. Since cleverness already screwed people here, I'd rather be more straightforward: allow to specific the percentage as a float in [0.0, 100.0], and also allow to specify a minimum count directly. Like "call something junk if it appears in more than 0.231 of the positions and also appears at least 42 times". That would be a 2-tuple rather than a single number, with default (1.0, 2) to match current behavior. However, I'm wary of introducing a generalization in the absence of experience saying people would use it. Is this the right kind of parametrization? Is this even the right kind of way to go about auto-detecting junk? I know it worked great for the original use case that drove it, but I haven't seen anyone say they want a notion of auto-junk detection for other uses - just that they _don't_ want the wholly inappropriate current auto-junk detection in (some or all) of their uses. IOW, it's hard to generalize confidently from a sample of one :-( > Implementation: Add a new parameter named 'common' or 'threshold' or > whatever that defaults to 1. I'd call it "autojunk", cuz that's what it would do. Also a useful syntactic overlap with the name of the current "isjunk" argument. > After computing b2j without the heuristic, if > 'common' is not None, prune b2j as described above. > > My thinking here is that a user either knows or can easily find out the > length of b and the size of the intented or actual alphabet of b (the latter > is len(set(b)). So the user can conditionally call SequenceMatcher with > 'common' set to None or an int as appropriate, perhaps after some > experimentation. So the threshold is the only tuning parameter actually > needed, and it also allows the heuristic to be turned off. > > The reason I did not list this before is the problem with 2.7.1. F, unlike > option A, overtly changes the api, and some would object to that for 2.7 > even though is really is a bugfix. Shouldn't be a problem for a major release (like 2.7), but would be for a minor (bugfix) release (like 2.7.1). > However, option F will not not break code while the covert change of > option A could break code. So this may be the best option for a bad > situation. It is a minimal change that gives the user complete flexibility. Certainly an improvement, just unsure whether the flexibility provided will prove to be useful (beyond the ability to specify None to turn auto-junk off, which is the burning issue). > In other words, I see three options for 2.7.1+: > D. keep the current sometimes buggy behavior > A. covertly change the heuristic to mostly fix it but possibly break some > uses. > F. add a parameter, with a no-change default, that allows the user to select > a change if and when wanted. > > Another advantage of F is that then 2.7 and 3.2 would get the same change. A very good thing - if it's the right change ;-) > Other changes that apply regardless of the heuristic/api change: > > Update the code to use sets (newer than difflib) instead of dicts with > values set to 1. > > Directly expose the set of 'common' items as an additional attribute of > SequenceMatcher instances. Such instance attributes are currently > undocumented, so adding one can hardly be a problem. Add documention > thereof. Being able to see the effect of the heuristic when it is not turned > off might help people decide whether or not to use it, or how to tune the > threshold for smallish alphabets where 1 is too small. Wholly agreed. junkdict (after turning it into a set) should also be exposed - when someone passes in a fancy regexp matcher for the isjunk argument, they can be surprised at what their regexp matches. Being able to see the results can be helpful there too, for debugging. From basherwo at ncsu.edu Mon Jul 12 03:43:00 2010 From: basherwo at ncsu.edu (Bruce Sherwood) Date: Sun, 11 Jul 2010 19:43:00 -0600 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> Message-ID: On the notion that IDLE is fatally flawed and is driving away potential users of Python (to put the statements in their most extreme form): It seems that there are (at least) two very different communities people have in mind. I can appreciate that highly expert programmers may find IDLE insufficient for their needs, and it might even be the case that these people should be advised not to even try IDLE. But I want to stress as strongly as possible that for a very large community of nonexpert users of Python, IDLE has been extremely important and works very well for their purposes. Editing is pretty much like what they're already used to in word processors, which is certainly not the case with professional tools such as vim or Emacs. That said, yes, there are some significant deficiencies with the current IDLE (which is why there's a VIDLE, for instance). I'm very glad to hear from Martin that Guilherme is free to commit his important bug fixes and improvements. I'm afraid that neither Guilherme nor I understood the procedures and thought that we had to wait for others to act. Bruce Sherwood -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Mon Jul 12 05:30:27 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 11 Jul 2010 23:30:27 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <4C3A103A.1070504@v.loewis.de> References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> <4C3A103A.1070504@v.loewis.de> Message-ID: On 7/11/2010 2:40 PM, "Martin v. L?wis" wrote: Guido >>> There clearly are *some* folks who care enough about IDLE to submit >>> bug reports and fixes. How about we empower these people by giving at >>> least one of them commit privileges? IDLE development has often been >>> done by people who aren't otherwise contributing to the core, and we >>> surely should trust those folks with commit privileges. It appears Guiherme Polo already has commit privileges and just needs help exercising them, which I have offered, along with Martin's encouragement. Multiple IDLE maintainers would be even better. Mark Lawrence >> Can I take a really big liberty and volunteer Terry Reedy for the job. Thank you for the nomination. > If Terry would volunteer himself, he'd get commit access in no time. What I specifically want right now is Commit Authorization Privilege, especially for IDLE, but in general would be fine. I am thinking about working working one or more 'beginners' who are 'shy' about acting independently, or who are not yet authorized to do so, and who would like help getting their feet 'wet', so to speak. I think I would enjoy this sort of pair development. In regard to IDLE, who, short of Guido, is in charge? Is there a design doc? It appears that several people have ideas in their heads, such as 'keep it simple'. Abstractly, I agree with that, but who decides what is simple, to the point of vetoing something as 'too complex'? -- Terry Jan Reedy From ronaldoussoren at mac.com Mon Jul 12 07:50:47 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Mon, 12 Jul 2010 07:50:47 +0200 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <4C39C626.40400@codebykevin.com> References: <7EF8C99A-4163-461A-922B-1FF3003D89E2@mac.com> <4C39C626.40400@codebykevin.com> Message-ID: On 11 Jul, 2010, at 15:24, Kevin Walzer wrote: > On 7/11/10 5:03 AM, Ronald Oussoren wrote: >> > >> >> The OSX issues al seem to be related to general Tk or Tkinter bugs >> on OSX. I know to little about Tk and Tkinter to seriously work on >> those. > > Ronald, > > How about http://bugs.python.org/issue6075? I first submitted that patch in May '09, and updated it in October '09 on request from you, and it's just sat there since... If I read the patch correctly it replaces the existing 8.4 support by support for 8.5. That would not be acceptable because it would result in a non-functional version of IDLE for anyone that hasn't installed a custom copy of Tk. The "checkForAppKit" function doesn't work with the system version of Tk that Python currently links to. Does the patch work with the system version of Tk 8.4 on OSX? Ronald > > --Kevin > > > -- > Kevin Walzer > Code by Kevin > http://www.codebykevin.com -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From ronaldoussoren at mac.com Mon Jul 12 07:52:48 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Mon, 12 Jul 2010 07:52:48 +0200 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <9773.1278869716@parc.com> References: <9773.1278869716@parc.com> Message-ID: <612FF368-08DD-4A7B-AD20-5C543E418259@mac.com> On 11 Jul, 2010, at 19:35, Bill Janssen wrote: > Tal Einat wrote: > >> Although several people say that they think having IDLE in the stdlib >> is important, the fact is that IDLE is considered quite unimportant by >> most of the Python community. Having IDLE in the stdlib may be >> convenient for a few people, but most never use it and don't care >> about it. I think that in its current state, IDLE may still be helpful >> for learning Python, but it is more likely to drive away users who run >> into its various quirks and problems. And for experienced Python >> developers, very few actually use IDLE, and those who do could easily >> install it if it weren't part of the stdlib. > > I agree with you on this, Tal. On OS X, this is particularly > aggravating, as the Apple-supplied Python doesn't seem to include a > working version, and installing MacPython leads to other problems (see, > for instance, the thread at > http://groups.google.com/group/nltk-users/browse_thread/thread/e14b647243ca5b66). Apple doesn't ship IDLE.app, but does ship the rest of the code. It should be fairly easy to create a small IDLE.app using the python.org source tree that uses /usr/bin/python. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From martin at v.loewis.de Mon Jul 12 08:05:06 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 12 Jul 2010 08:05:06 +0200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> <4C3A103A.1070504@v.loewis.de> Message-ID: <4C3AB092.8030902@v.loewis.de> > What I specifically want right now is Commit Authorization Privilege, > especially for IDLE, Not sure who could grant that, but as far as I can: you have it. Regards, Martin From apt.shansen at gmail.com Mon Jul 12 08:24:38 2010 From: apt.shansen at gmail.com (Stephen Hansen) Date: Sun, 11 Jul 2010 23:24:38 -0700 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <4C3A5983.3010002@holdenweb.com> References: <4C3A5983.3010002@holdenweb.com> Message-ID: On Sun, Jul 11, 2010 at 4:53 PM, Steve Holden wrote: > Stephen Hansen wrote: > > On Sat, Jul 10, 2010 at 9:23 PM, Guilherme Polo > > wrote: > > > > By "never had a problem" do you mean using some of the latest > versions > > ? Here, running "idle" from a mac terminal and trying to type: print > > "hi" crashes when entering the quotation mark. > > > > > > Huh? Works fine for me. Python 2.6.1, OSX 10.6.3, intel. > > > One of the good things about the python-dev community is its commitment > to test-driven development. If you are prepared to define "fine" as > 'successfully runs \'print "hello"\'' then I guess we should be > perfectly happy about IDLE. > Er, how hostile. My point is, the poster made an assertion-- that you couldn't do the simple act as launching idle from a command line, and printing Hi. Maybe they can't, I have no idea. I know I can. I know that I have also opened random python files, saved them, and ran them with IDLE. I don't use IDLE beyond that though: I live in TextMate on my mac. My point was not, "IDLE is perfect". My point was, "You've claimed you can't even print out a word in IDLE, so its utterly and completely non-functional" -- and that assertion surprises me and I challenge. I don't define IDLE as "fine", because I'm not qualified to speak to its larger aspects-- as I only rarely use it. But the level of utter brokenness that the poster I was replying to spoke of, I've never seen. Across multiple versions of Python, IDLE, and OSX. > From the lurking crowd-- Please don't consider removing IDLE until there > > is a compelling replacement ready. It's better to have a limited IDE > > that works everywhere (even if in a limited fashion-- people are free to > > try out one of the many excellent full-featured Python IDE's out there > > after they advance to that point) then not. > > > 1: I refuse to see why we need a "compelling replacement" for a piece of > software whose performance might be actively deterring people from > taking up the language. ["Have you thought about Python?" "Yeah, but I > tried it {meaning "I downloaded some random Python release and tried > IDLE, which by modern standards appears completely lame"} and it > sucked". If this is our standard for "compelling" then it appears the > command-line interpreter is the competition. > The claim that IDLE is "actively deterring" people from taking up Python is in my opinion unsupported. I know a lot of people who have and do use it, and I am personally (in my own experience) unaware of anyone who is actively deterred from using Python because of it. Therefore, I see no negative, and only a positive of IDLE's presence-- and so I'd want a compelling replacement available before that positive was wiped out. Perhaps your experience is different. So be it: but -- uh, really, Hostile. I was just sharing my own experience with using and talking to people who use IDLE. I've found it -- on the mac, but on other platforms as well -- an adequate but limited sort of IDE. I've found more issues with it with the people I know who use windows then mac (in particular, details of when the subprocess runs). But my comment was simply: it has constantly worked for me in the limited use I make of it, and I have a positive experience with the people I know that have used it. If your experience is different, that's fine. Perhaps your experience is more broad, more compelling, and representative of more people. But I, personally, would consider it a significant loss if IDLE went the way of the dodo or a third-party module. -- Stephen -------------- next part -------------- An HTML attachment was scrubbed... URL: From asmodai at in-nomine.org Mon Jul 12 10:06:38 2010 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Mon, 12 Jul 2010 10:06:38 +0200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A5983.3010002@holdenweb.com> Message-ID: <20100712080638.GM70600@nexus.in-nomine.org> -On [20100712 08:26], Stephen Hansen (apt.shansen at gmail.com) wrote: >But I, personally, would consider it a significant loss if IDLE went the way of >the dodo or a third-party module.? Why would it be a significant loss if it went the way of a third party module? Clearly right now it's not being maintained as well as the rest of Python. Maybe that's a clear indicator that it's better maintained externally instead of in the main tree. And for what it is worth, I personally never used it beyond one time looking at it in disgust and neither do I know Pythonistas around me that use it. Bpython and ipython do get installed a lot though, even on Windows. And all these people, no matter their proficiency in programming, use an editor or IDE of some sort, but not IDLE. So I would not mourn to see IDLE get moved out of the main repository as I do not see the added value or benefit, not even for training (and since you're going to set up a training environment anyway, it can only be described as lazy if you are adamant on having it included in the base distribution). Just my EUR 0,02. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Born from the Dark, in the black Cloak of Night... From martin at v.loewis.de Mon Jul 12 10:18:11 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 12 Jul 2010 10:18:11 +0200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <20100712080638.GM70600@nexus.in-nomine.org> References: <4C3A5983.3010002@holdenweb.com> <20100712080638.GM70600@nexus.in-nomine.org> Message-ID: <4C3ACFC3.8070601@v.loewis.de> Am 12.07.2010 10:06, schrieb Jeroen Ruigrok van der Werven: > -On [20100712 08:26], Stephen Hansen (apt.shansen at gmail.com) wrote: >> But I, personally, would consider it a significant loss if IDLE went the way of >> the dodo or a third-party module. > > Why would it be a significant loss if it went the way of a third party > module? Clearly right now it's not being maintained as well as the rest of > Python. I start disagreeing here already. It *is* as well maintained as the rest of Python - at least, there are many other parts of Python that get the same attention (or less). IDLE has been working for years, in a certain feature set. It just doesn't need much "maintenance", except for updates when the world around it is changing. > Maybe that's a clear indicator that it's better maintained > externally instead of in the main tree. That's an unfounded theory, at best. Two experiments to pro > So I would not mourn to see IDLE get moved out of the main repository as I > do not see the added value or benefit Now that's a different issue. You are not using it, fine. Does that mean it should be removed? If we removed all modules that somebody is not using, the standard library would be empty. Regards, Martin From kbk at shore.net Mon Jul 12 11:20:49 2010 From: kbk at shore.net (Kurt B. Kaiser) Date: Mon, 12 Jul 2010 05:20:49 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: (Bruce Sherwood's message of "Sun, 11 Jul 2010 19:43:00 -0600") References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> Message-ID: <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> On Sun, Jul 11 2010, Bruce Sherwood wrote: > On the notion that IDLE is fatally flawed and is driving away > potential users of Python (to put the statements in their most extreme > form): > > It seems that there are (at least) two very different communities > people have in mind. I can appreciate that highly expert programmers > may find IDLE insufficient for their needs, and it might even be the > case that these people should be advised not to even try IDLE. But I > want to stress as strongly as possible that for a very large community > of nonexpert users of Python, IDLE has been extremely important and > works very well for their purposes. Editing is pretty much like what > they're already used to in word processors, which is certainly not the > case with professional tools such as vim or Emacs. Particularly on Windows, there is an expectation that a contemporary programming language comes with a GUI. Using the Windows Python shell isn't very satisfactory, largely because of the poor file handling capability in that environment. IDLE provides a simple solution, suitable for beginners as well as experts. It provides one-keystroke save/run in a fresh environment not associated with the GUI. I have run IDLE inside IDLE (though I'm not positive that will work right now). How many IDEs can do that? I've tried to channel Guido, and my general approach has been to provide as much functionality as needed, but to hide the more advanced features to avoid clutter. When the 'experts' need them, they will find them. I'm mystified about the comments that the GUI is ugly. It is minimal. On XP, it looks exactly like an XP window with a simple menubar. Those who haven't looked at it for awhile may not be aware of the recent advances made by Tk in native look and feel. What is ugly? And IDLE is cross-platform. There are perhaps deficiencies on the Mac, but Ron has made good progress. I haven't been able to come up with a Mac yet, so I can't really comment. IDLE works on 2.7 and 3.x. Running IDLE on the trunk is a decent test of Python and tkinter during development. It seems that people who do Scheme write Scheme implementations, while people who do Python write IDEs :-) Usually, they keep adding features and buttons until it looks like Eclipse. That's fine. But why turn IDLE into another example of that style and diminish diversity in doing so? On a netbook, screen space is at a premium, and a simple interface has advantages. There are many IDEs listed on the wiki if people are looking for the more complex style, but I'd suggest that they aren't appropriate for beginners. And by beginners, I include elementary school students. IDLE works with "extensions". Plug-ins, if you will. The Option dialog needs enhancement to support extension selection, and I believe there is a patch to do that. More extensions would be a good thing. The .idlerc directory determines IDLE's configuration, including the help source urls available on the Help menu. By downloading .idlerc or via indirection, a student environment can be set up including the day's lesson and which extensions are activated. Call tips and method name completion can be "hidden" by increasing the time delay before they pop up. > That said, yes, there are some significant deficiencies with the > current IDLE (which is why there's a VIDLE, for instance). I'm very > glad to hear from Martin that Guilherme is free to commit his > important bug fixes and improvements. I'm afraid that neither > Guilherme nor I understood the procedures and thought that we had to > wait for others to act. Tal has made a couple of comments about not being able to run multiple IDLEs at the same time, and he mentions http://bugs.python.org/issue1529142 This bug has been closed for over a year, and you can run as many IDLEs as you like. You can run 2.7 and 3.x at the same time. I would have expected Tal to know that. Also, the current right click edit action on Windows is to only open an edit window; no shell. And it uses the subprocess! So, some of the comments on this thread are not up to date. The reason that bug languished for two years was because first, it was a bit of a hack, and second, Windows was problematic in that it reused sockets and often left zombie subprocesses behind which couldn't be killed except with the task manager. This causes real problems with students - they lose confidence in the tool. Scherer and Weeble put together a patch using ephemeral ports which nailed the problem, and I checked it in right away and forward/backported it. As I recollect, much of what Scherer did in VIDLE related to running multiple IDLE copies. For that reason, the VIDLE changes have to be evaluated carefully to determine what has already been incorporated. I believe I added a couple of his smaller changes, also. Bruce, speaking of Dave Scherer, we've been trying to get a Python Contributor Agreement from him. Is that something you can help us with? -- KBK From stefan at bytereef.org Mon Jul 12 11:31:51 2010 From: stefan at bytereef.org (Stefan Krah) Date: Mon, 12 Jul 2010 11:31:51 +0200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> <20100711213801.4d391849@pitrou.net> Message-ID: <20100712093151.GA17171@yoda.bytereef.org> Jesse Noller wrote: > On Sun, Jul 11, 2010 at 8:22 PM, geremy condra wrote: > >>> (This seems to me like an area where a judicious application of PSF funds might help; if every > >>> single bug were actively triaged and responded to, even if it weren't reviewed, and patch > >>> contributors were directed to take specific steps to elicit a response or a review, the fact that > >>> patch reviews take a while might not be so bad.) > >> > >> The operative word being "judicious". It is not obvious who should get > >> funded, and for what tasks. > >> Some specific issues (like email in 3.x) are large enough that they can > >> be the sole focus of a fund grant. But I'm not sure triaging can apply. > > > > I'm mulling over starting a monthly triage sprint under the auspices of > > Jesse Noeller's PSF sponsored sprints in the hopes of making this a > > little more fun. I'd appreciate comments on the idea. [responding to Geremy] I'm with Georg on this. If triaging needs a monetary incentive because it is tedious work, so does committing. A lot of the abandoned issues aren't very glamorous either. Also, from the work that Mark Lawrence has been doing on the tracker in the past few weeks, it's apparent that a dedicated person can achieve a lot without pay. Due to his tracker reshuffling, many issues got closed, several bug reporters responded after years, etc. Thanks, Mark! Stefan Krah From kbk at shore.net Mon Jul 12 11:46:59 2010 From: kbk at shore.net (Kurt B. Kaiser) Date: Mon, 12 Jul 2010 05:46:59 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <20100712080638.GM70600@nexus.in-nomine.org> (Jeroen Ruigrok van der Werven's message of "Mon, 12 Jul 2010 10:06:38 +0200") References: <4C3A5983.3010002@holdenweb.com> <20100712080638.GM70600@nexus.in-nomine.org> Message-ID: <87oced2fsc.fsf@hydra.hampton.thirdcreek.com> On Mon, Jul 12 2010, Jeroen Ruigrok van der Werven wrote: > -On [20100712 08:26], Stephen Hansen (apt.shansen at gmail.com) wrote: >>But I, personally, would consider it a significant loss if IDLE went >>the way of the dodo or a third-party module.? > > Why would it be a significant loss if it went the way of a third party > module? Clearly right now it's not being maintained as well as the > rest of Python. Maybe that's a clear indicator that it's better > maintained externally instead of in the main tree. > > And for what it is worth, I personally never used it beyond one time > looking at it in disgust Apparently you haven't looked recently. If you'd describe specifically what is disgusting on the current version, perhaps we can do something about it. > and neither do I know Pythonistas around me that use it. Bpython and > ipython do get installed a lot though, even on Windows. And all these > people, no matter their proficiency in programming, use an editor or > IDE of some sort, but not IDLE. Well, I use it :-) I used to use emacs to develop IDLE, but as IDLE became more capable, I stopped using emacs (except to fix IDLE when I break it completely :) When I find something I really miss, I add it. But I don't just add features to check off a list. Search on the status bar and jump to definition are next. > > So I would not mourn to see IDLE get moved out of the main repository > as I do not see the added value or benefit, not even for training (and > since you're going to set up a training environment anyway, it can > only be described as lazy if you are adamant on having it included in > the base distribution). On Windows, IDLE opens when you right click / edit a .py. Very useful. On linux, the packagers generally split IDLE off into a separate package so Python can be installed without Tcl/Tk. That doesn't mean it should be removed from the tarball; their package build tools build several packages from the single tarball at the same time. Second guessing them by having two tarballs just increases the work for everyone. This could be done on Windows, but then you wouldn't have a GUI to use right after running the Python installer. Minimal installations are not so important on Windows. -- KBK From solipsis at pitrou.net Mon Jul 12 12:10:40 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 12 Jul 2010 12:10:40 +0200 Subject: [Python-Dev] Removing IDLE from the standard library References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> Message-ID: <20100712121040.41f16654@pitrou.net> On Mon, 12 Jul 2010 05:20:49 -0400 "Kurt B. Kaiser" wrote: > > I'm mystified about the comments that the GUI is ugly. It is minimal. > On XP, it looks exactly like an XP window with a simple menubar. Those > who haven't looked at it for awhile may not be aware of the recent > advances made by Tk in native look and feel. What is ugly? Ok, I've just tried IDLE (on py3k) for the first time in years. Under Linux, the look is ugly and outdated; it uses some kind of Motif-like widgets. Usability is inconsistent with the rest of the desktop, and looks generally subpar (for example, you have to click on a menu to open it; the file open dialog is antiquated and doesn't allow me to use keyboard shortcuts). The editor looks decent, though there doesn't seem to be a simple (e.g. Shift+Tab) way of unindenting a block of code. Regards Antoine. From solipsis at pitrou.net Mon Jul 12 12:21:08 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 12 Jul 2010 12:21:08 +0200 Subject: [Python-Dev] Removing IDLE from the standard library References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <20100712121040.41f16654@pitrou.net> Message-ID: <20100712122108.62fa0f3c@pitrou.net> After a few keystrokes in the interactive interpreter, I got the following traceback: Traceback (most recent call last): File "Lib/idlelib/idle.py", line 11, in idlelib.PyShell.main() File "/home/antoine/py3k/__svn__/Lib/idlelib/PyShell.py", line 1420, in main root.mainloop() File "/home/antoine/py3k/__svn__/Lib/tkinter/__init__.py", line 1009, in mainloop self.tk.mainloop(n) UnicodeDecodeError: 'utf8' codec can't decode byte 0xc0 in position 0: invalid start byte The keystrokes were ['d', 'i', Ctrl+Space], which briefly displayed the completion popup before crashing. Regards Antoine. From debatem1 at gmail.com Mon Jul 12 12:37:05 2010 From: debatem1 at gmail.com (geremy condra) Date: Mon, 12 Jul 2010 06:37:05 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <20100712093151.GA17171@yoda.bytereef.org> References: <4C3A0F85.1090404@v.loewis.de> <20100711213801.4d391849@pitrou.net> <20100712093151.GA17171@yoda.bytereef.org> Message-ID: On Mon, Jul 12, 2010 at 5:31 AM, Stefan Krah wrote: > Jesse Noller wrote: >> On Sun, Jul 11, 2010 at 8:22 PM, geremy condra wrote: >> >>> (This seems to me like an area where a judicious application of PSF funds might help; if every >> >>> single bug were actively triaged and responded to, even if it weren't reviewed, and patch >> >>> contributors were directed to take specific steps to elicit a response or a review, the fact that >> >>> patch reviews take a while might not be so bad.) >> >> >> >> The operative word being "judicious". It is not obvious who should get >> >> funded, and for what tasks. >> >> Some specific issues (like email in 3.x) are large enough that they can >> >> be the sole focus of a fund grant. But I'm not sure triaging can apply. >> > >> > I'm mulling over starting a monthly triage sprint under the auspices of >> > Jesse Noeller's PSF sponsored sprints in the hopes of making this a >> > little more fun. I'd appreciate comments on the idea. > > [responding to Geremy] > > I'm with Georg on this. If triaging needs a monetary incentive because it > is tedious work, so does committing. A lot of the abandoned issues aren't > very glamorous either. I'm not sure what you mean by "monetary incentive". I was considering handing a t-shirt or beer token to the most productive sprinters, but that's about the limit of it, and I suspect that would come out of my pocket. I'd also emphasize that I am exactly as far as I stated on this: I'm mulling it over and asking for feedback. If it turns out that there are other things that python-dev feels are more necessary but similarly unglamorous, then I'll think about doing that instead. > Also, from the work that Mark Lawrence has been doing on the tracker in > the past few weeks, it's apparent that a dedicated person can achieve a > lot without pay. Indeed, although I'm again unsure what pay has to do with this. > Due to his tracker reshuffling, many issues got closed, several bug > reporters responded after years, etc. Thanks, Mark! +1 Geremy Condra From ezio.melotti at gmail.com Mon Jul 12 12:39:15 2010 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Mon, 12 Jul 2010 13:39:15 +0300 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <4C3A5A23.2070903@v.loewis.de> References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> <4C3A103A.1070504@v.loewis.de> <4C3A5A23.2070903@v.loewis.de> Message-ID: <4C3AF0D3.5090303@gmail.com> On 12/07/2010 2.56, "Martin v. L?wis" wrote: >> As for assigning bugs, I've been told to use the maintainer.rst list, so >> either the list is wrong, or I've had finger problems. If it's the >> latter I again say sorry. > > I see. What copy have you been using specifically? I think I need to > remove myself from these lists. > > Regards, > Martin Indeed it's not clear if the names that appear in the maintainers.rst list are supposed to be added only to the nosy list or if it's possible to add them to the "assigned to" field too. A way to avoid confusion is to mark the names that should be added to the "Assigned to" field with a '*', and add the others to the nosy list only. E.g.: unicodedata loewis, lemburg, ezio.melotti* would mean "You can add loewis and lemburg to the nosy list and assign the issue to ezio.melotti". Otherwise we can just decide that those names should just be added to the nosy list and let them assign the issue to themselves if they want to fix it. Best Regards, Ezio Melotti From taleinat at gmail.com Mon Jul 12 12:52:31 2010 From: taleinat at gmail.com (Tal Einat) Date: Mon, 12 Jul 2010 13:52:31 +0300 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> Message-ID: Hi Kurt, I'm glad you've joined this discussion. My point is that whatever the reason, for the past five years (at least) nearly every issue related to IDLE has taken years to be resolved, and many have still not been resolved. As a result the current state of IDLE is quite poor. To be perfectly clear, I'm am not blaming anyone for this. I just think this is something the development community should be more aware of and decide how to deal with. On Mon, Jul 12, 2010 at 12:20 PM, Kurt B. Kaiser wrote: > Particularly on Windows, there is an expectation that a contemporary > programming language comes with a GUI. ?Using the Windows Python shell > isn't very satisfactory, largely because of the poor file handling > capability in that environment. ?IDLE provides a simple solution, > suitable for beginners as well as experts. ?It provides one-keystroke > save/run in a fresh environment not associated with the GUI. ?I have run > IDLE inside IDLE (though I'm not positive that will work right now). > How many IDEs can do that? [snip] > On a netbook, screen space is at a premium, and a simple interface has > advantages. ?There are many IDEs listed on the wiki if people are > looking for the more complex style, but I'd suggest that they aren't > appropriate for beginners. ?And by beginners, I include elementary > school students. I agree that IDLE is great specifically thanks to its simplicity, and should be kept simple. I think something like IDLE can be great for teaching and learning Python (and other uses as well) and has a place in the stdlib. I'm simply not sure if IDLE in its current state is fit for this purpose. > I'm mystified about the comments that the GUI is ugly. It is minimal. > On XP, it looks exactly like an XP window with a simple menubar. Those > who haven't looked at it for awhile may not be aware of the recent > advances made by Tk in native look and feel. What is ugly? IDLE may be somewhat close to looking like a native application, but it is not quite there. Some examples: * Users expect to be able to copy/paste using the mouse via the right-click context menu, which is impossible in IDLE (in both shell and editor windows). * The selection jumping to the end of the line unexpectedly doesn't happen in any other GUI text editor. * The config dialog looks archaic. As for the native look&feel, this has still not been integrated into IDLE, though it seems now it will happen soon. This has already been discussed here. > IDLE works with "extensions". ?Plug-ins, if you will. ?The Option dialog > needs enhancement to support extension selection, and I believe there is > a patch to do that. ?More extensions would be a good thing. IDLE extensions are a pain to work with and badly supported. Installing an extension requires manually placing it in the idlelib directory and manually adding its config options to a certain text file in that directory. In practice nobody ever takes the effort to try installing an IDLE extension, such as my SearchBar extension. There has been discussion on improving this situation, but again this never actually happened. I tried to move this along with a patch which adds an extension config dialog, which has been sitting in the issue tracker ignored for two years. > Tal has made a couple of comments about not being able to run multiple > IDLEs at the same time, and he mentions > > http://bugs.python.org/issue1529142 > > This bug has been closed for over a year, and you can run as many IDLEs > as you like. ?You can run 2.7 and 3.x at the same time. ?I would have > expected Tal to know that. I clearly mentioned that as an example of a fix for a major usability issue which took three years to resolve after a workaround was first posted. It is worth noting that most people still don't use the versions of Python (and IDLE) where this fix has been integrated. > The reason that bug languished for two years was because first, it was a > bit of a hack, and second, Windows was problematic in that it reused > sockets and often left zombie subprocesses behind which couldn't be > killed except with the task manager. ?This causes real problems with > students - they lose confidence in the tool. > > Scherer and Weeble put together a patch using ephemeral ports which > nailed the problem, and I checked it in right away and > forward/backported it. True, but it took three whole years for this to happen. And this is for a glaring, annoying and limiting issue that nearly every user of IDLE has encountered. From taleinat at gmail.com Mon Jul 12 13:01:54 2010 From: taleinat at gmail.com (Tal Einat) Date: Mon, 12 Jul 2010 14:01:54 +0300 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <4C3A48AE.9010707@v.loewis.de> References: <217B30F6-06FE-4237-90C2-540CD6BB6A28@twistedmatrix.com> <4C3A48AE.9010707@v.loewis.de> Message-ID: On Mon, Jul 12, 2010 at 1:41 AM, "Martin v. L?wis" wrote: >> My point is that I don't think I am exaggerating IDLE's flaws. I'm not >> saying that it is no longer usable or useful, but I am saying that its >> current state is not "okay". > > So can you produce a list of patches that you think can be accepted as-is? That's not a fair question! There have been several such patches, but most will likely need revision since they have been sitting around untouched for so long. And there would have been many more patches if the existing ones would have been addressed. Getting a few current patches accepted is not the reason I posted here. - Tal Einat From taleinat at gmail.com Mon Jul 12 13:07:05 2010 From: taleinat at gmail.com (Tal Einat) Date: Mon, 12 Jul 2010 14:07:05 +0300 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <4C3A493E.8090308@v.loewis.de> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A493E.8090308@v.loewis.de> Message-ID: On Mon, Jul 12, 2010 at 1:44 AM, "Martin v. L?wis" wrote: >> FWIW this is why I started IDLE-Spoon (well, continued Noam Raphael's >> project of the same name, in a sense). The idea was to have a fork of >> IDLE with new features which need to be tried out by "beta testers" to >> iron out all of the glitches before making it into the main version, >> like IDLE-fork back in the beginning of the decade. However I have >> utterly failed in promoting this project and getting "beta testers" on >> board, at least partially due to the lack of interest from the >> community (and admittedly my lack of PR skills). > > I think such a thing must inherently fail - precisely for these reasons. > > It's much better to release proposed new features along with Python, > and wait for feedback. Users won't start trying things out until after > the release. This is a general problem, and lead Barry Warsaw to believe > that "release candidates" are an utter waste of time. That's debatable, and I disagree. IDLE-fork was a great success, for example. If IDLE actually had many users today, certainly some of them would be interested in trying out a version with usability fixes and improvements. Waiting for a new release of Python can take over a year. Furthermore, backwards compatibility issues and support by third party libraries can delay migration to a newer version of Python even further. - Tal Einat From nyamatongwe at gmail.com Mon Jul 12 13:14:08 2010 From: nyamatongwe at gmail.com (Neil Hodgson) Date: Mon, 12 Jul 2010 21:14:08 +1000 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> Message-ID: Kurt B. Kaiser: > I'm mystified about the comments that the GUI is ugly. ?It is minimal. > On XP, it looks exactly like an XP window with a simple menubar. ?Those > who haven't looked at it for awhile may not be aware of the recent > advances made by Tk in native look and feel. ?What is ugly? While Tk has improved at emulating native appearance, there are still many differences. On the main editing screen of IDLE, the most noticeable issue is that there is no horizontal scroll bar even though the text will move left when you move the caret beyond the rightmost visible character. The scrollbar and status bar have an appearance that looks to be from Windows 2000, not Windows XP and there is no resizing gripper on the right side of the status bar. The tear off menus are ugly as well as being non-standard on all three major platforms. Use the "Configure IDLE..." and an "idle" dialog appears that also looks to be from Windows 2000. I know Tk can do better than this as Git Gui (the Tk (8.5.8) program I use most often) at least shows XP themed buttons, scrollbars and other controls. However, the "idle" dialog (as well as Git Gui) shows the largest remaining problem for Tk user interfaces: keyboard navigation. When the "idle" dialog opens, try doing anything with the keyboard. Chances are nothing will happen. If you press Tab 16 times (yes, 16!) a focus rectangle will finally show on the "Bold" check box. Another Tab takes you to the "Indentation Width" slider. After that you don't see the focus until it wraps around to "Bold" again. The Enter key doesn't trigger OK and the Escape key doesn't let you escape. The Find and Replace dialogs are better as focus works as do Enter and Escape but none of the buttons have mnemonics. This may all sound like picking nits but details and consistency are important in user interfaces and this is just looking at the most easily discovered problems. Neil From stefan at bytereef.org Mon Jul 12 13:17:07 2010 From: stefan at bytereef.org (Stefan Krah) Date: Mon, 12 Jul 2010 13:17:07 +0200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> <20100711213801.4d391849@pitrou.net> <20100712093151.GA17171@yoda.bytereef.org> Message-ID: <20100712111707.GA19594@yoda.bytereef.org> geremy condra wrote: > On Mon, Jul 12, 2010 at 5:31 AM, Stefan Krah wrote: > > Jesse Noller wrote: > >> On Sun, Jul 11, 2010 at 8:22 PM, geremy condra wrote: > >> >>> (This seems to me like an area where a judicious application of PSF funds might help; if every > >> >>> single bug were actively triaged and responded to, even if it weren't reviewed, and patch > >> >>> contributors were directed to take specific steps to elicit a response or a review, the fact that > >> >>> patch reviews take a while might not be so bad.) > >> >> > >> >> The operative word being "judicious". It is not obvious who should get > >> >> funded, and for what tasks. > >> >> Some specific issues (like email in 3.x) are large enough that they can > >> >> be the sole focus of a fund grant. But I'm not sure triaging can apply. > >> > > >> > I'm mulling over starting a monthly triage sprint under the auspices of > >> > Jesse Noeller's PSF sponsored sprints in the hopes of making this a > >> > little more fun. I'd appreciate comments on the idea. > > > > [responding to Geremy] > > > > I'm with Georg on this. If triaging needs a monetary incentive because it > > is tedious work, so does committing. A lot of the abandoned issues aren't > > very glamorous either. > > I'm not sure what you mean by "monetary incentive". I was considering > handing a t-shirt or beer token to the most productive sprinters, but > that's about the limit of it, and I suspect that would come out of my > pocket. Sorry for misinterpreting your intentions. I was reading this in the context of "judicious application of PSF funds". What you are describing is of course quite judicious. To me, "PSF sponsored" sounded like there would be a lot more money involved. Stefan Krah From debatem1 at gmail.com Mon Jul 12 13:31:08 2010 From: debatem1 at gmail.com (geremy condra) Date: Mon, 12 Jul 2010 07:31:08 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <20100712111707.GA19594@yoda.bytereef.org> References: <4C3A0F85.1090404@v.loewis.de> <20100711213801.4d391849@pitrou.net> <20100712093151.GA17171@yoda.bytereef.org> <20100712111707.GA19594@yoda.bytereef.org> Message-ID: On Mon, Jul 12, 2010 at 7:17 AM, Stefan Krah wrote: > geremy condra wrote: >> On Mon, Jul 12, 2010 at 5:31 AM, Stefan Krah wrote: >> > Jesse Noller wrote: >> >> On Sun, Jul 11, 2010 at 8:22 PM, geremy condra wrote: >> >> >>> (This seems to me like an area where a judicious application of PSF funds might help; if every >> >> >>> single bug were actively triaged and responded to, even if it weren't reviewed, and patch >> >> >>> contributors were directed to take specific steps to elicit a response or a review, the fact that >> >> >>> patch reviews take a while might not be so bad.) >> >> >> >> >> >> The operative word being "judicious". It is not obvious who should get >> >> >> funded, and for what tasks. >> >> >> Some specific issues (like email in 3.x) are large enough that they can >> >> >> be the sole focus of a fund grant. But I'm not sure triaging can apply. >> >> > >> >> > I'm mulling over starting a monthly triage sprint under the auspices of >> >> > Jesse Noeller's PSF sponsored sprints in the hopes of making this a >> >> > little more fun. I'd appreciate comments on the idea. >> > >> > [responding to Geremy] >> > >> > I'm with Georg on this. If triaging needs a monetary incentive because it >> > is tedious work, so does committing. A lot of the abandoned issues aren't >> > very glamorous either. >> >> I'm not sure what you mean by "monetary incentive". I was considering >> handing a t-shirt or beer token to the most productive sprinters, but >> that's about the limit of it, and I suspect that would come out of my >> pocket. > > Sorry for misinterpreting your intentions. I was reading this in the context > of "judicious application of PSF funds". What you are describing is of course > quite judicious. > > To me, "PSF sponsored" sounded like there would be a lot more money involved. Nope, as long as I can keep the costs under $100 or so (I'm the worlds biggest cheapskate, shouldn't be a problem) I doubt I'll ask for anything except that people promote and attend. Geremy Condra From rdmurray at bitdance.com Mon Jul 12 13:42:16 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 12 Jul 2010 07:42:16 -0400 Subject: [Python-Dev] maintainers.rst enhancement (was: Removing IDLE from the standard library) In-Reply-To: <4C3AF0D3.5090303@gmail.com> References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> <4C3A103A.1070504@v.loewis.de> <4C3A5A23.2070903@v.loewis.de> <4C3AF0D3.5090303@gmail.com> Message-ID: <20100712114216.3032B21B0D5@kimball.webabinitio.net> On Mon, 12 Jul 2010 13:39:15 +0300, Ezio Melotti wrote: > On 12/07/2010 2.56, "Martin v. L=F6wis" wrote: > >> As for assigning bugs, I've been told to use the maintainer.rst list, so > >> either the list is wrong, or I've had finger problems. If it's the > >> latter I again say sorry. > > > > I see. What copy have you been using specifically? I think I need to > > remove myself from these lists. > > > > Regards, > > Martin > > Indeed it's not clear if the names that appear in the maintainers.rst > list are supposed to be added only to the nosy list or if it's > possible to add them to the "assigned to" field too. A way to avoid > confusion is to mark the names that should be added to the "Assigned > to" field with a '*', and add the others to the nosy list only. Actually it is clear. The text at the top of the file says the names are there "to be added to the nosy list". It does not mention assigning bugs. This was actually discussed at least briefly when the file was created. We probably should have included "*don't* assign tickets, nosy the maintainer and let them pick it up if they want to" in the intro text. > E.g.: > unicodedata loewis, lemburg, ezio.melotti* > would mean "You can add loewis and lemburg to the nosy list and assign > the issue to ezio.melotti". Otherwise we can just decide that those I like this suggestion, but obviously we need to let those developers who wish to do this "star" themselves. If there are no objections to this change to maintainers.rst, I'll start the ball rolling by marking myself for email, and adjusting the introductory text accordingly. -- R. David Murray www.bitdance.com From fuzzyman at voidspace.org.uk Mon Jul 12 13:45:11 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 12 Jul 2010 12:45:11 +0100 Subject: [Python-Dev] maintainers.rst enhancement In-Reply-To: <20100712114216.3032B21B0D5@kimball.webabinitio.net> References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> <4C3A103A.1070504@v.loewis.de> <4C3A5A23.2070903@v.loewis.de> <4C3AF0D3.5090303@gmail.com> <20100712114216.3032B21B0D5@kimball.webabinitio.net> Message-ID: <4C3B0047.2000203@voidspace.org.uk> On 12/07/2010 12:42, R. David Murray wrote: > [snip...] > >> E.g.: >> unicodedata loewis, lemburg, ezio.melotti* >> would mean "You can add loewis and lemburg to the nosy list and assign >> the issue to ezio.melotti". Otherwise we can just decide that those >> > I like this suggestion, but obviously we need to let those developers > who wish to do this "star" themselves. > > If there are no objections to this change to maintainers.rst, I'll > start the ball rolling by marking myself for email, and adjusting > the introductory text accordingly. > Sounds good to me. You can leave me starred for unittest. Michael > -- > R. David Murray www.bitdance.com > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From rdmurray at bitdance.com Mon Jul 12 13:49:29 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 12 Jul 2010 07:49:29 -0400 Subject: [Python-Dev] maintainers.rst enhancement (was: Removing IDLE from the standard library) In-Reply-To: References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> <4C3A103A.1070504@v.loewis.de> <4C3A5A23.2070903@v.loewis.de> Message-ID: <20100712114929.BD550217E77@kimball.webabinitio.net> On Mon, 12 Jul 2010 01:07:54 +0100, Mark Lawrence wrote: > I've been working from this:- > http://svn.python.org/view/*checkout*/python/branches/py3k/Misc/maintainers.rst > > It strikes me as being so sadly outdated that it's getting less than > useless, or I assume that it's the same old case of not enough > volunteers? Why did I bother. :) As someone else mentioned, it is relatively new. It was created partially in the hopes that the huge gaps in coverage would encourage people to step forward and volunteer to be maintainers, and this has indeed been happening slowly. For example, Alexander Belopolsky recently became a committer and picked up datetime, time, and pickle. Another issue with the file that may make it look a tad outdated is that I think it hasn't been clear to everyone that it is designed to hold *tracker ids* rather than *committer ids*. I noticed the other day that I had to translate from committer id to tracker id for someone (I forget who, and I didn't have time to update the file at the time). -- R. David Murray www.bitdance.com From rdmurray at bitdance.com Mon Jul 12 13:51:34 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 12 Jul 2010 07:51:34 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> <4C3A103A.1070504@v.loewis.de> Message-ID: <20100712115134.2E06C1AB4F7@kimball.webabinitio.net> On Mon, 12 Jul 2010 00:13:21 +0100, Mark Lawrence wrote: > On 11/07/2010 19:40, "Martin v. L=F6wis" wrote: > >>> There clearly are *some* folks who care enough about IDLE to submit > >>> bug reports and fixes. How about we empower these people by giving at > >>> least one of them commit privileges? IDLE development has often been > >>> done by people who aren't otherwise contributing to the core, and we > >>> surely should trust those folks with commit privileges. > >>> > >> > >> Can I take a really big liberty and volunteer Terry Reedy for the job. > > > > It doesn't work that way. You can't volunteer somebody else (*). > > > > If Terry would volunteer himself, he'd get commit access in no time. > > > > Regards, > > Martin > > > > (*) so when you assign bugs to me, it probably means that they get less > > attention, not more. > > Martin, > > Thanks for your response. > > IIRC Terry Reedy has already volunteered to do this, if I'm incorrect > I'll apologise right now to both of you. > > As for assigning bugs, I've been told to use the maintainer.rst list, so > either the list is wrong, or I've had finger problems. If it's the > latter I again say sorry. I suggested you use maintainers.rst to find people to add to the nosy list, not to assign bugs to. But I can understand your confusion about that, given the name of the file, and the lack of complete process documentation. -- R. David Murray www.bitdance.com From kbk at shore.net Mon Jul 12 14:12:10 2010 From: kbk at shore.net (Kurt B. Kaiser) Date: Mon, 12 Jul 2010 08:12:10 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <20100712121040.41f16654@pitrou.net> (Antoine Pitrou's message of "Mon, 12 Jul 2010 12:10:40 +0200") References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <20100712121040.41f16654@pitrou.net> Message-ID: <87fwzo3nmt.fsf@hydra.hampton.thirdcreek.com> On Mon, Jul 12 2010, Antoine Pitrou wrote: > On Mon, 12 Jul 2010 05:20:49 -0400 > "Kurt B. Kaiser" wrote: >> >> I'm mystified about the comments that the GUI is ugly. It is minimal. >> On XP, it looks exactly like an XP window with a simple menubar. Those >> who haven't looked at it for awhile may not be aware of the recent >> advances made by Tk in native look and feel. What is ugly? > > Ok, I've just tried IDLE (on py3k) for the first time in years. Under > Linux, the look is ugly and outdated; it uses some kind of Motif-like > widgets. That's because Linux isn't using Tk 8.5 yet. Debian defaults to Tk 8.5 in experimental since March. Most Linux is Tk 8.4. On Windows, our installer is using 8.5. Here's a screenshot from my eeePC, 1024 x 600: http://images.rant.ml1.net/idle.gif Quoting Tal: ============================================ > Perhaps, but the point is that these bugs remain. Certainly this isn't > because just you, out of the entire Python development community, know > little about Tk and Tkinter. > Using Tkinter is a major reason that maintaining and further > developing IDLE is difficult. For example, it took me many hours just > to get a working Tkinter scrolled frame widget, having had to write it > from scratch and struggle with the under-documented Canvas widget. > Another example is that integration of the new ttk (a.k.a. Tile) > widget set, which supports native look&feel on various platforms and > adds modern widgets, has still not been integrated despite being > available in Tk for years and despite considerable effort being > invested into it. ============================================= Tal, you've got some catching up to do, yourself. Tile went into Tk in 8.5, two years ago. Linux is behind, we don't control that, but our Windows installer and tkinter is cutting edge! Thanks, Martin! I don't know what version is running on the Mac: http://blog.markroseman.com/2007/12/tcltk-85-first.html I've got an ancient Python 2.5.1 running in Gnome/gNewSense using Tk 8.4. I think you guys are exaggerating. It's not /all/ that ugly, not by a long shot. It's just not native look and feel (yet). > Usability is inconsistent with the rest of the desktop, and > looks generally subpar (for example, you have to click on a menu to > open it; On Linux and Windows, f will drop the file menu. > the file open dialog is antiquated and doesn't allow me to use > keyboard shortcuts). Looks much better in 8.5 and the shortcuts work. http://images.rant.ml1.net/idle2.gif > The editor looks decent, though there doesn't seem to be a simple (e.g. > Shift+Tab) way of unindenting a block of code. Pull down the format menu. The dedent hotkey is Ctrl+[ This possibly could be changed to Shift+Tab by the user, but there may be conflicts with completion. -- KBK From solipsis at pitrou.net Mon Jul 12 14:22:00 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 12 Jul 2010 14:22:00 +0200 Subject: [Python-Dev] Removing IDLE from the standard library References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <20100712121040.41f16654@pitrou.net> <87fwzo3nmt.fsf@hydra.hampton.thirdcreek.com> Message-ID: <20100712142200.6b57ef8d@pitrou.net> On Mon, 12 Jul 2010 08:12:10 -0400 "Kurt B. Kaiser" wrote: > > > > Ok, I've just tried IDLE (on py3k) for the first time in years. Under > > Linux, the look is ugly and outdated; it uses some kind of Motif-like > > widgets. > > That's because Linux isn't using Tk 8.5 yet. Debian defaults to Tk 8.5 > in experimental since March. Most Linux is Tk 8.4. No, that's wrong. Tk here is 8.6: $ ldd build/lib.linux-x86_64-3.2/_tkinter.so [...] libtk8.6.so.0 => /usr/lib64/libtk8.6.so.0 (0x00007f4eb9259000) libtcl8.6.so.0 => /usr/lib64/libtcl8.6.so.0 (0x00007f4eb8f07000) [...] $ ./python -c "import tkinter; print(tkinter.TkVersion)" 8.6 > > the file open dialog is antiquated and doesn't allow me to use > > keyboard shortcuts). > > Looks much better in 8.5 and the shortcuts work. > > http://images.rant.ml1.net/idle2.gif Well, this is a Windows screenshot. I'm under Linux. From debatem1 at gmail.com Mon Jul 12 14:30:40 2010 From: debatem1 at gmail.com (geremy condra) Date: Mon, 12 Jul 2010 08:30:40 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <87fwzo3nmt.fsf@hydra.hampton.thirdcreek.com> References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <20100712121040.41f16654@pitrou.net> <87fwzo3nmt.fsf@hydra.hampton.thirdcreek.com> Message-ID: On Mon, Jul 12, 2010 at 8:12 AM, Kurt B. Kaiser wrote: > On Mon, Jul 12 2010, Antoine Pitrou wrote: > >> On Mon, 12 Jul 2010 05:20:49 -0400 >> "Kurt B. Kaiser" wrote: >>> >>> I'm mystified about the comments that the GUI is ugly. ?It is minimal. >>> On XP, it looks exactly like an XP window with a simple menubar. ?Those >>> who haven't looked at it for awhile may not be aware of the recent >>> advances made by Tk in native look and feel. ?What is ugly? >> >> Ok, I've just tried IDLE (on py3k) for the first time in years. Under >> Linux, the look is ugly and outdated; it uses some kind of Motif-like >> widgets. > > That's because Linux isn't using Tk 8.5 yet. ?Debian defaults to Tk 8.5 > in experimental since March. ?Most Linux is Tk 8.4. FYI, I'm at 8.5 here, running Ubuntu 9.10, haven't knowingly updated tk. > On Windows, our installer is using 8.5. ?Here's a screenshot from my > eeePC, 1024 x 600: > > http://images.rant.ml1.net/idle.gif You may have meant to put a different image up, this one is windows. > Quoting Tal: > ============================================ > >> Perhaps, but the point is that these bugs remain. Certainly this isn't >> because just you, out of the entire Python development community, know >> little about Tk and Tkinter. > >> Using Tkinter is a major reason that maintaining and further >> developing IDLE is difficult. For example, it took me many hours just >> to get a working Tkinter scrolled frame widget, having had to write it >> from scratch and struggle with the under-documented Canvas widget. >> Another example is that integration of the new ttk (a.k.a. Tile) >> widget set, which supports native look&feel on various platforms and >> adds modern widgets, has still not been integrated despite being >> available in Tk for years and despite considerable effort being >> invested into it. > ============================================= > > Tal, you've got some catching up to do, yourself. ?Tile went into Tk in > 8.5, two years ago. ?Linux is behind, we don't control that, but our > Windows installer and tkinter is cutting edge! ?Thanks, Martin! ?I don't > know what version is running on the Mac: > > http://blog.markroseman.com/2007/12/tcltk-85-first.html > > > I've got an ancient Python 2.5.1 running in Gnome/gNewSense using Tk > 8.4. ?I think you guys are ?exaggerating. ?It's not /all/ that ugly, not by a > long shot. ?It's just not native look and feel (yet). No offense, but I've been specifically asked not to do demos with IDLE because it looked 'unprofessional'. Given the constraint of working within tkinter that may not be something you can work around, but I'm sure you can see that from a certain perspective that's beside the point. >> Usability is inconsistent with the rest of the desktop, and >> looks generally subpar (for example, you have to click on a menu to >> open it; > > On Linux and Windows, f ?will drop the file menu. > > >> the file open dialog is antiquated and doesn't allow me to use >> keyboard shortcuts). > > Looks much better in 8.5 and the shortcuts work. > > http://images.rant.ml1.net/idle2.gif > >> The editor looks decent, though there doesn't seem to be a simple (e.g. >> Shift+Tab) way of unindenting a block of code. > > Pull down the format menu. ?The dedent hotkey is Ctrl+[ > > This possibly could be changed to Shift+Tab by the user, but there may > be conflicts with completion. > > -- > KBK > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/debatem1%40gmail.com > From thebrasse at brasse.org Mon Jul 12 14:30:40 2010 From: thebrasse at brasse.org (=?ISO-8859-1?Q?Mattias_Br=E4ndstr=F6m?=) Date: Mon, 12 Jul 2010 14:30:40 +0200 Subject: [Python-Dev] Getting an optional parameter instead of creating a socket internally In-Reply-To: <20100711190652.1937.530899525.divmod.xquotient.5@localhost.localdomain> References: <4BC395DA.3030702@jcea.es> <4BC3AA8C.70109@jcea.es> <20100413020319.2779.2069488812.divmod.xquotient.401@localhost.localdomain> <4C39DF2F.5000309@jcea.es> <20100711190652.1937.530899525.divmod.xquotient.5@localhost.localdomain> Message-ID: On Sun, Jul 11, 2010 at 9:06 PM, wrote: > On 03:11 pm, jcea at jcea.es wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On 13/04/10 04:03, exarkun at twistedmatrix.com wrote: >> >>> On 12 Apr, 11:19 pm, jcea at jcea.es wrote: >>> >>>> -----BEGIN PGP SIGNED MESSAGE----- >>>> Hash: SHA1 >>>> >>>> On 04/13/2010 12:47 AM, Antoine Pitrou wrote: >>>> >>>>> Jesus Cea jcea.es> writes: >>>>> >>>>>> >>>>>> PS: "socket.setdefaulttimeout()" is not enough, because it could >>>>>> shutdown a perfectly functional connection, just because it was idle >>>>>> for >>>>>> too long. >>>>>> >>>>> >>>>> The socket timeout doesn't shutdown anything. It just puts a limit on >>>>> how much >>>>> time recv() and send() can block. Then it's up to you to detect >>>>> whether the >>>>> server is still alive (for example by pinging it through whatever >>>>> means the >>>>> application protocol gives you). >>>>> >>>> >>>> A regular standard library (let say, poplib) would abort, after getting >>>> the timeout exception. >>>> >>>>> 4. Modify client libraries to accept a new optional socket-like object >>>>>> as an optional parameter. This would allow things like transparent >>>>>> compression or encryption, or to replace the socket connection by >>>>>> anything else (read/write to shared memory or database, for example). >>>>>> >>>>> >>>>> This could be useful too. >>>>> >>>> >>>> I have been thinking about this for years. Do you actually think this >>>> could be formally proposed?. >>>> >>> >>> Every once in a while I make a little bit more progress on the PEP I'm >>> working on for this. If you want to talk more about this, you can find >>> me in #python-dev or #twisted on freenode. >>> >>> Jean-Paul >>> >> >> Jean-Paul, I would like to have this for 3.2. How is the PEP going?. >> > > It's still little more than an outline. You can see it here: > > http://twistedmatrix.com/trac/wiki/ProtocolPEP > > And if you're interested in helping, we can figure out a way to do that > (you can have edit permission on the wiki or we can move the document > elsewhere, whatever). > > Jean-Paul > > This seems like an interesting idea to me. I would like to figure out some way I could help with the PEP. If you move the document, could you please keep me updated on the new location? :.:: mattias > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/thebrasse%40brasse.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdmurray at bitdance.com Mon Jul 12 14:42:25 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 12 Jul 2010 08:42:25 -0400 Subject: [Python-Dev] IRC culture (was: Removing IDLE from the standard library) In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> Message-ID: <20100712124225.5A2E3218BAA@kimball.webabinitio.net> On Sun, 11 Jul 2010 23:51:35 +0100, Mark Lawrence wrote: > I have been attempting to fill this hole and have been faced with > animosity from people who "hang out" on the python-dev IRC channel. I > thought it was a complete and utter waste of space, so I don't intend > going back. I would like things fixed, not a cosy little "who's round > is it next" mentality from the triage team. IMHO if they spent more It was clear from a message you sent to me, that I didn't see until after your visit to the channel, that you don't have any experience on IRC. IRC is its own unique medium, with its own mores, conventions, and culture. That you perceived hostility was probably due to the nature of the medium and its communication via short sentences and intertwined conversations. And yes, the IRC channel is our "office water cooler" where we come to chat with each other about things unrelated to our coding work, as well as serious talk about the coding and bug triage work (some of which takes place *while* we're chatting about things like the World Cup Final). It's a community, and we hang out there because we find it fun to do so. We often tease each other mercilessly, and an outsider would probably wonder what the heck was going on if they didn't stick around long enough to get the flavor of the community. But we also do a lot of good communicating about bugs and code, helping each other to improve the quality of Python. I thought the conversation when you arrived was mostly positive, and we were trying to share our (somewhat disjointed, as we admitted) wisdom about what works best when doing triage. Antoine did lead off with a specific criticism, which was unfortunate and doubtless set a bad tone for you, and his mini-rant could have been more politely phrased given that you were a newcomer. But I use the term "mini-rant" descriptively...that is part of the IRC style of communication, for better or worse. As several people have pointed out, currently there is a dearth of good documentation about the Python workflow. I think Jesse's sprint effort is going to help improve this, and I know Brett Cannon really wants to have time to work the docs over thoroughly. But in the meantime, what we have is "institutional knowledge" locked up in people's heads. The python-dev mailing list is one way to get access to that knowledge, as is the tracker-discuss list for triage in particular...and the IRC channel is a great way to get access to that knowledge (like, for example, the fact that maintainers.rst is not out of date :), if you are comfortable with IRC style communication. If you don't find the IRC channel a useful place, there's no reason for you to hang out there. We were offering you the opportunity to experience the camaraderie and mutual help that we find there, and I'm very sorry that you instead found the experience offputting. It is not an exclusive club (far from it) and you would be welcome to return. As I also said to you in a private message, the non-exclusivity goes both ways...there is no *formal* "triage team", and only some of us who do triage work hang out on IRC, and only some of us who hang out on #python-dev do triage work. Further, many of the people who chat regularly on the IRC channel are committers, which is one of the reasons why it can be a rich resource while doing triage. Often enough, bugs get closed that way.[*] -- R. David Murray www.bitdance.com But, to be honest, I remember that Arfrever asked about committing the patch for a particular bug on at least three different days before someone finally had the time to do it. It was very appropriate for that bugfix to go in before the release, and he was very patient, and it did get done. From kw at codebykevin.com Mon Jul 12 14:41:37 2010 From: kw at codebykevin.com (Kevin Walzer) Date: Mon, 12 Jul 2010 08:41:37 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <7EF8C99A-4163-461A-922B-1FF3003D89E2@mac.com> <4C39C626.40400@codebykevin.com> Message-ID: <4C3B0D81.7070803@codebykevin.com> > > If I read the patch correctly it replaces the existing 8.4 support by support for 8.5. That would not be acceptable because it would result in a non-functional version of IDLE for anyone that hasn't installed a custom copy of Tk. > Not quite. It doesn't specify a version of Tk to run; it checks instead for whether Tk is built on Cocoa or Carbon. (It needs 8.5 to run on Cocoa.) > The "checkForAppKit" function doesn't work with the system version of Tk that Python currently links to. The checkForAppKit function queries the Tk windowing server for some information about its implementation. If it find 'AppKit,' then Cocoa's running: if no, then Carbon's running. If Carbon, then the code falls back to the current implementation. > > Does the patch work with the system version of Tk 8.4 on OSX? I just tested it against Python 2.5/Tk 8.4.7 (the system install on Leopard), which runs on Carobn, and it works fine. I've also tested it against 2.6 with Tk-Cocoa, and again it runs fine. I don't have access to other combinations of Python and Tk, such as Python 3.1, but I think these results indicate it's safe to apply. --Kevin -- Kevin Walzer Code by Kevin http://www.codebykevin.com From breamoreboy at yahoo.co.uk Mon Jul 12 14:50:48 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 12 Jul 2010 13:50:48 +0100 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> <20100711213801.4d391849@pitrou.net> <20100712093151.GA17171@yoda.bytereef.org> Message-ID: On 12/07/2010 11:37, geremy condra wrote: > On Mon, Jul 12, 2010 at 5:31 AM, Stefan Krah wrote: >> Jesse Noller wrote: >>> On Sun, Jul 11, 2010 at 8:22 PM, geremy condra wrote: >>>>>> (This seems to me like an area where a judicious application of PSF funds might help; if every >>>>>> single bug were actively triaged and responded to, even if it weren't reviewed, and patch >>>>>> contributors were directed to take specific steps to elicit a response or a review, the fact that >>>>>> patch reviews take a while might not be so bad.) >>>>> >>>>> The operative word being "judicious". It is not obvious who should get >>>>> funded, and for what tasks. >>>>> Some specific issues (like email in 3.x) are large enough that they can >>>>> be the sole focus of a fund grant. But I'm not sure triaging can apply. >>>> >>>> I'm mulling over starting a monthly triage sprint under the auspices of >>>> Jesse Noeller's PSF sponsored sprints in the hopes of making this a >>>> little more fun. I'd appreciate comments on the idea. >> >> [responding to Geremy] >> >> I'm with Georg on this. If triaging needs a monetary incentive because it >> is tedious work, so does committing. A lot of the abandoned issues aren't >> very glamorous either. > > I'm not sure what you mean by "monetary incentive". I was considering > handing a t-shirt or beer token to the most productive sprinters, but > that's about the limit of it, and I suspect that would come out of my > pocket. Any chance of getting me a t-shirt autographed by the one and only BDFL? :) > > I'd also emphasize that I am exactly as far as I stated on this: I'm > mulling it over and asking for feedback. If it turns out that there > are other things that python-dev feels are more necessary but > similarly unglamorous, then I'll think about doing that instead. > >> Also, from the work that Mark Lawrence has been doing on the tracker in >> the past few weeks, it's apparent that a dedicated person can achieve a >> lot without pay. > > Indeed, although I'm again unsure what pay has to do with this. > >> Due to his tracker reshuffling, many issues got closed, several bug >> reporters responded after years, etc. Thanks, Mark! > > +1 > > Geremy Condra Geremy, Stefan, Jesse and anyone that I might have missed, thanks for your kind responses, its given me quite a lift. For the record note that I only got going because of a post on c.l.py from Terry Reedy, and that he too has been doing similar work on the issue tracker, my round Terry. :) Kindest regards. Mark Lawrence From exarkun at twistedmatrix.com Mon Jul 12 14:58:35 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Mon, 12 Jul 2010 12:58:35 -0000 Subject: [Python-Dev] Getting an optional parameter instead of creating a socket internally In-Reply-To: References: <4BC395DA.3030702@jcea.es> <4BC3AA8C.70109@jcea.es> <20100413020319.2779.2069488812.divmod.xquotient.401@localhost.localdomain> <4C39DF2F.5000309@jcea.es> <20100711190652.1937.530899525.divmod.xquotient.5@localhost.localdomain> Message-ID: <20100712125835.1650.180484553.divmod.xquotient.1@localhost.localdomain> On 12:30 pm, thebrasse at brasse.org wrote: >On Sun, Jul 11, 2010 at 9:06 PM, wrote: >> >>It's still little more than an outline. You can see it here: >> >> http://twistedmatrix.com/trac/wiki/ProtocolPEP >> >>And if you're interested in helping, we can figure out a way to do >>that >>(you can have edit permission on the wiki or we can move the document >>elsewhere, whatever). >> >>Jean-Paul >This seems like an interesting idea to me. I would like to figure out >some >way I could help with the PEP. If you move the document, could you >please >keep me updated on the new location? Sure. If it moves, there'll be a pointer from its existing location to the new one. Jean-Paul From solipsis at pitrou.net Mon Jul 12 15:19:27 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 12 Jul 2010 15:19:27 +0200 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> Message-ID: <20100712151927.456cbed3@pitrou.net> On Mon, 12 Jul 2010 00:36:33 +0200 "Martin v. L?wis" wrote: > > I think Martin has always supported me in some way and I really > > appreciate that. But, maybe because I won commit privileges solely > > based on GSoC work, I felt other developers wouldn't approve my > > commits without previous discussion and that is the major reason for > > not committing most of my patches. > [...] > > So: go ahead. I won't have the time to review your changes in detail, > and nobody else will, so just put them in, and we wait for people to > complain (in which case I'd hope that you would be around to respond > quickly - as you did in the past). +1. Don't be afraid. We are quite good at pointing out mistakes after the fact :) Regards Antoine. From ncoghlan at gmail.com Mon Jul 12 16:07:59 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 Jul 2010 00:07:59 +1000 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: <201007120942.40778.steve@pearwood.info> References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> Message-ID: On Mon, Jul 12, 2010 at 9:42 AM, Steven D'Aprano wrote: > On Sun, 11 Jul 2010 09:37:22 pm Eric Smith wrote: >> > re2 comparison is interesting from the point of if it should be >> > included in stdlib. >> >> Is "it" re2 or regex? I don't see having 2 regular expression engines >> in the stdlib. > > There's precedence though... the old regex engine and the new re engine > were side-by-side for many years before regex was deprecated and > finally removed in 2.5. Hypothetically, re2 could similarly be added to > the standard library while re is deprecated. re2 deliberately omits some features for efficiency reasons, hence is not even on the table as a possible replacement for the standard library version. If someone is in a position where re2 can solve their problems with the re module, they should also be in a position where they can track it down for themselves. MRAB's module offers a superset of re's features rather than a subset though, so once it has had more of a chance to bake on PyPI it may be worth another look. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From malsburg at gmail.com Mon Jul 12 16:19:19 2010 From: malsburg at gmail.com (Titus von der Malsburg) Date: Mon, 12 Jul 2010 16:19:19 +0200 Subject: [Python-Dev] PEP 3148 ready for pronouncement [ACCEPTED] Message-ID: <20100712141919.GA3441@gmail.com> Hi I learned about the futures PEP only today. I saw the example on http://code.google.com/p/pythonfutures/ One thing that worries me is that this approach seems to bypass the usual exception handling mechanism of Python. In particular I'm wondering why you have to do things like: if future.exception() is not None: ... This reminds me a lot of how things are done in C but it's not very pythonic. Wouldn't it be possible and nicer to raise the exception -- if there was one inside the asynchronous job -- when the result of the future is accessed? try: print('%r page is %d bytes' % (url, len(future.result()))) except FutureError: print('%r generated an exception: %s' % (url, future.exception())) Best, Titus From motoom at xs4all.nl Mon Jul 12 16:16:11 2010 From: motoom at xs4all.nl (Michiel Overtoom) Date: Mon, 12 Jul 2010 16:16:11 +0200 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: <4C3B23AB.3000108@xs4all.nl> Tal Einat wrote: > I would like to propose removing IDLE from the standard library. I use IDLE every day. It does everything I want an IDE to do, it looks simple and doesn't waste screen real estate like some other IDEs do, it supports proportionally spaced fonts correctly, its syntax coloring is easy configurable (not a nightmare like some scintilla-based IDEs), and it's instantly available on any PC on which I install Python. I'd like to keep it in the stdlib. Greetings, From tseaver at palladion.com Mon Jul 12 16:25:09 2010 From: tseaver at palladion.com (Tres Seaver) Date: Mon, 12 Jul 2010 10:25:09 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tal Einat wrote: >> I would like to propose removing IDLE from the standard library. >> >> I have been using IDLE since 2002 and have been doing my best to help >> maintain and further develop IDLE since 2005. > > I'm surprised by the amount of interest this has raised already. To > answer a few questions that were raised: > > In recent years I have worked up many patches, both bugfixes and new > features and improvements. Getting any attention to these was > non-trivial, and getting patches accepted (or an explanation why they > are rejected in some cases) almost always took many months, sometimes > years, and some are still unresolved. It has been very frustrating. > > When I ran into bugs I fixed them and submitted a patch. I have also > done so for quite a few bugs reported by others. However, there are > currently several bugs in the tracker which nobody is taking any > notice of. IIRC most of the recent bugs are related to OSX or 64-bit > Windows. > > To those who mention that IDLE is "okay" or "not going uphill", my > grandfather would say "if you aren't running forwards, you are falling > behind." You should know how IDLE looks to programmers seeing it for > the first time -- IDLE's quirky and old-fashioned looks and interface > are a major turnoff for new users. As a result I have stopped > recommending it to coworkers, despite personally liking IDLE, instead > recommending the basic command-line or IPython for interactive work, > and any other IDE or text editor for development. > > I too prefer IDLE to the basic command line, and think that something > like IDLE is well-suited for learning/teaching Python. I also think an > interpreter with a nice GUI can be far superior to a text-only > interpreter. However, I've mostly lost hope for IDLE, and am currently > hoping that something else takes its place. > > The fact is that for many years little effort has gone into developing > and maintaining IDLE, and I believe being tucked in a corner of the > Python codebase is a major reason for this. I really don't see why > IDLE has to be part of the standard library, what's wrong with IDLE > being an externally maintained application? > > Yes, IDLE still works (mostly), but us few who continue to use it > could do so even if it weren't part of the standard library. I wonder if moving it out of stdlib might actually help improve its development velocity: maybe if it were managed via bitbucket, with user-visible forks to known fixes, etc., it would get "caught up" to people's expectations. Perhaps I'm really suggesting that there be an 'idle2' project nn bitbucket, as a "friendly fork" of the mostly freeze-dried version in stdlib. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkw7JcAACgkQ+gerLs4ltQ73RACfTcPaDXPFlg8EWnBxYj3qfWwg qswAn3Ws/FvYqLLiYGvgzEpd1sIpWuWJ =ZlSp -----END PGP SIGNATURE----- From kw at codebykevin.com Mon Jul 12 16:34:50 2010 From: kw at codebykevin.com (Kevin Walzer) Date: Mon, 12 Jul 2010 10:34:50 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <4C3B23AB.3000108@xs4all.nl> References: <4C3B23AB.3000108@xs4all.nl> Message-ID: <4C3B280A.90102@codebykevin.com> On 7/12/10 10:16 AM, Michiel Overtoom wrote: > Tal Einat wrote: > >> I would like to propose removing IDLE from the standard library. > > I use IDLE every day. It does everything I want an IDE to do, it looks > simple and doesn't waste screen real estate like some other IDEs do, it > supports proportionally spaced fonts correctly, its syntax coloring is > easy configurable (not a nightmare like some scintilla-based IDEs), and > it's instantly available on any PC on which I install Python. > > I'd like to keep it in the stdlib. +1 to keeping it in the stdlib, in case my earlier comments didn't make that clear. My own reasons for submitting patches to help with Tkinter on the Mac are pretty typical: the Cocoa-based implementation of Tk broke some things in IDLE because of different methods for menu placement, etc. I wanted this fixed, and since I might be the only Python person with the knowledge about Carbon vs. Cocoa internals in Tk to do it, I dug in, attempted to grok IDLE's rather baroque code, and eventually came up with some patches. Scratching my own itch, IOW. I'm currently using Aquamacs/Emacs for my Python development because of some other issues with Tk-Cocoa (external to IDLE) on Leopard, but once I upgrade my OS, I plan to return to using IDLE for everyday Python development. Since Tkinter is my GUI toolkit of choice, I like to work in that environment. Also, I strongly urge Guilherme (or someone) to commit his changes to IDLE. While I like IDLE and feel it is a very useful IDE for Python work even for someone who is past the beginner stage, I also agree that its current UI is dated. Adding the themed Tk widgets would likely make it more pleasant to use, as well as showing what can be done with the new widgets. (The existence and widespread use of archaic Tk extensions like Tix within Tkinter isn't a recommendation for the toolkit beyond backwards compatibility.) --Kevin -- Kevin Walzer Code by Kevin http://www.codebykevin.com From ncoghlan at gmail.com Mon Jul 12 16:42:17 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 Jul 2010 00:42:17 +1000 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <20100712151927.456cbed3@pitrou.net> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> Message-ID: On Mon, Jul 12, 2010 at 11:19 PM, Antoine Pitrou wrote: > +1. Don't be afraid. We are quite good at pointing out mistakes after > the fact :) Just make sure to subscribe to python-checkins and keep an eye out for replies to your commits. Most post hoc review comments come in as replies to the commit notifications :) Hmm, do we mention that part of the process in the dev docs anywhere?... and it turns out we do, as Brett included it in the Intro to Development article. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Mon Jul 12 16:48:35 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 Jul 2010 00:48:35 +1000 Subject: [Python-Dev] PEP 3148 ready for pronouncement [ACCEPTED] In-Reply-To: <20100712141919.GA3441@gmail.com> References: <20100712141919.GA3441@gmail.com> Message-ID: On Tue, Jul 13, 2010 at 12:19 AM, Titus von der Malsburg wrote: > This reminds me a lot of how things are done in C but it's not very > pythonic. ?Wouldn't it be possible and nicer to raise the exception -- > if there was one inside the asynchronous job -- when the result of the > future is accessed? That's what actually happens, so you can code it either way (either just calling result() and dealing with any exception that was raised, or else checking for a non-None exception value directly). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From malsburg at gmail.com Mon Jul 12 16:59:09 2010 From: malsburg at gmail.com (Titus von der Malsburg) Date: Mon, 12 Jul 2010 16:59:09 +0200 Subject: [Python-Dev] PEP 3148 ready for pronouncement [ACCEPTED] In-Reply-To: References: <20100712141919.GA3441@gmail.com> Message-ID: <20100712145909.GB3801@gmail.com> On Tue, Jul 13, 2010 at 12:48:35AM +1000, Nick Coghlan wrote: > On Tue, Jul 13, 2010 at 12:19 AM, Titus von der Malsburg > That's what actually happens, so you can code it either way That's great! None of the examples I found used the pythonic exception style, that's why I assumed that checking the "return value" is the only possibility. Reading the PEP carefully would have helped. :-) Titus From fuzzyman at voidspace.org.uk Mon Jul 12 17:07:34 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 12 Jul 2010 16:07:34 +0100 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> Message-ID: <4C3B2FB6.3050400@voidspace.org.uk> On 12/07/2010 15:42, Nick Coghlan wrote: > On Mon, Jul 12, 2010 at 11:19 PM, Antoine Pitrou wrote: > >> +1. Don't be afraid. We are quite good at pointing out mistakes after >> the fact :) >> > Just make sure to subscribe to python-checkins and keep an eye out for > replies to your commits. Most post hoc review comments come in as > replies to the commit notifications :) > That mailing list (python-checkins) is way too high traffic for many committers to monitor. I hope people making comments on checkins also email the committer directly. All the best, Michael > Hmm, do we mention that part of the process in the dev docs > anywhere?... and it turns out we do, as Brett included it in the Intro > to Development article. > > Cheers, > Nick. > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From kbk at shore.net Mon Jul 12 17:11:23 2010 From: kbk at shore.net (Kurt B. Kaiser) Date: Mon, 12 Jul 2010 11:11:23 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: (Tal Einat's message of "Mon, 12 Jul 2010 14:07:05 +0300") References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A493E.8090308@v.loewis.de> Message-ID: <878w5g3fc4.fsf@hydra.hampton.thirdcreek.com> On Mon, Jul 12 2010, Tal Einat wrote: > On Mon, Jul 12, 2010 at 1:44 AM, "Martin v. L?wis" wrote: >>> FWIW this is why I started IDLE-Spoon (well, continued Noam Raphael's >>> project of the same name, in a sense). The idea was to have a fork of >>> IDLE with new features which need to be tried out by "beta testers" to >>> iron out all of the glitches before making it into the main version, >>> like IDLE-fork back in the beginning of the decade. However I have >>> utterly failed in promoting this project and getting "beta testers" on >>> board, at least partially due to the lack of interest from the >>> community (and admittedly my lack of PR skills). >> >> I think such a thing must inherently fail - precisely for these reasons. >> >> It's much better to release proposed new features along with Python, >> and wait for feedback. Users won't start trying things out until after >> the release. This is a general problem, and lead Barry Warsaw to believe >> that "release candidates" are an utter waste of time. > > That's debatable, and I disagree. IDLE-fork was a great success, for example. We had major contributions from David Scherer, Guido, and Stephen Gava. But a key factor in its success was that I took it upon myself to keep IDLEfork sync'd with core IDLE using a cvs vendor branch and frequent merges. Once the project was completed, I arranged with SF to move the IDLEfork repository, including history, back into Python. This was not done with Noam's branch. As a result, it gradually drifted to the point where it became essentially unmergable. Once we switch to Hg, we should be able to use distributed vc and topic branches to facilitate community participation without the need for a separate project. None of this is automatic, of course, it requires diligence, planning, and clean topic patches. > > If IDLE actually had many users today, certainly some of them would be > interested in trying out a version with usability fixes and > improvements. Waiting for a new release of Python can take over a > year. Furthermore, backwards compatibility issues and support by third > party libraries can delay migration to a newer version of Python even > further. There's no reason why we couldn't release interim IDLE testing packages from somewhere other than python.org (for those that don't want to track IDLE's Hg repo). To do this successfully, we would need to avoid using new Python features being introduced during that development cycle, i.e. IDLE would be compatible with the previous Python release. -- KBK From taleinat at gmail.com Mon Jul 12 17:14:16 2010 From: taleinat at gmail.com (Tal Einat) Date: Mon, 12 Jul 2010 18:14:16 +0300 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> Message-ID: Kurt B. Kaiser wrote: >> Using Tkinter is a major reason that maintaining and further >> developing IDLE is difficult. For example, it took me many hours just >> to get a working Tkinter scrolled frame widget, having had to write it >> from scratch and struggle with the under-documented Canvas widget. >> Another example is that integration of the new ttk (a.k.a. Tile) >> widget set, which supports native look&feel on various platforms and >> adds modern widgets, has still not been integrated despite being >> available in Tk for years and despite considerable effort being >> invested into it. > > Tal, you've got some catching up to do, yourself. Tile went into Tk in > 8.5, two years ago. I was referring to the integration of the new ttk widgets into IDLE, which still hasn't happened despite two years having gone by. Sorry for not being clear on that point. - Tal Einat From glyph at twistedmatrix.com Mon Jul 12 17:18:21 2010 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Mon, 12 Jul 2010 11:18:21 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <4C3AD3A8.8090203@netwok.org> References: <4C3A0F85.1090404@v.loewis.de> <4C3A1938.4020304@v.loewis.de> <1E32216D-8C3A-4BD8-8E5A-127E98B863AB@twistedmatrix.com> <4C3AD3A8.8090203@netwok.org> Message-ID: <7C265E82-D5DF-496B-8303-62B2B28E254A@twistedmatrix.com> On Jul 12, 2010, at 4:34 AM, ?ric Araujo wrote: >> Plus, is a useful resource >> for core developers with only a little bit of free time to do a >> review. > > Title: ?Review Tickets, By Order You Should Review Them In? > I haven?t found a description of this order, can you explain? Thanks. Part of the reason that the report is worded that way is that we may decide that the order should be different, but it will still be the order that you should review them in :). Right now the order is "amount of time since last change, sorted from highest to lowest". In other words, first come, first serve, by last activity. From fuzzyman at voidspace.org.uk Mon Jul 12 17:18:38 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 12 Jul 2010 16:18:38 +0100 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> Message-ID: <4C3B324E.2050005@voidspace.org.uk> On 12/07/2010 15:07, Nick Coghlan wrote: > On Mon, Jul 12, 2010 at 9:42 AM, Steven D'Aprano wrote: > >> On Sun, 11 Jul 2010 09:37:22 pm Eric Smith wrote: >> >>>> re2 comparison is interesting from the point of if it should be >>>> included in stdlib. >>>> >>> Is "it" re2 or regex? I don't see having 2 regular expression engines >>> in the stdlib. >>> >> There's precedence though... the old regex engine and the new re engine >> were side-by-side for many years before regex was deprecated and >> finally removed in 2.5. Hypothetically, re2 could similarly be added to >> the standard library while re is deprecated. >> > re2 deliberately omits some features for efficiency reasons, hence is > not even on the table as a possible replacement for the standard > library version. If someone is in a position where re2 can solve their > problems with the re module, they should also be in a position where > they can track it down for themselves. > > If it has *partial* compatibility, and big enough performance improvements for common cases, it could perhaps be used where the regex doesn't use unsupported features. This would have some extra cost in the compile phase, but would mean Python could ship with two regex engines but only one interface exposed to the programmer... Michael > MRAB's module offers a superset of re's features rather than a subset > though, so once it has had more of a chance to bake on PyPI it may be > worth another look. > > Cheers, > Nick. > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From basherwo at ncsu.edu Mon Jul 12 17:12:51 2010 From: basherwo at ncsu.edu (Bruce Sherwood) Date: Mon, 12 Jul 2010 09:12:51 -0600 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> Message-ID: On Mon, Jul 12, 2010 at 3:20 AM, Kurt B. Kaiser wrote: > > > As I recollect, much of what Scherer did in VIDLE related to running > multiple IDLE copies. > > For that reason, the VIDLE changes have to be evaluated carefully to > determine what has already been incorporated. I believe I added a > couple of his smaller changes, also. > > Bruce, speaking of Dave Scherer, we've been trying to get a Python > Contributor Agreement from him. Is that something you can help us with? > > -- > KBK > I don't recall that VIDLE has anything to do with running multiple IDLE copies. What's in VIDLE is a lot of bug fixes and some improvements. For example, you can configure it to not require having to save a file, which makes it as easy to use for quick testing as the shell. Another improvement, of high importance for our physics students and other novice programmers, is to bring an error display forward. Many students fill the screen with the edit window and the error message could be hidden, leaving them puzzled as to why their program wasn't running. Stuff like that, all aimed at issues we had seen with a large population of science and engineering students. I'll say again that IDLE/VIDLE is an excellent environment for novices. Issues identified here about look and feel etc. have just not been of any importance in that context. If you'll send me what you want from Scherer, I can contact him. Bruce Sherwood -------------- next part -------------- An HTML attachment was scrubbed... URL: From kbk at shore.net Mon Jul 12 17:23:04 2010 From: kbk at shore.net (Kurt B. Kaiser) Date: Mon, 12 Jul 2010 11:23:04 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <20100712142200.6b57ef8d@pitrou.net> (Antoine Pitrou's message of "Mon, 12 Jul 2010 14:22:00 +0200") References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <20100712121040.41f16654@pitrou.net> <87fwzo3nmt.fsf@hydra.hampton.thirdcreek.com> <20100712142200.6b57ef8d@pitrou.net> Message-ID: <874og43esn.fsf@hydra.hampton.thirdcreek.com> On Mon, Jul 12 2010, Antoine Pitrou wrote: > On Mon, 12 Jul 2010 08:12:10 -0400 > "Kurt B. Kaiser" wrote: >> > >> > Ok, I've just tried IDLE (on py3k) for the first time in years. Under >> > Linux, the look is ugly and outdated; it uses some kind of Motif-like >> > widgets. >> >> That's because Linux isn't using Tk 8.5 yet. Debian defaults to Tk 8.5 >> in experimental since March. Most Linux is Tk 8.4. > > No, that's wrong. Tk here is 8.6: > > $ ldd build/lib.linux-x86_64-3.2/_tkinter.so > [...] > libtk8.6.so.0 => /usr/lib64/libtk8.6.so.0 (0x00007f4eb9259000) > libtcl8.6.so.0 => /usr/lib64/libtcl8.6.so.0 (0x00007f4eb8f07000) > [...] > > $ ./python -c "import tkinter; print(tkinter.TkVersion)" > 8.6 What distro are you using? Tk8.6 is still in beta. Still looks crummy? Bummer. >> > the file open dialog is antiquated and doesn't allow me to use >> > keyboard shortcuts). >> >> Looks much better in 8.5 and the shortcuts work. >> >> http://images.rant.ml1.net/idle2.gif > > Well, this is a Windows screenshot. I'm under Linux. Fine, I showed it as an example of the improvement in 8.5. Most people, I think, are using Windows or Macs. Looks decent, don't you think? I don't have Tk8.5 running under Linux yet, much less 8.6. But the Tk release docs claim improvement under X11. It's possible we may have to use Tk theming to get full benefit of the Tile improvements in X11. -- KBK From kbk at shore.net Mon Jul 12 17:32:27 2010 From: kbk at shore.net (Kurt B. Kaiser) Date: Mon, 12 Jul 2010 11:32:27 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: (geremy condra's message of "Mon, 12 Jul 2010 08:30:40 -0400") References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <20100712121040.41f16654@pitrou.net> <87fwzo3nmt.fsf@hydra.hampton.thirdcreek.com> Message-ID: <87zkxw1zsk.fsf@hydra.hampton.thirdcreek.com> On Mon, Jul 12 2010, geremy condra wrote: > No offense, but I've been specifically asked not to do demos with IDLE > because it looked 'unprofessional'. Given the constraint of working > within tkinter that may not be something you can work around, but I'm > sure you can see that from a certain perspective that's beside the > point. Right, I stay away from working on Tk. It's not even a tkinter issue. I hope it will continue to get better, but I imagine it's a moving target for the Tk developers. In your case, just bring up Wing. That'll impress them. As long as it's not on netbook :-) -- KBK From reid.kleckner at gmail.com Mon Jul 12 17:36:16 2010 From: reid.kleckner at gmail.com (Reid Kleckner) Date: Mon, 12 Jul 2010 15:36:16 +0000 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> Message-ID: On Mon, Jul 12, 2010 at 9:20 AM, Kurt B. Kaiser wrote: > Also, the current right click edit action on Windows is to only open an > edit window; no shell. ?And it uses the subprocess! ?So, some of the > comments on this thread are not up to date. > > The reason that bug languished for two years was because first, it was a > bit of a hack, and second, Windows was problematic in that it reused > sockets and often left zombie subprocesses behind which couldn't be > killed except with the task manager. ?This causes real problems with > students - they lose confidence in the tool. > > Scherer and Weeble put together a patch using ephemeral ports which > nailed the problem, and I checked it in right away and > forward/backported it. That's great news! I TAed a freshman Python class this January, and Windows users ran into this problem a lot. Mostly when hitting 'x' in the upper right. Fortunately, some quick searching led me to the Python tracker where I found the workaround. :) (Somwhat off-topic): Another pain point students had was accidentally shadowing stdlib modules, like random. Renaming the file didn't solve the problem either, because it left behind .pycs, which I had to help them delete. Overall, I would say that IDLE worked very well in that situation, so while it does have its quirks, it worked very well for us. Imagine trying to get students started with Eclipse or NetBeans. Yuck! Reid From solipsis at pitrou.net Mon Jul 12 17:39:52 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 12 Jul 2010 17:39:52 +0200 Subject: [Python-Dev] New regex module for 3.2? References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3B324E.2050005@voidspace.org.uk> Message-ID: <20100712173952.2b0be599@pitrou.net> On Mon, 12 Jul 2010 16:18:38 +0100 Michael Foord wrote: > On 12/07/2010 15:07, Nick Coghlan wrote: > > On Mon, Jul 12, 2010 at 9:42 AM, Steven D'Aprano wrote: > > > >> On Sun, 11 Jul 2010 09:37:22 pm Eric Smith wrote: > >> > >>>> re2 comparison is interesting from the point of if it should be > >>>> included in stdlib. > >>>> > >>> Is "it" re2 or regex? I don't see having 2 regular expression engines > >>> in the stdlib. > >>> > >> There's precedence though... the old regex engine and the new re engine > >> were side-by-side for many years before regex was deprecated and > >> finally removed in 2.5. Hypothetically, re2 could similarly be added to > >> the standard library while re is deprecated. > >> > > re2 deliberately omits some features for efficiency reasons, hence is > > not even on the table as a possible replacement for the standard > > library version. If someone is in a position where re2 can solve their > > problems with the re module, they should also be in a position where > > they can track it down for themselves. > > > > > > If it has *partial* compatibility, and big enough performance > improvements for common cases, it could perhaps be used where the regex > doesn't use unsupported features. This would have some extra cost in the > compile phase, but would mean Python could ship with two regex engines > but only one interface exposed to the programmer... You're forgetting the maintenance cost. Regards Antoine. From janssen at parc.com Mon Jul 12 17:44:10 2010 From: janssen at parc.com (Bill Janssen) Date: Mon, 12 Jul 2010 08:44:10 PDT Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A5983.3010002@holdenweb.com> Message-ID: <86474.1278949450@parc.com> Stephen Hansen wrote: > On Sun, Jul 11, 2010 at 4:53 PM, Steve Holden wrote: > > > Stephen Hansen wrote: > > > On Sat, Jul 10, 2010 at 9:23 PM, Guilherme Polo > > > wrote: > > > > > > By "never had a problem" do you mean using some of the latest > > versions > > > ? Here, running "idle" from a mac terminal and trying to type: print > > > "hi" crashes when entering the quotation mark. > > > > > > > > > Huh? Works fine for me. Python 2.6.1, OSX 10.6.3, intel. > > > > > One of the good things about the python-dev community is its commitment > > to test-driven development. If you are prepared to define "fine" as > > 'successfully runs \'print "hello"\'' then I guess we should be > > perfectly happy about IDLE. > > > > Er, how hostile. > > My point is, the poster made an assertion-- that you couldn't do the simple > act as launching idle from a command line, and printing Hi. Maybe they > can't, I have no idea. > > I know I can. I know that I have also opened random python files, saved > them, and ran them with IDLE. I don't use IDLE beyond that though: I live in > TextMate on my mac. > > My point was not, "IDLE is perfect". My point was, "You've claimed you can't > even print out a word in IDLE, so its utterly and completely non-functional" > -- and that assertion surprises me and I challenge. Steve, you encouraged me to try it again. From an xterm on OS X 10.5.8, it launches fine (long as you know where it is -- /System/Library/Frameworks/Python.framework/Versions/Current/bin/idle). Seems to work OK for what it is, too. Same for Terminal. When I start it from the *shell* buffer in Cocoa GNU Emacs, though, the idle window pops up under the Emacs window, which is why I didn't see it over the weekend. Bill From solipsis at pitrou.net Mon Jul 12 17:42:26 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 12 Jul 2010 17:42:26 +0200 Subject: [Python-Dev] Removing IDLE from the standard library References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <20100712121040.41f16654@pitrou.net> <87fwzo3nmt.fsf@hydra.hampton.thirdcreek.com> <20100712142200.6b57ef8d@pitrou.net> <874og43esn.fsf@hydra.hampton.thirdcreek.com> Message-ID: <20100712174226.2c692abb@pitrou.net> On Mon, 12 Jul 2010 11:23:04 -0400 "Kurt B. Kaiser" wrote: > > What distro are you using? Tk8.6 is still in beta. It's Mandriva 2010.1 > Still looks crummy? Bummer. Yes. > Fine, I showed it as an example of the improvement in 8.5. Most people, > I think, are using Windows or Macs. Looks decent, don't you think? Yes, it does. If indeed the main target population for IDLE is Windows and Mac users (since Linux users have a lot of alternatives almost by default), then I suppose the situation is not that bad. Regards Antoine. From timwintle at gmail.com Mon Jul 12 17:48:03 2010 From: timwintle at gmail.com (Tim Wintle) Date: Mon, 12 Jul 2010 16:48:03 +0100 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: <4C3B324E.2050005@voidspace.org.uk> References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3B324E.2050005@voidspace.org.uk> Message-ID: <1278949683.7284.12.camel@tim-laptop> On Mon, 2010-07-12 at 16:18 +0100, Michael Foord wrote: > On 12/07/2010 15:07, Nick Coghlan wrote: > > On Mon, Jul 12, 2010 at 9:42 AM, Steven D'Aprano wrote: > > > > re2 deliberately omits some features for efficiency reasons, hence is > > not even on the table as a possible replacement for the standard > > library version. If someone is in a position where re2 can solve their > > problems with the re module, they should also be in a position where > > they can track it down for themselves. > > > > If it has *partial* compatibility, and big enough performance > improvements for common cases, it could perhaps be used where the regex > doesn't use unsupported features. This would have some extra cost in the > compile phase, but would mean Python could ship with two regex engines > but only one interface exposed to the programmer... I'm not sure how common those cases are - I played around with RE2 a few months ago and found that the majority of regular expressions in my own code were noticeably slower running under RE2 than python's re module - RE2 just puts much nicer theoretical bounds on cases that were (in my code at least) unusual. The really good use case for RE2 is for applications where users can write regular expressions as input (exactly what RE2 was designed for) - but I'm not sure how common those applications are. Tim Wintle From ncoghlan at gmail.com Mon Jul 12 17:52:02 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 Jul 2010 01:52:02 +1000 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <4C3B2FB6.3050400@voidspace.org.uk> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> Message-ID: On Tue, Jul 13, 2010 at 1:07 AM, Michael Foord wrote: > That mailing list (python-checkins) is way too high traffic for many > committers to monitor. I hope people making comments on checkins also email > the committer directly. Not normally, no - there's no easy way to connect a checkin message to a committer's email address, so it's usually just a matter of hitting "Reply" and sending the review comment to the list. With a new committer I'll make the effort to cc them directly in case they aren't subscribed yet, but I expect everyone else to be monitor the checkins list. If the list is too high volume, filter it down to just those issues with your checkin name in the "From" or "To" fields (your own checkin messages will have the former, while replies should have the latter, even though the email address in both cases will actually be python-checkins at python.org. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From collinw at gmail.com Mon Jul 12 17:54:44 2010 From: collinw at gmail.com (Collin Winter) Date: Mon, 12 Jul 2010 08:54:44 -0700 Subject: [Python-Dev] commit privs In-Reply-To: <20100711182831.56462244@pitrou.net> References: <20100711182831.56462244@pitrou.net> Message-ID: On Sun, Jul 11, 2010 at 9:28 AM, Antoine Pitrou wrote: > On Sun, 11 Jul 2010 13:23:13 +0000 > Reid Kleckner wrote: >> >> I'm also expecting to be doing more work merging unladen-swallow into >> the py3k-jit branch, so I was wondering if I could get commit >> privileges for that. > > It sounds good to me. Also, thanks for your threading patches! +1 from me. From kbk at shore.net Mon Jul 12 18:02:52 2010 From: kbk at shore.net (Kurt B. Kaiser) Date: Mon, 12 Jul 2010 12:02:52 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: (Neil Hodgson's message of "Mon, 12 Jul 2010 21:14:08 +1000") References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> Message-ID: <87vd8k1ydv.fsf@hydra.hampton.thirdcreek.com> On Mon, Jul 12 2010, Neil Hodgson wrote: > On the main editing screen of IDLE, the most noticeable issue is > that there is no horizontal scroll bar even though the text will move > left when you move the caret beyond the rightmost visible character. That was a design decision by Guido to encourage keeping to 80 columns. > The scrollbar and status bar have an appearance that looks to be from > Windows 2000, not Windows XP Tk issue, maybe we can address it with theming. > and there is no resizing gripper on the right side of the status > bar. Although the corner can be dragged, that would be useful. As I recollect, we special-cased it so there is a space for the Mac gripper! > The tear off menus are ugly as well as being non-standard on all three > major platforms. Well, would you discard them? They can (occasionally) be useful. > Use the "Configure IDLE..." and an "idle" dialog appears that also > looks to be from Windows 2000. I know Tk can do better than this as > Git Gui (the Tk (8.5.8) program I use most often) at least shows XP > themed buttons, scrollbars and other controls. However, the "idle" > dialog (as well as Git Gui) shows the largest remaining problem for Tk > user interfaces: keyboard navigation. When the "idle" dialog opens, > try doing anything with the keyboard. Chances are nothing will happen. > If you press Tab 16 times (yes, 16!) a focus rectangle will finally > show on the "Bold" check box. Another Tab takes you to the > "Indentation Width" slider. After that you don't see the focus until > it wraps around to "Bold" again. The Enter key doesn't trigger OK and > the Escape key doesn't let you escape. > > The Find and Replace dialogs are better as focus works as do Enter > and Escape but none of the buttons have mnemonics. > > This may all sound like picking nits but details and consistency > are important in user interfaces and this is just looking at the most > easily discovered problems. No, these are worthwhile improvements, and not too difficult. -- Thanks, KBK From ncoghlan at gmail.com Mon Jul 12 18:14:26 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 Jul 2010 02:14:26 +1000 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <20100712174226.2c692abb@pitrou.net> References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <20100712121040.41f16654@pitrou.net> <87fwzo3nmt.fsf@hydra.hampton.thirdcreek.com> <20100712142200.6b57ef8d@pitrou.net> <874og43esn.fsf@hydra.hampton.thirdcreek.com> <20100712174226.2c692abb@pitrou.net> Message-ID: On Tue, Jul 13, 2010 at 1:44 AM, Bill Janssen wrote: > Steve, you encouraged me to try it again. ?From an xterm on OS X 10.5.8, > it launches fine (long as you know where it is -- > /System/Library/Frameworks/Python.framework/Versions/Current/bin/idle). > Seems to work OK for what it is, too. ?Same for Terminal. ?When I start > it from the *shell* buffer in Cocoa GNU Emacs, though, the idle window > pops up under the Emacs window, which is why I didn't see it over the > weekend. FWIW, "./python -m idlelib,idle" launched the shell window for me on two development installs. "python -m idlelib.idle" didn't work intially on the system Python (since Kubuntu leaves idle out by default), but it worked fine once I installed the idle package. (Oh, and I can confirm it still looks like a Tcl/Tk app on Kubuntu rather than a KDE app even with Tk 8.5) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From taleinat at gmail.com Mon Jul 12 18:29:38 2010 From: taleinat at gmail.com (Tal Einat) Date: Mon, 12 Jul 2010 19:29:38 +0300 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <878w5g3fc4.fsf@hydra.hampton.thirdcreek.com> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A493E.8090308@v.loewis.de> <878w5g3fc4.fsf@hydra.hampton.thirdcreek.com> Message-ID: On Mon, Jul 12, 2010 at 6:11 PM, Kurt B. Kaiser wrote: > On Mon, Jul 12 2010, Tal Einat wrote: > >> On Mon, Jul 12, 2010 at 1:44 AM, "Martin v. L?wis" wrote: >>>> FWIW this is why I started IDLE-Spoon (well, continued Noam Raphael's >>>> project of the same name, in a sense). The idea was to have a fork of >>>> IDLE with new features which need to be tried out by "beta testers" to >>>> iron out all of the glitches before making it into the main version, >>>> like IDLE-fork back in the beginning of the decade. However I have >>>> utterly failed in promoting this project and getting "beta testers" on >>>> board, at least partially due to the lack of interest from the >>>> community (and admittedly my lack of PR skills). >>> >>> I think such a thing must inherently fail - precisely for these reasons. >>> >>> It's much better to release proposed new features along with Python, >>> and wait for feedback. Users won't start trying things out until after >>> the release. This is a general problem, and lead Barry Warsaw to believe >>> that "release candidates" are an utter waste of time. >> >> That's debatable, and I disagree. IDLE-fork was a great success, for example. > > We had major contributions from David Scherer, Guido, and Stephen Gava. > > But a key factor in its success was that I took it upon myself to keep > IDLEfork sync'd with core IDLE using a cvs vendor branch and frequent > merges. ?Once the project was completed, I arranged with SF to move the > IDLEfork repository, including history, back into Python. > > This was not done with Noam's branch. ?As a result, it gradually drifted > to the point where it became essentially unmergable. Actually, Noam's branch was pretty much merged back as is -- that's where IDLE's auto-completion functionality came from. The later changes he made on that branch were never merged, as you mentioned, because Noam never bothered. I have been maintaining my own fork of IDLE for several years and manually keeping it in sync with IDLE (this was simple). The difference is that there was no single major new feature I was working on, such as the addition of a sub-process in IDLE-fork or Noam's addition of auto-completion. I was mostly making relatively minor fixes and changes which were not interrelated. I saw no reason to have them all merged back at once, so I posted patches as soon as I felt they were ready, and did the best I could to get them accepted. I eventually gave up on this process because every patch took far too long to be addressed and finally accepted or rejected, and I realized that most of the work I had done would never be merged back into the mainstream version of IDLE. From fuzzyman at voidspace.org.uk Mon Jul 12 19:39:09 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 12 Jul 2010 18:39:09 +0100 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> Message-ID: <4C3B533D.1080706@voidspace.org.uk> On 12/07/2010 16:52, Nick Coghlan wrote: > On Tue, Jul 13, 2010 at 1:07 AM, Michael Foord > wrote: > > >> That mailing list (python-checkins) is way too high traffic for many >> committers to monitor. I hope people making comments on checkins also email >> the committer directly. >> > Not normally, no - there's no easy way to connect a checkin message to > a committer's email address, There's a one-to-one mapping somewhere. > so it's usually just a matter of hitting > "Reply" and sending the review comment to the list. With a new > committer I'll make the effort to cc them directly in case they aren't > subscribed yet, but I expect everyone else to be monitor the checkins > list. > Without contacting committers, rather than firing into the black depths of a high traffic list, I strongly suspect that much of the feedback is lost. All the best, Michael > If the list is too high volume, filter it down to just those issues > with your checkin name in the "From" or "To" fields (your own checkin > messages will have the former, while replies should have the latter, > even though the email address in both cases will actually be > python-checkins at python.org. > > Cheers, > Nick. > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From Petre.Galan at gmail.com Mon Jul 12 19:35:09 2010 From: Petre.Galan at gmail.com (Petre Galan) Date: Mon, 12 Jul 2010 17:35:09 +0000 (UTC) Subject: [Python-Dev] More C API abstraction for user defined types Message-ID: Hello, Defining a user type which implements number protocol should be treated as a number (long) in PyArg_ParseTuple using "l" format. In getargs.c: case 'l': {/* long int */ long *p = va_arg(*p_va, long *); long ival; if (float_argument_error(arg)) return converterr("integer", arg, msgbuf, bufsize); ival = PyLong_AsLong(arg); if (ival == -1 && PyErr_Occurred()) return converterr("integer", arg, msgbuf, bufsize); else *p = ival; break; } ival should not be resolved through PyLong_AsLong, but through functionality/interface like PyNumber_Long, thus allowing more diversity in accessing the PyArg_Parse interfaces. Petre From ianb at colorstudy.com Mon Jul 12 20:21:54 2010 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 12 Jul 2010 11:21:54 -0700 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: On Sun, Jul 11, 2010 at 3:38 PM, Ron Adam wrote: > There might be another alternative. > > Both idle and pydoc are applications (are there others?) that are in the > standard library. As such, they or parts of them, are possibly importable > to other projects. That restricts changes because a committer needs to > consider the chances that a change may break something else. > > I suggest they be moved out of the lib directory, but still be included > with python. (Possibly in the tools directory.) That removes some of the > backward compatibility restrictions or at least makes it clear there isn't a > need for backward compatibility. > I also like this idea. This means Python comes with an IDE "out of he box" but without the overhead of a management and release process that is built for something very different than a GUI program (the standard library). This would mean that IDLE would be in site-packages, could easily be upgraded using normal tools, and maybe most importantly it could have its own community tools and development process that is more casual (and can more easily integrate new contributors) and higher velocity of changes and releases. Python releases would then ship the most recent stable release of IDLE. -- Ian Bicking | http://blog.ianbicking.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Mon Jul 12 20:36:03 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 12 Jul 2010 19:36:03 +0100 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: <4C3B6093.70809@voidspace.org.uk> On 12/07/2010 19:21, Ian Bicking wrote: > On Sun, Jul 11, 2010 at 3:38 PM, Ron Adam > wrote: > > There might be another alternative. > > Both idle and pydoc are applications (are there others?) that are > in the standard library. As such, they or parts of them, are > possibly importable to other projects. That restricts changes > because a committer needs to consider the chances that a change > may break something else. > > I suggest they be moved out of the lib directory, but still be > included with python. (Possibly in the tools directory.) That > removes some of the backward compatibility restrictions or at > least makes it clear there isn't a need for backward compatibility. > > > I also like this idea. This means Python comes with an IDE "out of he > box" but without the overhead of a management and release process that > is built for something very different than a GUI program (the standard > library). This would mean that IDLE would be in site-packages, could > easily be upgraded using normal tools, and maybe most importantly it > could have its own community tools and development process that is > more casual (and can more easily integrate new contributors) and > higher velocity of changes and releases. Python releases would then > ship the most recent stable release of IDLE. > IDLE itself is probably stable enough that being able to "upgrade in place" is not a high priority. That could change based on this thread of course. I would *really* support this approach with "pip" once distutils2 is complete and integrated into the standard library. I would really like Python to come with a capable package manager "out of the box" but understand your reluctance to tie pip to the Python release schedule. Having pip installed into site-packages by default, so that it *can* be upgraded in place, would be win-win as far as I can tell. Slight thread hijacking I realise... :-) All the best, Michael > -- > Ian Bicking | http://blog.ianbicking.org > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dickinsm at gmail.com Mon Jul 12 21:03:31 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 12 Jul 2010 20:03:31 +0100 Subject: [Python-Dev] A grammatical oddity: trailing commas in argument lists. In-Reply-To: References: Message-ID: On Sat, Jul 10, 2010 at 1:22 AM, Nick Coghlan wrote: > +1 for fixing it from me, unless any of the other implementations object. > > @Mark: my comment on the tracker issue had an implied "...unless you > really want to" on the end :) Thanks! Patch at http://bugs.python.org/issue9232 Mark From glyph at twistedmatrix.com Mon Jul 12 21:48:12 2010 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Mon, 12 Jul 2010 15:48:12 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> Message-ID: <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> On Jul 12, 2010, at 11:36 AM, Reid Kleckner wrote: > (Somwhat off-topic): Another pain point students had was accidentally > shadowing stdlib modules, like random. Renaming the file didn't solve > the problem either, because it left behind .pycs, which I had to help > them delete. I feel your pain. It seems like every third person who starts playing with Twisted starts off by making a file called 'twisted.py' and then getting really confused by the behavior. I would love it if this could be fixed, but I haven't yet thought of a solution that would be less confusing than the problem itself. -------------- next part -------------- An HTML attachment was scrubbed... URL: From phd at phd.pp.ru Mon Jul 12 22:12:01 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Tue, 13 Jul 2010 00:12:01 +0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> Message-ID: <20100712201201.GA21730@phd.pp.ru> On Mon, Jul 12, 2010 at 03:48:12PM -0400, Glyph Lefkowitz wrote: > I feel your pain. It seems like every third person who starts playing with Twisted starts off by making a file called 'twisted.py' and then getting really confused by the behavior. I would love it if this could be fixed, but I haven't yet thought of a solution that would be less confusing than the problem itself. Doesn't absolute import help? Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From tjreedy at udel.edu Mon Jul 12 23:01:42 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 12 Jul 2010 17:01:42 -0400 Subject: [Python-Dev] maintainers.rst enhancement In-Reply-To: <20100712114216.3032B21B0D5@kimball.webabinitio.net> References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> <4C3A103A.1070504@v.loewis.de> <4C3A5A23.2070903@v.loewis.de> <4C3AF0D3.5090303@gmail.com> <20100712114216.3032B21B0D5@kimball.webabinitio.net> Message-ID: On 7/12/2010 7:42 AM, R. David Murray wrote: Another 'enhancement' might be to have a program occasionally email people with the items they are currently signed up for, to encourage editing. > -- > R. David Murray www.bitdance.com -- Terry Jan Reedy From debatem1 at gmail.com Mon Jul 12 23:13:40 2010 From: debatem1 at gmail.com (geremy condra) Date: Mon, 12 Jul 2010 17:13:40 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <87zkxw1zsk.fsf@hydra.hampton.thirdcreek.com> References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <20100712121040.41f16654@pitrou.net> <87fwzo3nmt.fsf@hydra.hampton.thirdcreek.com> <87zkxw1zsk.fsf@hydra.hampton.thirdcreek.com> Message-ID: On Mon, Jul 12, 2010 at 11:32 AM, Kurt B. Kaiser wrote: > On Mon, Jul 12 2010, geremy condra wrote: > >> No offense, but I've been specifically asked not to do demos with IDLE >> because it looked 'unprofessional'. Given the constraint of working >> within tkinter that may not be something you can work around, but I'm >> sure you can see that from a certain perspective that's beside the >> point. > > Right, I stay away from working on Tk. ?It's not even a tkinter issue. > I hope it will continue to get better, but I imagine it's a moving > target for the Tk developers. > > In your case, just bring up Wing. ?That'll impress them. ?As long as > it's not on netbook :-) The point isn't to impress them. The point is to not embarrass us. I just do it in gedit and ipython now, and I use an eee 901 for all my presentations and a good bit of my work and it seems to be fine, which is not paying much of a complement to IDLE. Geremy Condra From ncoghlan at gmail.com Mon Jul 12 23:19:49 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 Jul 2010 07:19:49 +1000 Subject: [Python-Dev] More C API abstraction for user defined types In-Reply-To: References: Message-ID: On Tue, Jul 13, 2010 at 3:35 AM, Petre Galan wrote: > ival should not be resolved through PyLong_AsLong, but through > functionality/interface like PyNumber_Long, thus allowing more diversity in > accessing the PyArg_Parse interfaces. Sounds like a reasonable idea to me, but it needs to be a posted as a feature request on the tracker. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From martin at v.loewis.de Mon Jul 12 23:20:21 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 12 Jul 2010 23:20:21 +0200 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <217B30F6-06FE-4237-90C2-540CD6BB6A28@twistedmatrix.com> <4C3A48AE.9010707@v.loewis.de> Message-ID: <4C3B8715.9000605@v.loewis.de> Am 12.07.2010 13:01, schrieb Tal Einat: > On Mon, Jul 12, 2010 at 1:41 AM, "Martin v. L?wis" wrote: >>> My point is that I don't think I am exaggerating IDLE's flaws. I'm not >>> saying that it is no longer usable or useful, but I am saying that its >>> current state is not "okay". >> >> So can you produce a list of patches that you think can be accepted as-is? > > That's not a fair question! > > There have been several such patches, but most will likely need > revision since they have been sitting around untouched for so long. > And there would have been many more patches if the existing ones would > have been addressed. > > Getting a few current patches accepted is not the reason I posted here. Ok. Then I guess I cannot help further - I certainly don't support removal of IDLE from the standard library. Regards, Martin From tjreedy at udel.edu Mon Jul 12 23:21:22 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 12 Jul 2010 17:21:22 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <87oced2fsc.fsf@hydra.hampton.thirdcreek.com> References: <4C3A5983.3010002@holdenweb.com> <20100712080638.GM70600@nexus.in-nomine.org> <87oced2fsc.fsf@hydra.hampton.thirdcreek.com> Message-ID: On 7/12/2010 5:46 AM, Kurt B. Kaiser wrote: > On Windows, IDLE opens when you right click / edit a .py. Very useful. On my xp machine with 3.1.2, it edit .py opens with notepad. Perhaps the installer just copies forward the association from long ago, before IDLE was available, or at least so usable. I have thought of changing that, but I do not know what the replacement incantation would be. -- Terry Jan Reedy From fdrake at acm.org Mon Jul 12 23:33:49 2010 From: fdrake at acm.org (Fred Drake) Date: Mon, 12 Jul 2010 17:33:49 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <20100712201201.GA21730@phd.pp.ru> References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> <20100712201201.GA21730@phd.pp.ru> Message-ID: On Mon, Jul 12, 2010 at 4:12 PM, Oleg Broytman wrote: > ? Doesn't absolute import help? Not when both modules are at the top level; both acceptably provide the same name. The let's-play-with-it script just wasn't *intended* to be a module. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein From fuzzyman at voidspace.org.uk Mon Jul 12 23:42:50 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 12 Jul 2010 22:42:50 +0100 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> <20100712201201.GA21730@phd.pp.ru> Message-ID: <4C3B8C5A.3080308@voidspace.org.uk> On 12/07/2010 22:33, Fred Drake wrote: > On Mon, Jul 12, 2010 at 4:12 PM, Oleg Broytman wrote: > >> Doesn't absolute import help? >> > Not when both modules are at the top level; both acceptably provide > the same name. The let's-play-with-it script just wasn't *intended* > to be a module. > > > I'm sure Brett will love this idea, but if it was impossible to reimport the script being executed as __main__ with a different name it would solve these problems. Michael Foord > -Fred > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From brian at sweetapp.com Mon Jul 12 23:36:18 2010 From: brian at sweetapp.com (Brian Quinlan) Date: Tue, 13 Jul 2010 07:36:18 +1000 Subject: [Python-Dev] PEP 3148 ready for pronouncement [ACCEPTED] In-Reply-To: <20100712145909.GB3801@gmail.com> References: <20100712141919.GA3441@gmail.com> <20100712145909.GB3801@gmail.com> Message-ID: On 13 Jul 2010, at 00:59, Titus von der Malsburg wrote: > On Tue, Jul 13, 2010 at 12:48:35AM +1000, Nick Coghlan wrote: >> On Tue, Jul 13, 2010 at 12:19 AM, Titus von der Malsburg >> That's what actually happens, so you can code it either way > > That's great! None of the examples I found used the pythonic > exception style, that's why I assumed that checking the "return value" > is the only possibility. Reading the PEP carefully would have helped. > :-) I'd add that it would feel more natural to me to write: try: print('%r page is %d bytes' % (url, len(future.result()))) - except FutureError: - print('%r generated an exception: %s' % (url, future.exception())) + except FutureError as e: + print('%r generated an exception: %s' % (url, e)) Cheers, Brian From martin at v.loewis.de Mon Jul 12 23:43:44 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 12 Jul 2010 23:43:44 +0200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A5983.3010002@holdenweb.com> <20100712080638.GM70600@nexus.in-nomine.org> <87oced2fsc.fsf@hydra.hampton.thirdcreek.com> Message-ID: <4C3B8C90.7010604@v.loewis.de> Am 12.07.2010 23:21, schrieb Terry Reedy: > On 7/12/2010 5:46 AM, Kurt B. Kaiser wrote: > >> On Windows, IDLE opens when you right click / edit a .py. Very useful. > > On my xp machine with 3.1.2, it edit .py opens with notepad. Perhaps the > installer just copies forward the association from long ago, before IDLE > was available, or at least so usable. I have thought of changing that, > but I do not know what the replacement incantation would be. There should be an "Edit with IDLE" (sic) context menu item. Regards, Martin From fdrake at acm.org Mon Jul 12 23:47:31 2010 From: fdrake at acm.org (Fred Drake) Date: Mon, 12 Jul 2010 17:47:31 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <4C3B8C5A.3080308@voidspace.org.uk> References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> <20100712201201.GA21730@phd.pp.ru> <4C3B8C5A.3080308@voidspace.org.uk> Message-ID: On Mon, Jul 12, 2010 at 5:42 PM, Michael Foord wrote: > I'm sure Brett will love this idea, but if it was impossible to reimport the > script being executed as __main__ with a different name it would solve these > problems. Indeed! And I'd be quite content with such a solution, since I consider scripts and modules to be distinct. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein From martin at v.loewis.de Mon Jul 12 23:49:15 2010 From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 12 Jul 2010 23:49:15 +0200 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <4C3B533D.1080706@voidspace.org.uk> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> Message-ID: <4C3B8DDB.8070109@v.loewis.de> >> Not normally, no - there's no easy way to connect a checkin message to >> a committer's email address, > > There's a one-to-one mapping somewhere. Unfortunately, no: we don't have email addresses of all committers. Regards, Martin From fuzzyman at voidspace.org.uk Mon Jul 12 23:51:04 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 12 Jul 2010 22:51:04 +0100 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> <20100712201201.GA21730@phd.pp.ru> <4C3B8C5A.3080308@voidspace.org.uk> Message-ID: <4C3B8E48.8010805@voidspace.org.uk> On 12/07/2010 22:47, Fred Drake wrote: > On Mon, Jul 12, 2010 at 5:42 PM, Michael Foord > wrote: > >> I'm sure Brett will love this idea, but if it was impossible to reimport the >> script being executed as __main__ with a different name it would solve these >> problems. >> > Indeed! And I'd be quite content with such a solution, since I > consider scripts and modules to be distinct. > It seems to bite every newbie at some point. The change would have to go through the usual deprecation cycle I guess. All the best, Michael Forod > > -Fred > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From ncoghlan at gmail.com Mon Jul 12 23:52:46 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 Jul 2010 07:52:46 +1000 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <4C3B533D.1080706@voidspace.org.uk> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> Message-ID: On Tue, Jul 13, 2010 at 3:39 AM, Michael Foord wrote: >> Not normally, no - there's no easy way to connect a checkin message to >> a committer's email address, > > There's a one-to-one mapping somewhere. That "somewhere" isn't readily available when I hit reply to the checkin email though (for me, "somewhere" is generally my python-dev archive - I think the actual mapping file is a private one on the SVN server somewhere). >> so it's usually just a matter of hitting >> "Reply" and sending the review comment to the list. With a new >> committer I'll make the effort to cc them directly in case they aren't >> subscribed yet, but I expect everyone else to be monitor the checkins >> list. >> > > Without contacting committers, rather than firing into the black depths of a > high traffic list, I strongly suspect that much of the feedback is lost. Not that I've noticed. If someone doesn't respond to a comment I've made on python-checkins I'll follow up with them directly and point out that monitoring python-checkins for responses after making a commit is an obligation of making use of commit privileges, just as much as of one as running (at least relevant parts of) the test suite before committing and then checking the buildbots to see you haven't broken anything on other platforms. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From benjamin at python.org Mon Jul 12 23:57:05 2010 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 12 Jul 2010 16:57:05 -0500 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <4C3B8DDB.8070109@v.loewis.de> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B8DDB.8070109@v.loewis.de> Message-ID: 2010/7/12 "Martin v. L?wis" : >>> Not normally, no - there's no easy way to connect a checkin message to >>> a committer's email address, >> >> There's a one-to-one mapping somewhere. > > Unfortunately, no: we don't have email addresses of all committers. What about the python-committers mailing list? That has at least all the active ones, correct? -- Regards, Benjamin From solipsis at pitrou.net Mon Jul 12 23:59:20 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 12 Jul 2010 23:59:20 +0200 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> <20100712201201.GA21730@phd.pp.ru> <4C3B8C5A.3080308@voidspace.org.uk> Message-ID: <20100712235920.146ffb37@pitrou.net> On Mon, 12 Jul 2010 17:47:31 -0400 Fred Drake wrote: > On Mon, Jul 12, 2010 at 5:42 PM, Michael Foord > wrote: > > I'm sure Brett will love this idea, but if it was impossible to reimport the > > script being executed as __main__ with a different name it would solve these > > problems. > > Indeed! And I'd be quite content with such a solution, since I > consider scripts and modules to be distinct. Except that modules can often be executed as scripts... Antoine. From tjreedy at udel.edu Tue Jul 13 00:00:09 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 12 Jul 2010 18:00:09 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <4C3B8C90.7010604@v.loewis.de> References: <4C3A5983.3010002@holdenweb.com> <20100712080638.GM70600@nexus.in-nomine.org> <87oced2fsc.fsf@hydra.hampton.thirdcreek.com> <4C3B8C90.7010604@v.loewis.de> Message-ID: On 7/12/2010 5:43 PM, "Martin v. L?wis" wrote: > Am 12.07.2010 23:21, schrieb Terry Reedy: >> On 7/12/2010 5:46 AM, Kurt B. Kaiser wrote: >> >>> On Windows, IDLE opens when you right click / edit a .py. Very useful. >> >> On my xp machine with 3.1.2, it edit .py opens with notepad. Perhaps the >> installer just copies forward the association from long ago, before IDLE >> was available, or at least so usable. I have thought of changing that, >> but I do not know what the replacement incantation would be. > > There should be an "Edit with IDLE" (sic) context menu item. I agree, and thought about requesting such, but there is not and never has been for me that I know of. Actually, I would like Edit with IDLE x.y and Run with x.y so it is easy to test a file with different versions. XP with updates, install for everyone, make install the default Python. Should I open a tracker issue? -- Terry Jan Reedy From fuzzyman at voidspace.org.uk Tue Jul 13 00:04:30 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 12 Jul 2010 23:04:30 +0100 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> Message-ID: <4C3B916E.7050801@voidspace.org.uk> On 12/07/2010 22:52, Nick Coghlan wrote: > [snip...] >>> so it's usually just a matter of hitting >>> "Reply" and sending the review comment to the list. With a new >>> committer I'll make the effort to cc them directly in case they aren't >>> subscribed yet, but I expect everyone else to be monitor the checkins >>> list. >>> >>> >> Without contacting committers, rather than firing into the black depths of a >> high traffic list, I strongly suspect that much of the feedback is lost. >> > Not that I've noticed. If someone doesn't respond to a comment I've > made on python-checkins I'll follow up with them directly and point > out that monitoring python-checkins for responses after making a > commit is an obligation of making use of commit privileges, just as > much as of one as running (at least relevant parts of) the test suite > before committing and then checking the buildbots to see you haven't > broken anything on other platforms. > > Given how high traffic python-checkins is I don't consider that a reasonable place to send follow-up and nor do I consider it the responsibility of committers to monitor it. As you said earlier this *isn't* in our standard dev procedures and nor do I think it should be. If you can't find an email address then either python-comitters or python-dev would be a better place to send feedback. Michael > Cheers, > Nick. > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From ncoghlan at gmail.com Tue Jul 13 00:05:24 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 Jul 2010 08:05:24 +1000 Subject: [Python-Dev] avoiding accidental shadowing of top-level libraries by the main module Message-ID: On Tue, Jul 13, 2010 at 7:47 AM, Fred Drake wrote: > On Mon, Jul 12, 2010 at 5:42 PM, Michael Foord > wrote: >> I'm sure Brett will love this idea, but if it was impossible to reimport the >> script being executed as __main__ with a different name it would solve these >> problems. > > Indeed! ?And I'd be quite content with such a solution, since I > consider scripts and modules to be distinct. And here I've been busily blurring that distinction for years ;) (actually, the whole "name == '__main__'" idiom meant the distinction was already pretty blurry long before I got involved) I take it the concrete proposal here is if the filename of a new module matches either __main__.__file__ or __main__.__cached__, then that module should be ignored completely for import purposes allowing a module with the same name later on sys.path to be found? I'm not sure I like that, I'd be more inclined to just return the __main__ module in that case rather than letting the search continue further down sys.path (although I agree the current semantics of getting two copies of the same module under different names in this case are less than ideal). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From fuzzyman at voidspace.org.uk Tue Jul 13 00:06:49 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 12 Jul 2010 23:06:49 +0100 Subject: [Python-Dev] avoiding accidental shadowing of top-level libraries by the main module In-Reply-To: References: Message-ID: <4C3B91F9.7060906@voidspace.org.uk> On 12/07/2010 23:05, Nick Coghlan wrote: > On Tue, Jul 13, 2010 at 7:47 AM, Fred Drake wrote: > >> On Mon, Jul 12, 2010 at 5:42 PM, Michael Foord >> wrote: >> >>> I'm sure Brett will love this idea, but if it was impossible to reimport the >>> script being executed as __main__ with a different name it would solve these >>> problems. >>> >> Indeed! And I'd be quite content with such a solution, since I >> consider scripts and modules to be distinct. >> > And here I've been busily blurring that distinction for years ;) > > (actually, the whole "name == '__main__'" idiom meant the distinction > was already pretty blurry long before I got involved) > > I take it the concrete proposal here is if the filename of a new > module matches either __main__.__file__ or __main__.__cached__, then > that module should be ignored completely for import purposes allowing > a module with the same name later on sys.path to be found? > An explicit error being raised instead would be just as good. Michael > I'm not sure I like that, I'd be more inclined to just return the > __main__ module in that case rather than letting the search continue > further down sys.path (although I agree the current semantics of > getting two copies of the same module under different names in this > case are less than ideal). > > Cheers, > Nick. > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From martin at v.loewis.de Tue Jul 13 00:11:02 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Tue, 13 Jul 2010 00:11:02 +0200 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B8DDB.8070109@v.loewis.de> Message-ID: <4C3B92F6.4020807@v.loewis.de> Am 12.07.2010 23:57, schrieb Benjamin Peterson: > 2010/7/12 "Martin v. L?wis" : >>>> Not normally, no - there's no easy way to connect a checkin message to >>>> a committer's email address, >>> >>> There's a one-to-one mapping somewhere. >> >> Unfortunately, no: we don't have email addresses of all committers. > > What about the python-committers mailing list? That has at least all > the active ones, correct? Probably. Also, Dirkjan is maintaining a list for the Mercurial migration. So it is probably the case that one could be constructed if desired. It's just that "we" (the people maintaining the subversion access control) don't have such a list. Regards, Martin From ncoghlan at gmail.com Tue Jul 13 00:13:01 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 Jul 2010 08:13:01 +1000 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <4C3B916E.7050801@voidspace.org.uk> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B916E.7050801@voidspace.org.uk> Message-ID: On Tue, Jul 13, 2010 at 8:04 AM, Michael Foord wrote: > Given how high traffic python-checkins is I don't consider that a reasonable > place to send follow-up and nor do I consider it the responsibility of > committers to monitor it. As you said earlier this *isn't* in our standard > dev procedures and nor do I think it should be. If you can't find an email > address then either python-comitters or python-dev would be a better place > to send feedback. Umm, yeah it is - Brett's Intro to Development *is* the documentation of the workflow procedures, and it says that committers are expected to subscribe to both python-checkins and python-committers. And, as I said, I've been working this way for years, and haven't seen any of my comments or anyone else's made on python-checkins go unaddressed (although we've occasionally had to follow up by finding the committer's email address and adding it directly, that's fairly rare). It *really* isn't very hard to ignore most of the traffic on that list and just look at your own commits (and any responses). (I don't do that myself unless I'm busy, as I quite like doing after the fact reviews of commits) Bringing the whole of python-dev into a python-checkins discussion is only necessary if there are any actual disagreements regarding a commit (99% of what we spot in post review is just typos in documentation, comments and text strings, as well as minor things like poor choices of internal variable names). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Tue Jul 13 00:23:04 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 Jul 2010 08:23:04 +1000 Subject: [Python-Dev] avoiding accidental shadowing of top-level libraries by the main module In-Reply-To: <4C3B91F9.7060906@voidspace.org.uk> References: <4C3B91F9.7060906@voidspace.org.uk> Message-ID: On Tue, Jul 13, 2010 at 8:06 AM, Michael Foord wrote: > An explicit error being raised instead would be just as good. Ah, refusing the temptation to guess. So the idea would be, when attempting to import __main__ under it's original name: 3.2 issue a DeprecationWarning 3.3 raise ImportError It still strikes me as wrong for the base import implementation, but PEP 302 may provide a mechanism for an interpreter shell to add its own "newbie protection" hooks (and that would have the virtue of being available much earlier and with a much lower chance of breaking anything). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From eric at trueblade.com Tue Jul 13 00:29:20 2010 From: eric at trueblade.com (Eric Smith) Date: Mon, 12 Jul 2010 18:29:20 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B8DDB.8070109@v.loewis.de> Message-ID: <4C3B9740.9020906@trueblade.com> On 7/12/2010 5:57 PM, Benjamin Peterson wrote: > 2010/7/12 "Martin v. L?wis": >>>> Not normally, no - there's no easy way to connect a checkin message to >>>> a committer's email address, >>> >>> There's a one-to-one mapping somewhere. >> >> Unfortunately, no: we don't have email addresses of all committers. > > What about the python-committers mailing list? That has at least all > the active ones, correct? I put some effort into this a while ago and failed. Not all committers are subscribed, and for those that are we don't have a way of mapping them to names. Ideally I'd like to see a table with tracker id, username, email addresses, real name. But without a way of automatically keeping it up-to-date, I don't think it will happen. Eric. From john.arbash.meinel at gmail.com Tue Jul 13 00:26:03 2010 From: john.arbash.meinel at gmail.com (John Arbash Meinel) Date: Mon, 12 Jul 2010 17:26:03 -0500 Subject: [Python-Dev] Intended behavior of backlash in raw strings Message-ID: <4C3B967B.1030604@gmail.com> I'm trying to determine if this is intended behavior: >>> r"\"" '\\"' >>> r'\'' "\\'" Normally, the quote would end the string, but it gets escaped by the preceding '\'. However, the preceding slash is interpreted as 'not a backslash' because of the raw indicator, so it gets left in verbatim. Note that it works anywhere: >>> r"testing \" backslash and quote" 'testing \\" backslash and quote' It happens that this is the behavior I want, but it seemed just as likely to be an error. I tested it with python2.5 and 2.6 and got the same results. Is this something I can count on? Or is it undefined behavior and I should really not be doing it? John =:-> From fuzzyman at voidspace.org.uk Tue Jul 13 00:27:48 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 12 Jul 2010 23:27:48 +0100 Subject: [Python-Dev] avoiding accidental shadowing of top-level libraries by the main module In-Reply-To: <20100712235920.146ffb37@pitrou.net> References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> <20100712201201.GA21730@phd.pp.ru> <4C3B8C5A.3080308@voidspace.org.uk> <20100712235920.146ffb37@pitrou.net> Message-ID: <4C3B96E4.5010507@voidspace.org.uk> On 12/07/2010 22:59, Antoine Pitrou wrote: > On Mon, 12 Jul 2010 17:47:31 -0400 > Fred Drake wrote: > > >> On Mon, Jul 12, 2010 at 5:42 PM, Michael Foord >> wrote: >> >>> I'm sure Brett will love this idea, but if it was impossible to reimport the >>> script being executed as __main__ with a different name it would solve these >>> problems. >>> >> Indeed! And I'd be quite content with such a solution, since I >> consider scripts and modules to be distinct. >> > Except that modules can often be executed as scripts... > Allowing a module-executed-as-script to be in sys.modules twice with different names and classes is still a recipe for problems. For example see this blog entry from Phil Hassey, an experienced Python developer, caused by this issue: http://www.philhassey.com/blog/2009/07/16/oddball-python-252-import-issue/ For scripts that need to be in sys.modules with their "real" name (e.g. modules executed with python -m) the following trivial workaround is possible: if __name__ == '__main__': sys.modules[real_name] = sys.modules['__main__'] That avoids the problem of error messages like: TypeError: unbound method foo() must be called with A instance as first argument (got A instance instead) (Two different classes with the same name created - one from __main__ and one from real_name.) Use cases for *genuinely* reimporting the same module with different names (as different module objects rather than aliases) are relatively rare, and the problem of modules *accidentally* reimporting themselves not that rare. All the best, Michael > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From fuzzyman at voidspace.org.uk Tue Jul 13 00:29:49 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 12 Jul 2010 23:29:49 +0100 Subject: [Python-Dev] avoiding accidental shadowing of top-level libraries by the main module In-Reply-To: References: <4C3B91F9.7060906@voidspace.org.uk> Message-ID: <4C3B975D.5020501@voidspace.org.uk> On 12/07/2010 23:23, Nick Coghlan wrote: > On Tue, Jul 13, 2010 at 8:06 AM, Michael Foord > wrote: > >> An explicit error being raised instead would be just as good. >> > Ah, refusing the temptation to guess. > > So the idea would be, when attempting to import __main__ under it's > original name: > 3.2 issue a DeprecationWarning > 3.3 raise ImportError > > That's what I was suggesting. > It still strikes me as wrong for the base import implementation, but > PEP 302 may provide a mechanism for an interpreter shell to add its > own "newbie protection" hooks (and that would have the virtue of being > available much earlier and with a much lower chance of breaking > anything). > That would be a nice thing to have for earlier versions of Python. Personally I think the confusion the problem it causes would be nicer fixed. All the best, Michael > Cheers, > Nick. > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From fuzzyman at voidspace.org.uk Tue Jul 13 00:31:14 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 12 Jul 2010 23:31:14 +0100 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A5983.3010002@holdenweb.com> <20100712080638.GM70600@nexus.in-nomine.org> <87oced2fsc.fsf@hydra.hampton.thirdcreek.com> <4C3B8C90.7010604@v.loewis.de> Message-ID: <4C3B97B2.2000505@voidspace.org.uk> On 12/07/2010 23:00, Terry Reedy wrote: > On 7/12/2010 5:43 PM, "Martin v. L?wis" wrote: >> Am 12.07.2010 23:21, schrieb Terry Reedy: >>> On 7/12/2010 5:46 AM, Kurt B. Kaiser wrote: >>> >>>> On Windows, IDLE opens when you right click / edit a .py. Very useful. >>> >>> On my xp machine with 3.1.2, it edit .py opens with notepad. Perhaps >>> the >>> installer just copies forward the association from long ago, before >>> IDLE >>> was available, or at least so usable. I have thought of changing that, >>> but I do not know what the replacement incantation would be. >> >> There should be an "Edit with IDLE" (sic) context menu item. > > I agree, and thought about requesting such, but there is not and never > has been for me that I know of. Actually, I would like Edit with IDLE > x.y and Run with x.y so it is easy to test a file with different > versions. > I have "Edit with IDLE" as a context menu option for Windows 7 - so it does exist *sometimes*. Either an XP issue or a "your system" issue I guess. Michael > XP with updates, install for everyone, make install the default Python. > > Should I open a tracker issue? > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From p.f.moore at gmail.com Tue Jul 13 00:35:04 2010 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 12 Jul 2010 23:35:04 +0100 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A5983.3010002@holdenweb.com> <20100712080638.GM70600@nexus.in-nomine.org> <87oced2fsc.fsf@hydra.hampton.thirdcreek.com> <4C3B8C90.7010604@v.loewis.de> Message-ID: On 12 July 2010 23:00, Terry Reedy wrote: > On 7/12/2010 5:43 PM, "Martin v. L?wis" wrote: >> >> There should be an "Edit with IDLE" (sic) context menu item. > > I agree, and thought about requesting such, but there is not and never has > been for me that I know of. There is for me. I think what Martin meant was that the standard install puts one there, and if you don't have one it looks like your install is broken somehow. > Actually, I would like Edit with IDLE x.y and > Run with x.y so it is easy to test a file with different versions. I'm not sure I would, too much menu clutter. But as an install-time option I'd be OK with it. > XP with updates, install for everyone, make install the default Python. > > Should I open a tracker issue? If you can reproduce it, I guess so. But as I say, I have the entry (Win7, 2.6.3, install for everyone, default Python). So it's not a universal issue. Paul. From kbk at shore.net Tue Jul 13 00:36:51 2010 From: kbk at shore.net (Kurt B. Kaiser) Date: Mon, 12 Jul 2010 18:36:51 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: (Tal Einat's message of "Mon, 12 Jul 2010 19:29:38 +0300") References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A493E.8090308@v.loewis.de> <878w5g3fc4.fsf@hydra.hampton.thirdcreek.com> Message-ID: <87eif81g58.fsf@hydra.hampton.thirdcreek.com> On Mon, Jul 12 2010, Tal Einat wrote: > I have been maintaining my own fork of IDLE for several years and > manually keeping it in sync with IDLE (this was simple). The > difference is that there was no single major new feature I was working > on, such as the addition of a sub-process in IDLE-fork or Noam's > addition of auto-completion. I was mostly making relatively minor > fixes and changes which were not interrelated. I saw no reason to have > them all merged back at once, so I posted patches as soon as I felt > they were ready, and did the best I could to get them accepted. I > eventually gave up on this process because every patch took far too > long to be addressed and finally accepted or rejected, and I realized > that most of the work I had done would never be merged back into the > mainstream version of IDLE. There were several contributing factors. I decided to stop committing new features in 2.7 and focus on bugs for several reasons. First, IDLE3 needed work to get it running smoothly. Second, committing, forward porting and running the (manual) functional tests on a bunch of small features was a bit of a pain. Third, leaving the new features to IDLE3 was a draw to get people to use the new version. Then, about two years ago, I got buried with PSF/PyCon issues. If you'll look back in the IDLE NEWS, you'll see I was giving your patches quite a bit of attention. So never say never. -- KBK From glyph at twistedmatrix.com Tue Jul 13 00:44:03 2010 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Mon, 12 Jul 2010 18:44:03 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> <20100712201201.GA21730@phd.pp.ru> <4C3B8C5A.3080308@voidspace.org.uk> Message-ID: <22A7F10B-4E4D-44C2-B4F5-0F0EB87578DC@twistedmatrix.com> On Jul 12, 2010, at 5:47 PM, Fred Drake wrote: > On Mon, Jul 12, 2010 at 5:42 PM, Michael Foord > wrote: >> I'm sure Brett will love this idea, but if it was impossible to reimport the >> script being executed as __main__ with a different name it would solve these >> problems. > > Indeed! And I'd be quite content with such a solution, since I > consider scripts and modules to be distinct. but ... isn't the whole point of 'python -m' to make scripts and modules _not_ be distinct? -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric at trueblade.com Tue Jul 13 00:48:04 2010 From: eric at trueblade.com (Eric Smith) Date: Mon, 12 Jul 2010 18:48:04 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <4C3B916E.7050801@voidspace.org.uk> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B916E.7050801@voidspace.org.uk> Message-ID: <4C3B9BA4.7050500@trueblade.com> On 7/12/2010 6:04 PM, Michael Foord wrote: > Given how high traffic python-checkins is I don't consider that a > reasonable place to send follow-up and nor do I consider it the > responsibility of committers to monitor it. As you said earlier this > *isn't* in our standard dev procedures and nor do I think it should be. > If you can't find an email address then either python-comitters or > python-dev would be a better place to send feedback. Maybe reply-to on the checkin messages could be set to python-dev. Not sure if that's a mailman feature, though. Eric. From martin at v.loewis.de Tue Jul 13 00:50:35 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 13 Jul 2010 00:50:35 +0200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A5983.3010002@holdenweb.com> <20100712080638.GM70600@nexus.in-nomine.org> <87oced2fsc.fsf@hydra.hampton.thirdcreek.com> <4C3B8C90.7010604@v.loewis.de> Message-ID: <4C3B9C3B.9080801@v.loewis.de> Am 13.07.2010 00:00, schrieb Terry Reedy: > On 7/12/2010 5:43 PM, "Martin v. L?wis" wrote: >> Am 12.07.2010 23:21, schrieb Terry Reedy: >>> On 7/12/2010 5:46 AM, Kurt B. Kaiser wrote: >>> >>>> On Windows, IDLE opens when you right click / edit a .py. Very useful. >>> >>> On my xp machine with 3.1.2, it edit .py opens with notepad. Perhaps the >>> installer just copies forward the association from long ago, before IDLE >>> was available, or at least so usable. I have thought of changing that, >>> but I do not know what the replacement incantation would be. >> >> There should be an "Edit with IDLE" (sic) context menu item. > > I agree, and thought about requesting such You misunderstand. There should be such a menu item *today*, *on your machine*. If it's not there, either the installation procedure went wrong, or you opted out of having it. > Should I open a tracker issue? Please, no. Regards, Martin From rrr at ronadam.com Tue Jul 13 01:01:31 2010 From: rrr at ronadam.com (Ron Adam) Date: Mon, 12 Jul 2010 18:01:31 -0500 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: On 07/12/2010 01:21 PM, Ian Bicking wrote: > On Sun, Jul 11, 2010 at 3:38 PM, Ron Adam > wrote: > > There might be another alternative. > > Both idle and pydoc are applications (are there others?) that are in > the standard library. As such, they or parts of them, are possibly > importable to other projects. That restricts changes because a > committer needs to consider the chances that a change may break > something else. > > I suggest they be moved out of the lib directory, but still be > included with python. (Possibly in the tools directory.) That > removes some of the backward compatibility restrictions or at least > makes it clear there isn't a need for backward compatibility. > > > I also like this idea. This means Python comes with an IDE "out of he > box" but without the overhead of a management and release process that > is built for something very different than a GUI program (the standard > library). This would mean that IDLE would be in site-packages, could > easily be upgraded using normal tools, and maybe most importantly it > could have its own community tools and development process that is more > casual (and can more easily integrate new contributors) and higher > velocity of changes and releases. Python releases would then ship the > most recent stable release of IDLE. Yes, if you follow the guide lines for the rest of the library, anything that is removed needs to be depreciated first and anything thats added needs to be carefully looked at to be sure it doesn't break anything that may depend on it. That is good for the rest of the standard library but really slows things down for an application like idle. Just removing those restrictions would make things a lot simpler and speed things up considerably I think. The site-packages directory is still in the lib path and so things there are still importable. That is why I suggested the tools directory. Another place would be in the same directory the python executable lives. But the exact location isn't really the important thing, having a clear policy on how the upgrade policy differs from the python library is what is needed. Ron Ron From tjreedy at udel.edu Tue Jul 13 01:08:07 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 12 Jul 2010 19:08:07 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <4C3AB092.8030902@v.loewis.de> References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> <4C3A103A.1070504@v.loewis.de> <4C3AB092.8030902@v.loewis.de> Message-ID: On 7/12/2010 2:05 AM, "Martin v. L?wis" wrote: >> What I specifically want right now is Commit Authorization Privilege, >> especially for IDLE, > > Not sure who could grant that, but as far as I can: you have it. If I were approved to commit patches directly, then by implication I should be able to approve others doing the same. That is occasionally done now by others, but I wanted to be clear that for the present, the latter is all I could and would do. -- Terry Jan Reedy From solipsis at pitrou.net Tue Jul 13 01:08:31 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 13 Jul 2010 01:08:31 +0200 Subject: [Python-Dev] python-checkins In-Reply-To: <4C3B916E.7050801@voidspace.org.uk> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B916E.7050801@voidspace.org.uk> Message-ID: <1278976111.3134.7.camel@localhost.localdomain> Le lundi 12 juillet 2010 ? 23:04 +0100, Michael Foord a ?crit : > Given how high traffic python-checkins is I don't consider that a > reasonable place to send follow-up and nor do I consider it the > responsibility of committers to monitor it. You don't have to receive e-mail from it. Just take a look at the archives from time to time after you have done some commits. In a threaded view, it's easy to spot the few messages which aren't commit notifications, since they are the only one not at the top-level. Regatds Antoine. From collinwinter at google.com Tue Jul 13 01:11:14 2010 From: collinwinter at google.com (Collin Winter) Date: Mon, 12 Jul 2010 16:11:14 -0700 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: <4C3B324E.2050005@voidspace.org.uk> References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3B324E.2050005@voidspace.org.uk> Message-ID: On Mon, Jul 12, 2010 at 8:18 AM, Michael Foord wrote: > On 12/07/2010 15:07, Nick Coghlan wrote: >> >> On Mon, Jul 12, 2010 at 9:42 AM, Steven D'Aprano >> ?wrote: >> >>> >>> On Sun, 11 Jul 2010 09:37:22 pm Eric Smith wrote: >>> >>>>> >>>>> re2 comparison is interesting from the point of if it should be >>>>> included in stdlib. >>>>> >>>> >>>> Is "it" re2 or regex? I don't see having 2 regular expression engines >>>> in the stdlib. >>>> >>> >>> There's precedence though... the old regex engine and the new re engine >>> were side-by-side for many years before regex was deprecated and >>> finally removed in 2.5. Hypothetically, re2 could similarly be added to >>> the standard library while re is deprecated. >>> >> >> re2 deliberately omits some features for efficiency reasons, hence is >> not even on the table as a possible replacement for the standard >> library version. If someone is in a position where re2 can solve their >> problems with the re module, they should also be in a position where >> they can track it down for themselves. >> >> > > If it has *partial* compatibility, and big enough performance improvements > for common cases, it could perhaps be used where the regex doesn't use > unsupported features. This would have some extra cost in the compile phase, > but would mean Python could ship with two regex engines but only one > interface exposed to the programmer... FWIW, this has all been discussed before: http://aspn.activestate.com/ASPN/Mail/Message/python-dev/3829265. In particular, I still believe that, "it's not obvious that enough Python regexes would benefit from re2's performance/restrictions tradeoff to make such a hybrid system worthwhile in the long term. (There is no representative corpus of real-world Python regexes weighted for dynamic execution frequency to use in assessing such tradeoffs empirically like there is for JavaScript.)" Collin >> MRAB's module offers a superset of re's features rather than a subset >> though, so once it has had more of a chance to bake on PyPI it may be >> worth another look. >> >> Cheers, >> Nick. >> >> > > > -- > http://www.ironpythoninaction.com/ > http://www.voidspace.org.uk/blog > > READ CAREFULLY. By accepting and reading this email you agree, on behalf of > your employer, to release me from all obligations and waivers arising from > any and all NON-NEGOTIATED agreements, licenses, terms-of-service, > shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, > non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have > entered into with your employer, its partners, licensors, agents and > assigns, in perpetuity, without prejudice to my ongoing rights and > privileges. You further represent that you have the authority to release me > from any BOGUS AGREEMENTS on behalf of your employer. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/collinwinter%40google.com > From nyamatongwe at gmail.com Tue Jul 13 01:37:32 2010 From: nyamatongwe at gmail.com (Neil Hodgson) Date: Tue, 13 Jul 2010 09:37:32 +1000 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <87vd8k1ydv.fsf@hydra.hampton.thirdcreek.com> References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <87vd8k1ydv.fsf@hydra.hampton.thirdcreek.com> Message-ID: Kurt B. Kaiser: >> The tear off menus are ugly as well as being non-standard on all three >> major platforms. > > Well, would you discard them? They can (occasionally) be useful. Yes, I would replace the menus with ones missing the tear line. Most of the GUI toolkits experimented with tear-offs (Mac in late 80s, GTK+ up until 2002) and dropped them or hid them in a rarely visited API. The idea initially appeared reasonable ("I can have the Run and Check commands available with a single click") but was found to be too confusing in use. IDLE, because it uses a separate top-level window for each file and shell suffers more than most applications. A menu is torn off from one window and always applies to that window but shows no visual affinity with that window: its window is not even activated when a menu command acts on it. Neil From fuzzyman at voidspace.org.uk Tue Jul 13 01:42:31 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 13 Jul 2010 00:42:31 +0100 Subject: [Python-Dev] python-checkins replies In-Reply-To: <4C3B9BA4.7050500@trueblade.com> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B916E.7050801@voidspace.org.uk> <4C3B9BA4.7050500@trueblade.com> Message-ID: <4C3BA867.6050906@voidspace.org.uk> On 12/07/2010 23:48, Eric Smith wrote: > On 7/12/2010 6:04 PM, Michael Foord wrote: >> Given how high traffic python-checkins is I don't consider that a >> reasonable place to send follow-up and nor do I consider it the >> responsibility of committers to monitor it. As you said earlier this >> *isn't* in our standard dev procedures and nor do I think it should be. >> If you can't find an email address then either python-comitters or >> python-dev would be a better place to send feedback. > > Maybe reply-to on the checkin messages could be set to python-dev. Not > sure if that's a mailman feature, though. I think this would be a good idea. It would be nice to have on-topic traffic here. :-) Michael > > Eric. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From greg.ewing at canterbury.ac.nz Tue Jul 13 02:25:28 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 13 Jul 2010 12:25:28 +1200 Subject: [Python-Dev] PEP 3148 ready for pronouncement [ACCEPTED] In-Reply-To: <20100712145909.GB3801@gmail.com> References: <20100712141919.GA3441@gmail.com> <20100712145909.GB3801@gmail.com> Message-ID: <4C3BB278.5010206@canterbury.ac.nz> Titus von der Malsburg wrote: > None of the examples I found used the pythonic > exception style, that's why I assumed that checking the "return value" > is the only possibility. Reading the PEP carefully would have helped. > :-) I had to read the pep fairly carefully before I noticed this too, so perhaps it could be made more prominent. -- Greg From steve at pearwood.info Tue Jul 13 02:30:30 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 13 Jul 2010 10:30:30 +1000 Subject: [Python-Dev] avoiding accidental shadowing of top-level libraries by the main module In-Reply-To: References: Message-ID: <201007131030.31035.steve@pearwood.info> On Tue, 13 Jul 2010 08:05:24 am Nick Coghlan wrote: > On Tue, Jul 13, 2010 at 7:47 AM, Fred Drake wrote: > > On Mon, Jul 12, 2010 at 5:42 PM, Michael Foord > > > > wrote: > >> I'm sure Brett will love this idea, but if it was impossible to > >> reimport the script being executed as __main__ with a different > >> name it would solve these problems. > > > > Indeed! ?And I'd be quite content with such a solution, since I > > consider scripts and modules to be distinct. > > And here I've been busily blurring that distinction for years ;) > > (actually, the whole "name == '__main__'" idiom meant the distinction > was already pretty blurry long before I got involved) I would hate it if that distinction was un-blurred. Most of my modules include a section "if __name__ == '__main__': run_tests(), and some of them do significantly more than that. A few of them import themselves so they can pass the module object to another module. > I take it the concrete proposal here is if the filename of a new > module matches either __main__.__file__ or __main__.__cached__, then > that module should be ignored completely for import purposes allowing > a module with the same name later on sys.path to be found? > > I'm not sure I like that, I'd be more inclined to just return the > __main__ module in that case rather than letting the search continue > further down sys.path (although I agree the current semantics of > getting two copies of the same module under different names in this > case are less than ideal). Yes, that's a weird corner case. I don't see any advantage to keeping that behaviour. -- Steven D'Aprano From alexander.belopolsky at gmail.com Tue Jul 13 02:37:14 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 12 Jul 2010 20:37:14 -0400 Subject: [Python-Dev] How to block a module import Message-ID: I thought that in order to block a module from being imported, one would need to assign None to the corresponding entry in sys.modules. However, it looks like the code in test.support uses 0 instead of None: def _save_and_block_module(name, orig_modules): """Helper function to save and block a module in sys.modules Return value is True if the module was in sys.modules and False otherwise.""" saved = True try: orig_modules[name] = sys.modules[name] except KeyError: saved = False sys.modules[name] = 0 # <--- I would expect None here return saved In my experiments, 0 is not equivalent to None: >>> import sys, time >>> sys.modules['time'] = 0 >>> __import__('time') 0 >>> sys.modules['time'] = None >>> __import__('time') Traceback (most recent call last): File "", line 1, in ImportError: No module named time Am I missing something? From greg.ewing at canterbury.ac.nz Tue Jul 13 03:00:18 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 13 Jul 2010 13:00:18 +1200 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> <20100712201201.GA21730@phd.pp.ru> Message-ID: <4C3BBAA2.6000901@canterbury.ac.nz> Fred Drake wrote: > Not when both modules are at the top level; both acceptably provide > the same name. The let's-play-with-it script just wasn't *intended* > to be a module. I wonder whether this kind of problem would be less prevalent if the tutorials etc. encouraged naming top-level scripts with some extension *other* than .py? -- Greg From stephen at xemacs.org Tue Jul 13 03:05:16 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 13 Jul 2010 10:05:16 +0900 Subject: [Python-Dev] python-checkins In-Reply-To: <1278976111.3134.7.camel@localhost.localdomain> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B916E.7050801@voidspace.org.uk> <1278976111.3134.7.camel@localhost.localdomain> Message-ID: <87k4p0i437.fsf@uwakimon.sk.tsukuba.ac.jp> Antoine Pitrou writes: > You don't have to receive e-mail from it. Just take a look at the > archives from time to time after you have done some commits. > In a threaded view, it's easy to spot the few messages which aren't > commit notifications, since they are the only one not at the > top-level. It also is possible to arrange that the author (From field) of the commit message is the author of the commit, rather than the bot. Reply-To could be set to bot plus author. Once the Mercurial transition is done, this would be more useful as "everybody" would have an email address as part of their committer ID. I realize that there is probably good reason for the current approach (including but not limited to lack of real email addresses for some committers), but if people really find it that great an imposition to filter messages themselves with procmail or MUA features, this would make the author view of the archives useful to them. (Although that destroys threading in the summary, you still have links to followups in the message view.) From kbk at shore.net Tue Jul 13 03:34:30 2010 From: kbk at shore.net (Kurt B. Kaiser) Date: Mon, 12 Jul 2010 21:34:30 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library; Scherer Agreement In-Reply-To: (Bruce Sherwood's message of "Mon, 12 Jul 2010 09:12:51 -0600") References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> Message-ID: <878w5g17x5.fsf_-_@hydra.hampton.thirdcreek.com> On Mon, Jul 12 2010, Bruce Sherwood wrote: > I don't recall that VIDLE has anything to do with running multiple IDLE > copies. Well, I stole the ephemeral port idea from him! > What's in VIDLE is a lot of bug fixes and some improvements. For > example, you can configure it to not require having to save a file, > which makes it as easy to use for quick testing as the shell. Another > improvement, of high importance for our physics students and other > novice programmers, is to bring an error display forward. Many > students fill the screen with the edit window and the error message > could be hidden, leaving them puzzled as to why their program wasn't > running. Stuff like that, all aimed at issues we had seen with a large > population of science and engineering students. > > I'll say again that IDLE/VIDLE is an excellent environment for novices. > Issues identified here about look and feel etc. have just not been of any > importance in that context. > I'd like to get all that incorporated so you can go back to dealing directly with us. Your students are the people we want to reach! > If you'll send me what you want from Scherer, I can contact him. I'm copying our PSF Secretary/Administrator, Pat Campbell. She can work with you to get David Scherer's Contributor Agreement. We really appreciate your assistance with this, he's one of the last missing agreements. -- KBK From steve at holdenweb.com Tue Jul 13 03:42:12 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 12 Jul 2010 21:42:12 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <87vd8k1ydv.fsf@hydra.hampton.thirdcreek.com> Message-ID: Neil Hodgson wrote: > Kurt B. Kaiser: > >>> The tear off menus are ugly as well as being non-standard on all three >>> major platforms. >> Well, would you discard them? They can (occasionally) be useful. > > Yes, I would replace the menus with ones missing the tear line. > Most of the GUI toolkits experimented with tear-offs (Mac in late 80s, > GTK+ up until 2002) and dropped them or hid them in a rarely visited > API. The idea initially appeared reasonable ("I can have the Run and > Check commands available with a single click") but was found to be too > confusing in use. > > IDLE, because it uses a separate top-level window for each file and > shell suffers more than most applications. A menu is torn off from one > window and always applies to that window but shows no visual affinity > with that window: its window is not even activated when a menu command > acts on it. > I agree, the tear-off menus are an anachronism. I'd also like a pony in the form of easily-changeable sets of keystroke mappings. I have never found Alt-P and its cousins either memorable or comfortable. I won't raise an issue for this. There are enough IDLE tickets without it :-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From kbk at shore.net Tue Jul 13 04:01:12 2010 From: kbk at shore.net (Kurt B. Kaiser) Date: Mon, 12 Jul 2010 22:01:12 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: (Terry Reedy's message of "Mon, 12 Jul 2010 18:00:09 -0400") References: <4C3A5983.3010002@holdenweb.com> <20100712080638.GM70600@nexus.in-nomine.org> <87oced2fsc.fsf@hydra.hampton.thirdcreek.com> <4C3B8C90.7010604@v.loewis.de> Message-ID: <87pqysywbb.fsf@hydra.hampton.thirdcreek.com> On Mon, Jul 12 2010, Terry Reedy wrote: > On 7/12/2010 5:43 PM, "Martin v. L?wis" wrote: >> Am 12.07.2010 23:21, schrieb Terry Reedy: >>> On 7/12/2010 5:46 AM, Kurt B. Kaiser wrote: >>> >>>> On Windows, IDLE opens when you right click / edit a .py. Very useful. >>> >>> On my xp machine with 3.1.2, it edit .py opens with notepad. Perhaps the >>> installer just copies forward the association from long ago, before IDLE >>> was available, or at least so usable. I have thought of changing that, >>> but I do not know what the replacement incantation would be. >> >> There should be an "Edit with IDLE" (sic) context menu item. > > I agree, and thought about requesting such, but there is not and never > has been for me that I know of. Actually, I would like Edit with IDLE > x.y and Run with x.y so it is easy to test a file with different > versions. > > XP with updates, install for everyone, make install the default Python. It should be the second item on the right click context menu when hovering over a .py. If not, check your Windows Explorer / Tools / Folder Options / File Types / Advanced. There should be an "Edit with IDLE" in the dialog selection box. Perhaps you overrode it at some point. I've found that whatever version of IDLE is installed last grabs this function. If you build from svn, you don't get it, of course, unless you set it manually. And, you can use the manual setting to select a specific Python/IDLE version. > > Should I open a tracker issue? No, it's your system configuration, 99% probable. -- KBK From tjreedy at udel.edu Tue Jul 13 04:28:00 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 12 Jul 2010 22:28:00 -0400 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: References: Message-ID: On 7/11/2010 11:02 PM, Tim Peters wrote: >> The heuristic lowered the reported match ratio from .96 to .88, which >> would be bad when one wanted the unaltered value. > BTW, it's not clear whether ratio() computes a _useful_ value in the > presence of junk, however that may be defined. I agree, which is one reason why one should be to disable auto-junking. There are a number of statistical methods for analyzing similarity matrices, analogous to correlation matrices, except that entries are in [0.0,1.0] rather than [-1.0,1.0]. For my Ph.D. thesis, I did such analyses for sets of species. Similarity measures should ususally be symmetric and increase with greater matching. The heuristic can cause both to fail. > I suspect nobody cares ;-) There are multiple possible definitions of similarity for sets (and arguments thereabout). I am sure the same is true for sequences. But I consider the definition for .ratio, without the heuristic, to be sensible. I would consider using it should the occasion arise. > It certainly was the intent that nothing would be > called junk unless it appeared at least twice, so the "n>= 200" > clause ensures that 1% of n is at least 2. Since 2 cannot be greater than something that is at least 2, you ensured that nothing would be called junk unless it appears as least thrice. > However, I'm wary of introducing a generalization in the absence of > experience saying people would use it. Is this the right kind of > parametrization? Is this even the right kind of way to go about > auto-detecting junk? I know it worked great for the original use case > that drove it, but I haven't seen anyone say they want a notion of > auto-junk detection for other uses - just that they _don't_ want the > wholly inappropriate current auto-junk detection in (some or all) of > their uses. > > IOW, it's hard to generalize confidently from a sample of one :-( > >> Implementation: Add a new parameter named 'common' or 'threshold' or >> whatever that defaults to 1. > > I'd call it "autojunk", cuz that's what it would do. Also a useful > syntactic overlap with the name of the current "isjunk" argument. I like that. I am now leaning toward the following? G (I hope, this time, for 'go' ;-). For 2.7.1, 3.1.3, and 3.2, add 'autojunk = True' to the constructor signature. This is the minimal change that fixes the bug of no choice while keeping the default as is. So it is a minimal violation of the usual stricture against API changes in bugfix releases. I would doc this as "Use an internal heuristic that identifies 'common' items as junk." and separately describe the 'current heuristic', leaving open the possibility of changing it. Possible suboption: enforce 'autojunk in (True,False)' so the user cannot forget that it is a switch and not a tuning parameter. In 3.2, expose as an attribute a tuple 'hueristic' or '_heuristic' with the tuning parameters for the current heuristic. Adding the _ would indicate that is it a private, expert-only, use at your own risk, subject to change attribute. If we agree on this much, we can then discuss what the tuple should be for 3.2. >> Other changes that apply regardless of the heuristic/api change: >> >> Update the code to use sets (newer than difflib) instead of dicts with >> values set to 1. >> Directly expose the set of 'common' items as an additional attribute of >> SequenceMatcher instances. Such instance attributes are currently >> undocumented, so adding one can hardly be a problem. Add documention >> thereof. Being able to see the effect of the heuristic when it is not turned >> off might help people decide whether or not to use it, or how to tune the >> threshold for smallish alphabets where 1 is too small. > > Wholly agreed. junkdict (after turning it into a set) should also be > exposed - when someone passes in a fancy regexp matcher for the isjunk > argument, they can be surprised at what their regexp matches. Being > able to see the results can be helpful there too, for debugging. I meant to include junkdict also. -- Terry Jan Reedy From alexander.belopolsky at gmail.com Tue Jul 13 04:30:51 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 12 Jul 2010 22:30:51 -0400 Subject: [Python-Dev] Python equivalents in stdlib Was: Include datetime.py in stdlib or not? In-Reply-To: References: <4C34F828.3080100@voidspace.org.uk> Message-ID: On Thu, Jul 8, 2010 at 9:44 PM, Alexander Belopolsky wrote: .. > and it still requires that '_pickle' is disabled to pass pickle tests. > I have found the problem in test_datetime. Restoring sys.modules has to be done in-place. With this fix, test_datetime looks as follows: ===== import unittest import sys from test.support import import_fresh_module, run_unittest TESTS = 'test.datetimetester' pure_tests = import_fresh_module(TESTS, fresh=['datetime', '_strptime'], blocked=['_datetime']) fast_tests = import_fresh_module(TESTS, fresh=['datetime', '_datetime', '_strptime']) test_modules = [pure_tests, fast_tests] test_suffixes = ["_Pure", "_Fast"] for module, suffix in zip(test_modules, test_suffixes): for name, cls in module.__dict__.items(): if isinstance(cls, type) and issubclass(cls, unittest.TestCase): name += suffix cls.__name__ = name globals()[name] = cls def setUp(self, module=module, setup=cls.setUp): self._save_sys_modules = sys.modules.copy() sys.modules[TESTS] = module sys.modules['datetime'] = module.datetime_module sys.modules['_strptime'] = module.datetime_module._strptime setup(self) def tearDown(self, teardown=cls.tearDown): teardown(self) sys.modules.__init__(self._save_sys_modules) cls.setUp = setUp cls.tearDown = tearDown def test_main(): run_unittest(__name__) if __name__ == "__main__": test_main() ===== I think this is as good as it gets. I am going to update the patch at http://bugs.python.org/issue7989 . From kbk at shore.net Tue Jul 13 04:49:52 2010 From: kbk at shore.net (Kurt B. Kaiser) Date: Mon, 12 Jul 2010 22:49:52 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: (Terry Reedy's message of "Mon, 12 Jul 2010 19:08:07 -0400") References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> <4C3A103A.1070504@v.loewis.de> <4C3AB092.8030902@v.loewis.de> Message-ID: <87lj9gyu27.fsf@hydra.hampton.thirdcreek.com> On Mon, Jul 12 2010, Terry Reedy wrote: > On 7/12/2010 2:05 AM, "Martin v. L?wis" wrote: >>> What I specifically want right now is Commit Authorization Privilege, >>> especially for IDLE, >> >> Not sure who could grant that, but as far as I can: you have it. > > If I were approved to commit patches directly, then by implication I > should be able to approve others doing the same. That is occasionally > done now by others, but I wanted to be clear that for the present, the > latter is all I could and would do. I've not had experience with patches from Terry. I don't think there are any in IDLE, at least not acknowledged in NEWS. I'd suggest giving him Tracker access and start out working with the patch submitters to update their patches to Py3, followed by a patch review and triage. He could unset all the bugs/patches assigned to me. I don't consider that I "own" IDLE. I've put quite a bit of time into it, and expect to continue. But I'd welcome help! I only hope to keep to roughly the same stylistic philosophy, and hope I'm still channeling Guido accurately. In addition, like the rest of the library, IDLE should be an example of good Python code. But there is some cruft in there. Some just old and/or unfinished, and some, like Multicall.py, rather obscure. Multicall is needed because Tk forgets what modifier keys were depressed when an event is generated. I'm hoping Tk will grow a memory so Multicall can be simplified. -- KBK From tjreedy at udel.edu Tue Jul 13 04:54:07 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 12 Jul 2010 22:54:07 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <4C3B9C3B.9080801@v.loewis.de> References: <4C3A5983.3010002@holdenweb.com> <20100712080638.GM70600@nexus.in-nomine.org> <87oced2fsc.fsf@hydra.hampton.thirdcreek.com> <4C3B8C90.7010604@v.loewis.de> <4C3B9C3B.9080801@v.loewis.de> Message-ID: On 7/12/2010 6:50 PM, "Martin v. L?wis" wrote: > Am 13.07.2010 00:00, schrieb Terry Reedy: >> On 7/12/2010 5:43 PM, "Martin v. L?wis" wrote: >>> There should be an "Edit with IDLE" (sic) context menu item. >> >> I agree, and thought about requesting such > > You misunderstand. To the contrary. You misunderstood my humor ;-). > There should be such a menu item *today*, *on your machine*. Yes, I understood that claim. > If it's not there, either the installation procedure went wrong, It has never indicated that there was a problem. > or you opted out of having it. I do not recall having ever seen an option to include or exclude 'Edit with IDLE'. I will check more carefully on my next install. I still wonder whether the installer is paying too much attention in this regard to the previous install. One thing it does not do, however, is pay enough attention to where the current install is. If I already have /xxx/python31, then it should at least default to putting a bugfix releases in the same place. -- Terry Jan Reedy From kbk at shore.net Tue Jul 13 05:00:06 2010 From: kbk at shore.net (Kurt B. Kaiser) Date: Mon, 12 Jul 2010 23:00:06 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: (Steve Holden's message of "Mon, 12 Jul 2010 21:42:12 -0400") References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <87vd8k1ydv.fsf@hydra.hampton.thirdcreek.com> Message-ID: <871vb8ytl5.fsf@hydra.hampton.thirdcreek.com> On Mon, Jul 12 2010, Steve Holden wrote: > I agree, the tear-off menus are an anachronism. OK, thanks for the input. I use them rarely, myself. > I'd also like a pony in the form of easily-changeable sets of > keystroke mappings. I have never found Alt-P and its cousins either > memorable or comfortable. Is Options / Configure IDLE / Keys what you are looking for? There are four built-in sets currently, and you can define your own based on one of those. -- KBK From tjreedy at udel.edu Tue Jul 13 06:32:18 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 13 Jul 2010 00:32:18 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <87lj9gyu27.fsf@hydra.hampton.thirdcreek.com> References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> <4C3A103A.1070504@v.loewis.de> <4C3AB092.8030902@v.loewis.de> <87lj9gyu27.fsf@hydra.hampton.thirdcreek.com> Message-ID: On 7/12/2010 10:49 PM, Kurt B. Kaiser wrote: > I've not had experience with patches from Terry. I don't think there are > any in IDLE, at least not acknowledged in NEWS. You posts in the last day have told me a lot more about you. Let me introduce myself to you in turn. I have been involved with Python and active on the lists for most of about 13 years. One of my first posts was to dub Python 'executable pseudocode'. I just helped with the preparation of a simple patch that fixes an IDLE annoyance I have. At the moment, I need for someone else to commit it. Will you do it? http://bugs.python.org/issue9222 > I'd suggest giving him Tracker access I have had that for perhaps five years. My first tracker post was http://bugs.python.org/issue593696 In that, I dianosed a problem and suggested a fix, which Neal Norwitz checked in. In the last month, I have touched perhaps 200 issues; about 20 are closed as a result. There are a few more that I think should be committed and closed. Overall, I have posted message to nearly 500 issues and adjusted headers on some number more. (I do not see a way to search for that with the provided form.) None the less, I have not said a word, until yesterday, in response to Martin's offer, which surprised me, about commit privileges for me. Last January, I turned down an offer, based on my tracker record, to be sponsored for such. I then thought it premature. I could hardly blame anyone who thought such today; I partly agree myself. > He could unset all the bugs/patches assigned to me. So could you. Or you could find a real beginner to be your go-fer. Or you could ask a super-admin to save everyone (except himself) some time and do it with one SQL statement. (I am presuming that Roundup uses a real database under the web facade.) > I've put quite a bit of time into [IDLE] and expect to continue. I am really glad to hear that. -- Terry Jan Reedy From memilanuk at gmail.com Tue Jul 13 06:34:04 2010 From: memilanuk at gmail.com (Monte Milanuk) Date: Mon, 12 Jul 2010 21:34:04 -0700 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: On 7/11/10 7:31 AM, Tal Einat wrote: > However, I still think IDLE is not currently in a state that it should > be suggested for use by beginners. Being one of those beginners... here's my $0.02 worth. IDLE being instantly available on pretty much anything I care to try my hand at python on... my Macbook, my desktop PC, Linux (inside Virtualbox), or even via PortablePython on my usb thumb drive... is priceless. Wherever I go, it just works. The defaults are fairly reasonable and comfortable on the platforms I've used it on. I don't have all the sidebars and bottom bars and gizmos and gadgets in my way - just a shell to try things in, and a basic editor to write code in. That said... some documentation (i.e. with examples) of using some of the other features (debugger, class & path browsers, etc.) would be nice. I think I posted a while back that some basic niceties for the editor like line numbers and 'show whitespace' would make things a little handier for the newbies out there. Just reading through this thread, it seems like the problem seems to be those who know how to fix the problems or make the enhancements... generally have 'moved on' more advanced editors or IDEs, and those who use IDLE regularly are often those who lack the knowledge of how to do bug reports, etc. and probably would be very discouraged to see it untouched for extended periods. Not sure there's an easy reconciliation between the two. The idea of a plain 'release' version and a possible enhanced 'development' version does have some interest for me personally... especially if 1) it stays easy to install/use, and 2) the enhancements get regularly rolled into the release versions. YMMV, Monte From fdrake at acm.org Tue Jul 13 07:31:47 2010 From: fdrake at acm.org (Fred Drake) Date: Tue, 13 Jul 2010 01:31:47 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <20100712235920.146ffb37@pitrou.net> References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> <20100712201201.GA21730@phd.pp.ru> <4C3B8C5A.3080308@voidspace.org.uk> <20100712235920.146ffb37@pitrou.net> Message-ID: I wrote: > Indeed! ?And I'd be quite content with such a solution, since I > consider scripts and modules to be distinct. On Mon, Jul 12, 2010 at 5:59 PM, Antoine Pitrou wrote: > Except that modules can often be executed as scripts... Rest assured, I'm well aware of the history, and don't seriously expect the situation to change. Not all of us consider modules and scripts synonymous; I've considered them distinct for quite some time. The problem with a script being importable as a module, masking the intended module, is simply a symptom of this misfeature that has been identified as commonly biting newcomers. Glyph Lefkowitz wrote: > but ... isn't the whole point of 'python -m' to make scripts and modules _not_ be distinct? That was never how I understood the intention. The point was to make modules that could be used as scripts easier to use as scripts. There have been proposals for main-functions in modules that would be invoked as a default entry point; those make -m more attractive. I'd be more likely to use -m if those were part of the package. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein From martin at v.loewis.de Tue Jul 13 08:41:31 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 13 Jul 2010 08:41:31 +0200 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <4C3B9BA4.7050500@trueblade.com> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B916E.7050801@voidspace.org.uk> <4C3B9BA4.7050500@trueblade.com> Message-ID: <4C3C0A9B.6050306@v.loewis.de> Am 13.07.2010 00:48, schrieb Eric Smith: > On 7/12/2010 6:04 PM, Michael Foord wrote: >> Given how high traffic python-checkins is I don't consider that a >> reasonable place to send follow-up and nor do I consider it the >> responsibility of committers to monitor it. As you said earlier this >> *isn't* in our standard dev procedures and nor do I think it should be. >> If you can't find an email address then either python-comitters or >> python-dev would be a better place to send feedback. > > Maybe reply-to on the checkin messages could be set to python-dev. Not > sure if that's a mailman feature, though. Somebody correct me if I'm wrong: I think we had this, at one point, and then switched it to the status quo. I can surely try switching it back if desired. Regards, Martin From greg at krypto.org Tue Jul 13 09:28:26 2010 From: greg at krypto.org (Gregory P. Smith) Date: Tue, 13 Jul 2010 00:28:26 -0700 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: <4C362C8C.80707@mrabarnett.plus.com> References: <4C362C8C.80707@mrabarnett.plus.com> Message-ID: On Thu, Jul 8, 2010 at 12:52 PM, MRAB wrote: > Hi all, > > I re-implemented the re module, adding new features and speed > improvements. It's available at: > > http://pypi.python.org/pypi/regex > > under the name "regex" so that it can be tried alongside "re". > > I'd be interested in any comments or feedback. How does it compare with > "re" in terms of speed on real-world data? The benchmarks suggest it > should be faster, or at worst comparable. > > How much interest would there be in putting it in Python 3.2? A big +1 from me. I'm all for putting it in Python 3.2 alphas and beta1 as a *replacement* for the existing 're' module in order to get it visibility and a shakeout and make this available to the world sooner rather than later. This module was designed for compatibility for exactly that reason. Lets give it a high publicity fair shake it iron out any unknown compatibility issues early on so that we can decide how to proceed. Also, +1 to Antoine's point on volunteering to be the maintainer of it for the next couple major releases. (I do expect someone to balk at this idea with "but python 3.x doesn't have enough users yet for a beta cycle to shake out hidden regular expression engine assumption bugs" to which I'll respond in advance with "thats a good thing; we can add a regular expression sniffer to 2to3 that flags potential problems in code being ported over") -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg at krypto.org Tue Jul 13 09:29:52 2010 From: greg at krypto.org (Gregory P. Smith) Date: Tue, 13 Jul 2010 00:29:52 -0700 Subject: [Python-Dev] commit privs In-Reply-To: <20100711182831.56462244@pitrou.net> References: <20100711182831.56462244@pitrou.net> Message-ID: On Sun, Jul 11, 2010 at 9:28 AM, Antoine Pitrou wrote: > On Sun, 11 Jul 2010 13:23:13 +0000 > Reid Kleckner wrote: > > > > I'm also expecting to be doing more work merging unladen-swallow into > > the py3k-jit branch, so I was wondering if I could get commit > > privileges for that. > > It sounds good to me. Also, thanks for your threading patches! > > Regards +1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dirkjan at ochtman.nl Tue Jul 13 09:56:16 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Tue, 13 Jul 2010 09:56:16 +0200 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: <4C3B92F6.4020807@v.loewis.de> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B8DDB.8070109@v.loewis.de> <4C3B92F6.4020807@v.loewis.de> Message-ID: On Tue, Jul 13, 2010 at 00:11, "Martin v. L?wis" wrote: >>>> There's a one-to-one mapping somewhere. >>> >>> Unfortunately, no: we don't have email addresses of all committers. >> >> What about the python-committers mailing list? That has at least all >> the active ones, correct? > > Probably. Also, Dirkjan is maintaining a list for the Mercurial migration. Indeed: http://hg.python.org/pymigr/file/tip/author-map?style=raw I think I took email addresses from the public key list that Martin maintains for SVN etc and then augmented that with stuff from the mailing list archives and Google. The list is by no means guaranteed to be correct and fully up-to-date, but it's complete (save for maybe new committers from the last few weeks). Cheers, Dirkjan From hodgestar+pythondev at gmail.com Tue Jul 13 10:03:47 2010 From: hodgestar+pythondev at gmail.com (Simon Cross) Date: Tue, 13 Jul 2010 10:03:47 +0200 Subject: [Python-Dev] Intended behavior of backlash in raw strings In-Reply-To: <4C3B967B.1030604@gmail.com> References: <4C3B967B.1030604@gmail.com> Message-ID: On Tue, Jul 13, 2010 at 12:26 AM, John Arbash Meinel wrote: >>>> r"testing \" backslash and quote" > 'testing \\" backslash and quote' > > It happens that this is the behavior I want, but it seemed just as > likely to be an error. I tested it with python2.5 and 2.6 and got the > same results. The behaviour does appear to be explicitly documented in the language reference: """ When an 'r' or 'R' prefix is present, a character following a backslash is included in the string without change, and all backslashes are left in the string. For example, the string literal r"\n" consists of two characters: a backslash and a lowercase 'n'. String quotes can be escaped with a backslash, but the backslash remains in the string; for example, r"\"" is a valid string literal consisting of two characters: a backslash and a double quote; r"\" is not a valid string literal (even a raw string cannot end in an odd number of backslashes). """ Schiavo Simon From barzhomi at gmail.com Tue Jul 13 10:20:02 2010 From: barzhomi at gmail.com (=?KOI8-R?B?4sHWxc4g8tbF1dTTy8nK?=) Date: Tue, 13 Jul 2010 15:20:02 +0700 Subject: [Python-Dev] Problem with executing python interpretetor on special build of win server 2003 Message-ID: Hello. Actual problem in the next, when i trying to execute portable python on my build then nothing happens, and the error code is 128. What does mean this code 128? From dickinsm at gmail.com Tue Jul 13 10:44:12 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 13 Jul 2010 09:44:12 +0100 Subject: [Python-Dev] More C API abstraction for user defined types In-Reply-To: References: Message-ID: On Mon, Jul 12, 2010 at 10:19 PM, Nick Coghlan wrote: > On Tue, Jul 13, 2010 at 3:35 AM, Petre Galan wrote: >> ival should not be resolved through PyLong_AsLong, but through >> functionality/interface like PyNumber_Long +1, but I'd prefer it if PyNumber_Index were used, rather than PyNumber_Long. It shouldn't be possible to pass a Decimal instance to something expecting an integer argument. Mark From phd at phd.pp.ru Tue Jul 13 12:15:48 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Tue, 13 Jul 2010 14:15:48 +0400 Subject: [Python-Dev] Problem with executing python interpretetor on special build of win server 2003 In-Reply-To: References: Message-ID: <20100713101548.GA19436@phd.pp.ru> Hello. We'are sorry but we cannot help you. This mailing list is to work on developing Python (fixing bugs and adding new features to Python itself); if you're having problems using Python, please find another forum. Probably python-list (comp.lang.python) news group/mailing list is the best place. See http://www.python.org/community/ for other lists/news groups/fora. Thank you for understanding. On Tue, Jul 13, 2010 at 03:20:02PM +0700, ????? ????????? wrote: > Hello. Actual problem in the next, when i trying to execute portable > python on my build then nothing happens, and the error code is 128. > What does mean this code 128? > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/phd%40phd.pp.ru Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From fuzzyman at voidspace.org.uk Tue Jul 13 12:45:20 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 13 Jul 2010 11:45:20 +0100 Subject: [Python-Dev] avoiding accidental shadowing of top-level libraries by the main module In-Reply-To: <201007131030.31035.steve@pearwood.info> References: <201007131030.31035.steve@pearwood.info> Message-ID: <4C3C43C0.2060703@voidspace.org.uk> On 13/07/2010 01:30, Steven D'Aprano wrote: > On Tue, 13 Jul 2010 08:05:24 am Nick Coghlan wrote: > >> On Tue, Jul 13, 2010 at 7:47 AM, Fred Drake wrote: >> >>> On Mon, Jul 12, 2010 at 5:42 PM, Michael Foord >>> >>> wrote: >>> >>>> I'm sure Brett will love this idea, but if it was impossible to >>>> reimport the script being executed as __main__ with a different >>>> name it would solve these problems. >>>> >>> Indeed! And I'd be quite content with such a solution, since I >>> consider scripts and modules to be distinct. >>> >> And here I've been busily blurring that distinction for years ;) >> >> (actually, the whole "name == '__main__'" idiom meant the distinction >> was already pretty blurry long before I got involved) >> > I would hate it if that distinction was un-blurred. Most of my modules > include a section "if __name__ == '__main__': run_tests(), and some of > them do significantly more than that. A few of them import themselves > so they can pass the module object to another module. > > > Reimporting yourself (and creating a second version of the module with new versions of all the classes / constants / functions / etc) doesn't seem like a good way of doing that though. If you need the module object you can always do: module = sys.modules[__name__] Michael Foord >> I take it the concrete proposal here is if the filename of a new >> module matches either __main__.__file__ or __main__.__cached__, then >> that module should be ignored completely for import purposes allowing >> a module with the same name later on sys.path to be found? >> >> I'm not sure I like that, I'd be more inclined to just return the >> __main__ module in that case rather than letting the search continue >> further down sys.path (although I agree the current semantics of >> getting two copies of the same module under different names in this >> case are less than ideal). >> > Yes, that's a weird corner case. I don't see any advantage to keeping > that behaviour. > > > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From solipsis at pitrou.net Tue Jul 13 13:06:25 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 13 Jul 2010 13:06:25 +0200 Subject: [Python-Dev] Removing IDLE from the standard library References: <3B9E6F16-EC11-42B3-A06B-18453A514BE9@dabeaz.com> <4C3A103A.1070504@v.loewis.de> <4C3AB092.8030902@v.loewis.de> Message-ID: <20100713130625.5381959a@pitrou.net> On Mon, 12 Jul 2010 19:08:07 -0400 Terry Reedy wrote: > On 7/12/2010 2:05 AM, "Martin v. L?wis" wrote: > >> What I specifically want right now is Commit Authorization Privilege, > >> especially for IDLE, > > > > Not sure who could grant that, but as far as I can: you have it. > > If I were approved to commit patches directly, then by implication I > should be able to approve others doing the same. That is occasionally > done now by others, but I wanted to be clear that for the present, the > latter is all I could and would do. There is no formal approval process. If you post a review of a patch saying "the patch is ok" and the patch submitter is also a committer, he can then commit it himself. There is no need for more bureaucracy, and you don't need commit access to do this. Regards Antoine. From vlastimil.brom at gmail.com Tue Jul 13 13:39:45 2010 From: vlastimil.brom at gmail.com (Vlastimil Brom) Date: Tue, 13 Jul 2010 13:39:45 +0200 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: <4C362C8C.80707@mrabarnett.plus.com> References: <4C362C8C.80707@mrabarnett.plus.com> Message-ID: 2010/7/8 MRAB : > Hi all, > > I re-implemented the re module, adding new features and speed > improvements. It's available at: > > ? ?http://pypi.python.org/pypi/regex > > under the name "regex" so that it can be tried alongside "re". > > I'd be interested in any comments or feedback. How does it compare with > "re" in terms of speed on real-world data? The benchmarks suggest it > should be faster, or at worst comparable. > > How much interest would there be in putting it in Python 3.2? > Hi, please, let me apologize for posting here, not being a python developer; I'd like to support the inclusion of the new regex library in the standard lib. I use it since the early development versions in my internal app for searching, modifying, ordering, extracting data from text - mainly using the manually created regex patterns. I see, that it is only one specific usecase, and the app isn't time or space critical (input texts up to a few MB, mostly smaller; the processing time is often rather negligible compared to the gui presentation, styling etc.) However, I see it as a great enhancement both in terms of regex features (listed on http://pypi.python.org/pypi/regex ) as well as the behaviour in some cornercases, which aren't effectively usable in the current re (e.g. nested subexpressions with quantifiers - while many of these are more effectively solved with the added possesive quantifiers). I think, this is a far more feature complete engine, which doesn't induce any significant drawbacks (IMO) comparing to the current re and is backward compatible. (The mentioned exception in the scoped flags might be fixable by allowing only explicit scoping (?flags)...(?-flags) or using the current paren, if possible (?flag:...) and treating the bare flag setting parens as global; however, I would consider it quite misleading in the current re, if these flags are set at some other place than at the beginning of the pattern. I don't see the readability enhanced in any way with these flags set at the end, while they should apply from the beginning of the pattern.) Having seen MRABs commitment in the development phase in both - bugfixes and feature additions - including some rather complex ones (in my opinion) like unicode properties, I am also confident, that he could be a competent maintainer of this package in the standardlib as well. just my biased opinion... Regards, Vlastimil Brom From ncoghlan at gmail.com Tue Jul 13 14:30:58 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 Jul 2010 22:30:58 +1000 Subject: [Python-Dev] python-checkins In-Reply-To: <87k4p0i437.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B916E.7050801@voidspace.org.uk> <1278976111.3134.7.camel@localhost.localdomain> <87k4p0i437.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Tue, Jul 13, 2010 at 11:05 AM, Stephen J. Turnbull wrote: > Antoine Pitrou writes: > > ?> You don't have to receive e-mail from it. Just take a look at the > ?> archives from time to time after you have done some commits. > ?> In a threaded view, it's easy to spot the few messages which aren't > ?> commit notifications, since they are the only one not at the > ?> top-level. > > It also is possible to arrange that the author (From field) of the > commit message is the author of the commit, rather than the bot. > Reply-To could be set to bot plus author. ?Once the Mercurial > transition is done, this would be more useful as "everybody" would > have an email address as part of their committer ID. I was actually going to suggest something along those lines post transition, since Hg will have an email address for everyone - leave push notifications coming from the bot (so existing filters based on the bot address keep working), but include the push author in the reply-to field. Given its limited remaining lifespan, I don't see any point in messing with the current SVN setup though. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From cben at users.sf.net Tue Jul 13 10:11:07 2010 From: cben at users.sf.net (Beni Cherniavsky-Paskin) Date: Tue, 13 Jul 2010 11:11:07 +0300 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library In-Reply-To: References: <4C39DFD3.6040304@codebykevin.com> Message-ID: On Sun, Jul 11, 2010 at 18:20, Tal Einat wrote: > The (hopefully) compelling arguments were others, such as the sentence > following the one you quoted: > > "I think that in its current state, IDLE may still be helpful for > learning Python, but it is more likely to drive away users who run > into its various quirks and problems." > > Having taught a few Python courses myself, I must say that while not perfect [I had to apologize for some IDLE issues, which shamed me into filing bugs/patches for some of them], it is *better* for interactive use than most other IDEs! That's IMHO the #1 is the real reason that makes it ideal for teaching, not the fact that it's bundled with Python. I wouldn't mind telling people "install Python and X" instead of "install Python", but very few Python environments do multiline history correctly :-(. The only one I know that beats IDLE is Dreampie (designed by Noam Raphael, a long time IDLE contributor). However, without an editor and F5 it's not a winner for teaching. For the record, I'm -42 on removing IDLE from the distribution (unless you have a better replacement?), but +1 on widening commit access and setting up one obvious way for users to try bleeding-edge IDLE. -0 on extracting it from the stdlib (it's one way to implement the above, not sure if best way). But here's a thought: why not make IDLE an early adopter of Mercurial? It seems to me that could ease a lot of the issues: - contributors will be able to publish their changes without waiting for official commiters - contributors will be able to maintain "beefed up" IDLE branches with much less pain - trying out bleeding-edge branches would be much simpler - I expect a de-facto "stable" IDLE branch will emerge from the community -- Beni Cherniavsky-Paskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Jul 13 14:48:02 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 Jul 2010 22:48:02 +1000 Subject: [Python-Dev] How to block a module import In-Reply-To: References: Message-ID: On Tue, Jul 13, 2010 at 10:37 AM, Alexander Belopolsky wrote: > In my experiments, 0 is not equivalent to None: > >>>> import sys, time >>>> sys.modules['time'] = 0 >>>> __import__('time') > 0 >>>> sys.modules['time'] = None >>>> __import__('time') > Traceback (most recent call last): > ?File "", line 1, in > ImportError: No module named time > > ?Am I missing something? I don't think so. I suspect I got away with this mistake because the imports I was trying to block in the test suite were all C acceleration imports of the form "from import *", and those will blow up anyway since zero doesn't have __dict__ or __all__ attributes. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Tue Jul 13 14:52:29 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 Jul 2010 22:52:29 +1000 Subject: [Python-Dev] More C API abstraction for user defined types In-Reply-To: References: Message-ID: On Tue, Jul 13, 2010 at 6:44 PM, Mark Dickinson wrote: > On Mon, Jul 12, 2010 at 10:19 PM, Nick Coghlan wrote: >> On Tue, Jul 13, 2010 at 3:35 AM, Petre Galan wrote: >>> ival should not be resolved through PyLong_AsLong, but through >>> functionality/interface like PyNumber_Long > > +1, but I'd prefer it if PyNumber_Index were used, rather than PyNumber_Long. > It shouldn't be possible to pass a Decimal instance to something expecting an > integer argument. Oops, I misread the suggestion. Indeed, PyNumber_Index is the correct API function for this purpose. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Tue Jul 13 15:00:53 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 Jul 2010 23:00:53 +1000 Subject: [Python-Dev] avoiding accidental shadowing of top-level libraries by the main module In-Reply-To: <4C3B96E4.5010507@voidspace.org.uk> References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> <20100712201201.GA21730@phd.pp.ru> <4C3B8C5A.3080308@voidspace.org.uk> <20100712235920.146ffb37@pitrou.net> <4C3B96E4.5010507@voidspace.org.uk> Message-ID: > (Two different classes with the same name created - one from __main__ and > one from real_name.) Use cases for *genuinely* reimporting the same module > with different names (as different module objects rather than aliases) are > relatively rare, and the problem of modules *accidentally* reimporting > themselves not that rare. That particular issue can be resolved by automatically doing the shadowing at the sys.module level though (i.e. insert into sys.modules under the real module name as well as __main__ for runpy._run_module_as_main and just strip the directory and extension details from __file__ to determine where to insert the second reference for a directly executed script file). Making sure both __main__ and the corresponding importable name refers to the same module object seems reasonable. Trying to special case shadowing detection just because the shadowing module happens to also be the main module seems ugly as hell :) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From fdrake at acm.org Tue Jul 13 15:29:52 2010 From: fdrake at acm.org (Fred Drake) Date: Tue, 13 Jul 2010 09:29:52 -0400 Subject: [Python-Dev] avoiding accidental shadowing of top-level libraries by the main module In-Reply-To: References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> <20100712201201.GA21730@phd.pp.ru> <4C3B8C5A.3080308@voidspace.org.uk> <20100712235920.146ffb37@pitrou.net> <4C3B96E4.5010507@voidspace.org.uk> Message-ID: On Tue, Jul 13, 2010 at 9:00 AM, Nick Coghlan wrote: > Making sure both __main__ and the corresponding importable name refers > to the same module object seems reasonable. One detail that may not have been obvious when I described the persistent object problem; when class references are pickled, they should be for the "real" name of the module, not __main__. Loading the file as __main__ and adding an alias in sys.modules simply isn't sufficient. Existing instances could be loaded, but new instances would cause references to the classes defined in __main__. To some degree, this sort of problem can be easily handled using a "don't do that" approach, and this isn't likely to burn someone just learning Python on the first day. (Well, maybe in the afternoon, once they've got the basics down.) -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein From reid.kleckner at gmail.com Tue Jul 13 16:07:40 2010 From: reid.kleckner at gmail.com (Reid Kleckner) Date: Tue, 13 Jul 2010 14:07:40 +0000 Subject: [Python-Dev] commit privs In-Reply-To: References: <20100711182831.56462244@pitrou.net> Message-ID: Thanks for the support! Georg Brandl authorized my SSH keys for SVN access. Reid On Tue, Jul 13, 2010 at 7:29 AM, Gregory P. Smith wrote: > > On Sun, Jul 11, 2010 at 9:28 AM, Antoine Pitrou wrote: >> >> On Sun, 11 Jul 2010 13:23:13 +0000 >> Reid Kleckner wrote: >> > >> > I'm also expecting to be doing more work merging unladen-swallow into >> > the py3k-jit branch, so I was wondering if I could get commit >> > privileges for that. >> >> It sounds good to me. Also, thanks for your threading patches! >> >> Regards > > +1 > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/reid.kleckner%40gmail.com > > From reid.kleckner at gmail.com Tue Jul 13 16:17:36 2010 From: reid.kleckner at gmail.com (Reid Kleckner) Date: Tue, 13 Jul 2010 14:17:36 +0000 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> Message-ID: On Mon, Jul 12, 2010 at 2:07 PM, Nick Coghlan wrote: > MRAB's module offers a superset of re's features rather than a subset > though, so once it has had more of a chance to bake on PyPI it may be > worth another look. I feel like the new module is designed to replace the current re module, and shouldn't need to spend time in PyPI. A faster regex library isn't going to motivate users to add external dependencies to their projects. Reid From fuzzyman at voidspace.org.uk Tue Jul 13 16:20:23 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 13 Jul 2010 15:20:23 +0100 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> Message-ID: <4C3C7627.80203@voidspace.org.uk> On 13/07/2010 15:17, Reid Kleckner wrote: > On Mon, Jul 12, 2010 at 2:07 PM, Nick Coghlan wrote: > >> MRAB's module offers a superset of re's features rather than a subset >> though, so once it has had more of a chance to bake on PyPI it may be >> worth another look. >> > I feel like the new module is designed to replace the current re > module, and shouldn't need to spend time in PyPI. A faster regex > library isn't going to motivate users to add external dependencies to > their projects. > > If the backwards compatibility issues can be addressed and MRAB is willing to remain as maintainer then the advantages seem well worth it to me. Michael > Reid > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ From solipsis at pitrou.net Tue Jul 13 16:35:44 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 13 Jul 2010 16:35:44 +0200 Subject: [Python-Dev] New regex module for 3.2? References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> Message-ID: <20100713163544.5f3a60d2@pitrou.net> On Tue, 13 Jul 2010 15:20:23 +0100 Michael Foord wrote: > On 13/07/2010 15:17, Reid Kleckner wrote: > > On Mon, Jul 12, 2010 at 2:07 PM, Nick Coghlan wrote: > > > >> MRAB's module offers a superset of re's features rather than a subset > >> though, so once it has had more of a chance to bake on PyPI it may be > >> worth another look. > >> > > I feel like the new module is designed to replace the current re > > module, and shouldn't need to spend time in PyPI. A faster regex > > library isn't going to motivate users to add external dependencies to > > their projects. > > > > > If the backwards compatibility issues can be addressed and MRAB is > willing to remain as maintainer then the advantages seem well worth it > to me. To me as well. The code needs a full review before integrating, though. Regards Antoine. From alexander.belopolsky at gmail.com Tue Jul 13 17:25:23 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 13 Jul 2010 11:25:23 -0400 Subject: [Python-Dev] Peculiar import code in pickle.py Message-ID: When pickle.py needs to import a module by name, it goes through a peculiar dance of __import__(module, level=0) mod = sys.modules[module] As far as I can tell, unless builtins.__import__ is overridden or sys.modules clobbered by user code, the above should be equivalent to mod = __import__(module, level=0) Note that the optimized _pickle implementation does not do the sys.modules lookup and simply uses the module returned by __import__(..). This code goes back to 1999, so there was probably a good reason back then to write it this way. Presently, however, it seems to be just another obscure difference between C and Python implementation of pickle. From solipsis at pitrou.net Tue Jul 13 17:34:18 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 13 Jul 2010 17:34:18 +0200 Subject: [Python-Dev] Peculiar import code in pickle.py References: Message-ID: <20100713173418.22e42f15@pitrou.net> On Tue, 13 Jul 2010 11:25:23 -0400 Alexander Belopolsky wrote: > When pickle.py needs to import a module by name, it goes through a > peculiar dance of > > __import__(module, level=0) > mod = sys.modules[module] > > As far as I can tell, unless builtins.__import__ is overridden or > sys.modules clobbered by user code, the above should be equivalent to > > mod = __import__(module, level=0) Only for top-level modules: >>> __import__("distutils.core", level=0) >>> sys.modules["distutils.core"] Regards Antoine. From alexander.belopolsky at gmail.com Tue Jul 13 17:46:11 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 13 Jul 2010 11:46:11 -0400 Subject: [Python-Dev] Peculiar import code in pickle.py In-Reply-To: <20100713173418.22e42f15@pitrou.net> References: <20100713173418.22e42f15@pitrou.net> Message-ID: On Tue, Jul 13, 2010 at 11:34 AM, Antoine Pitrou wrote: > On Tue, 13 Jul 2010 11:25:23 -0400 .. > Only for top-level modules: > >>>> __import__("distutils.core", level=0) > '/home/antoine/py3k/__svn__/Lib/distutils/__init__.py'> >>>> sys.modules["distutils.core"] > '/home/antoine/py3k/__svn__/Lib/distutils/core.py'> That's right, but I believe the recommended way to achieve that behavior is to supply a dummy fromlist: >>> __import__("distutils.core", fromlist=["dummy"], level=0) That's what C implementation does AFAICT. From fuzzyman at voidspace.org.uk Tue Jul 13 18:22:24 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 13 Jul 2010 17:22:24 +0100 Subject: [Python-Dev] Peculiar import code in pickle.py In-Reply-To: References: <20100713173418.22e42f15@pitrou.net> Message-ID: <4C3C92C0.6040606@voidspace.org.uk> On 13/07/2010 16:46, Alexander Belopolsky wrote: > On Tue, Jul 13, 2010 at 11:34 AM, Antoine Pitrou wrote: > >> On Tue, 13 Jul 2010 11:25:23 -0400 >> > .. > >> Only for top-level modules: >> >> >>>>> __import__("distutils.core", level=0) >>>>> >> > '/home/antoine/py3k/__svn__/Lib/distutils/__init__.py'> >> >>>>> sys.modules["distutils.core"] >>>>> >> > '/home/antoine/py3k/__svn__/Lib/distutils/core.py'> >> > That's right, but I believe the recommended way to achieve that > behavior is to supply a dummy fromlist: > > >>>> __import__("distutils.core", fromlist=["dummy"], level=0) >>>> > '/Users/sasha/Work/python-svn/py3k/Lib/distutils/core.py'> > > That's what C implementation does AFAICT. > I find the "little dance" much more readable. All the best, Michael > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From fuzzyman at voidspace.org.uk Tue Jul 13 18:25:48 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 13 Jul 2010 17:25:48 +0100 Subject: [Python-Dev] avoiding accidental shadowing of top-level libraries by the main module In-Reply-To: References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> <20100712201201.GA21730@phd.pp.ru> <4C3B8C5A.3080308@voidspace.org.uk> <20100712235920.146ffb37@pitrou.net> <4C3B96E4.5010507@voidspace.org.uk> Message-ID: <4C3C938C.2060409@voidspace.org.uk> On 13/07/2010 14:00, Nick Coghlan wrote: >> (Two different classes with the same name created - one from __main__ and >> one from real_name.) Use cases for *genuinely* reimporting the same module >> with different names (as different module objects rather than aliases) are >> relatively rare, and the problem of modules *accidentally* reimporting >> themselves not that rare. >> > That particular issue can be resolved by automatically doing the > shadowing at the sys.module level though (i.e. insert into sys.modules > under the real module name as well as __main__ for > runpy._run_module_as_main and just strip the directory and extension > details from __file__ to determine where to insert the second > reference for a directly executed script file). > Sure - there are trivial workarounds which is why I don't think there are *many* genuine use cases for a module reimporting itself with a different name. Michael > Making sure both __main__ and the corresponding importable name refers > to the same module object seems reasonable. Trying to special case > shadowing detection just because the shadowing module happens to also > be the main module seems ugly as hell :) > > Cheers, > Nick. > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From nagle at animats.com Tue Jul 13 18:44:20 2010 From: nagle at animats.com (John Nagle) Date: Tue, 13 Jul 2010 09:44:20 -0700 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: Message-ID: <4C3C97E4.9000703@animats.com> On Sun, Jul 11, 2010 at 18:20, Tal Einat wrote: > > The (hopefully) compelling arguments were others, such as the sentence > > following the one you quoted: > > > > "I think that in its current state, IDLE may still be helpful for > > learning Python, but it is more likely to drive away users who run > > into its various quirks and problems." The underlying problem is that, given the way Python is currently maintained, there are only two options: 1) part of the main Python distribution and synchronized with it, and 2) supported (or not) by some third party who may or may not produce a working version in sync with what the Python distribution is doing. There's no intermediate tier, like "Python extras", with a code base under common source control and some quality control. (PyPi is mostly a link farm to projects elsewhere, not a source repository.) IDLE belongs in such an intermediate tier. There are no dependencies on IDLE. Nothing will stop working because a version of IDLE isn't available. There are a reasonable number of modules not in the Python distribution that are more mission-critical to large numbers of users than IDLE. IDLE is useful, but not critical. Given limited resources, it's necessary to cut back in some areas. I'd vote IDLE off the island. John Nagle From benjamin at python.org Tue Jul 13 19:57:42 2010 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 13 Jul 2010 12:57:42 -0500 Subject: [Python-Dev] Peculiar import code in pickle.py In-Reply-To: References: <20100713173418.22e42f15@pitrou.net> Message-ID: 2010/7/13 Alexander Belopolsky : > On Tue, Jul 13, 2010 at 11:34 AM, Antoine Pitrou wrote: >> On Tue, 13 Jul 2010 11:25:23 -0400 > .. >> Only for top-level modules: >> >>>>> __import__("distutils.core", level=0) >> > '/home/antoine/py3k/__svn__/Lib/distutils/__init__.py'> >>>>> sys.modules["distutils.core"] >> > '/home/antoine/py3k/__svn__/Lib/distutils/core.py'> > > That's right, but I believe the recommended way to achieve that > behavior is to supply a dummy fromlist: > >>>> __import__("distutils.core", fromlist=["dummy"], level=0) > '/Users/sasha/Work/python-svn/py3k/Lib/distutils/core.py'> No! That's not recommended and a complete hack. The "dance" or importlib.import_module is preferred. -- Regards, Benjamin From jackdied at gmail.com Tue Jul 13 20:10:50 2010 From: jackdied at gmail.com (Jack Diederich) Date: Tue, 13 Jul 2010 14:10:50 -0400 Subject: [Python-Dev] Peculiar import code in pickle.py In-Reply-To: References: <20100713173418.22e42f15@pitrou.net> Message-ID: On Tue, Jul 13, 2010 at 1:57 PM, Benjamin Peterson wrote: > 2010/7/13 Alexander Belopolsky : >> On Tue, Jul 13, 2010 at 11:34 AM, Antoine Pitrou wrote: >>> On Tue, 13 Jul 2010 11:25:23 -0400 >> .. >>> Only for top-level modules: >>> >>>>>> __import__("distutils.core", level=0) >>> >> '/home/antoine/py3k/__svn__/Lib/distutils/__init__.py'> >>>>>> sys.modules["distutils.core"] >>> >> '/home/antoine/py3k/__svn__/Lib/distutils/core.py'> >> >> That's right, but I believe the recommended way to achieve that >> behavior is to supply a dummy fromlist: >> >>>>> __import__("distutils.core", fromlist=["dummy"], level=0) >> > '/Users/sasha/Work/python-svn/py3k/Lib/distutils/core.py'> > > No! That's not recommended and a complete hack. The "dance" or > importlib.import_module is preferred. A complete hack with a long pedigree: module = __import__(modname, None, None, 'python2.4 is silly, revisit this line in 2.5') I think that line in a code base of mine didn't get altered until 2.6.something. Hack-ily, -Jack From alexander.belopolsky at gmail.com Tue Jul 13 20:34:37 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 13 Jul 2010 14:34:37 -0400 Subject: [Python-Dev] Peculiar import code in pickle.py In-Reply-To: References: <20100713173418.22e42f15@pitrou.net> Message-ID: On Tue, Jul 13, 2010 at 1:57 PM, Benjamin Peterson wrote: .. > No! That's not recommended and a complete hack. The "dance" or > importlib.import_module is preferred. Nevertheless, "a complete hack" is what PyImport_Import does: PyObject * PyImport_Import(PyObject *module_name) { static PyObject *silly_list = NULL; .. /* Call the __import__ function with the proper argument list * Always use absolute import here. */ r = PyObject_CallFunction(import, "OOOOi", module_name, globals, globals, silly_list, 0, NULL); .. } and _pickle.c uses PyImport_Import() and thus is different form pickle.py which uses the double-lookup dance. As a result, the two implementations are subtly different. They cannot be both right. It should be easy to "fix" _pickle.c to do the sys.modules lookup, but I am not sure this is right. From alexander.belopolsky at gmail.com Tue Jul 13 21:52:44 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 13 Jul 2010 15:52:44 -0400 Subject: [Python-Dev] Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On Thu, Jul 8, 2010 at 3:09 PM, Brett Cannon wrote: .. > I can say that all the VM representatives have all said they like the idea. This is encouraging. Here is an update on the status of datetime.py. I believe it is mostly ready to move from sandbox to py3k/Lib. The patch is available on the tracker at http://bugs.python.org/file17978/issue9206b.diff and on Rietveld at http://codereview.appspot.com/1824041 . Unfortunately since many changes are just file renames will minor code modifications, neither the patch nor Rietveld give a good overview of the proposed commit. I will try to give this overview here: 1. Copy datetime.py from sandbox to py3k/Lib. (I need some help from an SVN expert on how to do that.) The diff between sandbox version and proposed commit is just ======================================================== --- ../py3k-datetime/datetime.py 2010-07-07 20:12:56.000000000 -0400 +++ Lib/datetime.py 2010-07-08 21:15:52.000000000 -0400 @@ -1555,7 +1555,6 @@ @classmethod def strptime(cls, date_string, format): 'string, format -> new datetime parsed from a string (like time.strptime()).' - import _strptime return _strptime._strptime_datetime(cls, date_string, format) def utcoffset(self): @@ -1874,6 +1873,13 @@ timezone.min = timezone(timezone._minoffset) timezone.max = timezone(timezone._maxoffset) +try: + from _datetime import * +except ImportError: + pass + +import _strptime + ======================================================== The "from _datetime import *" is the standard fast implementation override and "import _strptime" had to be moved from function level to module level after class definitions due to circular dependency of _strptime on datetime. The best place to review the entire datetime.py is on Rietveld at http://codereview.appspot.com/1824041/diff/1/4. I have only one remaining issue with this code - it leaves a number of "private" _xyz functions in the datetime module that are neither used nor overridden by the C implementation. In my view, this is not a big issue, but it can be dealt with by either moving module level functions to class namespace (making them static or class methods as appropriate) or by deleting them explicitly in the else clause of the try statement that imports the fast overrides. 2. Rename datetimemodule.c to _datetimemodule.c. The code changes are trivial: module name and the name of the init function. http://codereview.appspot.com/1824041/diff/1/7 3. Rename test_datetime.py to datetimetester.py. The testing strategy implements Nick Coghlan's idea of importing the test classes from a tester module with and without _datetime and injecting them into test_datetime namespace under different names. http://mail.python.org/pipermail/python-dev/2010-July/101598.html The changes required to the code are minimal: http://codereview.appspot.com/1824041/diff/1/5 . 4. Add new test_datetime.py that contains somewhat elaborate import machinery to create duplicates of each test case from datetimetester.py running with and without datetime acceleration. Unfortunately, test_datetime.py did not make it to Rietveld, but it is small enough to include here: ================================================ import unittest import sys from test.support import import_fresh_module, run_unittest TESTS = 'test.datetimetester' pure_tests = import_fresh_module(TESTS, fresh=['datetime', '_strptime'], blocked=['_datetime']) fast_tests = import_fresh_module(TESTS, fresh=['datetime', '_datetime', '_strptime']) test_modules = [pure_tests, fast_tests] test_suffixes = ["_Pure", "_Fast"] for module, suffix in zip(test_modules, test_suffixes): for name, cls in module.__dict__.items(): if isinstance(cls, type) and issubclass(cls, unittest.TestCase): name += suffix cls.__name__ = name globals()[name] = cls def setUp(self, module=module, setup=cls.setUp): self._save_sys_modules = sys.modules.copy() sys.modules[TESTS] = module sys.modules['datetime'] = module.datetime_module sys.modules['_strptime'] = module.datetime_module._strptime setup(self) def tearDown(self, teardown=cls.tearDown): teardown(self) sys.modules.__init__(self._save_sys_modules) cls.setUp = setUp cls.tearDown = tearDown def test_main(): run_unittest(__name__) if __name__ == "__main__": test_main() ================================================ Since this change does not introduce any new features, I think it is safe to commit it and make further improvements to datetime.py (if any) once it is in the main tree. From merwok at netwok.org Tue Jul 13 21:56:25 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Tue, 13 Jul 2010 21:56:25 +0200 Subject: [Python-Dev] python-checkins In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B916E.7050801@voidspace.org.uk> <1278976111.3134.7.camel@localhost.localdomain> <87k4p0i437.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4C3CC4E9.1000508@netwok.org> > I was actually going to suggest something along those lines post > transition, since Hg will have an email address for everyone - leave > push notifications coming from the bot (so existing filters based on > the bot address keep working), but include the push author in the > reply-to field. IIRC, you can?t know who pushed without kludgy hackery. Also, the one to push is not necessarily the author of the commit(s). If that was just a braino and you meant setting the committer as reply-to, it seems a good idea. Note that nothing in Mercurial forces you to have a parsable ?Name ? user name, it?s just a good practice. Dirkjan?s mapping uses a dummy tools at python.org address for unknown IDs, which probably means the other tools he?s writing depend on an email address. That would need to be in the dev policy. Regards From barry at python.org Tue Jul 13 22:10:04 2010 From: barry at python.org (Barry Warsaw) Date: Tue, 13 Jul 2010 16:10:04 -0400 Subject: [Python-Dev] python-checkins In-Reply-To: <4C3CC4E9.1000508@netwok.org> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B916E.7050801@voidspace.org.uk> <1278976111.3134.7.camel@localhost.localdomain> <87k4p0i437.fsf@uwakimon.sk.tsukuba.ac.jp> <4C3CC4E9.1000508@netwok.org> Message-ID: <20100713161004.5b084810@heresy> On Jul 13, 2010, at 09:56 PM, ?ric Araujo wrote: >Note that nothing in Mercurial forces you to have a parsable >?Name ? user name, it?s just a good practice. Dirkjan?s mapping >uses a dummy tools at python.org address for unknown IDs, which probably >means the other tools he?s writing depend on an email address. That >would need to be in the dev policy. Bazaar has a facility for digitally signing commits, which I always enable. While this is a local configuration, some projects I contribute to have merge hooks which check the digital signatures and refuse the push if the revisions are not signed with a known gpg key. Does Mercurial have a similar feature? If so, I would suggest that we enable that and require committers to use registered gpg keys to sign their commits. We'd always have a verifiable chain back to a responsible party, and committers would be responsible for any changes or patches they merge on behalf of others. IME the overhead is pretty trivial, but then I'm quite comfortable with gpg concepts and tools. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From brett at python.org Tue Jul 13 22:29:00 2010 From: brett at python.org (Brett Cannon) Date: Tue, 13 Jul 2010 13:29:00 -0700 Subject: [Python-Dev] python-checkins replies In-Reply-To: <4C3BA867.6050906@voidspace.org.uk> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B916E.7050801@voidspace.org.uk> <4C3B9BA4.7050500@trueblade.com> <4C3BA867.6050906@voidspace.org.uk> Message-ID: On Mon, Jul 12, 2010 at 16:42, Michael Foord wrote: > On 12/07/2010 23:48, Eric Smith wrote: > >> On 7/12/2010 6:04 PM, Michael Foord wrote: >> >>> Given how high traffic python-checkins is I don't consider that a >>> reasonable place to send follow-up and nor do I consider it the >>> responsibility of committers to monitor it. As you said earlier this >>> *isn't* in our standard dev procedures and nor do I think it should be. >>> If you can't find an email address then either python-comitters or >>> python-dev would be a better place to send feedback. >>> >> >> Maybe reply-to on the checkin messages could be set to python-dev. Not >> sure if that's a mailman feature, though. >> > I think this would be a good idea. It would be nice to have on-topic > traffic here. :-) > > Or python-committers since this is discussing code already checked in and thus is somewhat committer-specific. This also has the perk of being easier to spot (don't know about the rest of you but my python-committers filter makes those emails much more obvious than python-dev traffic). -Brett > Michael > > >> Eric. >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk >> > > > -- > http://www.ironpythoninaction.com/ > http://www.voidspace.org.uk/blog > > READ CAREFULLY. By accepting and reading this email you agree, on behalf of > your employer, to release me from all obligations and waivers arising from > any and all NON-NEGOTIATED agreements, licenses, terms-of-service, > shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, > non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have > entered into with your employer, its partners, licensors, agents and > assigns, in perpetuity, without prejudice to my ongoing rights and > privileges. You further represent that you have the authority to release me > from any BOGUS AGREEMENTS on behalf of your employer. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Tue Jul 13 22:34:58 2010 From: brett at python.org (Brett Cannon) Date: Tue, 13 Jul 2010 13:34:58 -0700 Subject: [Python-Dev] How to block a module import In-Reply-To: References: Message-ID: On Tue, Jul 13, 2010 at 05:48, Nick Coghlan wrote: > On Tue, Jul 13, 2010 at 10:37 AM, Alexander Belopolsky > wrote: > > In my experiments, 0 is not equivalent to None: > > > >>>> import sys, time > >>>> sys.modules['time'] = 0 > >>>> __import__('time') > > 0 > >>>> sys.modules['time'] = None > >>>> __import__('time') > > Traceback (most recent call last): > > File "", line 1, in > > ImportError: No module named time > > > > Am I missing something? > > I don't think so. I suspect I got away with this mistake because the > imports I was trying to block in the test suite were all C > acceleration imports of the form "from import *", and those > will blow up anyway since zero doesn't have __dict__ or __all__ > attributes. > Nick's right; 0 fails on an import * to pull anything in of interest. And as I said on python-checkins -- you can ignore that email, Alexander -- there is a historical reason because in Python 2 if you tried an implicit relative import a value of None met to do an absolute import. This doesn't have that effect in py3k as explicit relative imports are the only way to do relative imports (and luckily importlib does the proper thing for this as well =). -------------- next part -------------- An HTML attachment was scrubbed... URL: From dirkjan at ochtman.nl Tue Jul 13 22:45:53 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Tue, 13 Jul 2010 22:45:53 +0200 Subject: [Python-Dev] python-checkins In-Reply-To: <20100713161004.5b084810@heresy> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B916E.7050801@voidspace.org.uk> <1278976111.3134.7.camel@localhost.localdomain> <87k4p0i437.fsf@uwakimon.sk.tsukuba.ac.jp> <4C3CC4E9.1000508@netwok.org> <20100713161004.5b084810@heresy> Message-ID: This is getting a little off-topic, but let me just respond to this... On Tue, Jul 13, 2010 at 22:10, Barry Warsaw wrote: > Does Mercurial have a similar feature? ?If so, I would suggest that we enable > that and require committers to use registered gpg keys to sign their commits. > We'd always have a verifiable chain back to a responsible party, and > committers would be responsible for any changes or patches they merge on > behalf of others. ?IME the overhead is pretty trivial, but then I'm quite > comfortable with gpg concepts and tools. I wrote something on Stack Overflow about this today, which I reproduce here: - You could verify that whoever is pushing the cset is also the committer (by matching http or ssh authentication). This is somewhat limiting because it can be useful when people push other developer's changesets. - You could use the pgp extension (from hgext) to explicitly sign changesets after committing, but it's kind of a drag if you want to do it for every changeset. In Mercurial, we only do this for releases. - http://bitbucket.org/mg/commitsigs is another extension, which takes a different tack to signing (I believe it doesn't sign the commit metadata, only the file tree, which lets it sign before the commit is finished, meaning it doesn't take up an extra cset). - Mozilla uses a pushlog which just tracks who pushed what. This lets you look in the commit history on the server (but only there) to see who pushed what group of changesets, giving you a better paper trail than you normally get. This can also be provided by changegroup notifications, if you include the guy who did the push in the email (this is what Python will do once their conversion is done). Note that, if you're going to require that each cset is signed, each non-committer contributor also has to have this facility, which IMO raises the bar significantly. I think I added the pushing user to the commit mails to provide just this kind of paper trail. Given the tamper-proofness of the SHA1 changeset ID's (and yes, hg will move to some newer hash algorithm at some point before SHA1 becomes too easy to crack), I don't think signing each cset adds much value. Cheers, Dirkjan From brett at python.org Tue Jul 13 22:52:29 2010 From: brett at python.org (Brett Cannon) Date: Tue, 13 Jul 2010 13:52:29 -0700 Subject: [Python-Dev] Peculiar import code in pickle.py In-Reply-To: References: <20100713173418.22e42f15@pitrou.net> Message-ID: On Tue, Jul 13, 2010 at 11:34, Alexander Belopolsky < alexander.belopolsky at gmail.com> wrote: > On Tue, Jul 13, 2010 at 1:57 PM, Benjamin Peterson > wrote: > .. > > No! That's not recommended and a complete hack. The "dance" or > > importlib.import_module is preferred. > > Nevertheless, "a complete hack" is what PyImport_Import does: > > PyObject * > PyImport_Import(PyObject *module_name) > { > static PyObject *silly_list = NULL; > .. > /* Call the __import__ function with the proper argument list > * Always use absolute import here. */ > r = PyObject_CallFunction(import, "OOOOi", module_name, globals, > globals, silly_list, 0, NULL); > .. > } > > and _pickle.c uses PyImport_Import() and thus is different form > pickle.py which uses the double-lookup dance. As a result, the two > implementations are subtly different. They cannot be both right. It > should be easy to "fix" _pickle.c to do the sys.modules lookup, but I > am not sure this is right. Pulling from sys.modules is the correct way to do this. There are subtle issues when using a bunk fromlist argument (empty modules, double initialization, etc.). If one does not use importlib.import_module -- written *specifically* to prevent people from doing the nasty hack with the fromlist -- then you should use the sys.modules approach, period. If import.c is not doing this then it should get fixed. You can assign me the issue if you want. I say this every time I give an import talk and it has been brought up here before but obviously not everyone catches it (which is understandable as I think when it came up on python-dev it was at the tail end of a discussion), so I am just going to repeat myself: Do not put junk in fromlist if you call __import__ directly! Use importlib.import_module! Or if you have a *really* good reason to not use it, then use ``__import__(name); module = sys.modules[name]``. I have stopped fixing bugs related to this in import.c because of the annoying issues it causes and I expect the correct approach to gain traction at some point (plus get importlib bootstrapped in so I don't have to care about import.c anymore). -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.belopolsky at gmail.com Tue Jul 13 22:58:48 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 13 Jul 2010 16:58:48 -0400 Subject: [Python-Dev] How to block a module import In-Reply-To: References: Message-ID: On Tue, Jul 13, 2010 at 4:34 PM, Brett Cannon wrote: .. > Nick's right; 0 fails on an import * to pull anything in of interest. but if the imported module has try: import blocked_module except ImportError: do_something_important() then import_fresh_module() will create a broken module with do_something_important() not executed. > And as > I said on python-checkins -- you can ignore that email, Alexander Too late. I already replied. :-) >-- there > is a historical reason because in Python 2 if you tried an implicit relative > import a value of None met to do an absolute import. This doesn't have that > effect in py3k as explicit relative imports are the only way to do relative > imports (and luckily importlib does the proper thing for this as well =). Is there a problem with this change for 3.x? From ncoghlan at gmail.com Tue Jul 13 23:02:40 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 14 Jul 2010 07:02:40 +1000 Subject: [Python-Dev] avoiding accidental shadowing of top-level libraries by the main module In-Reply-To: <4C3C938C.2060409@voidspace.org.uk> References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> <20100712201201.GA21730@phd.pp.ru> <4C3B8C5A.3080308@voidspace.org.uk> <20100712235920.146ffb37@pitrou.net> <4C3B96E4.5010507@voidspace.org.uk> <4C3C938C.2060409@voidspace.org.uk> Message-ID: On Wed, Jul 14, 2010 at 2:25 AM, Michael Foord wrote: > Sure - there are trivial workarounds which is why I don't think there are > *many* genuine use cases for a module reimporting itself with a different > name. My concerns aren't about a module reimporting itself directly, they're about the case where a utility module is invoked as __main__ but is also imported normally somewhere else in a program (e.g. pdb is invoked as a top-level debugger, but is also imported directly for some reason). Currently that works as a non-circular import and will only cause hassles if there is top-level state in the affected module that absolutely must be a singleton within a given application. Either change (disallowing it completely as you suggest, or making it a circular import, as I suggest) runs the risk of breaking code that currently appears to work correctly. Fred's point about the practice of changing __name__ in the main module corrupting generated pickles is one I hadn't thought of before though. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Tue Jul 13 23:10:34 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 14 Jul 2010 07:10:34 +1000 Subject: [Python-Dev] How to block a module import In-Reply-To: References: Message-ID: > Nick's right; 0 fails on an import * to pull anything in of interest. And as > I said on python-checkins -- you can ignore that email, Alexander -- there > is a historical reason because in Python 2 if you tried an implicit relative > import a value of None met to do an absolute import. This doesn't have that > effect in py3k as explicit relative imports are the only way to do relative > imports (and luckily importlib does the proper thing for this as well =). Ah, thank you - I knew we had a reason for doing it that way, I just couldn't remember what it was :) @Alexander: yes, it should be changed to None for 2.7, 3.1 and 3.2, since none of those allow implicit relative imports. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From alexander.belopolsky at gmail.com Tue Jul 13 23:20:42 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 13 Jul 2010 17:20:42 -0400 Subject: [Python-Dev] Peculiar import code in pickle.py In-Reply-To: References: <20100713173418.22e42f15@pitrou.net> Message-ID: On Tue, Jul 13, 2010 at 4:52 PM, Brett Cannon wrote: .. > Pulling from sys.modules is the correct way to do this. There are subtle > issues when using a bunk fromlist argument (empty modules, double > initialization, etc.). If one does not use importlib.import_module -- > written *specifically* to prevent people from doing the nasty hack with the > fromlist -- then you should use the sys.modules approach, period. If > import.c is not doing this then it should get fixed. You can assign me the > issue if you want. Please see http://bugs.python.org/issue9252 . From tim.peters at gmail.com Tue Jul 13 23:20:24 2010 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 13 Jul 2010 17:20:24 -0400 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: References: Message-ID: [Tim] >> ... >> BTW, it's not clear whether ratio() computes a _useful_ value in the >> presence of junk, however that may be defined. [Terry Reedy] > I agree, which is one reason why one should be to disable auto-junking. Yup. > There are a number of statistical methods for analyzing similarity matrices, > analogous to correlation matrices, except that entries are in [0.0,1.0] > rather than [-1.0,1.0]. For my Ph.D. thesis, I did such analyses for sets of > species. Similarity measures should ususally be symmetric and increase with > greater matching. The heuristic can cause both to fail. Two things about that. First, ratio() is here _defined_ to be (when comparing sequence a and b): 2.0 * number_of_matches / (len(a) + len(b)) The denominator is not affected by any junk heuristics, so this ratio does indeed increase directly with "greater matching". But I don't know exactly what you mean by "greater matching" - I get the impression you think that's an inherent property of the sequences, but, as before, I don't see any meaning for it independent of the specific matching algorithm used. The current implementation of quick_ratio() may be closer to what you seem to be thinking. That views both sequences as multisets, and considers number_of_matches to be the cardinality of their intersection. That measure ignores the possibility of junk, and also ignores the order in which elements appear - it's wholly independent of the matching algorithm. But it returns 1.0 when the second sequence is any permutation of the elements in the first sequence, so throws away any notion of ordering. Second, it's common as mud for ratio() to give a different result depending on the order of SequenceMatcher's arguments, junk or no junk. This is mostly because it's a good thing for SequenceMatcher to run in finite time ;-) It's not just comparing sequences in the abstract, it's doing so in the context of producing a one-pass "left to right" edit sequence transforming the first sequence into the second, using only "insert" and "delete" operations. If the longest matching block between sequences X and Y is M, X and Y are effectively split into: X = A + M + B Y = C + M + D and then the same process is recursively applied to the pairs (A, C) and (B, D). If, for example, there are many matches between blocks A and D, they'll never be seen - a one-pass edit sequence restricted to "insert" and "delete" has to view M as a wall, transforming A into C entirely before it can skip over the matching M and start thinking about how to transform B into D. For that reason, when there are multiple maximal matching blocks, exactly which one is picked to be M can have percolating effects on how many additional matches are found _later_ (BTW, it's also a reason for why a notion of junk can improve the subjective _quality_ of results as judged by humans: if M is composed of things all of which "look interesting" to people, people tend to think the match is intuitive). The method find_longest_match() uses to pick a winner is documented: Return (i,j,k) such that a[i:i+k] is equal to b[j:j+k], where alo <= i <= i+k <= ahi blo <= j <= j+k <= bhi and for all (i',j',k') meeting those conditions, k >= k' i <= i' and if i == i', j <= j' In other words, of all maximal matching blocks, return one that starts earliest in a, and of all those maximal matching blocks that start earliest in a, return the one that starts earliest in b. This has consequences. Here's a tiny example: X = "abbb" Y = "bbab" All maximal matching blocks between X and Y have length 2. "bb" appears twice in X and once in "Y". "ab" appears once in each. Pass X first, and M="ab" is picked as the winner. That breaks the sequences into: X = "" + "ab" + "bb" Y = "bb" + "ab" + "" and no further matches are found between "" and "bb", or between "bb" and "". ratio() thus returns 0.5. But pass Y first, and M="bb" is picked as the winner, breaking the sequences into: X = "a" + "bb" + "b" Y = "" + "bb" + "ab" and an _additional_ match (on "b") is later found when comparing the suffixes ("b" and "ab"). ratio() thus returns 0.75 in that case. I can conceive of trying all possible winners (and so on recursively), and then backtracking to pick a winner at each branch that maximizes the total number of matches - but "it's a good thing for SequenceMatcher to run in finite time" ;-) > There are multiple possible definitions of similarity for sets (and > arguments thereabout). I am sure the same is true for sequences. But I > consider the definition for .ratio, without the heuristic, to be sensible. I > would consider using it should the occasion arise. If you ever used difflib's file-comparison functions, they used ratio() extensively under the covers. The most interesting case when producing diffs for human consumption is when two blocks have _no_ matching lines in common. difflib's file comparators look for the two most similar lines (highest ratio()) in that case, trying to guess where (&, later, how) a human may have edited a line. This produces terrifically useful & intuitive output - when it works ;-) >> It certainly was the intent that nothing would be >> called junk unless it appeared at least twice, so the "n>= 200" >> clause ensures that 1% of n is at least 2. > Since 2 cannot be greater than something that is at least 2, you ensured > that nothing would be called junk unless it appears as least thrice. OK, staring at the code the minimum is actually 4 times. It's true that if n >= 200 and len(indices) * 100 > n: can't succeed when len(indices) is 2, but when it's 3 we've _previously_ seen 3 instances of the element and are currently looking at the fourth. ... >> I'd call it "autojunk", cuz that's what it would do. ?Also a useful >> syntactic overlap with the name of the current "isjunk" argument. > I like that. I am now leaning toward the following? > > G (I hope, this time, for 'go' ;-). For 2.7.1, 3.1.3, and 3.2, add 'autojunk > = True' to the constructor signature. This is the minimal change that fixes > the bug of no choice while keeping the default as is. So it is a minimal > violation of the usual stricture against API changes in bugfix releases. I'm not current on the rules for bugfix releases, but way back when I'm afraid this wouldn't be allowed. The problem is that anyone using the new argument in 2.7.1 would be creating code that would blow up if run under 2.7. Seems that the 6 people who care ;-) about this case would happily agree to live with that, but I have to defer to those more current on bugfix release practice. > I would doc this as "Use an internal heuristic that identifies 'common' items > as junk." and separately describe the 'current heuristic', leaving open the > possibility of changing it. > > Possible suboption: enforce 'autojunk in (True,False)' so the user cannot > forget that it is a switch and not a tuning parameter. Don't think that part is needed. > In 3.2, expose as an attribute a tuple 'hueristic' or '_heuristic' with the > tuning parameters for the current heuristic. Adding the _ would indicate > that is it a private, expert-only, use at your own risk, subject to change > attribute. > > If we agree on this much, we can then discuss what the tuple should be for > 3.2. Think the most pressing thing is to give people a way to turn the damn thing off. An ugly way would be to trigger on an unlikely input-output behavior of the existing isjunk argument. For example, if isjunk("what's the airspeed velocity of an unladen swallow?") returned "don't use auto junk!" and 2.7.1 recognized that as meaning "don't use auto junk", code could be written under 2.7.1 that didn't blow up under 2.7. It could _behave_ differently, although that's true of any way of disabling the auto-junk heuristics. From patcam at python.org Tue Jul 13 22:19:15 2010 From: patcam at python.org (Pat Campbell) Date: Tue, 13 Jul 2010 16:19:15 -0400 Subject: [Python-Dev] [Idle-dev] Removing IDLE from the standard library; Scherer Agreement In-Reply-To: <878w5g17x5.fsf_-_@hydra.hampton.thirdcreek.com> References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <878w5g17x5.fsf_-_@hydra.hampton.thirdcreek.com> Message-ID: Hi Bruce: Please click on the link below to find the contributor agreement form along with instructions on how to send it. http://www.python.org/psf/contrib/ If you have any questions, please let me know. Thanks, Pat Campbell PSF Secretary & Administrator On Mon, Jul 12, 2010 at 9:34 PM, Kurt B. Kaiser wrote: > On Mon, Jul 12 2010, Bruce Sherwood wrote: > > > I don't recall that VIDLE has anything to do with running multiple IDLE > > copies. > > Well, I stole the ephemeral port idea from him! > > > What's in VIDLE is a lot of bug fixes and some improvements. For > > example, you can configure it to not require having to save a file, > > which makes it as easy to use for quick testing as the shell. Another > > improvement, of high importance for our physics students and other > > novice programmers, is to bring an error display forward. Many > > students fill the screen with the edit window and the error message > > could be hidden, leaving them puzzled as to why their program wasn't > > running. Stuff like that, all aimed at issues we had seen with a large > > population of science and engineering students. > > > > I'll say again that IDLE/VIDLE is an excellent environment for novices. > > Issues identified here about look and feel etc. have just not been of any > > importance in that context. > > > > I'd like to get all that incorporated so you can go back to dealing > directly with us. Your students are the people we want to reach! > > > If you'll send me what you want from Scherer, I can contact him. > > I'm copying our PSF Secretary/Administrator, Pat Campbell. She can work > with you to get David Scherer's Contributor Agreement. We really > appreciate your assistance with this, he's one of the last missing > agreements. > > -- > KBK > -- Pat Campbell PSF Administrator/Secretary patcam at python.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Jul 13 23:22:08 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 14 Jul 2010 07:22:08 +1000 Subject: [Python-Dev] notifications form Hg to python-checkins Message-ID: On Wed, Jul 14, 2010 at 5:56 AM, ?ric Araujo wrote: >> I was actually going to suggest something along those lines post >> transition, since Hg will have an email address for everyone - leave >> push notifications coming from the bot (so existing filters based on >> the bot address keep working), but include the push author in the >> reply-to field. > > IIRC, you can?t know who pushed without kludgy hackery. Also, the one to > push is not necessarily the author of the commit(s). If that was just a > braino and you meant setting the committer as reply-to, it seems a good > idea. No, I meant push. There's a separate discussion where it was pointed out that publishing each commit as a separate email makes python-checkins even chattier than it is already (this point came up after Tarek pushed a distutils2 changeset containing ~100 separate commits), so the suggestion was made to combine all commit notifications for a given push into a single email. However, I don't remember if that idea was suggested specifically to Dirkjan as a change to the notification email. If not, I'm suggesting it now :) Note that the current distutils2 emails to python-checkins already say "Tarek Ziade pushed..." and then include the committer details inside the email, and that's what we want - the developer pushing the change into the central repository is always going to be one of the core devs, but the original creator of the change set could have been anyone. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From fdrake at acm.org Tue Jul 13 23:34:58 2010 From: fdrake at acm.org (Fred Drake) Date: Tue, 13 Jul 2010 17:34:58 -0400 Subject: [Python-Dev] avoiding accidental shadowing of top-level libraries by the main module In-Reply-To: References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> <20100712201201.GA21730@phd.pp.ru> <4C3B8C5A.3080308@voidspace.org.uk> <20100712235920.146ffb37@pitrou.net> <4C3B96E4.5010507@voidspace.org.uk> <4C3C938C.2060409@voidspace.org.uk> Message-ID: On Tue, Jul 13, 2010 at 5:02 PM, Nick Coghlan wrote: > Fred's point about the practice of changing __name__ in the main > module corrupting generated pickles is one I hadn't thought of before > though. Not sure about changing __name__ anywhere... I don't do that. When an "application" is defined more by operations on persistent data then what top-level script is run, we (Zope Corp.) often have many different __main__s that run with the same database; there's generally not any point in thinking of the script as a module at all; the fact that it's available as sys.modules['__main__'] isn't important. We generally avoid the problem by applying the don't-do-that rule. Our actual scripts are always generated, and don't live in module-space. Understanding what not to do is an important aspect of teaching, though. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein From glyph at twistedmatrix.com Tue Jul 13 23:36:12 2010 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Tue, 13 Jul 2010 17:36:12 -0400 Subject: [Python-Dev] avoiding accidental shadowing of top-level libraries by the main module In-Reply-To: References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> <20100712201201.GA21730@phd.pp.ru> <4C3B8C5A.3080308@voidspace.org.uk> <20100712235920.146ffb37@pitrou.net> <4C3B96E4.5010507@voidspace.org.uk> <4C3C938C.2060409@voidspace.org.uk> Message-ID: On Jul 13, 2010, at 5:02 PM, Nick Coghlan wrote: > My concerns aren't about a module reimporting itself directly, they're > about the case where a utility module is invoked as __main__ but is > also imported normally somewhere else in a program (e.g. pdb is > invoked as a top-level debugger, but is also imported directly for > some reason). Currently that works as a non-circular import and will > only cause hassles if there is top-level state in the affected module > that absolutely must be a singleton within a given application. Either > change (disallowing it completely as you suggest, or making it a > circular import, as I suggest) runs the risk of breaking code that > currently appears to work correctly. > > Fred's point about the practice of changing __name__ in the main > module corrupting generated pickles is one I hadn't thought of before > though. It's not just pickle; anything that requires __name__ (or __module__) to be accurate for introspection or debugging is also problematic. I have long considered it a 'best practice' (ugh, I hate that phrase, but I can't think of what else to call it) to _always_ do this type of shadowing, and avoid defining _any_ names in the __name__ == '__main__' case, so that there's no ambiguity: -------------- next part -------------- An HTML attachment was scrubbed... URL: From dirkjan at ochtman.nl Tue Jul 13 23:45:45 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Tue, 13 Jul 2010 23:45:45 +0200 Subject: [Python-Dev] notifications form Hg to python-checkins In-Reply-To: References: Message-ID: On Tue, Jul 13, 2010 at 23:22, Nick Coghlan wrote: > No, I meant push. There's a separate discussion where it was pointed > out that publishing each commit as a separate email makes > python-checkins even chattier than it is already (this point came up > after Tarek pushed a distutils2 changeset containing ~100 separate > commits), so the suggestion was made to combine all commit > notifications for a given push into a single email. However, I don't > remember if that idea was suggested specifically to Dirkjan as a > change to the notification email. If not, I'm suggesting it now :) As I recall, that's what I first implemented, but it was shot down by some of the committers... anyway, I'd be happy to change it back. Cheers, Dirkjan From ncoghlan at gmail.com Tue Jul 13 23:52:43 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 14 Jul 2010 07:52:43 +1000 Subject: [Python-Dev] avoiding accidental shadowing of top-level libraries by the main module In-Reply-To: References: <29559_1278893635_o6C0DslL000282_4C3A5983.3010002@holdenweb.com> <87sk3p2gzy.fsf@hydra.hampton.thirdcreek.com> <1D4152AC-DC35-4039-8970-22A0D5521C85@twistedmatrix.com> <20100712201201.GA21730@phd.pp.ru> <4C3B8C5A.3080308@voidspace.org.uk> <20100712235920.146ffb37@pitrou.net> <4C3B96E4.5010507@voidspace.org.uk> <4C3C938C.2060409@voidspace.org.uk> Message-ID: On Wed, Jul 14, 2010 at 7:34 AM, Fred Drake wrote: > On Tue, Jul 13, 2010 at 5:02 PM, Nick Coghlan wrote: >> Fred's point about the practice of changing __name__ in the main >> module corrupting generated pickles is one I hadn't thought of before >> though. > > Not sure about changing __name__ anywhere... I don't do that. The interpreter does though (when it sets it to "__main__" instead of what it would have been if the module had been imported rather than executed). That fact is convincing me that we should leave this alone for now. A module imported as __main__ and a module imported under it's real name are *different*, and this is reflected in pickles they generate and anything else that remembers the values of __name__. Obscuring that by treating them as equivalent in some contexts (such as name shadowing) while they remain different in other significant ways is unlikely to help overall. I must say, this whole discussion is actually making me far more sympathetic to the ideas behind PEP 299 (which proposed __main__ functions as a replacement for __name__ == "__main__" blocks). Brett's rejected PEP 3122 and my own accepted PEP 366 were both a direct result of the interpreter lying about the real value of __name__. I don't have a particular good way forward though, and the one thing I can say about the status quo is that at least everyone is accustomed to having to work around it. And back to the topic of inadvertent shadowing when teaching new users: PEP 3147 should actually help in that regard, as the interpreter will now ignore any implicitly created .pyc files if the corresponding .py file has been removed. Only explicitly created .pyc files that live where the source file would have been can completely substitute for a missing source file now. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Wed Jul 14 00:00:16 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 14 Jul 2010 08:00:16 +1000 Subject: [Python-Dev] notifications form Hg to python-checkins In-Reply-To: References: Message-ID: On Wed, Jul 14, 2010 at 7:45 AM, Dirkjan Ochtman wrote: > On Tue, Jul 13, 2010 at 23:22, Nick Coghlan wrote: >> No, I meant push. There's a separate discussion where it was pointed >> out that publishing each commit as a separate email makes >> python-checkins even chattier than it is already (this point came up >> after Tarek pushed a distutils2 changeset containing ~100 separate >> commits), so the suggestion was made to combine all commit >> notifications for a given push into a single email. However, I don't >> remember if that idea was suggested specifically to Dirkjan as a >> change to the notification email. If not, I'm suggesting it now :) > > As I recall, that's what I first implemented, but it was shot down by > some of the committers... anyway, I'd be happy to change it back. Having seen the per-commit approach in action, I'd personally like it changed back to per-push notifications, but you may want to start a thread on python-committers about it (given the number of tangents it has spawned, I wouldn't be surprised if a lot of people have stopped following this discussion by now). My rationale is as follows: - for small changesets with only one or two changes, per-push or per-commit emails don't make much difference - for large changesets, per-commit can be very noisy (as Tarek's 100+ notifications showed) while per-push stays the same I seem to have a vague recollection of the discussion you mention - do you remember if the specific objection was to a lack of a list of filenames affected in the subject line for the old per-push emails? Listing all the files affected by the push in the subject line would definitely still be desirable. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From tjreedy at udel.edu Wed Jul 14 03:45:25 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 13 Jul 2010 21:45:25 -0400 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: References: Message-ID: Summary: adding an autojunk heuristic to difflib without also adding a way to turn it off was a bug because it disabled running code. 2.6 and 3.1 each have, most likely, one final version each. Don't fix for these but add something to the docs explaining the problem and future fix. 2.7 will have several more versions over several years and will be used by newcomers who might encounter the problem but not know to diagnose it and patch a private copy of the module. So it should have a fix. Solutions thought of so far. 1. Modify the heuristic to somewhat fix the problem. Bad (unacceptable) because this would silently change behavior and could break tests. 2. Add a parameter that defaults to using the heuristic but allows turning it off. Perhaps better, but code that used the new API would crash if run on 2.7.0 3. Tim Peters > Think the most pressing thing is to give people a way to turn the damn > thing off. An ugly way would be to trigger on an unlikely > input-output behavior of the existing isjunk argument. For example, > if > > isjunk("what's the airspeed velocity of an unladen swallow?") > > returned > > "don't use auto junk!" > > and 2.7.1 recognized that as meaning "don't use auto junk", code could > be written under 2.7.1 that didn't blow up under 2.7. It could > _behave_ differently, although that's true of any way of disabling the > auto-junk heuristics. Ugly, but perhaps crazy brilliant. Use of such a hack would obviously be temporary. Perhaps its use could be made to issue a -3 warning if such were enabled. I would simplify the suggestion to something like isjunk("disable!heuristic") == True so one could pass lambda s:s=="disable!heuristic" It should be something easy to document and write. This issue is the only place such a string should appear, so it should be safe. Tim and Antoine: if you two can agree on what to do for 2.7, Eli and I will code it. This suggestion amounts to a suggestion that the fix for 2.7 be decoupled from a better fix for 3.2. I agree. The latter can be discussed once 2.7 is settled. [copied to the tracker] -- Terry Jan Reedy From g.rodola at gmail.com Wed Jul 14 08:35:33 2010 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Wed, 14 Jul 2010 08:35:33 +0200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <4C3C97E4.9000703@animats.com> References: <4C3C97E4.9000703@animats.com> Message-ID: One of the main problems with IDLE is the lack of tabs for editing multiple files within the same window. Having that alone would be a great improvement. --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil From g.brandl at gmx.net Wed Jul 14 10:10:02 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 14 Jul 2010 10:10:02 +0200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> Message-ID: Am 12.07.2010 00:51, schrieb Mark Lawrence: > I have been attempting to fill this hole and have been faced with > animosity from people who "hang out" on the python-dev IRC channel. I > thought it was a complete and utter waste of space, so I don't intend > going back. I agree with everything David said about this. My personal opinion is that you've done great work on the tracker, and like a few others, I've "rediscovered" a few issues I wanted to fix thanks to your "stirring up the silt". I don't think you have reason to be offended by criticism (which was even pointed out to you as such). Try hanging around a little bit longer, take nothing too seriously, and see if you still get nothing of value from #python-dev. > I would like things fixed, not a cosy little "who's round > is it next" mentality from the triage team. IMHO if they spent more > time doing things, and less time talking crap via IRC, things might get > done. Sure, and if it was work time, we probably would do this ;). As it is right now, this is volunteer time, and I would say that we're entitled to do whatever helps us getting done the (not always exciting) work, and our IRC crap talk, if that's what it is, happens to be among that. cheers, Georg From g.brandl at gmx.net Wed Jul 14 10:15:51 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 14 Jul 2010 10:15:51 +0200 Subject: [Python-Dev] notifications form Hg to python-checkins In-Reply-To: References: Message-ID: Am 14.07.2010 00:00, schrieb Nick Coghlan: > On Wed, Jul 14, 2010 at 7:45 AM, Dirkjan Ochtman wrote: >> On Tue, Jul 13, 2010 at 23:22, Nick Coghlan wrote: >>> No, I meant push. There's a separate discussion where it was pointed >>> out that publishing each commit as a separate email makes >>> python-checkins even chattier than it is already (this point came up >>> after Tarek pushed a distutils2 changeset containing ~100 separate >>> commits), so the suggestion was made to combine all commit >>> notifications for a given push into a single email. However, I don't >>> remember if that idea was suggested specifically to Dirkjan as a >>> change to the notification email. If not, I'm suggesting it now :) >> >> As I recall, that's what I first implemented, but it was shot down by >> some of the committers... anyway, I'd be happy to change it back. I think I was one of the "shooters" :) > Having seen the per-commit approach in action, I'd personally like it > changed back to per-push notifications, but you may want to start a > thread on python-committers about it (given the number of tangents it > has spawned, I wouldn't be surprised if a lot of people have stopped > following this discussion by now). > > My rationale is as follows: > - for small changesets with only one or two changes, per-push or > per-commit emails don't make much difference True. > - for large changesets, per-commit can be very noisy (as Tarek's 100+ > notifications showed) while per-push stays the same I assume you mean large pushes, consisting of tens of changesets? In this case, I see several problems with a single email: * it can get huge, and therefore a pain to load, and view in mail clients * it's needlessly clumsy to reply to single changes, since you have to go and delete the thousands of lines inbetween * there's no indication of what was pushed in the subject I also don't think we will see pushes like Tarek's 100+ one very often for Python. Usually changes will be bug fixes, and unless the committer is offline I expect them to be pushed immediately. > I seem to have a vague recollection of the discussion you mention - do > you remember if the specific objection was to a lack of a list of > filenames affected in the subject line for the old per-push emails? > Listing all the files affected by the push in the subject line would > definitely still be desirable. No, I think we agreed to put the (first line of the) commit message there, which I think tells potential reviewers much better if they want to look at that changeset. Georg From g.brandl at gmx.net Wed Jul 14 10:21:01 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 14 Jul 2010 10:21:01 +0200 Subject: [Python-Dev] python-checkins replies In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B916E.7050801@voidspace.org.uk> <4C3B9BA4.7050500@trueblade.com> <4C3BA867.6050906@voidspace.org.uk> Message-ID: Am 13.07.2010 22:29, schrieb Brett Cannon: > Given how high traffic python-checkins is I don't consider that a > reasonable place to send follow-up and nor do I consider it the > responsibility of committers to monitor it. As you said earlier this > *isn't* in our standard dev procedures and nor do I think it should be. > If you can't find an email address then either python-comitters or > python-dev would be a better place to send feedback. > > > Maybe reply-to on the checkin messages could be set to python-dev. Not > sure if that's a mailman feature, though. > > I think this would be a good idea. It would be nice to have on-topic traffic > here. :-) > > > Or python-committers since this is discussing code already checked in and thus > is somewhat committer-specific. This also has the perk of being easier to spot > (don't know about the rest of you but my python-committers filter makes those > emails much more obvious than python-dev traffic). I think I've suggested this once, but it met some resistance IIRC (it supposedly made our development exclusive). I'm still +1 on the idea though, and +1 on python-committers. Georg From dirkjan at ochtman.nl Wed Jul 14 10:26:23 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Wed, 14 Jul 2010 10:26:23 +0200 Subject: [Python-Dev] notifications form Hg to python-checkins In-Reply-To: References: Message-ID: On Wed, Jul 14, 2010 at 10:15, Georg Brandl wrote: > I also don't think we will see pushes like Tarek's 100+ one very often for > Python. ?Usually changes will be bug fixes, and unless the committer is > offline I expect them to be pushed immediately. Depends. If we do feature branches in separate clones, the merges for those can potentially push large numbers of csets at once. Presumably, we could push some more complexity into the hook, and have it send emails per-group for groups of 5 and larger and per-cset for smaller groups. > No, I think we agreed to put the (first line of the) commit message there, > which I think tells potential reviewers much better if they want to look > at that changeset. Not sure there was actual consensus on that, but I still stand behind this point. Cheers, Dirkjan From eric at trueblade.com Wed Jul 14 10:36:34 2010 From: eric at trueblade.com (Eric Smith) Date: Wed, 14 Jul 2010 04:36:34 -0400 Subject: [Python-Dev] python-checkins replies In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B916E.7050801@voidspace.org.uk> <4C3B9BA4.7050500@trueblade.com> <4C3BA867.6050906@voidspace.org.uk> Message-ID: <4C3D7712.7050201@trueblade.com> On 7/14/2010 4:21 AM, Georg Brandl wrote: > Am 13.07.2010 22:29, schrieb Brett Cannon: > >> Given how high traffic python-checkins is I don't consider that a >> reasonable place to send follow-up and nor do I consider it the >> responsibility of committers to monitor it. As you said earlier this >> *isn't* in our standard dev procedures and nor do I think it should be. >> If you can't find an email address then either python-comitters or >> python-dev would be a better place to send feedback. >> >> >> Maybe reply-to on the checkin messages could be set to python-dev. Not >> sure if that's a mailman feature, though. >> >> I think this would be a good idea. It would be nice to have on-topic traffic >> here. :-) >> >> >> Or python-committers since this is discussing code already checked in and thus >> is somewhat committer-specific. This also has the perk of being easier to spot >> (don't know about the rest of you but my python-committers filter makes those >> emails much more obvious than python-dev traffic). > > I think I've suggested this once, but it met some resistance IIRC (it supposedly > made our development exclusive). That's why I think it should go on python-dev. If the code hadn't been checked in and you were asking "what do you think of solving this by using the following code", I think you'd put it on python-dev. I'd want the discussion of an actual checkin to occur in that same venue. > I'm still +1 on the idea though, and +1 on python-committers. That said, I'm +1 on the idea, but only +0 on python-dev. Eric. From mg at lazybytes.net Wed Jul 14 11:31:34 2010 From: mg at lazybytes.net (Martin Geisler) Date: Wed, 14 Jul 2010 11:31:34 +0200 Subject: [Python-Dev] python-checkins References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B916E.7050801@voidspace.org.uk> <1278976111.3134.7.camel@localhost.localdomain> <87k4p0i437.fsf@uwakimon.sk.tsukuba.ac.jp> <4C3CC4E9.1000508@netwok.org> <20100713161004.5b084810@heresy> Message-ID: <87iq4iwgsp.fsf@hbox.dyndns.org> Dirkjan Ochtman writes: > - http://bitbucket.org/mg/commitsigs is another extension, which takes > a different tack to signing (I believe it doesn't sign the commit > metadata, only the file tree, which lets it sign before the commit is > finished, meaning it doesn't take up an extra cset). It does signs the complete changeset hash including the commit message, username, etc, and it does this without creating an extra changeset. This is done by computing what the changeset hash would be without an embedded signature. This hash is then signed and embedded in the changeset. This is similar to how TCP checksums are computed. This increases the size of each changeset by about 2 KB of data which cannot be compressed -- this adds up over time and I would only advice people to use the extension if they are very paranoid or have special legal requirements. -- Martin Geisler Mercurial links: http://mercurial.ch/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From steve at holdenweb.com Wed Jul 14 13:03:29 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 14 Jul 2010 07:03:29 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> Message-ID: Georg Brandl wrote: > Am 12.07.2010 00:51, schrieb Mark Lawrence: > >> I have been attempting to fill this hole and have been faced with >> animosity from people who "hang out" on the python-dev IRC channel. I >> thought it was a complete and utter waste of space, so I don't intend >> going back. > > I agree with everything David said about this. My personal opinion is > that you've done great work on the tracker, and like a few others, I've > "rediscovered" a few issues I wanted to fix thanks to your "stirring up > the silt". I don't think you have reason to be offended by criticism > (which was even pointed out to you as such). Try hanging around a little > bit longer, take nothing too seriously, and see if you still get nothing > of value from #python-dev. > >> I would like things fixed, not a cosy little "who's round >> is it next" mentality from the triage team. IMHO if they spent more >> time doing things, and less time talking crap via IRC, things might get >> done. > > Sure, and if it was work time, we probably would do this ;). As it is > right now, this is volunteer time, and I would say that we're entitled > to do whatever helps us getting done the (not always exciting) work, > and our IRC crap talk, if that's what it is, happens to be among that. I tend to agree with you, but the fact that it can deter stalwarts like Mark perhaps indicate that a little caution should be applied when dealing with newcomers or "outsiders". Without, I trust, spoiling the fun for the regulars. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Wed Jul 14 13:05:33 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 14 Jul 2010 07:05:33 -0400 Subject: [Python-Dev] Peculiar import code in pickle.py In-Reply-To: References: <20100713173418.22e42f15@pitrou.net> Message-ID: Brett Cannon wrote: > > > On Tue, Jul 13, 2010 at 11:34, Alexander Belopolsky > > > wrote: > > On Tue, Jul 13, 2010 at 1:57 PM, Benjamin Peterson > > wrote: > .. > > No! That's not recommended and a complete hack. The "dance" or > > importlib.import_module is preferred. > > Nevertheless, "a complete hack" is what PyImport_Import does: > > PyObject * > PyImport_Import(PyObject *module_name) > { > static PyObject *silly_list = NULL; > .. > /* Call the __import__ function with the proper argument list > * Always use absolute import here. */ > r = PyObject_CallFunction(import, "OOOOi", module_name, globals, > globals, silly_list, 0, NULL); > .. > } > > and _pickle.c uses PyImport_Import() and thus is different form > pickle.py which uses the double-lookup dance. As a result, the two > implementations are subtly different. They cannot be both right. It > should be easy to "fix" _pickle.c to do the sys.modules lookup, but I > am not sure this is right. > > > Pulling from sys.modules is the correct way to do this. There are subtle > issues when using a bunk fromlist argument (empty modules, double > initialization, etc.). If one does not use importlib.import_module -- > written *specifically* to prevent people from doing the nasty hack with > the fromlist -- then you should use the sys.modules approach, period. If > import.c is not doing this then it should get fixed. You can assign me > the issue if you want. > > I say this every time I give an import talk and it has been brought up > here before but obviously not everyone catches it (which is > understandable as I think when it came up on python-dev it was at the > tail end of a discussion), so I am just going to repeat myself: > > Do not put junk in fromlist if you call __import__ directly! Use > importlib.import_module! Or if you have a *really* good reason to not > use it, then use ``__import__(name); module = sys.modules[name]``. > > I have stopped fixing bugs related to this in import.c because of the > annoying issues it causes and I expect the correct approach to gain > traction at some point (plus get importlib bootstrapped in so I don't > have to care about import.c anymore). > It's only a matter of time until someone decides to provide a C implementation of importlib ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From ncoghlan at gmail.com Wed Jul 14 14:15:43 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 14 Jul 2010 22:15:43 +1000 Subject: [Python-Dev] Peculiar import code in pickle.py In-Reply-To: References: <20100713173418.22e42f15@pitrou.net> Message-ID: On Wed, Jul 14, 2010 at 9:05 PM, Steve Holden wrote: >> I have stopped fixing bugs related to this in import.c because of the >> annoying issues it causes and I expect the correct approach to gain >> traction at some point (plus get importlib bootstrapped in so I don't >> have to care about import.c anymore). >> > It's only a matter of time until someone decides to provide a C > implementation of importlib ;-) A C accelerated version of importlib would probably be an awful lot cleaner than the current import implementation. While the import code isn't quite the mess that we sometimes make it out to be (it does basically work after all, and most of the "problems" lie in dim dark corners that 99% of developers will never get close to), but it has definitely suffered from an accumulation of features on top of a core approach that has been pushed far beyond what it was originally designed to support. That said, I believe the limiting factor in import speed is likely to remain the number of stat calls and other filesystem operations, so it will be interesting to find out just how significant a slowdown there is between import.c and importlib. If I'm right about the real source of bottlenecks in import performance, the difference may be surprisingly small. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From steve at pearwood.info Wed Jul 14 14:38:37 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 14 Jul 2010 22:38:37 +1000 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: References: Message-ID: <201007142238.39331.steve@pearwood.info> On Wed, 14 Jul 2010 11:45:25 am Terry Reedy wrote: > Summary: adding an autojunk heuristic to difflib without also adding > a way to turn it off was a bug because it disabled running code. > > 2.6 and 3.1 each have, most likely, one final version each. Don't fix > for these but add something to the docs explaining the problem and > future fix. > > 2.7 will have several more versions over several years and will be > used by newcomers who might encounter the problem but not know to > diagnose it and patch a private copy of the module. So it should have > a fix. Solutions thought of so far. > > 1. Modify the heuristic to somewhat fix the problem. Bad > (unacceptable) because this would silently change behavior and could > break tests. > > 2. Add a parameter that defaults to using the heuristic but allows > turning it off. Perhaps better, but code that used the new API would > crash if run on 2.7.0 > > 3. > Tim Peters [... snip crazy scheme...] 4. I normally dislike global flags, but this is one time it might be less-worse than the alternatives. Modify SequenceMatcher to test for the value of a global flag, defaulting to False if it doesn't exist. try: disable = disable_heuristic except NameError: disable = False if disable: # use the new, correct behaviour else: # stick to the old, backwards-compatible but wrong behaviour The flag will only exist if the caller explicitly creates it: import difflib difflib.disable_heuristic = True On 2.7.0 and older versions, creating the flag won't do anything useful, but nor will it cause an exception. It will be harmless. I think an explicit flag is better than relying on magic behaviour triggered by "unlikely" input. -- Steven D'Aprano From breamoreboy at yahoo.co.uk Wed Jul 14 15:06:54 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Wed, 14 Jul 2010 14:06:54 +0100 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> Message-ID: On 14/07/2010 09:10, Georg Brandl wrote: > Am 12.07.2010 00:51, schrieb Mark Lawrence: > >> I have been attempting to fill this hole and have been faced with >> animosity from people who "hang out" on the python-dev IRC channel. I >> thought it was a complete and utter waste of space, so I don't intend >> going back. > > I agree with everything David said about this. My personal opinion is > that you've done great work on the tracker, and like a few others, I've > "rediscovered" a few issues I wanted to fix thanks to your "stirring up > the silt". I don't think you have reason to be offended by criticism > (which was even pointed out to you as such). Try hanging around a little > bit longer, take nothing too seriously, and see if you still get nothing > of value from #python-dev. > >> I would like things fixed, not a cosy little "who's round >> is it next" mentality from the triage team. IMHO if they spent more >> time doing things, and less time talking crap via IRC, things might get >> done. > > Sure, and if it was work time, we probably would do this ;). As it is > right now, this is volunteer time, and I would say that we're entitled > to do whatever helps us getting done the (not always exciting) work, > and our IRC crap talk, if that's what it is, happens to be among that. > > cheers, > Georg > Georg, Thanks to your response and the earlier one from David I now understand how things work. I'm actually on IRC right now. I consider this matter done and dusted, so let's move on shall we? Kindest regards. Mark Lawrence. From ncoghlan at gmail.com Wed Jul 14 15:45:41 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 14 Jul 2010 23:45:41 +1000 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: <201007142238.39331.steve@pearwood.info> References: <201007142238.39331.steve@pearwood.info> Message-ID: On Wed, Jul 14, 2010 at 10:38 PM, Steven D'Aprano wrote: > 4. I normally dislike global flags, but this is one time it might be > less-worse than the alternatives. > > Modify SequenceMatcher to test for the value of a global flag, > defaulting to False if it doesn't exist. > > try: > ? ?disable = disable_heuristic > except NameError: > ? ?disable = False > if disable: > ? ?# use the new, correct behaviour > else: > ? ?# stick to the old, backwards-compatible but wrong behaviour > > > > The flag will only exist if the caller explicitly creates it: > > import difflib > difflib.disable_heuristic = True > > On 2.7.0 and older versions, creating the flag won't do anything useful, > but nor will it cause an exception. It will be harmless. > > I think an explicit flag is better than relying on magic behaviour > triggered by "unlikely" input. Why make it a global? A property on the SequenceMatcher object should be able to do the right thing. @property def use_autojunk(self): if self._use_autojunk is None: raise AttributeError("...") # Same error as 2.7.0 would raise return self._use_autojunk @use_autojunk.set def use_autojunk(self, val): self._use_autojunk = bool(val) Then use the self._use_autojunk flag to decide whether or not to use the heuristic on 2.7.1+. Code that sets the flag would behave the same on both 2.7.1+ and on 2.7.0, it would just fail to turn the heuristic off in 2.7.0. Code that only read the flag without setting it would also behave the same on the whole 2.7.x series (i.e. raising AttributeError if the flag had not been explicitly set by the application) A flag on the object rather than a per-call flag may actually be the better API here anyway. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From Petre.Galan at gmail.com Wed Jul 14 15:50:44 2010 From: Petre.Galan at gmail.com (Petre Galan) Date: Wed, 14 Jul 2010 13:50:44 +0000 (UTC) Subject: [Python-Dev] More C API abstraction for user defined types References: Message-ID: Nick Coghlan gmail.com> writes: > > On Tue, Jul 13, 2010 at 6:44 PM, Mark Dickinson gmail.com> wrote: > > On Mon, Jul 12, 2010 at 10:19 PM, Nick Coghlan gmail.com> wrote: > >> On Tue, Jul 13, 2010 at 3:35 AM, Petre Galan gmail.com> wrote: > >>> ival should not be resolved through PyLong_AsLong, but through > >>> functionality/interface like PyNumber_Long > > > > +1, but I'd prefer it if PyNumber_Index were used, rather than PyNumber_Long. > > It shouldn't be possible to pass a Decimal instance to something expecting an > > integer argument. > > Oops, I misread the suggestion. Indeed, PyNumber_Index is the correct > API function for this purpose. > > Cheers, > Nick. > No. The right interface is PyNumber_Long. The purpose of the PyNumber_Index (and nb_index slot) is as index in slicing. Petre From tjreedy at udel.edu Wed Jul 14 18:34:27 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 14 Jul 2010 12:34:27 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> Message-ID: On 7/14/2010 4:10 AM, Georg Brandl wrote: > Sure, and if it was work time, we probably would do this ;). As it is > right now, this is volunteer time, and I would say that we're entitled > to do whatever helps us getting done the (not always exciting) work, > and our IRC crap talk, if that's what it is, happens to be among that. Aha! IRC ('chat') is like a social hour or cocktail party, with multiple simultaneous conversations. Some people must like such events and even find relaxing. I tend to find them uncomfortable and a bit of a chore to mentally process, at least until I can get deep enough into a conversation with 1 or 2 people to tune everyone else out. So, while a bit of business *might* get conducted, that is not the primary purpose. So I can see that pushing to make it a business meeting would not be too welcome. What I would like is an online sprint with a temporary #python-triage channel, with at least one commit-developer present. Pending that, is there any time of day when more people with commit privileges are likely to be present. The other problem I have is being dropped or timed out, and not having/knowing a way to get auto-reconnected to the channel. Thus, I could miss a response even if I do get one. -- Terry Jan Reedy From solipsis at pitrou.net Wed Jul 14 18:58:15 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 14 Jul 2010 18:58:15 +0200 Subject: [Python-Dev] Removing IDLE from the standard library References: <4C3A0F85.1090404@v.loewis.de> Message-ID: <20100714185815.1be1bdcc@pitrou.net> On Wed, 14 Jul 2010 12:34:27 -0400 Terry Reedy wrote: > > So I can see that pushing to make it a business meeting would not be too > welcome. What I would like is an online sprint with a temporary > #python-triage channel, with at least one commit-developer present. It should be noted that #python-dev is quite low-traffic. There is less than one message per minute on average, and not often more than one conversation at a time. > Pending that, is there any time of day when more people with commit > privileges are likely to be present. Generally any time of day, since different people are on different timezones. > The other problem I have is being dropped or timed out, and not > having/knowing a way to get auto-reconnected to the channel. Thus, I > could miss a response even if I do get one. I never get drops or timeouts. I suppose any decent IRC client will handle connection issues for you. Regards Antoine. From steve at holdenweb.com Wed Jul 14 19:02:36 2010 From: steve at holdenweb.com (Steve Holden) Date: Wed, 14 Jul 2010 13:02:36 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> Message-ID: Terry Reedy wrote: > On 7/14/2010 4:10 AM, Georg Brandl wrote: > >> Sure, and if it was work time, we probably would do this ;). As it is >> right now, this is volunteer time, and I would say that we're entitled >> to do whatever helps us getting done the (not always exciting) work, >> and our IRC crap talk, if that's what it is, happens to be among that. > > Aha! IRC ('chat') is like a social hour or cocktail party, with multiple > simultaneous conversations. Some people must like such events and even > find relaxing. I tend to find them uncomfortable and a bit of a chore to > mentally process, at least until I can get deep enough into a > conversation with 1 or 2 people to tune everyone else out. So, while a > bit of business *might* get conducted, that is not the primary purpose. > > So I can see that pushing to make it a business meeting would not be too > welcome. What I would like is an online sprint with a temporary > #python-triage channel, with at least one commit-developer present. > > Pending that, is there any time of day when more people with commit > privileges are likely to be present. > > The other problem I have is being dropped or timed out, and not > having/knowing a way to get auto-reconnected to the channel. Thus, I > could miss a response even if I do get one. > Speak to the twisted guys. It's about a line-and-a-half of code to run a logger on an IRC channel. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From tjreedy at udel.edu Wed Jul 14 19:32:21 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 14 Jul 2010 13:32:21 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3C97E4.9000703@animats.com> Message-ID: On 7/14/2010 2:35 AM, Giampaolo Rodol? wrote: > One of the main problems with IDLE is the lack of tabs for editing > multiple files within the same window. > Having that alone would be a great improvement. Yes, the same as tabs for browsing was. This is firstly an unlying gui widget set issue. Tk does not, as far as I know, have a tabbed document widget. Ttk has a new Notebook widget, with tabs. The second issue is whether, starting with 3.2, IDLE can require the availability of ttk, meaning tcl/tk 8.5+ or the Tile add-on. I opened http://bugs.python.org/issue9262 -- Terry Jan Reedy From brian.curtin at gmail.com Wed Jul 14 19:44:42 2010 From: brian.curtin at gmail.com (Brian Curtin) Date: Wed, 14 Jul 2010 12:44:42 -0500 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> Message-ID: On Wed, Jul 14, 2010 at 11:34, Terry Reedy wrote: > On 7/14/2010 4:10 AM, Georg Brandl wrote: > > Sure, and if it was work time, we probably would do this ;). As it is >> right now, this is volunteer time, and I would say that we're entitled >> to do whatever helps us getting done the (not always exciting) work, >> and our IRC crap talk, if that's what it is, happens to be among that. >> > > Pending that, is there any time of day when more people with commit > privileges are likely to be present. There never seems to be a lack of committers online. There are plenty of us here in the US, along with numerous committers from a host of European countries and elsewhere, so committer availability tends to follow the sun. Knowing that you are in the US, I don't think you'll come across many dead-zones. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ggpolo at gmail.com Wed Jul 14 19:45:57 2010 From: ggpolo at gmail.com (Guilherme Polo) Date: Wed, 14 Jul 2010 14:45:57 -0300 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3C97E4.9000703@animats.com> Message-ID: 2010/7/14 Terry Reedy : > On 7/14/2010 2:35 AM, Giampaolo Rodol? wrote: >> >> One of the main problems with IDLE is the lack of tabs for editing >> multiple files within the same window. >> Having that alone would be a great improvement. > > Yes, the same as tabs for browsing was. > > This is firstly an unlying gui widget set issue. Tk does not, as far as I > know, have a tabbed document widget. Ttk has a new Notebook widget, with > tabs. > I have worked on this before, and I can tell you that simply changing to ttk widgets is the easiest part. My recommendation, that you are free to ignore (especially if you want to skip this previous work), is: as a first step change the EditWindow to act more like a EditPage. That is, you should be able to instantiate a EditWindow and include this new EditPage on it (as a child or something else that you may imagine). > -- > Terry Jan Reedy > -- -- Guilherme H. Polo Goncalves From tjreedy at udel.edu Wed Jul 14 20:32:17 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 14 Jul 2010 14:32:17 -0400 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: References: <201007142238.39331.steve@pearwood.info> Message-ID: On 7/14/2010 9:45 AM, Nick Coghlan wrote: > Code that sets the flag would behave the same on both 2.7.1+ and on > 2.7.0, it would just fail to turn the heuristic off in 2.7.0. Antoine Pitrou pointed out on the tracker http://bugs.python.org/issue2986 that such code would *not* 'behave the same'. It would return different results (thus failing tests, if any). If, as would usually be the case, it *needs* the heuristic turned off to get sensible results, it would fail silently and deceptively. He thinks it better to fail noisily with an exception. Any code that depends on an x.y.z bugfix for proper operation and does not include conditional workaround for prior releases, will fail when run on prior releases. Whether the failure is a bad result (silent) or bad exception (noisy) usually depends on what the buggy behavior was. This case is an exception in that a. There is no workaround. b. A complete fix requires some api change. c. We have a choice of changes; some make failures on previous x.y releases silent (as they have been), others make them noisy -- without doing anything extra to make them noisy[*]. d. Hence we have a choice of failure mode. Hence the exceptional debate. [*] For the normal case of bad behavior for some input, we change the code to give the correct behavior. Raising an exception with previous releases would require extra conditional code. Here, by the nature of heuristic, there is no completely correct fix except to be able to turn it off. (If there were a correct deterministic rule, it would not be a heuristic ;=). The mechanism for turning it off might or might not raise an exception 'for free', with no added code for that. -- Terry Jan Reedy From alexander.belopolsky at gmail.com Wed Jul 14 21:13:51 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 14 Jul 2010 15:13:51 -0400 Subject: [Python-Dev] Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On Tue, Jul 13, 2010 at 3:52 PM, Alexander Belopolsky wrote: >.. and "import _strptime" had to be moved from function level to > module level after class definitions due to circular dependency of > _strptime on datetime. This turned out to be not such a great idea. Importing _strptime at the module level even after all datetime initializations does not work because of another circular dependency: _strptime imports calendar which imports calendar which imports datetime. New patch is at http://bugs.python.org/file17978/issue9206c.diff From brett at python.org Wed Jul 14 21:22:24 2010 From: brett at python.org (Brett Cannon) Date: Wed, 14 Jul 2010 12:22:24 -0700 Subject: [Python-Dev] python-checkins replies In-Reply-To: <4C3D7712.7050201@trueblade.com> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B916E.7050801@voidspace.org.uk> <4C3B9BA4.7050500@trueblade.com> <4C3BA867.6050906@voidspace.org.uk> <4C3D7712.7050201@trueblade.com> Message-ID: On Wed, Jul 14, 2010 at 01:36, Eric Smith wrote: > On 7/14/2010 4:21 AM, Georg Brandl wrote: > >> Am 13.07.2010 22:29, schrieb Brett Cannon: >> >> Given how high traffic python-checkins is I don't consider >>> that a >>> reasonable place to send follow-up and nor do I consider it >>> the >>> responsibility of committers to monitor it. As you said >>> earlier this >>> *isn't* in our standard dev procedures and nor do I think it >>> should be. >>> If you can't find an email address then either >>> python-comitters or >>> python-dev would be a better place to send feedback. >>> >>> >>> Maybe reply-to on the checkin messages could be set to >>> python-dev. Not >>> sure if that's a mailman feature, though. >>> >>> I think this would be a good idea. It would be nice to have on-topic >>> traffic >>> here. :-) >>> >>> >>> Or python-committers since this is discussing code already checked in and >>> thus >>> is somewhat committer-specific. This also has the perk of being easier to >>> spot >>> (don't know about the rest of you but my python-committers filter makes >>> those >>> emails much more obvious than python-dev traffic). >>> >> >> I think I've suggested this once, but it met some resistance IIRC (it >> supposedly >> made our development exclusive). >> > > That's why I think it should go on python-dev. If the code hadn't been > checked in and you were asking "what do you think of solving this by using > the following code", I think you'd put it on python-dev. Actually, I probably wouldn't. =) When it gets to explicit code, a design decision has been made, so I do not need to worry about involving the general public in some low-level technical discussion that won't impact them. > I'd want the discussion of an actual checkin to occur in that same venue. Right, which is why I want python-committers. Otherwise it's just a glorified commit lock when we are cutting releases. > > > I'm still +1 on the idea though, and +1 on python-committers. >> > > That said, I'm +1 on the idea, but only +0 on python-dev. +1 on python-committers, +0 on python-dev. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Wed Jul 14 21:33:55 2010 From: brett at python.org (Brett Cannon) Date: Wed, 14 Jul 2010 12:33:55 -0700 Subject: [Python-Dev] Peculiar import code in pickle.py In-Reply-To: References: <20100713173418.22e42f15@pitrou.net> Message-ID: On Wed, Jul 14, 2010 at 05:15, Nick Coghlan wrote: > On Wed, Jul 14, 2010 at 9:05 PM, Steve Holden wrote: > >> I have stopped fixing bugs related to this in import.c because of the > >> annoying issues it causes and I expect the correct approach to gain > >> traction at some point (plus get importlib bootstrapped in so I don't > >> have to care about import.c anymore). > >> > > It's only a matter of time until someone decides to provide a C > > implementation of importlib ;-) > > A C accelerated version of importlib would probably be an awful lot > cleaner than the current import implementation. While the import code > isn't quite the mess that we sometimes make it out to be (it does > basically work after all, and most of the "problems" lie in dim dark > corners that 99% of developers will never get close to), but it has > definitely suffered from an accumulation of features on top of a core > approach that has been pushed far beyond what it was originally > designed to support. > So my dream is to finally get full compatibility for importlib in 3.3 (probably won't hit 3.2 as it requires me changing marshal.loads to take a file path argument) and then try to bootstrap it in. Now bootstrapping can be done with actually a minimal amount of C code as I can simply make the bytecode for importlib a literal in C, get that loaded, and then import importlib as found on the file system to allow people to override things. But obviously I could also identify the true bottlenecks through profiling and provide acceleration where useful. Trick is being reasonable about this so as to not put other VMs at a disadvantage by making the bootstrap solution too difficult to implement. > > That said, I believe the limiting factor in import speed is likely to > remain the number of stat calls and other filesystem operations, so it > will be interesting to find out just how significant a slowdown there > is between import.c and importlib. If I'm right about the real source > of bottlenecks in import performance, the difference may be > surprisingly small. So I started writing benchmark code in anticipation of needing to prove a minimal performance difference to justify bootstrapping importlib. Right now it only compares importing from sys.modules and built-in modules. You can run it with ``./python.exe -m importlib.test.benchmark``. If you add a `-b` option that will use the built-in __import__ implementation. I still need to benchmark loading source, bytecode, writing bytecode, and loading extensions. Maybe I will finish writing the benchmark code as the thing I do while at EuroPython (that and finally getting to reviewing http://bugs.python.org/issue2919 so that cProfile and profile can merge, unless someone beats me to it, in which case I would be grateful =). -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Wed Jul 14 21:37:27 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 14 Jul 2010 15:37:27 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <20100714185815.1be1bdcc@pitrou.net> References: <4C3A0F85.1090404@v.loewis.de> <20100714185815.1be1bdcc@pitrou.net> Message-ID: On 7/14/2010 12:58 PM, Antoine Pitrou wrote: > On Wed, 14 Jul 2010 12:34:27 -0400 > Terry Reedy wrote: >> >> So I can see that pushing to make it a business meeting would not be too >> welcome. What I would like is an online sprint with a temporary >> #python-triage channel, with at least one commit-developer present. > > It should be noted that #python-dev is quite low-traffic. There is less > than one message per minute on average, and not often more than one > conversation at a time. I will take a look. >> The other problem I have is being dropped or timed out, and not >> having/knowing a way to get auto-reconnected to the channel. Thus, I >> could miss a response even if I do get one. > > I never get drops or timeouts. I suppose any decent IRC client will > handle connection issues for you. Is Chatzilla not decent, or is there a better Windows client. I have it set to reconnect when disconnected and to 'msg NickServe identify' when connected, but I do not see any way to wait for that to complete before issuing 'join #python'. I found a box under #python for 'rejoin when kicked' but I do not know if that will work for disconnections. -- Terry Jan Reedy From solipsis at pitrou.net Wed Jul 14 22:01:15 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 14 Jul 2010 22:01:15 +0200 Subject: [Python-Dev] importlib References: <20100713173418.22e42f15@pitrou.net> Message-ID: <20100714220115.305350b0@pitrou.net> On Wed, 14 Jul 2010 12:33:55 -0700 Brett Cannon wrote: > > So I started writing benchmark code in anticipation of needing to prove a > minimal performance difference to justify bootstrapping importlib. Right now > it only compares importing from sys.modules and built-in modules. You can > run it with ``./python.exe -m importlib.test.benchmark``. If you add a `-b` > option that will use the built-in __import__ implementation. In what unit are the numbers? In any case, here my results under a Linux system: $ ./python -m importlib.test.benchmark sys.modules [ 323782 326183 326667 ] best is 326667 Built-in module [ 33600 33693 33610 ] best is 33693 $ ./python -m importlib.test.benchmark -b sys.modules [ 1297640 1315366 1292283 ] best is 1315366 Built-in module [ 58180 57708 58057 ] best is 58180 Regards Antoine. From g.rodola at gmail.com Wed Jul 14 22:24:30 2010 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Wed, 14 Jul 2010 22:24:30 +0200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3C97E4.9000703@animats.com> Message-ID: 2010/7/14 Guilherme Polo : > 2010/7/14 Terry Reedy : >> On 7/14/2010 2:35 AM, Giampaolo Rodol? wrote: >>> >>> One of the main problems with IDLE is the lack of tabs for editing >>> multiple files within the same window. >>> Having that alone would be a great improvement. >> >> Yes, the same as tabs for browsing was. >> >> This is firstly an unlying gui widget set issue. Tk does not, as far as I >> know, have a tabbed document widget. Ttk has a new Notebook widget, with >> tabs. >> > > I have worked on this before, and I can tell you that simply changing > to ttk widgets is the easiest part. My recommendation, that you are > free to ignore (especially if you want to skip this previous work), > is: as a first step change the EditWindow to act more like a EditPage. > That is, you should be able to instantiate a EditWindow and include > this new EditPage on it (as a child or something else that you may > imagine). > >> -- >> Terry Jan Reedy >> > > > > -- > -- Guilherme H. Polo Goncalves > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > http://code.google.com/p/python-ttk/wiki/Screenshots Modified IDLE screenshots look impressive. Has Ttk (or something similar) ever been considered for inclusion? I think the problem here is more Tkinter itself rather than IDLE. Does spending energies on working on something (IDLE) based on such an old and features-limited module as Tkinter really worth the effort? Some major lacks like tabbed browsing can be fixed somehow but there always will be a big impediment deriving from using such an old graphic library which can't provide all the facilities offered by wx, gtk, etc... Maybe it would be better to first upgrade/improve Tkinter somehow; provide a gui toolkit that is *really* competitive, then start to seriously work on IDLE. My 2 cents --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil From tim.peters at gmail.com Wed Jul 14 22:40:54 2010 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 14 Jul 2010 16:40:54 -0400 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: References: <201007142238.39331.steve@pearwood.info> Message-ID: [Steven D'Aprano] >> 4. I normally dislike global flags, but this is one time it might be >> less-worse than the alternatives. >> >> Modify SequenceMatcher to test for the value of a global flag, >> defaulting to False if it doesn't exist. >> ... >> The flag will only exist if the caller explicitly creates it: >> >> import difflib >> difflib.disable_heuristic = True >> ... A module global is a non-starter for me (for all the usual reasons about the potentially horrid sociopathic consequences of one piece of an app changing behavior for the entire app - fine if it's a small app wholly understood by the user, potential nightmares if it's a large app nobody understands <0.3 wink>). [Nick Coghlan] > Why make it a global? A property on the SequenceMatcher object should > be able to do the right thing. > ... The pragmatic drawback is that the heuristics we're trying to disable typically execute _during_ SequenceMatcher instance construction. That's why it makes most sense to convey the information to the constructor. Setting a property after the instance is created is typically too late (unless several other SequenceMatcher methods are rewritten to delay analyzing the second sequence). > A flag on the object rather than a per-call flag may actually be the > better API here anyway. The call in question here is the constructor (__init__), so there's no real difference between "on the object" and "per call" in this case. From g.rodola at gmail.com Wed Jul 14 22:48:31 2010 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Wed, 14 Jul 2010 22:48:31 +0200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3C97E4.9000703@animats.com> Message-ID: http://docs.python.org/dev/whatsnew/2.7.html#ttk-themed-widgets-for-tk Sorry, I realized just now that ttk is already included in Python 2.7 and 3.2. 2010/7/14 Giampaolo Rodol? : > 2010/7/14 Guilherme Polo : >> 2010/7/14 Terry Reedy : >>> On 7/14/2010 2:35 AM, Giampaolo Rodol? wrote: >>>> >>>> One of the main problems with IDLE is the lack of tabs for editing >>>> multiple files within the same window. >>>> Having that alone would be a great improvement. >>> >>> Yes, the same as tabs for browsing was. >>> >>> This is firstly an unlying gui widget set issue. Tk does not, as far as I >>> know, have a tabbed document widget. Ttk has a new Notebook widget, with >>> tabs. >>> >> >> I have worked on this before, and I can tell you that simply changing >> to ttk widgets is the easiest part. My recommendation, that you are >> free to ignore (especially if you want to skip this previous work), >> is: as a first step change the EditWindow to act more like a EditPage. >> That is, you should be able to instantiate a EditWindow and include >> this new EditPage on it (as a child or something else that you may >> imagine). >> >>> -- >>> Terry Jan Reedy >>> >> >> >> >> -- >> -- Guilherme H. Polo Goncalves >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com >> > > http://code.google.com/p/python-ttk/wiki/Screenshots > Modified IDLE screenshots look impressive. > Has Ttk (or something similar) ever been considered for inclusion? > I think the problem here is more Tkinter itself rather than IDLE. > Does spending energies on working on something (IDLE) based on such an > old and features-limited module as Tkinter really worth the effort? > > Some major lacks like tabbed browsing can be fixed somehow but there > always will be a big impediment deriving from using such an old > graphic library which can't provide all the facilities offered by wx, > gtk, etc... > > Maybe it would be better to first upgrade/improve Tkinter somehow; > provide a gui toolkit that is *really* competitive, then start to > seriously work on IDLE. > > > My 2 cents > > --- Giampaolo > http://code.google.com/p/pyftpdlib > http://code.google.com/p/psutil > From timothy.c.delaney at gmail.com Wed Jul 14 23:21:42 2010 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Thu, 15 Jul 2010 07:21:42 +1000 Subject: [Python-Dev] notifications form Hg to python-checkins In-Reply-To: References: Message-ID: On 14 July 2010 18:26, Dirkjan Ochtman wrote: > On Wed, Jul 14, 2010 at 10:15, Georg Brandl wrote: > > I also don't think we will see pushes like Tarek's 100+ one very often > for > > Python. Usually changes will be bug fixes, and unless the committer is > > offline I expect them to be pushed immediately. > > Depends. If we do feature branches in separate clones, the merges for > those can potentially push large numbers of csets at once. > > Presumably, we could push some more complexity into the hook, and have > it send emails per-group for groups of 5 and larger and per-cset for > smaller groups. > > > No, I think we agreed to put the (first line of the) commit message > there, > > which I think tells potential reviewers much better if they want to look > > at that changeset. > > Not sure there was actual consensus on that, but I still stand behind > this point. > > If development work was done in named branches, the hook could send one email per branch that was delivered, and to be safe, one email per changeset added to a main feature branch. Tim Delaney -------------- next part -------------- An HTML attachment was scrubbed... URL: From mal at egenix.com Thu Jul 15 00:43:49 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 15 Jul 2010 00:43:49 +0200 Subject: [Python-Dev] str(memoryview('abc')) != 'abc' in Python 2.7 Message-ID: <4C3E3DA5.5010500@egenix.com> Is this intended or should I open a bug report for it: >>> m = memoryview('abc') >>> m == 'abc' True >>> str(m) == 'abc' False >>> str(m) '' I would have expected str(m) == 'abc'. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 15 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2010-07-19: EuroPython 2010, Birmingham, UK 3 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From ncoghlan at gmail.com Thu Jul 15 01:09:31 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 15 Jul 2010 09:09:31 +1000 Subject: [Python-Dev] More C API abstraction for user defined types In-Reply-To: References: Message-ID: On Wed, Jul 14, 2010 at 11:50 PM, Petre Galan wrote: > No. The right interface is PyNumber_Long. The purpose of the PyNumber_Index (and > nb_index slot) is as index in slicing. Allowing other objects to say "I'm a real integer, treat me as one" is exactly what the nb_index slot is for (see PEP 357). The use as an index in slicing was just the primary initial use case and the one that was chosen as the name for the new slot because nb_int was already taken for the lossy transformation. Indexing is not (and never has been) intended to be the only use case. From ncoghlan at gmail.com Thu Jul 15 01:12:28 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 15 Jul 2010 09:12:28 +1000 Subject: [Python-Dev] python-checkins replies In-Reply-To: References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B916E.7050801@voidspace.org.uk> <4C3B9BA4.7050500@trueblade.com> <4C3BA867.6050906@voidspace.org.uk> <4C3D7712.7050201@trueblade.com> Message-ID: On Thu, Jul 15, 2010 at 5:22 AM, Brett Cannon wrote: > Actually, I probably wouldn't. =) When it gets to explicit code, a design > decision has been made, so I do not need to worry about involving the > general public in some low-level technical discussion that won't impact > them. Yep, that's my perspective as well. If my post-commit comments are more significant than typo fixes and internal naming suggestions, I'll take them back to python-dev manually. So for me, +1 on python-committers, +0 on python-dev or the status quo. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From greg.ewing at canterbury.ac.nz Thu Jul 15 01:26:44 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 15 Jul 2010 11:26:44 +1200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3C97E4.9000703@animats.com> Message-ID: <4C3E47B4.90309@canterbury.ac.nz> Giampaolo Rodol? wrote: > One of the main problems with IDLE is the lack of tabs for editing > multiple files within the same window. While tabs seem to work well for web browsing, I'm not so sure about using them for source editing. Often I want to display two or more files side by side, which can't be done if they're in different tabs of the same window. -- Greg From ncoghlan at gmail.com Thu Jul 15 01:19:20 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 15 Jul 2010 09:19:20 +1000 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: References: <201007142238.39331.steve@pearwood.info> Message-ID: On Thu, Jul 15, 2010 at 6:40 AM, Tim Peters wrote: > The call in question here is the constructor (__init__), so there's no > real difference between "on the object" and "per call" in this case. You're right, I was misremembering how SequenceMatcher works. Terry's summary of the situation seems correct to me - adding a new flag to the constructor signature would mean we're taking a silent failure ("the heuristic makes my code give the wrong answer on 2.7.0") and making it a noisy failure ("my code needs to be able to turn the heuristic off to get the right answer, so it will fail noisily on 2.7.0"). That's a far cry from the True/False mistake. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Thu Jul 15 01:23:06 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 15 Jul 2010 09:23:06 +1000 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <4C3E47B4.90309@canterbury.ac.nz> References: <4C3C97E4.9000703@animats.com> <4C3E47B4.90309@canterbury.ac.nz> Message-ID: On Thu, Jul 15, 2010 at 9:26 AM, Greg Ewing wrote: > Giampaolo Rodol? wrote: >> >> One of the main problems with IDLE is the lack of tabs for editing >> multiple files within the same window. > > While tabs seem to work well for web browsing, I'm not so > sure about using them for source editing. Often I want to > display two or more files side by side, which can't be > done if they're in different tabs of the same window. I use tabbed editors all the time (Kate, Notepad++) and find them to be excellent. Tastes will obviously vary though, since there are even people out there that use vim and emacs voluntarily ;) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From alexander.belopolsky at gmail.com Thu Jul 15 01:24:28 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 14 Jul 2010 19:24:28 -0400 Subject: [Python-Dev] How to spell PyInstance_NewRaw in py3k? In-Reply-To: References: Message-ID: I am reposting the same question again because it seems to have gone unnoticed. Antoine Pitrou and I had a brief discussion on the tracker, but did not reach an agreement on whether a more elaborate code is needed to replace PyInstance_NewRaw than a simple type->tp_alloc() call. I have reviewed the patch again and I am convinced that this issue comes into play only when 3.x loads 2.x pickles that contain instances of classic classes. (Specifically, this code is used to process INST and OBJ pickle opcodes that are not produced by 3.x.) This means that Antoine's concern that "tomorrow [object_new()] may entail additional operations" is not valid - there is no tomorrow for 2.x. :-) This also means that the type cannot inherit from anything other than object and thus cannot have funny tp_flags or tp_alloc that does not create a usable object. I would like to commit the patch as presented. If a corner case is discovered later where type->tp_alloc() is not sufficient, we can deal with it then. On Mon, Jun 28, 2010 at 3:59 PM, Alexander Belopolsky wrote: > Issue #5180 [1] presented an interesting challenge: how to unpickle > instances of old-style classes when a pickle created with 2.x is > loaded in 3.x python? ?The problem is that pickle protocol requires > that unpickled instances be created without calling the __init__ > method. ? This is necessary because pickle file may not contain > information about how __init__ method should be invoked. ?Instead, > implementations are required to bypass ?__init__ and populate > instance's __dict__ directly using data found in the pickle. > > Pure python implementation uses the following trick that happens to work in 3.x: > > class Empty: > ? ?pass > > pickled = Empty() > pickled.__class__ = Pickled > > This of course, creates a new-style class in 3.x, but if 3.x version > of Pickled behaves similarly to its 2.x predecessor, it should work. > > The cPickle implementation, on the other hand uses 2.x C API which is > not available in 3.x. ?Namely, the PyInstance_NewRaw function. ?In > order to fix the bug described in issue #5180, I had to emulate > PyInstance_NewRaw using type->tp_alloc. ?I considered an rejected the > idea to use tp_new instead. [2] > > Is this the right way to proceed? ?The patch is attached to the issue. [3] > > > [1] http://bugs.python.org/issue5180 > [2] http://bugs.python.org/issue5180#msg108846 > [3] http://bugs.python.org/file17792/issue5180.diff > From ncoghlan at gmail.com Thu Jul 15 01:26:05 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 15 Jul 2010 09:26:05 +1000 Subject: [Python-Dev] str(memoryview('abc')) != 'abc' in Python 2.7 In-Reply-To: <4C3E3DA5.5010500@egenix.com> References: <4C3E3DA5.5010500@egenix.com> Message-ID: I wouldn't assume so - memoryview is meant to eventually support more than just 1-D views of contiguous memory (see PEP 3118), so that conversion doesn't seem intuitive to me. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From tim.peters at gmail.com Thu Jul 15 01:32:11 2010 From: tim.peters at gmail.com (Tim Peters) Date: Wed, 14 Jul 2010 19:32:11 -0400 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: References: <201007142238.39331.steve@pearwood.info> Message-ID: [Nick Coghlan] > You're right, I was misremembering how SequenceMatcher works. > > Terry's summary of the situation seems correct to me - adding a new > flag to the constructor signature would mean we're taking a silent > failure ("the heuristic makes my code give the wrong answer on 2.7.0") > and making it a noisy failure ("my code needs to be able to turn the > heuristic off to get the right answer, so it will fail noisily on > 2.7.0"). Yup - exactly so. > That's a far cry from the True/False mistake. Which I'm sure refers to introducing True and False builtins in a bugfix release. That was _almost_ as bad as the sometimes-hated heuristic I added to SequenceMatcher ;-) Me, I like "fail noisily on 2.7.0" for this case - practicality beats purity, especially when so few people and programs are likely to be affected by the backwards-incompatible-but-only-for-those-who-need-to-use-it SequenceMatcher constructor signature change. From greg.ewing at canterbury.ac.nz Thu Jul 15 01:58:14 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 15 Jul 2010 11:58:14 +1200 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3C97E4.9000703@animats.com> <4C3E47B4.90309@canterbury.ac.nz> Message-ID: <4C3E4F16.3090605@canterbury.ac.nz> Nick Coghlan wrote: > I use tabbed editors all the time (Kate, Notepad++) and find them to > be excellent. Tastes will obviously vary though, since there are even > people out there that use vim and emacs voluntarily ;) It's probably all right if you have the ability to move tabs from one window to another. My perceptions may be coloured a bit from using Crimson Editor on Windows, where you only get *one* top-level window that you can use either in tabbed or MDI mode. :-( I do use xemacs on linux, and with the ability to display any buffer in any window at will, it's fine. -- Greg From barry at python.org Thu Jul 15 01:59:55 2010 From: barry at python.org (Barry Warsaw) Date: Wed, 14 Jul 2010 19:59:55 -0400 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: References: <20100624115048.4fd152e3@heresy> Message-ID: <20100714195955.4815f980@heresy> On Jun 25, 2010, at 08:35 AM, Nick Coghlan wrote: >I like the idea, but I think summarising the rest of this discussion >in its own (relatively short) PEP would be good (there are a few >things that are tricky - exact versioning scheme, PEP 384 forward >compatibility, impact on distutils, articulating the benefits for >distro packaging, etc). The first draft of PEP 3149 is ready for review. http://www.python.org/dev/peps/pep-3149/ Plain text attached here for your convenience. Comments, suggestions as always are welcome. Thanks to everyone who participated in the original discussion. -Barry PEP: 3149 Title: ABI version tagged .so files Version: $Revision: 81577 $ Last-Modified: $Date: 2010-05-27 19:54:25 -0400 (Thu, 27 May 2010) $ Author: Barry Warsaw Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 2010-07-09 Python-Version: 3.2 Post-History: 2010-07-14 Resolution: TBD Abstract ======== PEP 3147 [1]_ described an extension to Python's import machinery that improved the sharing of Python source code, by allowing more than one byte compilation file (.pyc) to be co-located with each source file. This PEP defines an adjunct feature which allows the co-location of extension module files (.so) in a similar manner. This optional, build-time feature will enable downstream distributions of Python to more easily provide more than one Python major version at a time. Background ========== PEP 3147 defined the file system layout for a pure-Python package, where multiple versions of Python are available on the system. For example, where the `alpha` package containing source modules `one.py` and `two.py` exist on a system with Python 3.2 and 3.3, the post-byte compilation file system layout would be:: alpha/ __pycache__/ __init__.cpython-32.pyc __init__.cpython-33.pyc one.cpython-32.pyc one.cpython-33.pyc two.cpython-32.pyc two.cpython-33.pyc __init__.py one.py two.py For packages with extension modules, a similar differentiation is needed for the module's .so files. Extension modules compiled for different Python major versions are incompatible with each other due to changes in the ABI. While PEP 384 [2]_ defines a stable ABI, it will minimize, but not eliminate extension module incompatibilities between Python major versions. Thus a mechanism for discriminating extension module file names is proposed. Rationale ========= Linux distributions such as Ubuntu [3]_ and Debian [4]_ provide more than one Python version at the same time to their users. For example, Ubuntu 9.10 Karmic Koala users can install Python 2.5, 2.6, and 3.1, with Python 2.6 being the default. In order to share as much as possible between the available Python versions, these distributions install third party (i.e. non-standard library) packages into `/usr/share/pyshared` and symlink to them from `/usr/lib/pythonX.Y/dist-packages`. The symlinks exist because in a pre-PEP 3147 world (i.e < Python 3.2), the `.pyc` files resulting from byte compilation by the various installed Pythons will name collide with each other. For Python versions >= 3.2, all pure-Python packages can be shared, because the `.pyc` files will no longer cause file system naming conflicts. Eliminating these symlinks makes for a simpler, more robust Python distribution. A similar situation arises with shared library extensions. Because extension modules are typically named `foo.so` for a `foo` extension module, these would also name collide if `foo` was provided for more than one Python version. There are several approaches that could be taken to avoid this, which will be explored below, but this PEP proposes a fairly simple compile-time option to allow extension modules to live in the same file system directory and avoid any name collisions. Proposal ======== A new configure option is added for building Python, called `--with-so-abi-tag`. This takes as an argument a unique, but arbitrary string, e.g.:: ./configure --with-so-abi-tag=cpython-32 This string is passed into the `Makefile` and affects two aspects of the Python build. First, it is compiled into `Python/dynload_shlib.c` where it defines some additional `.so` file names to search for when importing extension modules. Second, it modifies the `Makefile`'s `$SO` variable, which in turn controls the `distutils` module's default filename when compiling extension modules. When `--with-so-abi-tag` is not given to `configure` nothing changes in the way the Python executable is built, or acts. Thus, this configure switch is completely optional and has no effect if not used. What this allows is for distributions that want to distinguish among extension modules built for different versions of Python, but shared in the same file system path, to arrange for `.so` names that are unique and non-colliding. For example, let's say Python 3.2 was built with:: ./configure --with-so-abi-tag=cpython-32 and Python 3.3 was built with:: ./configure --with-so-abi-tag=cpython-33 For an arbitrary package `foo`, you would see these files when the distribution package was installed:: /usr/share/pyshared/foo.cpython-32.so /usr/share/pyshared/foo.cpython-33.so Proven approach =============== The approach described here is already proven, in a sense, on Debian and Ubuntu system where different extensions are used for debug builds of Python and extension modules. Debug builds on Windows also already use a different file extension for dynamic libraries. PEP 384 ======= PEP 384 defines a stable ABI for extension modules. Universal adoption of PEP 384 would eliminate the need for this PEP because all extension modules would be compatible with any Python version. In practice of course, it will be impossible to achieve universal adoption. Older extensions may not be ported to PEP 384, or an extension may require Python APIs outside of PEP 384 definition. Therefore there will always be a (hopefully diminishing, but never zero) need for ABI version tagged shared libraries. Further, it is anticipated that the stable ABI will evolve over time, meaning that existing PEP 384 compatible extension modules may be incompatible with future versions of Python. While a complete specification is reserved for PEP 384, here is a discussion of the relevant issues. PEP 384 describes a change to ``PyModule_Create()`` where ``3`` is passed as the API version if the extension was complied with ``Py_LIMITED_API``. This should be formalized into an official macro called ``PYTHON_ABI_VERSION`` to mirror ``PYTHON_API_VERSION``. If and when the ABI changes in an incompatible way, this version number would be bumped. To facilitate sharing, Python would be extended to search for extension modules with the ``PYTHON_ABI_VERSION`` number in its name. The prefix ``abi`` is reserved for Python's use. Thus for example, an initial implementation of PEP 384, compiled with `--with-so-abi-tag=cpython-xy` would search for the following file names when extension module `foo` is imported (in this order):: foo.abi3.so foo.cpython-xy.so foo.so The distutils [7]_ ``build_ext`` command would also have to be extended to compile to shared library files with the ``abi3`` tag, when the module author indicates that their extension supports that version of the ABI. This could be done in a backward compatible way by adding a keyword argument to the ``Extension`` class, such as:: Extension('foo', ['foo.c'], abi=3) Alternatives ============ In the initial python-dev thread [8]_ where this idea was first introduced, several alternatives were suggested. For completeness they are listed here, along with the reasons for not adopting them. Independent directories or symlinks ----------------------------------- Debian and Ubuntu could simply add a version-specific directory to ``sys.path`` that would contain just the extension modules for that version of Python. Or the symlink trick eliminated in PEP 3147 could be retained for just shared libraries. This approach is rejected because it propagates the essential complexity that PEP 3147 tries to avoid, and adds yet another directory to search for all modules, even when the number of extension modules is much fewer than the total number of Python packages. It also makes for more robust management when all of a package's module files live in the same directory, because it allows systems such as `dpkg` to detect file conflicts between distribution packages. Don't share packages with extension modules ------------------------------------------- It has been suggested that Python packages with extension modules not be shared among all supported Python versions on a distribution. Even with adoption of PEP 3149, extension modules will have to be compiled for every supported Python version, so perhaps sharing of such packages isn't useful anyway. Not sharing packages with extensions though is infeasible for several reasons. If a pure-Python package is shared in one version, should it suddenly be not-shared if the next release adds an extension module for speed? Also, even though all extension shared libraries will be compiled and distributed once for every supported Python, there's a big difference between duplicating the `.so` files and duplicating all `.py` files. The extra space increases the download time for such packages, and more immediately, increases the space pressures on already constrained distribution CD-ROMs. Reference implementation ======================== Work on this code is tracked in a Bazaar branch on Launchpad [5]_ until it's ready for merge into Python 3.2. The work-in-progress diff can also be viewed [6]_ and is updated automatically as new changes are uploaded. References ========== .. [1] PEP 3147 .. [2] PEP 384 .. [3] Ubuntu: .. [4] Debian: .. [5] https://code.edge.launchpad.net/~barry/python/sovers .. [6] https://code.edge.launchpad.net/~barry/python/sovers/+merge/29411 .. [7] http://docs.python.org/py3k/distutils/index.html .. [8] http://mail.python.org/pipermail/python-dev/2010-June/100998.html Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From ggpolo at gmail.com Thu Jul 15 02:29:15 2010 From: ggpolo at gmail.com (Guilherme Polo) Date: Wed, 14 Jul 2010 21:29:15 -0300 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <4C3E47B4.90309@canterbury.ac.nz> References: <4C3C97E4.9000703@animats.com> <4C3E47B4.90309@canterbury.ac.nz> Message-ID: 2010/7/14 Greg Ewing : > Giampaolo Rodol? wrote: >> >> One of the main problems with IDLE is the lack of tabs for editing >> multiple files within the same window. > > While tabs seem to work well for web browsing, I'm not so > sure about using them for source editing. Often I want to > display two or more files side by side, which can't be > done if they're in different tabs of the same window. > Adding tabs doesn't necessarily mean a single window, you should be able to continue using multiple windows with single tabs if that is your preference. > -- > Greg -- -- Guilherme H. Polo Goncalves From ggpolo at gmail.com Thu Jul 15 02:29:15 2010 From: ggpolo at gmail.com (Guilherme Polo) Date: Wed, 14 Jul 2010 21:29:15 -0300 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <4C3E47B4.90309@canterbury.ac.nz> References: <4C3C97E4.9000703@animats.com> <4C3E47B4.90309@canterbury.ac.nz> Message-ID: 2010/7/14 Greg Ewing : > Giampaolo Rodol? wrote: >> >> One of the main problems with IDLE is the lack of tabs for editing >> multiple files within the same window. > > While tabs seem to work well for web browsing, I'm not so > sure about using them for source editing. Often I want to > display two or more files side by side, which can't be > done if they're in different tabs of the same window. > Adding tabs doesn't necessarily mean a single window, you should be able to continue using multiple windows with single tabs if that is your preference. > -- > Greg -- -- Guilherme H. Polo Goncalves From stephen at xemacs.org Thu Jul 15 04:43:40 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 15 Jul 2010 11:43:40 +0900 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3C97E4.9000703@animats.com> <4C3E47B4.90309@canterbury.ac.nz> Message-ID: <87k4oxh3c3.fsf@uwakimon.sk.tsukuba.ac.jp> Guilherme Polo writes: > Adding tabs doesn't necessarily mean a single window, you should be > able to continue using multiple windows with single tabs if that is > your preference. But it's very important to be able to *move* tabs across windows or panes. For example, in XEmacs this is a somewhat frequent pattern for me: C-x C-f /some/long/path/foo.c # edit foo.c # work for a while ... oops, need to see what's up with foo.h C-x C-f foo.h # look Ma, no path! # want to see both simultaneously to work on several prototypes in succession C-x 5 b foo.c # pull up foo.c in a new X # window, use mouse to position That's quite painless. In many apps, however, you would have to select the foo.c tab, close it, bring up a new window, and open foo.c using the long path (presumably with a file browser interface, but often enough the default directory is wherever you started the editor, not most recently used file). Recent Files may help, but even there some apps don't register a file with Recent Files unless it's been changed. Since these considerations are somewhat subtle, I think it's worth mentioning for your consideration. Eg, it's *not* clear to me that Recent Files should contain every file you've looked at, but if it doesn't, it has the bad effect described above. So if you're not registering a file with recent files, there should be some other shortcut way to move its display (eg, in a tab) across windows. N.B. While I personally love Emacsen (so there, Nick Coghlan!), I don't recommend modeling anything in IDLE on Emacs UIs, which were designed by a guy with RSI who couldn't pay rent (until after he got the MacArthur) let alone buy a 640x480-graphics-capable monitor. :-) From nyamatongwe at gmail.com Thu Jul 15 05:59:30 2010 From: nyamatongwe at gmail.com (Neil Hodgson) Date: Thu, 15 Jul 2010 13:59:30 +1000 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <87k4oxh3c3.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4C3C97E4.9000703@animats.com> <4C3E47B4.90309@canterbury.ac.nz> <87k4oxh3c3.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: Stephen J. Turnbull: > But it's very important to be able to *move* tabs across windows or > panes. ?... > In many apps, however, you would have to select the foo.c tab, close > it, bring up a new window, and open foo.c using the long path > (presumably with a file browser interface, but often enough the > default directory is wherever you started the editor, not most > recently used file). The common GUI technique is to drag a tab from one window into another window. Drag onto the desktop for a new top level window. This is supported by, among others, Firefox; Chrome; gedit; and GNOME Terminal. Neil From brett at python.org Thu Jul 15 08:06:58 2010 From: brett at python.org (Brett Cannon) Date: Wed, 14 Jul 2010 23:06:58 -0700 Subject: [Python-Dev] importlib In-Reply-To: <20100714220115.305350b0@pitrou.net> References: <20100713173418.22e42f15@pitrou.net> <20100714220115.305350b0@pitrou.net> Message-ID: On Wed, Jul 14, 2010 at 13:01, Antoine Pitrou wrote: > On Wed, 14 Jul 2010 12:33:55 -0700 > Brett Cannon wrote: >> >> So I started writing benchmark code in anticipation of needing to prove a >> minimal performance difference to justify bootstrapping importlib. Right now >> it only compares importing from sys.modules and built-in modules. You can >> run it with ``./python.exe -m importlib.test.benchmark``. If you add a `-b` >> option that will use the built-in __import__ implementation. > > In what unit are the numbers? Imports/second. I'll fix the code to state that. > > In any case, here my results under a Linux system: > > $ ./python -m importlib.test.benchmark > sys.modules [ 323782 326183 326667 ] best is 326667 > Built-in module [ 33600 33693 33610 ] best is 33693 > > $ ./python -m importlib.test.benchmark -b > sys.modules [ 1297640 1315366 1292283 ] best is 1315366 > Built-in module [ 58180 57708 58057 ] best is 58180 And this is what might make evaluating importlib tough; while the performance is 25% of what it is for import.c, being able to import over 300,000 times/second is still damn fast. From tjreedy at udel.edu Thu Jul 15 08:24:34 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 15 Jul 2010 02:24:34 -0400 Subject: [Python-Dev] Issue 2986: difflib.SequenceMatcher is partly broken In-Reply-To: References: <201007142238.39331.steve@pearwood.info> Message-ID: On 7/14/2010 7:32 PM, Tim Peters wrote: > [Nick Coghlan] >> You're right, I was misremembering how SequenceMatcher works. >> >> Terry's summary of the situation seems correct to me - adding a new >> flag to the constructor signature would mean we're taking a silent >> failure ("the heuristic makes my code give the wrong answer on 2.7.0") >> and making it a noisy failure ("my code needs to be able to turn the >> heuristic off to get the right answer, so it will fail noisily on >> 2.7.0"). > > Yup - exactly so. > >> That's a far cry from the True/False mistake. > > Which I'm sure refers to introducing True and False builtins in a > bugfix release. That was _almost_ as bad as the sometimes-hated > heuristic I added to SequenceMatcher ;-) > > Me, I like "fail noisily on 2.7.0" for this case - practicality beats > purity, especially when so few people and programs are likely to be > affected by the > backwards-incompatible-but-only-for-those-who-need-to-use-it > SequenceMatcher constructor signature change. Then tomorrow I will work with Eli on a suggested 2.6/3.1 doc patch and 2.7 code+doc patch. -- Terry Jan Reedy From raymond.hettinger at gmail.com Thu Jul 15 08:39:46 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Wed, 14 Jul 2010 23:39:46 -0700 Subject: [Python-Dev] str(memoryview('abc')) != 'abc' in Python 2.7 In-Reply-To: <4C3E3DA5.5010500@egenix.com> References: <4C3E3DA5.5010500@egenix.com> Message-ID: <44B07BFE-94D7-4B81-BDDD-40E448A37E3C@gmail.com> On Jul 14, 2010, at 3:43 PM, M.-A. Lemburg wrote: > Is this intended or should I open a bug report for it: > >>>> m = memoryview('abc') >>>> m == 'abc' > True >>>> str(m) == 'abc' > False >>>> str(m) > '' > > I would have expected str(m) == 'abc'. That is also my expectation. A memoryview object is iterable, so str(memviewobj) should return ''.join(memviewobj). In your example, that would be: >>> list(m) ['a', 'b', 'c'] >>> ''.join(m) 'abc' Raymond From raymond.hettinger at gmail.com Thu Jul 15 08:48:08 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Wed, 14 Jul 2010 23:48:08 -0700 Subject: [Python-Dev] Unordered tuples/lists In-Reply-To: <4BF572C7.9040703@v.loewis.de> References: <201005182313.42935.me@gustavonarea.net> <4BF572C7.9040703@v.loewis.de> Message-ID: On May 20, 2010, at 10:35 AM, Martin v. L?wis wrote: >> I think it'd be useful enough to go in the standard library. Now that >> there's a sample implementation, should I still try to demonstrate why I >> believe it's worth adding to the stdlib and get support? > > Most definitely. Just in case it isn't clear: nobody else seems to think > this is useful (let alone useful enough to go into the standard > library). In addition, it's trivial to implement, more reason not to add it. Count me among those who don't think its useful. I would oppose putting it in the collections module. But, there's no harm in posting a recipe on ASPN to see if it gains any traction. Raymond From mal at egenix.com Thu Jul 15 10:03:59 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 15 Jul 2010 10:03:59 +0200 Subject: [Python-Dev] str(memoryview('abc')) != 'abc' in Python 2.7 In-Reply-To: References: <4C3E3DA5.5010500@egenix.com> Message-ID: <4C3EC0EF.6080007@egenix.com> Nick Coghlan wrote: > I wouldn't assume so - memoryview is meant to eventually support more > than just 1-D views of contiguous memory (see PEP 3118), so that > conversion doesn't seem intuitive to me. In the example I'm passing in a single dimension contiguous memory chunk to memoryview(), so in that case I would expect str() of that memory chunk to return the same value and not the repr() of the object. buffer() already works like that. For non-contiguous memory views, the situation is less clear. AFAIK, non-contiguous views are mostly used to provide a non-copying view on a subset of multi-dimensional data, e.g. access to a row or column of a matrix, so the str() should probably return a contiguous copy of the selected data. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 15 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2010-07-19: EuroPython 2010, Birmingham, UK 3 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From techtonik at gmail.com Thu Jul 15 11:10:32 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Thu, 15 Jul 2010 12:10:32 +0300 Subject: [Python-Dev] python-checkins replies In-Reply-To: <4C3BA867.6050906@voidspace.org.uk> References: <4C3A0F85.1090404@v.loewis.de> <11989_1278874767_o6BIxON2026376_EA883BC9-5C6B-4444-B0D7-1BA51815328F@twistedmatrix.com> <4C3A415F.6050709@v.loewis.de> <4C3A4771.5090501@v.loewis.de> <20100712151927.456cbed3@pitrou.net> <4C3B2FB6.3050400@voidspace.org.uk> <4C3B533D.1080706@voidspace.org.uk> <4C3B916E.7050801@voidspace.org.uk> <4C3B9BA4.7050500@trueblade.com> <4C3BA867.6050906@voidspace.org.uk> Message-ID: On Tue, Jul 13, 2010 at 2:42 AM, Michael Foord wrote: >>> >>> Given how high traffic python-checkins is I don't consider that a >>> reasonable place to send follow-up and nor do I consider it the >>> responsibility of committers to monitor it. As you said earlier this >>> *isn't* in our standard dev procedures and nor do I think it should be. >>> If you can't find an email address then either python-comitters or >>> python-dev would be a better place to send feedback. >> >> Maybe reply-to on the checkin messages could be set to python-dev. Not >> sure if that's a mailman feature, though. > > I think this would be a good idea. It would be nice to have on-topic traffic > here. :-) +1 FWIW. It is a good practice to make code reviews public. It teaches people the proper ways and encourages to participate in reviews. -- anatoly t. From phd at phd.pp.ru Thu Jul 15 11:23:40 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Thu, 15 Jul 2010 13:23:40 +0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3C97E4.9000703@animats.com> <4C3E47B4.90309@canterbury.ac.nz> Message-ID: <20100715092340.GB3386@phd.pp.ru> On Thu, Jul 15, 2010 at 09:23:06AM +1000, Nick Coghlan wrote: > I use tabbed editors all the time (Kate, Notepad++) and find them to > be excellent. Tastes will obviously vary though, since there are even > people out there that use vim and emacs voluntarily ;) Sorry for being a wet blanket but vim implements tabbed windows even in console (text) mode. (-: Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From solipsis at pitrou.net Thu Jul 15 12:29:12 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 15 Jul 2010 12:29:12 +0200 Subject: [Python-Dev] How to spell PyInstance_NewRaw in py3k? References: Message-ID: <20100715122912.546cbb9c@pitrou.net> On Wed, 14 Jul 2010 19:24:28 -0400 Alexander Belopolsky wrote: > I am reposting the same question again because it seems to have gone > unnoticed. Antoine Pitrou and I had a brief discussion on the > tracker, but did not reach an agreement on whether a more elaborate > code is needed to replace PyInstance_NewRaw than a simple > type->tp_alloc() call. > > I have reviewed the patch again and I am convinced that this issue > comes into play only when 3.x loads 2.x pickles that contain instances > of classic classes. (Specifically, this code is used to process INST > and OBJ pickle opcodes that are not produced by 3.x.) This means that > Antoine's concern that "tomorrow [object_new()] may entail additional > operations" is not valid - there is no tomorrow for 2.x. :-) But there *is* a tomorrow in 3.x and that's what we are talking about. Your code is meant to emulate object_new() in 3.x. > This > also means that the type cannot inherit from anything other than > object and thus cannot have funny tp_flags or tp_alloc that does not > create a usable object. Why can't it inherit from something else than object? From solipsis at pitrou.net Thu Jul 15 12:37:54 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 15 Jul 2010 12:37:54 +0200 Subject: [Python-Dev] str(memoryview('abc')) != 'abc' in Python 2.7 References: <4C3E3DA5.5010500@egenix.com> Message-ID: <20100715123754.63e1f76a@pitrou.net> On Thu, 15 Jul 2010 00:43:49 +0200 "M.-A. Lemburg" wrote: > Is this intended or should I open a bug report for it: > > >>> m = memoryview('abc') > >>> m == 'abc' > True > >>> str(m) == 'abc' > False > >>> str(m) > '' Well, I think this is intended. str(m) is the human-readable string representation of the memoryview. In 3.x, you would write bytes(m) instead to have its bytes contents. Or, better again, use m.tobytes() in both versions. Regards Antoine. From ncoghlan at gmail.com Thu Jul 15 13:55:20 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 15 Jul 2010 21:55:20 +1000 Subject: [Python-Dev] importlib In-Reply-To: References: <20100713173418.22e42f15@pitrou.net> <20100714220115.305350b0@pitrou.net> Message-ID: On Thu, Jul 15, 2010 at 4:06 PM, Brett Cannon wrote: >> In any case, here my results under a Linux system: >> >> $ ./python -m importlib.test.benchmark >> sys.modules [ 323782 326183 326667 ] best is 326667 >> Built-in module [ 33600 33693 33610 ] best is 33693 >> >> $ ./python -m importlib.test.benchmark -b >> sys.modules [ 1297640 1315366 1292283 ] best is 1315366 >> Built-in module [ 58180 57708 58057 ] best is 58180 > > And this is what might make evaluating importlib tough; while the > performance is 25% of what it is for import.c, being able to import > over 300,000 times/second is still damn fast. Yeah, I think the numbers where the filesystem gets involved are going to be more relevant. Modules that have already been cached and those built in to the executable aren't likely to dominate interpreter and application startup times (which is the main thing I'm worried about seeing regress). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Thu Jul 15 14:13:50 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 15 Jul 2010 22:13:50 +1000 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: <20100715092340.GB3386@phd.pp.ru> References: <4C3C97E4.9000703@animats.com> <4C3E47B4.90309@canterbury.ac.nz> <20100715092340.GB3386@phd.pp.ru> Message-ID: On Thu, Jul 15, 2010 at 7:23 PM, Oleg Broytman wrote: > ? Sorry for being a wet blanket but vim implements tabbed windows even in > console (text) mode. (-: Oh, I know vim and emacs are actually incredibly powerful once you learn how to use them. I'm just a child of the GUI generation and believe UIs should be readily discoverable in accordance with http://xkcd.com/627/. I've tried to apply that technique to both applications in the past and failed miserably (although I will admit I haven't had the inclination to even try to use either of them for years now). Neither really strikes me as just a text editor, but more a way of life ;) Anyway, to bring this back on topic... Neither Kate nor Notepad++ allow you to easily move documents between windows (you have to open them explicitly in the second window). I had never noticed the lack until I explicitly checked for it after Stephen's last email. I suspect whether you consider this a must have feature or not depends greatly on your personal workflow when coding. If IDLE were to adopt a tabbed view without easy migration between separate windows, it would have plenty of company. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From merwok at netwok.org Thu Jul 15 14:26:52 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Thu, 15 Jul 2010 14:26:52 +0200 Subject: [Python-Dev] notifications form Hg to python-checkins In-Reply-To: References: Message-ID: <4C3EFE8C.7060605@netwok.org> [Nick] >[me] >> IIRC, you can?t know who pushed without kludgy hackery. > > [...] Note that the current distutils2 emails to python-checkins > already say "Tarek Ziade pushed..." I looked at mail.py in the hooks repository and learned that hooks get an HGPUSHER variable in their environment. Nice! Regards From alexander.belopolsky at gmail.com Thu Jul 15 16:12:34 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 15 Jul 2010 10:12:34 -0400 Subject: [Python-Dev] How to spell PyInstance_NewRaw in py3k? In-Reply-To: <20100715122912.546cbb9c@pitrou.net> References: <20100715122912.546cbb9c@pitrou.net> Message-ID: On Thu, Jul 15, 2010 at 6:29 AM, Antoine Pitrou wrote: > On Wed, 14 Jul 2010 19:24:28 -0400 > Alexander Belopolsky wrote: .. >>?This means that >> Antoine's concern that "tomorrow [object_new()] may entail additional >> operations" is not valid ?- there is no tomorrow for 2.x. :-) > > But there *is* a tomorrow in 3.x and that's what we are talking about. > Your code is meant to emulate object_new() in 3.x. > Yes, I realized that after I hit the sent button. However, it is 2.x classic instances that are being unpickled and it won't be reasonable for 3.x objects that are expected to emulate them to do anything non-trivial. The need to be able to faithfully emulate classic instances unpickled from 2.x may be a valid constraint to future object_new() evolution. >> This >> also means that the type cannot inherit from anything other than >> object and thus cannot have funny tp_flags or tp_alloc that does not >> create a usable object. > > Why can't it inherit from something else than object? Because this would not be a reasonable way to forward port 2.x classic classes and expect them to interoperate with 2.x pickles. There are many ways to break unpickling of old pickles by modifying the class in the new version of code. The serious question for me is whether a valid tp_alloc can create a partially initialized object that will crash the interpreter when its method is called. I don't think this is the case because certainly you need to be able to delete the object if tp_new fails and that can call arbitrary code. From techtonik at gmail.com Thu Jul 15 18:13:52 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Thu, 15 Jul 2010 19:13:52 +0300 Subject: [Python-Dev] importlib In-Reply-To: References: <20100713173418.22e42f15@pitrou.net> <20100714220115.305350b0@pitrou.net> Message-ID: On Thu, Jul 15, 2010 at 2:55 PM, Nick Coghlan wrote: >>> In any case, here my results under a Linux system: >>> >>> $ ./python -m importlib.test.benchmark >>> sys.modules [ 323782 326183 326667 ] best is 326667 >>> Built-in module [ 33600 33693 33610 ] best is 33693 >>> >>> $ ./python -m importlib.test.benchmark -b >>> sys.modules [ 1297640 1315366 1292283 ] best is 1315366 >>> Built-in module [ 58180 57708 58057 ] best is 58180 >> >> And this is what might make evaluating importlib tough; while the >> performance is 25% of what it is for import.c, being able to import >> over 300,000 times/second is still damn fast. > > Yeah, I think the numbers where the filesystem gets involved are going > to be more relevant. Modules that have already been cached and those > built in to the executable aren't likely to dominate interpreter and > application startup times (which is the main thing I'm worried about > seeing regress). What about web-applications? Is that true that for FastCgi or mod_wsgi deamon mode interpreter and application is started only once per say 100 requests? -- anatoly t. From tjreedy at udel.edu Thu Jul 15 18:19:58 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 15 Jul 2010 12:19:58 -0400 Subject: [Python-Dev] Whither 'trunk' Message-ID: The 'trunk' branch appears to have been frozen 12 days ago when 2.7 was released. I presume py3k is now the main development branch. Correct? There are doc(s) on the site the directed people to the 'trunk' branch. If not updated (as seems from a python-list post today, but I asked the OP), it/they should be. -- Terry Jan Reedy From rrr at ronadam.com Thu Jul 15 19:23:41 2010 From: rrr at ronadam.com (Ron Adam) Date: Thu, 15 Jul 2010 12:23:41 -0500 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3C97E4.9000703@animats.com> <4C3E47B4.90309@canterbury.ac.nz> <20100715092340.GB3386@phd.pp.ru> Message-ID: On 07/15/2010 07:13 AM, Nick Coghlan wrote: > On Thu, Jul 15, 2010 at 7:23 PM, Oleg Broytman wrote: >> Sorry for being a wet blanket but vim implements tabbed windows even in >> console (text) mode. (-: > > Oh, I know vim and emacs are actually incredibly powerful once you > learn how to use them. I'm just a child of the GUI generation and > believe UIs should be readily discoverable in accordance with > http://xkcd.com/627/. I've tried to apply that technique to both > applications in the past and failed miserably (although I will admit I > haven't had the inclination to even try to use either of them for > years now). Neither really strikes me as just a text editor, but more > a way of life ;) > > Anyway, to bring this back on topic... > > Neither Kate nor Notepad++ allow you to easily move documents between > windows (you have to open them explicitly in the second window). I had > never noticed the lack until I explicitly checked for it after > Stephen's last email. I suspect whether you consider this a must have > feature or not depends greatly on your personal workflow when coding. > If IDLE were to adopt a tabbed view without easy migration between > separate windows, it would have plenty of company. My preference would to just have two tabs and/or panes per window. One for the source listing and one for the command interface. It's easy enough to have multiple idle windows open and cut and copy between them. The file window could have an "open in new window" option which would start a new idle instance. That keeps things simple and would be easy enough to understand by the majority of new/novice to computer/programming users. It also avoid having the command window shared by multiple programs which could cause all sorts of messy output or errors if multiple scripts are being run at the same time. Ron From g.rodola at gmail.com Thu Jul 15 20:45:53 2010 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Thu, 15 Jul 2010 20:45:53 +0200 Subject: [Python-Dev] profiler decorator - is it worth for inclusion? Message-ID: Today I was looking for a quick and dirty way to profile a method of a class. I was thinking that cProfile module had a decorator for this but I was wrong so I decided to write one based on hotshot. Would it be worth for inclusion? #!/usr/bin/env python import hotshot import hotshot.stats import tempfile import pstats def profile(sort='cumulative', lines=30, strip_dirs=False): """A decorator which profiles a callable. Example usage: >>> @profile() ... def factorial(n): ... n = abs(int(n)) ... if n < 1: ... n = 1 ... x = 1 ... for i in range(1, n+1): ... x = i * x ... return x ... >>> factorial(5) 1 function calls in 0.000 CPU seconds Ordered by: internal time, call count ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 profiler.py:60(factorial) 0 0.000 0.000 profile:0(profiler) 120 >>> """ def outer(fun): def inner(*args, **kwargs): file = tempfile.NamedTemporaryFile() prof = hotshot.Profile(file.name) try: ret = prof.runcall(fun, *args, **kwargs) finally: prof.close() stats = hotshot.stats.load(file.name) if strip_dirs: stats.strip_dirs() if isinstance(sort, tuple): stats.sort_stats(*sort) else: stats.sort_stats(sort) stats.print_stats(lines) return ret return inner return outer From brian.curtin at gmail.com Thu Jul 15 20:55:30 2010 From: brian.curtin at gmail.com (Brian Curtin) Date: Thu, 15 Jul 2010 13:55:30 -0500 Subject: [Python-Dev] profiler decorator - is it worth for inclusion? In-Reply-To: References: Message-ID: On Thu, Jul 15, 2010 at 13:45, Giampaolo Rodol? wrote: > Today I was looking for a quick and dirty way to profile a method of a > class. > I was thinking that cProfile module had a decorator for this but I was > wrong so I decided to write one based on hotshot. > Would it be worth for inclusion? Since hotshot is gone in 3.x, I'd guess the chances are probably slim. -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.rodola at gmail.com Thu Jul 15 21:24:39 2010 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Thu, 15 Jul 2010 21:24:39 +0200 Subject: [Python-Dev] profiler decorator - is it worth for inclusion? In-Reply-To: References: Message-ID: 2010/7/15 Brian Curtin : > On Thu, Jul 15, 2010 at 13:45, Giampaolo Rodol? wrote: >> >> Today I was looking for a quick and dirty way to profile a method of a >> class. >> I was thinking that cProfile module had a decorator for this but I was >> wrong so I decided to write one based on hotshot. >> Would it be worth for inclusion? > > Since hotshot is gone in 3.x, I'd guess the chances are probably slim. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com > > Here's one using cProfile instead. I was using hotshot because I wasn't aware of cProfile.Profile.runcall which is currently not documented (I'm going to file a bug report). def profile(sort='cumulative', lines=30, strip_dirs=True): """A decorator which profiles a callable. Example usage: >>> @profile() ... def factorial(n): ... n = abs(int(n)) ... if n < 1: ... n = 1 ... x = 1 ... for i in range(1, n+1): ... x = i * x ... return x ... >>> factorial(5) Thu Jul 15 20:58:21 2010 /tmp/tmpIDejr5 4 function calls in 0.000 CPU seconds Ordered by: internal time, call count ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 0.000 0.000 profiler.py:120(factorial) 1 0.000 0.000 0.000 0.000 {range} 1 0.000 0.000 0.000 0.000 {abs} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} 120 """ def outer(fun): def inner(*args, **kwargs): file = tempfile.NamedTemporaryFile() prof = cProfile.Profile() try: ret = prof.runcall(fun, *args, **kwargs) except: file.close() raise prof.dump_stats(file.name) stats = pstats.Stats(file.name) if strip_dirs: stats.strip_dirs() if isinstance(sort, tuple): stats.sort_stats(*sort) else: stats.sort_stats(sort) stats.print_stats(lines) file.close() return ret return inner return outer --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil From debatem1 at gmail.com Thu Jul 15 21:57:50 2010 From: debatem1 at gmail.com (geremy condra) Date: Thu, 15 Jul 2010 12:57:50 -0700 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3C97E4.9000703@animats.com> <4C3E47B4.90309@canterbury.ac.nz> <20100715092340.GB3386@phd.pp.ru> Message-ID: On Thu, Jul 15, 2010 at 5:13 AM, Nick Coghlan wrote: > On Thu, Jul 15, 2010 at 7:23 PM, Oleg Broytman wrote: >> ? Sorry for being a wet blanket but vim implements tabbed windows even in >> console (text) mode. (-: > > Oh, I know vim and emacs are actually incredibly powerful once you > learn how to use them. I'm just a child of the GUI generation and > believe UIs should be readily discoverable in accordance with > http://xkcd.com/627/. I've tried to apply that technique to both > applications in the past and failed miserably (although I will admit I > haven't had the inclination to even try to use either of them for > years now). Neither really strikes me as just a text editor, but more > a way of life ;) > > Anyway, to bring this back on topic... > > Neither Kate nor Notepad++ allow you to easily move documents between > windows (you have to open them explicitly in the second window). I had > never noticed the lack until I explicitly checked for it after > Stephen's last email. I suspect whether you consider this a must have > feature or not depends greatly on your personal workflow when coding. > If IDLE were to adopt a tabbed view without easy migration between > separate windows, it would have plenty of company. It might be, but I'll admit that my work habits (used to be all-vim-all-the-time) have changed substantially since finding the joy of gedit's tab system (and plugins). Something along these lines is probably necessary to be competitive, at least on Linux. Geremy Condra From nagle at animats.com Thu Jul 15 22:06:28 2010 From: nagle at animats.com (John Nagle) Date: Thu, 15 Jul 2010 13:06:28 -0700 Subject: [Python-Dev] importlib In-Reply-To: References: Message-ID: <4C3F6A44.7040209@animats.com> anatoly techtonik wrote: > What about web-applications? Is that true that for FastCgi or mod_wsgi > deamon mode interpreter and application is started only once per say > 100 requests? Yes. Only CGI programs reload on every use. FCGI/WSGI programs run more or less forever, and are only loaded once, reading requests and handling them. Some FCGI systems reload their applications once in a while, and ask the idle ones to exit after periods of no traffic, but that's a form of automatic performance tuning. And, of course, there are the permanently resident systems like Twisted. (Incidentally, if you run FCGI/WSGI programs under Apache, make sure that you don't put .fcgi scripts in a directory from which Apache will execute .cgi scripts. Apache will run the .fcgi script with CGI, and the WSGI Python module will detect this and run the script one time and exit for each request. Everything appears to work and performance is terrible.) If import performance is important in a production server environment, it's more useful to look at minimizing the number of loads rather than trying to make them faster. If and when Unladen Swallow finally gets going, startup times will get worse as the JIT compiler cranks, but run time will (hopefully) decrease. John Nagle From steve at holdenweb.com Thu Jul 15 23:04:19 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 15 Jul 2010 17:04:19 -0400 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3C97E4.9000703@animats.com> <4C3E47B4.90309@canterbury.ac.nz> <87k4oxh3c3.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: Neil Hodgson wrote: > Stephen J. Turnbull: > >> But it's very important to be able to *move* tabs across windows or >> panes. ... >> In many apps, however, you would have to select the foo.c tab, close >> it, bring up a new window, and open foo.c using the long path >> (presumably with a file browser interface, but often enough the >> default directory is wherever you started the editor, not most >> recently used file). > > The common GUI technique is to drag a tab from one window into > another window. Drag onto the desktop for a new top level > window. This is supported by, among others, Firefox; Chrome; gedit; > and GNOME Terminal. > Time used to be we would innovate, not imitate, but I suppose that convergence has changed things. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From brett at python.org Thu Jul 15 23:21:11 2010 From: brett at python.org (Brett Cannon) Date: Thu, 15 Jul 2010 14:21:11 -0700 Subject: [Python-Dev] Whither 'trunk' In-Reply-To: References: Message-ID: On Thu, Jul 15, 2010 at 09:19, Terry Reedy wrote: > The 'trunk' branch appears to have been frozen 12 days ago when 2.7 was > released. I presume py3k is now the main development branch. Correct? > Yes. > There are doc(s) on the site the directed people to the 'trunk' branch. If > not updated (as seems from a python-list post today, but I asked the OP), > it/they should be. What pages? I thought I updated the developer docs, but maybe I only did the dev FAQ. From amauryfa at gmail.com Fri Jul 16 00:16:50 2010 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Fri, 16 Jul 2010 00:16:50 +0200 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <20100714195955.4815f980@heresy> References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> Message-ID: Hello, 2010/7/15 Barry Warsaw : > The first draft of PEP 3149 is ready for review. I like it! I think it could mention the case where packages are not installed in the canonical directory, but placed elsewhere along the PYTHONPATH. This is how I deploy applications, for example, and the differences between python versions makes this a pain. The specific case of Windows should be mentioned: each foo.pyd contains the name of the python library (Python27.dll) it has been linked with; It must be rebuilt for a major version change. IMO the Windows installers provided by python.org should be built with a tag that contains this major number. > Thus for example, an initial implementation of PEP 384, compiled with > `--with-so-abi-tag=cpython-xy` would search for the following file > names when extension module `foo` is imported (in this order):: > > ? ?foo.abi3.so > ? ?foo.cpython-xy.so > ? ?foo.so Is this the correct order? IMO the so-abi-tag is more precise and the two first items should be swapped. PyPy would also benefit from this patch: it can now use extension modules, but the ABI is slightly different. By default, PyPy would load only modules containing the ABI tag, and refuse foo.so which is incompatible for sure. But the installations could still be shared between Python implementations. Cheers, -- Amaury Forgeot d'Arc From debatem1 at gmail.com Fri Jul 16 00:23:12 2010 From: debatem1 at gmail.com (geremy condra) Date: Thu, 15 Jul 2010 15:23:12 -0700 Subject: [Python-Dev] Removing IDLE from the standard library In-Reply-To: References: <4C3C97E4.9000703@animats.com> <4C3E47B4.90309@canterbury.ac.nz> <87k4oxh3c3.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Thu, Jul 15, 2010 at 2:04 PM, Steve Holden wrote: > Neil Hodgson wrote: >> Stephen J. Turnbull: >> >>> But it's very important to be able to *move* tabs across windows or >>> panes. ?... >>> In many apps, however, you would have to select the foo.c tab, close >>> it, bring up a new window, and open foo.c using the long path >>> (presumably with a file browser interface, but often enough the >>> default directory is wherever you started the editor, not most >>> recently used file). >> >> ? ?The common GUI technique is to drag a tab from one window into >> another window. Drag onto the desktop for a new top level >> ?window. This is supported by, among others, Firefox; Chrome; gedit; >> and GNOME Terminal. >> > Time used to be we would innovate, not imitate, but I suppose that > convergence has changed things. Nobody is being innovative with IDLE's UI right now. Some imitation would do it good, however. Geremy Condra From doko at ubuntu.com Fri Jul 16 12:40:44 2010 From: doko at ubuntu.com (Matthias Klose) Date: Fri, 16 Jul 2010 12:40:44 +0200 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <20100714195955.4815f980@heresy> References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> Message-ID: <4C40372C.9030101@ubuntu.com> On 15.07.2010 01:59, Barry Warsaw wrote: > PEP 384 describes a change to ``PyModule_Create()`` where ``3`` is > passed as the API version if the extension was complied with > ``Py_LIMITED_API``. This should be formalized into an official macro > called ``PYTHON_ABI_VERSION`` to mirror ``PYTHON_API_VERSION``. If > and when the ABI changes in an incompatible way, this version number > would be bumped. To facilitate sharing, Python would be extended to > search for extension modules with the ``PYTHON_ABI_VERSION`` number in > its name. The prefix ``abi`` is reserved for Python's use. > > Thus for example, an initial implementation of PEP 384, compiled with > `--with-so-abi-tag=cpython-xy` would search for the following file > names when extension module `foo` is imported (in this order):: > > foo.abi3.so > foo.cpython-xy.so > foo.so > > The distutils [7]_ ``build_ext`` command would also have to be > extended to compile to shared library files with the ``abi3`` tag, > when the module author indicates that their extension supports that > version of the ABI. This could be done in a backward compatible way > by adding a keyword argument to the ``Extension`` class, such as:: > > Extension('foo', ['foo.c'], abi=3) I like the proposal, but IMO it is too unspecific about the abi tag. Assume that an extension is built with such a configured python and then tried to run with an abi compatible configured python, but with a slightly different version tag, the extension won't be found. Differing file names per configuration should be avoided. Proposing 1) Remove the configure option and use the new naming using the tag for all configurations unconditionally. Everybody can expect the same name on every installation, and PEP 384 most likely will require using a tag too. As Amaury did point out, there is a use case in that PyPy can use this tag too to build extensions only usable by PyPy. 2) As PEP 3147 defines a non-configurable name for .pyc files, this PEP should define a non-configurable way for the tag. The tag should include all information which currently makes an extension ABI incompatible: - the python implementation (cpython, PyPy, ...) - the python version (3,2, 3.3, ...) - unicode configure option (--with-wide-unicode, 16 or 32) - platform information (necessary?) If this list changes for coming python versions, then it can be extended. Barry pointed out on irc chat that people might want to build experimental ABI incompatible versions, which should have its own tag. If this is wanted, please provide a configure option which lets extend/append to the tag. 3) In case that 1) is not acceptable, the --with-so-abi-tag option should be implemented in such a way that it isn't required to take any arguments, and in this case to default to the fixed naming schema described in 2). Matthias From daniel at stutzbachenterprises.com Fri Jul 16 15:43:50 2010 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Fri, 16 Jul 2010 08:43:50 -0500 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <4C40372C.9030101@ubuntu.com> References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> Message-ID: On Fri, Jul 16, 2010 at 5:40 AM, Matthias Klose wrote: > 2) As PEP 3147 defines a non-configurable name for .pyc files, this PEP > should define a non-configurable way for the tag. The tag should > include all information which currently makes an extension ABI > incompatible: > - the python implementation (cpython, PyPy, ...) > - the python version (3,2, 3.3, ...) > - unicode configure option (--with-wide-unicode, 16 or 32) > - platform information (necessary?) I'm not sure it's that easy to enumerate all of the ways to end up with an incompatible ABI. There are quite a lot of configure options listed with "configure --help", and it's not always obvious if an option changes the ABI. On top of that, there are compilation flags that can change the ABI, as Kristj?n discovered in the following thread: http://mail.python.org/pipermail/python-dev/2010-June/100583.html On the flip side, a fully enumerated ABI signature could be used to identify (in)compatible binary eggs, which is basically impossible now. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From doko at ubuntu.com Fri Jul 16 15:53:53 2010 From: doko at ubuntu.com (Matthias Klose) Date: Fri, 16 Jul 2010 15:53:53 +0200 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> Message-ID: <4C406471.3070707@ubuntu.com> On 16.07.2010 15:43, Daniel Stutzbach wrote: > On Fri, Jul 16, 2010 at 5:40 AM, Matthias Klose wrote: > >> 2) As PEP 3147 defines a non-configurable name for .pyc files, this PEP >> should define a non-configurable way for the tag. The tag should >> include all information which currently makes an extension ABI >> incompatible: >> - the python implementation (cpython, PyPy, ...) >> - the python version (3,2, 3.3, ...) >> - unicode configure option (--with-wide-unicode, 16 or 32) >> - platform information (necessary?) > > > I'm not sure it's that easy to enumerate all of the ways to end up with an > incompatible ABI. There are quite a lot of configure options listed with > "configure --help", and it's not always obvious if an option changes the > ABI. On top of that, there are compilation flags that can change the ABI, > as Kristj?n discovered in the following thread: right, I forgot about the debug builds, because it's already the standard on windows to build foo_d.so extensions, and I adopted it for Debian/Ubuntu too. > On the flip side, a fully enumerated ABI signature could be used to identify > (in)compatible binary eggs, which is basically impossible now. indeed. From vlastimil.brom at gmail.com Fri Jul 16 17:08:03 2010 From: vlastimil.brom at gmail.com (Vlastimil Brom) Date: Fri, 16 Jul 2010 17:08:03 +0200 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C36491C.2050300@mrabarnett.plus.com> <4C366EC2.3000203@mrabarnett.plus.com> Message-ID: 2010/7/9 Georg Brandl : > Am 09.07.2010 02:35, schrieb MRAB: > >> >> 1. Some of the inline flags are scoped; for example, putting "(?i)" at >> the end of a regex will now have no effect because it's no longer a >> global, all-or-nothing, flag. > > That is problematic. ?I've often seen people put these flags at the end > of a regex, probably for readability purposes. ?IMHO it would be better > to limit flag scoping to the explicit (?flags-flags: ) groups. > I just noticed the formulation on the reference page regular-expressions.info on this kind of flags: "(?i) Turn on case insensitivity for the remainder of the regular expression. (Older regex flavors may turn it on for the entire regex.)" and likewise for other flags. http://www.regular-expressions.info/refadv.html I am not sure, how "authoritative" this page by Jan Goyvaerts is for various implementations, but it looks like a very comprehensive reference. I think with a new regex implementation, not all of this "historical" semantics must be copied, unless there are major real usecases, which would be affected by this. Just a thought; Vlastimil Brom From status at bugs.python.org Fri Jul 16 18:07:39 2010 From: status at bugs.python.org (Python tracker) Date: Fri, 16 Jul 2010 18:07:39 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20100716160739.8581478121@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2010-07-09 - 2010-07-16) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2807 open (+44) / 18285 closed (+18) / 21092 total (+62) Open issues with patches: 1144 Average duration of open issues: 703 days. Median duration of open issues: 497 days. Open Issues Breakdown open 2765 (+42) languishing 14 ( +0) pending 27 ( +2) Issues Created Or Reopened (64) _______________________________ regrtest: add a minimal "progress bar" 2010-07-10 http://bugs.python.org/issue8560 reopened georg.brandl patch contextlib.ContextDecorator 2010-07-15 http://bugs.python.org/issue9110 reopened ncoghlan patch dict_keys purports to implement the Set ABC, but is missing th 2010-07-09 http://bugs.python.org/issue9212 created stutzbach range purports to implement the Sequence ABC, but is missing i 2010-07-09 http://bugs.python.org/issue9213 created stutzbach Most Set methods of KeysView and ItemsView do not work right 2010-07-09 http://bugs.python.org/issue9214 created stutzbach easy test_float crashes with assertion failure on Ubuntu buildbot. 2010-07-09 CLOSED http://bugs.python.org/issue9215 created mark.dickinson FIPS support for hashlib 2010-07-10 http://bugs.python.org/issue9216 created dmalcolm patch 2to3 crashes with some doctests 2010-07-14 http://bugs.python.org/issue9217 reopened tiagoantao pop multiple elements of a list at once 2010-07-10 http://bugs.python.org/issue9218 created jacobidiego Make reindent-rst.py preserve ^L 2010-07-10 CLOSED http://bugs.python.org/issue9219 created merwok easy Modules no longer usable as context managers 2010-07-10 CLOSED http://bugs.python.org/issue9220 created j1m 2to3 fails to parse a for construct 2010-07-10 CLOSED http://bugs.python.org/issue9221 created tiagoantao IDLE: Fix open/saveas 'Files of type' choices 2010-07-11 http://bugs.python.org/issue9222 created tjreedy patch, easy distutils Command docs linking 2010-07-11 CLOSED http://bugs.python.org/issue9223 created techtonik Distutls fails with MSVC++ 2008on Windows Vista 64bit 2010-07-11 CLOSED http://bugs.python.org/issue9224 created ipatrol Replace DUP_TOPX with DUP_TOP_TWO 2010-07-11 http://bugs.python.org/issue9225 created serprex patch erroneous behavior when creating classes inside a closure 2010-07-11 http://bugs.python.org/issue9226 created monsanto patch can't import Tkinter / use IDLE after installing Python 2.7 on 2010-07-11 http://bugs.python.org/issue9227 created michael.foord Make changes in the path and pathext on installation 2010-07-11 http://bugs.python.org/issue9228 created ipatrol memoryview of mmap object not working in 2.7 2010-07-12 CLOSED http://bugs.python.org/issue9229 created Matt.Gattis pdb.Pdb.checkline() throws AttributeError if called before sta 2010-07-12 http://bugs.python.org/issue9230 created __Vano patch IDLE interactive interpreter crash 2010-07-12 CLOSED http://bugs.python.org/issue9231 created pitrou Allow trailing comma in any function argument list. 2010-07-12 http://bugs.python.org/issue9232 created mark.dickinson patch json.load failure when C optimizations aren't built 2010-07-12 http://bugs.python.org/issue9233 created fdrake patch argparse: aliases for positional arguments (subparsers) 2010-07-12 http://bugs.python.org/issue9234 created asampson missing "import sys" in Tools/gdb/libpython.py 2010-07-12 CLOSED http://bugs.python.org/issue9235 created Yaroslav.Halchenko Invalid reads in fastsearch.h 2010-07-12 CLOSED http://bugs.python.org/issue9236 created skrah Add sys.call_tracing to on-line sys module documentation 2010-07-12 http://bugs.python.org/issue9237 created rocky zipfile incorrectly documented as not supporting archive comme 2010-07-12 http://bugs.python.org/issue9238 created rfk patch zipfile: truncating comment can corrupt the zipfile 2010-07-12 http://bugs.python.org/issue9239 created rfk patch different behaviour with for loop... interpreted vs scripted 2010-07-13 CLOSED http://bugs.python.org/issue9240 created jacobidiego SAXParseError on unicode (Japanese) file 2010-07-13 CLOSED http://bugs.python.org/issue9241 created gianzula unicodeobject.c: use of uninitialized values 2010-07-13 http://bugs.python.org/issue9242 created skrah sndhdr erroneously detects files as vox 2010-07-13 CLOSED http://bugs.python.org/issue9243 created jbit patch multiprocessing.pool: Worker crashes if result can't be encode 2010-07-13 http://bugs.python.org/issue9244 created asksol patch "from itertools imoprt *" crashes 2to3 2010-07-13 CLOSED http://bugs.python.org/issue9245 created ronaldoussoren os.getcwd() hardcodes max path len 2010-07-13 http://bugs.python.org/issue9246 created pitrou imp.reload() doesn't take import lock 2010-07-13 http://bugs.python.org/issue9247 created pitrou multiprocessing.pool: Proposal: "waitforslot" 2010-07-13 http://bugs.python.org/issue9248 created asksol patch struct.pack and Long Integer datatype should be 4, but is 8 by 2010-07-13 CLOSED http://bugs.python.org/issue9249 created hannes.reuter sys.modules changes size during iteration in regrtest module 2010-07-13 http://bugs.python.org/issue9250 created xdegaye patch Test for the import lock 2010-07-13 http://bugs.python.org/issue9251 created pitrou patch PyImport_Import calls __import__ with dummy fromlist 2010-07-13 http://bugs.python.org/issue9252 created belopolsky argparse: optional subparsers 2010-07-13 http://bugs.python.org/issue9253 created nvie __import__ docstring should recommend importlib.import_module( 2010-07-13 http://bugs.python.org/issue9254 created belopolsky document that the 'test' package is for core developers only 2010-07-13 http://bugs.python.org/issue9255 created brett.cannon easy plistlib should create non-na??ve datetime objects 2010-07-14 http://bugs.python.org/issue9256 created wiml cElementTree iterparse requires events as bytes; ElementTree u 2010-07-14 http://bugs.python.org/issue9257 created eric-talevich Typos in docs for methods kqueue and kevent of module 'select' 2010-07-14 CLOSED http://bugs.python.org/issue9258 created Retro Python 2.7 breaks assigned __exit__s 2010-07-14 CLOSED http://bugs.python.org/issue9259 reopened arielbyd A finer grained import lock 2010-07-14 http://bugs.python.org/issue9260 created pitrou patch include higher (../../) dirs fails 2010-07-14 CLOSED http://bugs.python.org/issue9261 created indiedan IDLE: Use tabbed shell and edit windows 2010-07-14 http://bugs.python.org/issue9262 created tjreedy Try to print repr() when an C-level assert fails (in the garba 2010-07-14 http://bugs.python.org/issue9263 created dmalcolm patch trace.py documentation is incomplete 2010-07-15 http://bugs.python.org/issue9264 created eli.bendersky patch Incorrect name passed as arg[0] when shell=True and executable 2010-07-15 http://bugs.python.org/issue9265 created narnie patch, needs review ctypes "ValueError: NULL pointer access" on Win7 x64 2010-07-15 http://bugs.python.org/issue9266 created ghazel Update pickle opcode documentation in pickletools for 3.x 2010-07-15 http://bugs.python.org/issue9267 created belopolsky easy Add annotation option to pickletools.dis 2010-07-15 http://bugs.python.org/issue9268 created belopolsky patch, easy Cannot pickle self-referencing sets 2010-07-15 http://bugs.python.org/issue9269 created belopolsky distutils.util.getplatform and sysconfig.getplatform differ 2010-07-16 http://bugs.python.org/issue9270 created bugs-python at vendor.thewrittenword.com Python throws `IOError: [Errno 27] File too large' on long fil 2010-07-16 http://bugs.python.org/issue9271 created Jason.Gross CGIHTTPServer poisons os.environ 2010-07-16 http://bugs.python.org/issue9272 created techtonik patch 2to to simultaneously do code AND doctests 2010-07-16 http://bugs.python.org/issue9273 created tiagoantao Issues Now Closed (83) ______________________ epoll and kqueue wrappers for the select module 935 days http://bugs.python.org/issue1657 tjreedy ConfigParser.py do not allow leading (and trailing) space in v 922 days http://bugs.python.org/issue1714 tjreedy patch, easy Grammar change to prevent shift/reduce problem with varargslis 889 days http://bugs.python.org/issue2009 mark.dickinson patch MozillaCookieJar ignore HttpOnly cookies 868 days http://bugs.python.org/issue2190 jjlee patch MozillaCookieJar does not support Firefox3 cookie files 852 days http://bugs.python.org/issue2277 jjlee patch Augmenting the Windows build to support code signing. 787 days http://bugs.python.org/issue2850 tjreedy Pickle stream for unicode object may contain non-ASCII charact 774 days http://bugs.python.org/issue2980 tjreedy Let SimpleXMLRPCServer pass client_address to called functions 765 days http://bugs.python.org/issue3058 tjreedy Wrong size calculation in posix_execve 761 days http://bugs.python.org/issue3070 haypo The ValueError raised by failing to unpack sequence should hav 761 days http://bugs.python.org/issue3071 merwok patch Suggest change to glossary explanation: "Duck Typing" 744 days http://bugs.python.org/issue3214 merwok patch "make check" suggest a testing target under GNU coding standar 646 days http://bugs.python.org/issue3758 merwok patch test_output_textcalendar fails on non-englisch locale 644 days http://bugs.python.org/issue4044 r.david.murray test_mboxmmdf_to_maildir fails on non-englisch locale 644 days http://bugs.python.org/issue4045 r.david.murray Clarification needed for subprocess convenience functions in P 574 days http://bugs.python.org/issue4697 georg.brandl Little improvement on urlparse module, urlparse function. 547 days http://bugs.python.org/issue4932 orsenthil patch datetime lacks concrete tzinfo implementation for UTC 25 days http://bugs.python.org/issue5094 merwok patch expose _CrtSetReportMode via the msvcrt module 526 days http://bugs.python.org/issue5116 tjreedy IDLE to support reindent.py 521 days http://bugs.python.org/issue5150 merwok SocketIO redefines RawIOBase.__del__ 502 days http://bugs.python.org/issue5347 benjamin.peterson Subprocess.getstatusoutput Fails Executing 'dir' Command on Wi 451 days http://bugs.python.org/issue5808 matthieu.labbe Move test outside of urlparse module 445 days http://bugs.python.org/issue5842 merwok test_(zipfile|zipimport|gzip|distutils|sqlite) fail if zlib is 424 days http://bugs.python.org/issue6026 afarrell socket.shudown documentation: on some platforms, closing one h 321 days http://bugs.python.org/issue6774 georg.brandl patch system proxy not used for https (on windows) 311 days http://bugs.python.org/issue6853 orsenthil patch test_telnetlib gives spurious output 296 days http://bugs.python.org/issue6960 jackdied 2to3 execfile conversion changes behavior 235 days http://bugs.python.org/issue7268 benjamin.peterson patch test_memoryview test_setitem_writable failures with Intel ICC 191 days http://bugs.python.org/issue7616 pitrou patch Lib/ctypes/test/test_pep3118.py should not shadow the memoryvi 159 days http://bugs.python.org/issue7840 BreamoreBoy Cross-reference ast.literal_eval() from eval() docs 146 days http://bugs.python.org/issue7935 georg.brandl Problem with urlparse in Windows XP after a drag and drop 145 days http://bugs.python.org/issue7965 orsenthil test_ntpath fails on Windows XP 143 days http://bugs.python.org/issue7991 austin987 python 2.6.4 installation is not working for the single user m 124 days http://bugs.python.org/issue8114 BreamoreBoy Outdated information 94 days http://bugs.python.org/issue8338 georg.brandl patch sqlite3.connect documentation is incorrect 82 days http://bugs.python.org/issue8456 georg.brandl Update documentation on doctest/unittest2 integration 73 days http://bugs.python.org/issue8564 merwok patch cPickle dumps(tuple) != dumps(loads(dumps(tuple))) 59 days http://bugs.python.org/issue8738 belopolsky Inconsistency in behaviour of urllib and urllib2 with file:// 48 days http://bugs.python.org/issue8801 orsenthil svnmerge errors in msgfmt.py 30 days http://bugs.python.org/issue8974 loewis patch Bug in cookiejar 31 days http://bugs.python.org/issue8975 merwok Provide parseable repr to datetime.timezone 25 days http://bugs.python.org/issue9000 merwok patch, easy Documentation for comparing dictionaries is out of date 14 days http://bugs.python.org/issue9132 orsenthil patch, easy x.update(red=5, blue=6, other=7) doesn't work, where x is a Mu 10 days http://bugs.python.org/issue9137 mark.dickinson patch 2.7 sysconfig should handle arch duplicates while building uni 10 days http://bugs.python.org/issue9164 ronaldoussoren Add math.isfinite() 6 days http://bugs.python.org/issue9165 mark.dickinson patch Deprecate use of more than 3 positional arguments in timedelta 8 days http://bugs.python.org/issue9169 belopolsky unittest assertEqual error output sub-optimal for single strin 5 days http://bugs.python.org/issue9174 michael.foord patch open() doc: default value for buffering is -1, not None 5 days http://bugs.python.org/issue9184 georg.brandl patch Improve CFLAGS handling 2 days http://bugs.python.org/issue9189 jyasskin test_float crashes with assertion failure on Ubuntu buildbot. 3 days http://bugs.python.org/issue9215 mark.dickinson Make reindent-rst.py preserve ^L 0 days http://bugs.python.org/issue9219 merwok easy Modules no longer usable as context managers 0 days http://bugs.python.org/issue9220 j1m 2to3 fails to parse a for construct 0 days http://bugs.python.org/issue9221 merwok distutils Command docs linking 0 days http://bugs.python.org/issue9223 techtonik Distutls fails with MSVC++ 2008on Windows Vista 64bit 0 days http://bugs.python.org/issue9224 srid memoryview of mmap object not working in 2.7 0 days http://bugs.python.org/issue9229 pitrou IDLE interactive interpreter crash 0 days http://bugs.python.org/issue9231 ezio.melotti missing "import sys" in Tools/gdb/libpython.py 2 days http://bugs.python.org/issue9235 georg.brandl Invalid reads in fastsearch.h 3 days http://bugs.python.org/issue9236 skrah different behaviour with for loop... interpreted vs scripted 0 days http://bugs.python.org/issue9240 eric.smith SAXParseError on unicode (Japanese) file 0 days http://bugs.python.org/issue9241 amaury.forgeotdarc sndhdr erroneously detects files as vox 0 days http://bugs.python.org/issue9243 haypo patch "from itertools imoprt *" crashes 2to3 0 days http://bugs.python.org/issue9245 benjamin.peterson struct.pack and Long Integer datatype should be 4, but is 8 by 2 days http://bugs.python.org/issue9249 hannes.reuter Typos in docs for methods kqueue and kevent of module 'select' 0 days http://bugs.python.org/issue9258 georg.brandl Python 2.7 breaks assigned __exit__s 0 days http://bugs.python.org/issue9259 merwok include higher (../../) dirs fails 0 days http://bugs.python.org/issue9261 merwok commands.getstatusoutput(): cmd.exe support 3099 days http://bugs.python.org/issue506100 matthieu.labbe reading shelves is really slow 2420 days http://bugs.python.org/issue849662 rhettinger logging handlers raise exception on level 2373 days http://bugs.python.org/issue876421 vinay.sajip fails building C++ extensions when configured --without-cxx 2330 days http://bugs.python.org/issue902065 georg.brandl patch test test_codecs failed 2050 days http://bugs.python.org/issue1076790 lemburg patch Windows deadlock with PyEval_ReleaseLock 1964 days http://bugs.python.org/issue1147646 amaury.forgeotdarc urllib.urlencode provides two features in one param 1711 days http://bugs.python.org/issue1349732 orsenthil easy urllib/urllib2: HTTPS over (Squid) Proxy fails 1616 days http://bugs.python.org/issue1424152 tjreedy patch DOM tree inconsistency in expat XML parser 1603 days http://bugs.python.org/issue1434090 georg.brandl patch, easy "reindent.py" exposes bug in tokenize 1583 days http://bugs.python.org/issue1447633 merwok gettext.py breaks on plural-forms header (PATCH) 1587 days http://bugs.python.org/issue1448060 merwok patch mimetools module getencoding error 1545 days http://bugs.python.org/issue1471985 georg.brandl support all of strftime(3) 1465 days http://bugs.python.org/issue1520662 belopolsky socket.getaddrinfo() should take an address tuple. 1206 days http://bugs.python.org/issue1685962 giampaolo.rodola Expose callback API in readline module 1161 days http://bugs.python.org/issue1713877 georg.brandl move intern to sys, make intern() optionally warn 1136 days http://bugs.python.org/issue1728741 benjamin.peterson patch Top Issues Most Discussed (10) ______________________________ 26 Parent process hanging in multiprocessing if children terminate 8 days open http://bugs.python.org/issue9205 12 Incorrect name passed as arg[0] when shell=True and executable 1 days open http://bugs.python.org/issue9265 11 A finer grained import lock 2 days open http://bugs.python.org/issue9260 11 contextlib.ContextDecorator 1 days open http://bugs.python.org/issue9110 10 Condition.wait() doesn't raise KeyboardInterrupt 49 days open http://bugs.python.org/issue8844 10 msvc9compiler.py: ValueError: [u'path'] 213 days open http://bugs.python.org/issue7511 9 os.getcwd causes infinite loop on solaris 9 days open http://bugs.python.org/issue9185 9 PyImport_ImportModuleNoBlock() may solve problems but causes ot 129 days open http://bugs.python.org/issue8098 9 Finding programs in PATH, adding shutil.which 3278 days open http://bugs.python.org/issue444582 8 Python 2.7 breaks assigned __exit__s 0 days closed http://bugs.python.org/issue9259 From pengyu.ut at gmail.com Fri Jul 16 18:32:25 2010 From: pengyu.ut at gmail.com (Peng Yu) Date: Fri, 16 Jul 2010 11:32:25 -0500 Subject: [Python-Dev] more details of list comprehension in tutorial than in language reference Message-ID: Hi, Since this is related to the document, I think that I should send this to the dev mailing list. Please let me know if this is not appropriate. If I don't miss anything, I feel that there are much less descriptions of list comprehensions in the language reference (python_2.6.5_reference.pdf, glossary) than in the tutorial (python_2.6.5_tutorial.pdf). I think that it deserves at least a subsection in the language reference to make the reference complete. May I suggest the maintainer of the document add such a subsection? -- Regards, Peng From fuzzyman at voidspace.org.uk Fri Jul 16 18:46:36 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 16 Jul 2010 17:46:36 +0100 Subject: [Python-Dev] more details of list comprehension in tutorial than in language reference In-Reply-To: References: Message-ID: <4C408CEC.5040606@voidspace.org.uk> On 16/07/2010 17:32, Peng Yu wrote: > Hi, > > Since this is related to the document, I think that I should send this > to the dev mailing list. Please let me know if this is not > appropriate. > > If I don't miss anything, I feel that there are much less descriptions > of list comprehensions in the language reference > (python_2.6.5_reference.pdf, glossary) than in the tutorial > (python_2.6.5_tutorial.pdf). I think that it deserves at least a > subsection in the language reference to make the reference complete. > May I suggest the maintainer of the document add such a subsection? > > If you would like to write an addition and post it to the Python bug tracker it would be (very) welcomed for consideration. The tutorial has no single "maintainer" but we are always looking to improve it. All the best, Michael Foord -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From g.brandl at gmx.net Fri Jul 16 19:08:04 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 16 Jul 2010 19:08:04 +0200 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C36491C.2050300@mrabarnett.plus.com> <4C366EC2.3000203@mrabarnett.plus.com> Message-ID: Am 16.07.2010 17:08, schrieb Vlastimil Brom: > 2010/7/9 Georg Brandl : >> Am 09.07.2010 02:35, schrieb MRAB: >> >>> >>> 1. Some of the inline flags are scoped; for example, putting "(?i)" at >>> the end of a regex will now have no effect because it's no longer a >>> global, all-or-nothing, flag. >> >> That is problematic. I've often seen people put these flags at the end >> of a regex, probably for readability purposes. IMHO it would be better >> to limit flag scoping to the explicit (?flags-flags: ) groups. >> > > I just noticed the formulation on the reference page > regular-expressions.info on this kind of flags: > "(?i) Turn on case insensitivity for the remainder of the regular > expression. (Older regex flavors may turn it on for the entire > regex.)" and likewise for other flags. > > http://www.regular-expressions.info/refadv.html > > I am not sure, how "authoritative" this page by Jan Goyvaerts is for > various implementations, but it looks like a very comprehensive > reference. Nevertheless, the authoritative reference for our regex engine is its docs, i.e. http://docs.python.org/library/re.html -- and that states clearly that inline flags apply to the whole regex. > I think with a new regex implementation, not all of this "historical" > semantics must be copied, unless there are major real usecases, which > would be affected by this. As I already said, I *have* seen this in real code. As MRAB indicated, this was the only silent change in semantics as compared to the old regex engine. If we replace re by regex, which I think is the only way to get the new features in the stdlib, changing this one aspect is a) not backwards compatible and b) in a subtle way that forces everyone to review his/her regular expressions. That's definitely not acceptable. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From rdmurray at bitdance.com Fri Jul 16 19:37:08 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 16 Jul 2010 13:37:08 -0400 Subject: [Python-Dev] more details of list comprehension in tutorial than in language reference In-Reply-To: References: Message-ID: <20100716173708.DCEB321B540@kimball.webabinitio.net> On Fri, 16 Jul 2010 11:32:25 -0500, Peng Yu wrote: > Since this is related to the document, I think that I should send this > to the dev mailing list. Please let me know if this is not > appropriate. It is not, really. Documentation issues should be posted as bugs in the bug tracker under the 'Documentation' component. > If I don't miss anything, I feel that there are much less descriptions > of list comprehensions in the language reference > (python_2.6.5_reference.pdf, glossary) than in the tutorial > (python_2.6.5_tutorial.pdf). I think that it deserves at least a > subsection in the language reference to make the reference complete. > May I suggest the maintainer of the document add such a subsection? The Language Reference is intentionally terse. If there is important syntactic/semantic information *missing* then it should be added, but if it is just lacking in the detail of the explanations, then the tutorial is the place for those, and you say that is OK. -- R. David Murray www.bitdance.com From solipsis at pitrou.net Fri Jul 16 21:18:23 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 16 Jul 2010 21:18:23 +0200 Subject: [Python-Dev] importlib In-Reply-To: References: <20100713173418.22e42f15@pitrou.net> <20100714220115.305350b0@pitrou.net> Message-ID: <20100716211823.4b3c4181@pitrou.net> On Wed, 14 Jul 2010 23:06:58 -0700 Brett Cannon wrote: > > > > In any case, here my results under a Linux system: > > > > $ ./python -m importlib.test.benchmark > > sys.modules [ 323782 326183 326667 ] best is 326667 > > Built-in module [ 33600 33693 33610 ] best is 33693 > > > > $ ./python -m importlib.test.benchmark -b > > sys.modules [ 1297640 1315366 1292283 ] best is 1315366 > > Built-in module [ 58180 57708 58057 ] best is 58180 > > And this is what might make evaluating importlib tough; while the > performance is 25% of what it is for import.c, being able to import > over 300,000 times/second is still damn fast. Yes, that's very encouraging. I guess the final test would be to take something like Mercurial, and time e.g. "hg version" both with the builtin-import, and with importlib enabled as default import mechanism. Regards Antoine. From brett at python.org Fri Jul 16 21:19:10 2010 From: brett at python.org (Brett Cannon) Date: Fri, 16 Jul 2010 12:19:10 -0700 Subject: [Python-Dev] importlib In-Reply-To: References: <20100713173418.22e42f15@pitrou.net> <20100714220115.305350b0@pitrou.net> Message-ID: I have updated the benchmark to now measure importing source w/o writing bytecode, importing source & writing bytecode, and importing bytecode w/ source (as I don't care about sourceless import performance). Now, before you look at these numbers, realize that I have not once tried to profile importlib to see where its hot spots are (only optimization I have done is cut down on the stat calls since Python 3.1 when I re-developed the ABCs). I'm sure if I profiled the code and wrote key bits in C these performance numbers would improve reasonably quickly. Anyway, on my 2.2 GHz MacBook, this leads to:: import.c sys.modules [ 223337 223036 223362 ] best is 223362 Built-in module [ 23347 23319 23331 ] best is 23347 Bytecode w/ source [ 6624 6607 6608 ] best is 6624 Source w/o bytecode [ 4643 4674 4655 ] best is 4674 Source writing bytecode [ 2063 2145 2204 ] best is 2204 importlib sys.modules [ 43423 43414 43426 ] best is 43426 Built-in module [ 9130 9115 9120 ] best is 9130 Bytecode w/ source [ 1554 1556 1556 ] best is 1556 Source w/o bytecode [ 1351 1351 1353 ] best is 1353 Source writing bytecode [ 786 843 810 ] best is 843 importlib / import.c: sys.modules 19% Built-in module 39% Bytecode w/ source 23% Source w/o bytecode 29% Source writing bytecode 38% What does this show? Stuff that requires a lot of I/O has the smallest performance difference (source writing bytecode), but where there is as little I/O as possible (bytecode w/ source) import.c wins as it has to do less. This is also why sys.modules is so damn fast; it's the smallest amount of C code you can run while importlib has standard Python calling overhead. It should also be pointed out that importlib has fully implemented PEP 302 and intentionally has the loaders using their own exposed PEP 302 APIs. This means there are a lot more methods calls than in the C version, along with a lot less corners cut in the name of performance. So while importlib will be slower simply because it's implemented in C, it will also be slower because the darn thing is actually written to follow the PEPs we have along with making it easier for people to subclass and benefit from the import code. Anyway, as I have said, I need to hit 100% compatibility when running the test suite -- run importlib.test.regrtest to see where it fails now; also read that file as it has notes as to why the known failures are happening -- before I start worrying about bootstrapping and performance and that will all be no sooner than Python 3.3. On Thu, Jul 15, 2010 at 04:55, Nick Coghlan wrote: > On Thu, Jul 15, 2010 at 4:06 PM, Brett Cannon wrote: > >> In any case, here my results under a Linux system: > >> > >> $ ./python -m importlib.test.benchmark > >> sys.modules [ 323782 326183 326667 ] best is 326667 > >> Built-in module [ 33600 33693 33610 ] best is 33693 > >> > >> $ ./python -m importlib.test.benchmark -b > >> sys.modules [ 1297640 1315366 1292283 ] best is 1315366 > >> Built-in module [ 58180 57708 58057 ] best is 58180 > > > > And this is what might make evaluating importlib tough; while the > > performance is 25% of what it is for import.c, being able to import > > over 300,000 times/second is still damn fast. > > Yeah, I think the numbers where the filesystem gets involved are going > to be more relevant. Modules that have already been cached and those > built in to the executable aren't likely to dominate interpreter and > application startup times (which is the main thing I'm worried about > seeing regress). > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Fri Jul 16 22:55:36 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 16 Jul 2010 16:55:36 -0400 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <4C34BF2C.8090600@netwok.org> Message-ID: On 7/7/2010 2:27 PM, Georg Brandl wrote: > Am 07.07.2010 19:53, schrieb ?ric Araujo: >>> I promised to write a PEP about that some time in the future. (Probably after >>> 3.2 final.) >> It seems that projects putting Sphinxy reST in their doc are using >> automatic doc generation. This is however not always the best way to >> make good doc, as demonstrated by Python?s hand-written, >> very-high-quality documentation. > I know, and this is what I originally intended for Sphinx. However, the calls > for automatic doc generation are very loud, and it's understandable that most > project can't afford writing their documentation twice. Neither can CPython, really, as evidenced by numerous examples that have shone up on the tracker. Let me add another one. A week ago, Eli Benderdky asked me for help adding missing pieces to the trace module doc. The result so far is http://bugs.python.org/issue9264 After getting that far, I noticed that there were already doc strings for some things that were not documented in the manual, which we added. I also noticed the the public methods already in the manual had not help strings, and hence no helpful help() output. In other words, the manual entries and doc strings were close to disjoint sets. Even when they do overlap as they should, violation of DRY is a maintenance nightmare when anything changes. It would be better if the manual could, to some extent, be docstrings plus additions. -- Terry Jan Reedy From g.brandl at gmx.net Fri Jul 16 23:25:13 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 16 Jul 2010 23:25:13 +0200 Subject: [Python-Dev] query: docstring formatting in python distutils code In-Reply-To: References: <20100707155358.GB26419@idyll.org> <4C34A6C4.2030205@voidspace.org.uk> <4C34BF2C.8090600@netwok.org> Message-ID: Am 16.07.2010 22:55, schrieb Terry Reedy: > On 7/7/2010 2:27 PM, Georg Brandl wrote: >> Am 07.07.2010 19:53, schrieb ?ric Araujo: >>>> I promised to write a PEP about that some time in the future. (Probably after >>>> 3.2 final.) > >>> It seems that projects putting Sphinxy reST in their doc are using >>> automatic doc generation. This is however not always the best way to >>> make good doc, as demonstrated by Python?s hand-written, >>> very-high-quality documentation. > >> I know, and this is what I originally intended for Sphinx. However, the calls >> for automatic doc generation are very loud, and it's understandable that most >> project can't afford writing their documentation twice. > > Neither can CPython, really, as evidenced by numerous examples that have > shone up on the tracker. Let me add another one. A week ago, Eli > Benderdky asked me for help adding missing pieces to the trace module > doc. The result so far is > http://bugs.python.org/issue9264 > > After getting that far, I noticed that there were already doc strings > for some things that were not documented in the manual, which we added. > I also noticed the the public methods already in the manual had not help > strings, and hence no helpful help() output. In other words, the manual > entries and doc strings were close to disjoint sets. And that's a pity -- very commendable that you work on this. But this is a single case, and most of our library is very well covered by the docs and docstrings. Additions (which usually are small amendments) are made only in the docs, not the docstring, as it should be given the nature of our docstrings. Added functionality is usually documented first in docstrings, which then get copied into the docs, and are padded from there. > Even when they do overlap as they should, violation of DRY is a > maintenance nightmare when anything changes. It would be better if the > manual could, to some extent, be docstrings plus additions. That sounds nice in theory, but opens another can of worms: what belongs in the docstring, what in the "addition"? How does it work for builtin types, for language features, and so on. Also, the problems I already outlined in my answer to Fred remain... Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From alexander.belopolsky at gmail.com Sat Jul 17 02:24:42 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Fri, 16 Jul 2010 20:24:42 -0400 Subject: [Python-Dev] Curious datetime method Message-ID: I always thought that date.today() was a date class method and its availability as a datetime method was an artifact of datetime inheritance from date. I thought datetime.today() would be just the same as date.today(). It turned out I was wrong. Instead, datetime.today() is more like datetime.now(). Almost. >>> datetime.today() datetime.datetime(2010, 7, 16, 20, 5, 10, 710143) The documentation says X.today() is equivalent to X.fromtimestamp(time.time()) and this is literary true: >>> class X(datetime): ... @classmethod ... def fromtimestamp(cls, t): ... return "whatever" ... >>> X.today() 'whatever' Unlike datetime.now() which gets time directly from OS, datetime.today() calls time.time(), gets timestamp as float and breaks it into integer datetime components in fromtimestamp(). This method is slow, relies on various poorly defined float to integer conversions and cannot produce anything other than local time as a naive datetime object. I wonder why would anyone want to use datetime.today() instead of datetime.now()? From comcombrandon2 at yahoo.com Sat Jul 17 04:38:40 2010 From: comcombrandon2 at yahoo.com (Brandon Hayden) Date: Fri, 16 Jul 2010 19:38:40 -0700 (PDT) Subject: [Python-Dev] A suggestion Message-ID: <625243.91818.qm@web120205.mail.ne1.yahoo.com> In the Python language, or any other language for that matter, I have never understood why they don't have a loop function. Here's what I mean, every time someone wants something to repeat itself, they have to write a while loop like this: x = 0 while x < 10: function() x = x + 1 When, instead of writing all of that, you could do something similar to this: function().10 Where you can just type in the number 10 right next to it. It makes no since. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Sat Jul 17 05:10:46 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 16 Jul 2010 23:10:46 -0400 Subject: [Python-Dev] A suggestion In-Reply-To: <625243.91818.qm@web120205.mail.ne1.yahoo.com> References: <625243.91818.qm@web120205.mail.ne1.yahoo.com> Message-ID: On 7/16/2010 10:38 PM, Brandon Hayden wrote: > In the Python language, or any other language for that matter, I have > never understood why they don't have a loop function. Here's what I > mean, every time someone wants something to repeat itself, they have to > write a while loop like this: > > x = 0 > while x < 10: > function() > x = x + 1 > > When, instead of writing all of that, you could do something similar to > this: > > function().10 > > Where you can just type in the number 10 right next to it. It makes no > since. Ask on python-list/clp/gmane.comp.python.general for an explanation. -- Terry Jan Reedy From alexander.belopolsky at gmail.com Sat Jul 17 06:06:27 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sat, 17 Jul 2010 00:06:27 -0400 Subject: [Python-Dev] A suggestion In-Reply-To: <625243.91818.qm@web120205.mail.ne1.yahoo.com> References: <625243.91818.qm@web120205.mail.ne1.yahoo.com> Message-ID: On Fri, Jul 16, 2010 at 10:38 PM, Brandon Hayden wrote: > In the Python language, or any other language for that matter, .. You brought this question to the wrong forum, but in Ruby you can do >> 10.times{f} to execute f 10 times. From steve at pearwood.info Sat Jul 17 10:19:11 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 17 Jul 2010 18:19:11 +1000 Subject: [Python-Dev] A suggestion In-Reply-To: <625243.91818.qm@web120205.mail.ne1.yahoo.com> References: <625243.91818.qm@web120205.mail.ne1.yahoo.com> Message-ID: <201007171819.12221.steve@pearwood.info> On Sat, 17 Jul 2010 12:38:40 pm Brandon Hayden wrote: > In the Python language, or any other language for that matter, I have > never understood why they don't have a loop function. Here's what I > mean, every time someone wants something to repeat itself, they have > to write a while loop like this: > > x = 0 > while x < 10: > function() > x = x + 1 Perhaps you should learn a little more Python before criticising it. The idiomatic way to write the above is: for x in range(10): function() Anyway, this is the wrong forum for this discussion. If you're still interested in it, please take it to comp.lang.python or tutor at python.org. -- Steven D'Aprano From steve at holdenweb.com Sat Jul 17 13:47:38 2010 From: steve at holdenweb.com (Steve Holden) Date: Sat, 17 Jul 2010 07:47:38 -0400 Subject: [Python-Dev] IDLE contributors and committers In-Reply-To: <20100717133325.5f204ee9@pitrou.net> References: <20100717133325.5f204ee9@pitrou.net> Message-ID: <4C41985A.4030201@holdenweb.com> On 7/17/2010 7:33 AM, Antoine Pitrou wrote: > > Hello, > > On Sun, 11 Jul 2010 02:05:22 +0300 > Tal Einat wrote: >> >> I would like to propose removing IDLE from the standard library. >> >> I have been using IDLE since 2002 and have been doing my best to help >> maintain and further develop IDLE since 2005. > > I haven't seen any conclusive statement or action to this thread. > Without being an IDLE user myself (for good reason), I think that if > IDLE should stay, active contributors such as Tal should be given commit > access and enough free rein to maintain and improve it. > > Otherwise there's no reason to continue claiming that IDLE is important > while discouraging such people's contributions. The current situation, > where several core developers support IDLE's continued inclusion but > none actually cares for the issues and patches in the tracker, is > schizophrenic. > > Regards > > Antoine. +1 There's no reason why Tal should be obstructed in his goal of making IDLE at least acceptable again. It's fairly obvious thaat there aren't any committers who have both the inclination /and/ the time to do this, so adding Tal (and other interested parties) as a developer makes a lot of sense. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From fuzzyman at voidspace.org.uk Sat Jul 17 13:50:50 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sat, 17 Jul 2010 12:50:50 +0100 Subject: [Python-Dev] IDLE contributors and committers In-Reply-To: <4C41985A.4030201@holdenweb.com> References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> Message-ID: <4C41991A.9040103@voidspace.org.uk> On 17/07/2010 12:47, Steve Holden wrote: > On 7/17/2010 7:33 AM, Antoine Pitrou wrote: > >> Hello, >> >> On Sun, 11 Jul 2010 02:05:22 +0300 >> Tal Einat wrote: >> >>> I would like to propose removing IDLE from the standard library. >>> >>> I have been using IDLE since 2002 and have been doing my best to help >>> maintain and further develop IDLE since 2005. >>> >> I haven't seen any conclusive statement or action to this thread. >> Without being an IDLE user myself (for good reason), I think that if >> IDLE should stay, active contributors such as Tal should be given commit >> access and enough free rein to maintain and improve it. >> >> Otherwise there's no reason to continue claiming that IDLE is important >> while discouraging such people's contributions. The current situation, >> where several core developers support IDLE's continued inclusion but >> none actually cares for the issues and patches in the tracker, is >> schizophrenic. >> >> Regards >> >> Antoine. >> > +1 > > There's no reason why Tal should be obstructed in his goal of making > IDLE at least acceptable again. It's fairly obvious thaat there aren't > any committers who have both the inclination /and/ the time to do this, > so adding Tal (and other interested parties) as a developer makes a lot > of sense. > Guilherme's *existing* patch for IDLE looks like it improves it a great deal, at the cost of potentially requiring Tk 8.5 (?). Can this just be committed? https://code.google.com/p/python-ttk/wiki/Screenshots Michael Foord > regards > Steve > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From breamoreboy at yahoo.co.uk Sat Jul 17 14:41:31 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 17 Jul 2010 13:41:31 +0100 Subject: [Python-Dev] IDLE contributors and committers In-Reply-To: <4C41991A.9040103@voidspace.org.uk> References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> <4C41991A.9040103@voidspace.org.uk> Message-ID: On 17/07/2010 12:50, Michael Foord wrote: > On 17/07/2010 12:47, Steve Holden wrote: >> On 7/17/2010 7:33 AM, Antoine Pitrou wrote: >>> Hello, >>> >>> On Sun, 11 Jul 2010 02:05:22 +0300 >>> Tal Einat wrote: >>>> I would like to propose removing IDLE from the standard library. >>>> >>>> I have been using IDLE since 2002 and have been doing my best to help >>>> maintain and further develop IDLE since 2005. >>> I haven't seen any conclusive statement or action to this thread. >>> Without being an IDLE user myself (for good reason), I think that if >>> IDLE should stay, active contributors such as Tal should be given commit >>> access and enough free rein to maintain and improve it. >>> >>> Otherwise there's no reason to continue claiming that IDLE is important >>> while discouraging such people's contributions. The current situation, >>> where several core developers support IDLE's continued inclusion but >>> none actually cares for the issues and patches in the tracker, is >>> schizophrenic. >>> >>> Regards >>> >>> Antoine. >> +1 >> >> There's no reason why Tal should be obstructed in his goal of making >> IDLE at least acceptable again. It's fairly obvious thaat there aren't >> any committers who have both the inclination /and/ the time to do this, >> so adding Tal (and other interested parties) as a developer makes a lot >> of sense. > > Guilherme's *existing* patch for IDLE looks like it improves it a great > deal, at the cost of potentially requiring Tk 8.5 (?). Can this just be > committed? > > https://code.google.com/p/python-ttk/wiki/Screenshots > > Michael Foord > >> regards >> Steve > > IIRC Terry Reedy is also interested in moving IDLE forward. Some help will certainly be needed to work on the 3 high, 80 normal and 13 low priority issues that are open against IDLE on the issue tracker. Kindest regards. Mark Lawrence. From eliben at gmail.com Sat Jul 17 15:23:16 2010 From: eliben at gmail.com (Eli Bendersky) Date: Sat, 17 Jul 2010 16:23:16 +0300 Subject: [Python-Dev] Markup of command-line options in Python's .rst documentation Message-ID: Hello, I'm currently working, together with Terry Reedy, on improving the documentation of the trace module, and I ran into a peculiar convention of marking command-line options which seems to be widespread. Consider the documentation of timeit, for instance: http://docs.python.org/dev/py3k/library/timeit.html The "--help" option appears as a hyperlink leading to http://docs.python.org/dev/py3k/using/cmdline.html#cmdoption--help, which is hardly relevant or useful. The same applies for several command-line options documented for the trace module (for example -m and -s). This is a result of the following markup (again, taking the timeit module as an example) in the relevant .rst file (Doc/library/timeit.rst): -h/:option:`--help` print a short usage message and exit The :option: markup seems to be translated by Sphinx into a link to the Python executable's own command line arguments. This creates the aforementioned problem in other modules as well, for example unittest. Is there really any merit in marking command-line options for modules with :option:, if it's only useful for Python's own options? Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Sat Jul 17 15:26:55 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sat, 17 Jul 2010 14:26:55 +0100 Subject: [Python-Dev] Markup of command-line options in Python's .rst documentation In-Reply-To: References: Message-ID: <4C41AF9F.6090408@voidspace.org.uk> On 17/07/2010 14:23, Eli Bendersky wrote: > Hello, > > I'm currently working, together with Terry Reedy, on improving the > documentation of the trace module, and I ran into a peculiar > convention of marking command-line options which seems to be widespread. > > Consider the documentation of timeit, for instance: > http://docs.python.org/dev/py3k/library/timeit.html > > The "--help" option appears as a hyperlink leading to > http://docs.python.org/dev/py3k/using/cmdline.html#cmdoption--help, > which is hardly relevant or useful. > > The same applies for several command-line options documented for the > trace module (for example -m and -s). This is a result of the > following markup (again, taking the timeit module as an example) in > the relevant .rst file (Doc/library/timeit.rst): > > -h/:option:`--help` > print a short usage message and exit > > The :option: markup seems to be translated by Sphinx into a link to > the Python executable's own command line arguments. This creates the > aforementioned problem in other modules as well, for example unittest. > Is there really any merit in marking command-line options for modules > with :option:, if it's only useful for Python's own options? > If it links to the wrong thing then the markup is incorrect (unless it is due to a regression in Sphinx but I think that is unlikely). Michael > Eli > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. -------------- next part -------------- An HTML attachment was scrubbed... URL: From merwok at netwok.org Sat Jul 17 15:44:04 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sat, 17 Jul 2010 15:44:04 +0200 Subject: [Python-Dev] Markup of command-line options in Python's .rst documentation In-Reply-To: References: Message-ID: <4C41B3A4.2000501@netwok.org> > The "--help" option appears as a hyperlink leading to > http://docs.python.org/dev/py3k/using/cmdline.html#cmdoption--help, which is > hardly relevant or useful. [...] > > -h/:option:`--help` > print a short usage message and exit I think this is a doc bug in Doc/documenting/markup.rst :cmdoption: and :option: are not clearly distinguished; the latter creates references to using/cmdline, the former is what you?re looking for for documenting trace.py. Regards From eliben at gmail.com Sat Jul 17 15:44:21 2010 From: eliben at gmail.com (Eli Bendersky) Date: Sat, 17 Jul 2010 16:44:21 +0300 Subject: [Python-Dev] Markup of command-line options in Python's .rst documentation In-Reply-To: <4C41AF9F.6090408@voidspace.org.uk> References: <4C41AF9F.6090408@voidspace.org.uk> Message-ID: On Sat, Jul 17, 2010 at 16:26, Michael Foord wrote: > On 17/07/2010 14:23, Eli Bendersky wrote: > > Hello, > > I'm currently working, together with Terry Reedy, on improving the > documentation of the trace module, and I ran into a peculiar convention of > marking command-line options which seems to be widespread. > > Consider the documentation of timeit, for instance: > http://docs.python.org/dev/py3k/library/timeit.html > > The "--help" option appears as a hyperlink leading to > http://docs.python.org/dev/py3k/using/cmdline.html#cmdoption--help, which > is hardly relevant or useful. > > The same applies for several command-line options documented for the trace > module (for example -m and -s). This is a result of the following markup > (again, taking the timeit module as an example) in the relevant .rst file > (Doc/library/timeit.rst): > > -h/:option:`--help` > print a short usage message and exit > > The :option: markup seems to be translated by Sphinx into a link to the > Python executable's own command line arguments. This creates the > aforementioned problem in other modules as well, for example unittest. Is > there really any merit in marking command-line options for modules with > :option:, if it's only useful for Python's own options? > > > If it links to the wrong thing then the markup is incorrect (unless it is > due to a regression in Sphinx but I think that is unlikely). > > Michael > Michael, What *should* it link to in case of modules, however? Is there some streamlined policy as to how module command line options should look and where they should be listed? From a cursory look on some documentation files, it's unlikely. Perhaps the answer is not to markup module options with :option: at all, because it's reserved for Python's own command-line options. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.rodola at gmail.com Sat Jul 17 16:03:36 2010 From: g.rodola at gmail.com (=?ISO-8859-1?Q?Giampaolo_Rodol=E0?=) Date: Sat, 17 Jul 2010 16:03:36 +0200 Subject: [Python-Dev] profiler decorator - is it worth for inclusion? In-Reply-To: References: Message-ID: Provided a patch on the tracker: http://bugs.python.org/issue9285 Further comments can be submitted there, if any. --- Giampaolo http://code.google.com/p/pyftpdlib http://code.google.com/p/psutil 2010/7/15 Giampaolo Rodol? : > 2010/7/15 Brian Curtin : >> On Thu, Jul 15, 2010 at 13:45, Giampaolo Rodol? wrote: >>> >>> Today I was looking for a quick and dirty way to profile a method of a >>> class. >>> I was thinking that cProfile module had a decorator for this but I was >>> wrong so I decided to write one based on hotshot. >>> Would it be worth for inclusion? >> >> Since hotshot is gone in 3.x, I'd guess the chances are probably slim. >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com >> >> > > > Here's one using cProfile instead. > I was using hotshot because I wasn't aware of cProfile.Profile.runcall > which is currently not documented (I'm going to file a bug report). > > > def profile(sort='cumulative', lines=30, strip_dirs=True): > ? ?"""A decorator which profiles a callable. > > ? ?Example usage: > > ? ?>>> @profile() > ? ?... def factorial(n): > ? ?... ? ? n = abs(int(n)) > ? ?... ? ? if n < 1: > ? ?... ? ? ? ? ? ? n = 1 > ? ?... ? ? x = 1 > ? ?... ? ? for i in range(1, n+1): > ? ?... ? ? ? ? ? ? x = i * x > ? ?... ? ? return x > ? ?... > ? ?>>> factorial(5) > ? ?Thu Jul 15 20:58:21 2010 ? ?/tmp/tmpIDejr5 > > ? ? ? ? ? ? 4 function calls in 0.000 CPU seconds > > ? ? ? Ordered by: internal time, call count > > ? ? ? ncalls ?tottime ?percall ?cumtime ?percall filename:lineno(function) > ? ? ? ? ? ?1 ? ?0.000 ? ?0.000 ? ?0.000 ? ?0.000 profiler.py:120(factorial) > ? ? ? ? ? ?1 ? ?0.000 ? ?0.000 ? ?0.000 ? ?0.000 {range} > ? ? ? ? ? ?1 ? ?0.000 ? ?0.000 ? ?0.000 ? ?0.000 {abs} > ? ? ? ? ? ?1 ? ?0.000 ? ?0.000 ? ?0.000 ? ?0.000 {method 'disable' of > '_lsprof.Profiler' objects} > > ? ?120 > ? ?""" > ? ?def outer(fun): > ? ? ? ?def inner(*args, **kwargs): > ? ? ? ? ? ?file = tempfile.NamedTemporaryFile() > ? ? ? ? ? ?prof = cProfile.Profile() > ? ? ? ? ? ?try: > ? ? ? ? ? ? ? ?ret = prof.runcall(fun, *args, **kwargs) > ? ? ? ? ? ?except: > ? ? ? ? ? ? ? ?file.close() > ? ? ? ? ? ? ? ?raise > > ? ? ? ? ? ?prof.dump_stats(file.name) > ? ? ? ? ? ?stats = pstats.Stats(file.name) > ? ? ? ? ? ?if strip_dirs: > ? ? ? ? ? ? ? ?stats.strip_dirs() > ? ? ? ? ? ?if isinstance(sort, tuple): > ? ? ? ? ? ? ? ?stats.sort_stats(*sort) > ? ? ? ? ? ?else: > ? ? ? ? ? ? ? ?stats.sort_stats(sort) > ? ? ? ? ? ?stats.print_stats(lines) > > ? ? ? ? ? ?file.close() > ? ? ? ? ? ?return ret > ? ? ? ?return inner > > ? ?return outer > > > --- Giampaolo > http://code.google.com/p/pyftpdlib > http://code.google.com/p/psutil > From taleinat at gmail.com Sat Jul 17 16:57:34 2010 From: taleinat at gmail.com (Tal Einat) Date: Sat, 17 Jul 2010 17:57:34 +0300 Subject: [Python-Dev] [Idle-dev] IDLE contributors and committers In-Reply-To: <4C41985A.4030201@holdenweb.com> References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> Message-ID: On Sat, Jul 17, 2010 at 2:47 PM, Steve Holden wrote: > On 7/17/2010 7:33 AM, Antoine Pitrou wrote: >> >> Hello, >> >> On Sun, 11 Jul 2010 02:05:22 +0300 >> Tal Einat wrote: >>> >>> I would like to propose removing IDLE from the standard library. >>> >>> I have been using IDLE since 2002 and have been doing my best to help >>> maintain and further develop IDLE since 2005. >> >> I haven't seen any conclusive statement or action to this thread. >> Without being an IDLE user myself (for good reason), I think that if >> IDLE should stay, active contributors such as Tal should be given commit >> access and enough free rein to maintain and improve it. >> >> Otherwise there's no reason to continue claiming that IDLE is important >> while discouraging such people's contributions. The current situation, >> where several core developers support IDLE's continued inclusion but >> none actually cares for the issues and patches in the tracker, is >> schizophrenic. >> > +1 > > There's no reason why Tal should be obstructed in his goal of making > IDLE at least acceptable again. It's fairly obvious thaat there aren't > any committers who have both the inclination /and/ the time to do this, > so adding Tal (and other interested parties) as a developer makes a lot > of sense. I would certainly accept this as a first step. Although I now use IDLE much less than I have in previous years, I would be willing to put in some time towards fixing the major current issues and integrating the few polished enhancements. However, in the long run just allowing "heavy" contributors such as myself commit rights won't be enough. There's definitely a need for one or more active maintainers of IDLE who can take care of incoming bug reports and patches. We may hope that at least one serious contributor who is given commit rights will take up this position naturally, but perhaps a more active approach would be beneficial? I also think that there is a need for a guiding hand for IDLE, as Guido is for Python. It took a bit of time until I "got" the goals and principles of IDLE (e.g. easy to learn, minimal and obvious interface) by having KBK explain them in detail and explain the drawbacks of certain proposed changes. Having some kind of central authority is especially important in order to keep IDLE on track because the active development of IDLE is slow and done by various contributors -- there is currently no central group of active developers making such decisions. This doesn't have to be one person who also takes care of bugs, patches and testing, it could be someone who is just readily available via the idle-dev mailing list and keeps up with development of IDLE. Going along these lines of thought, I reach my original conclusion: IDLE is somewhat a project of its own. Perhaps considering IDLE a daughter-project of Python is appropriate, and continuing to develop it as part of the Python codebase could be reasonable, if more active maintainers can be found. I certainly support continuing to package it as part of the standard distribution. From tjreedy at udel.edu Sat Jul 17 23:57:50 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 17 Jul 2010 17:57:50 -0400 Subject: [Python-Dev] IDLE contributors and committers In-Reply-To: References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> <4C41991A.9040103@voidspace.org.uk> Message-ID: On 7/17/2010 8:41 AM, Mark Lawrence wrote: > IIRC Terry Reedy is also interested in moving IDLE forward. Interested, yes. But until either a) I can commit patches, or b) there is someone who will respond to commit review recommendations with "No, here is why not" or "Yes, committed", I will work on other issues, such as doc issues, for which I can get responses. I am certainly reluctant to recruit others to help, as I did for #9222, if there will be no action indefinitely. -- Terry Jan Reedy From pengyu.ut at gmail.com Sun Jul 18 00:03:18 2010 From: pengyu.ut at gmail.com (Peng Yu) Date: Sat, 17 Jul 2010 17:03:18 -0500 Subject: [Python-Dev] mkdir -p in python Message-ID: I don't see that there is a function in the library that mimic the behavior of 'mkdir -p'. If 'makedirs' is used, it will generate an error if the file already exists. There are some functions available on the website to close the gap. But I'd prefer this is in the library. Is there any plan to add a function that mimic the behavior of 'mkdir -p'? http://code.activestate.com/recipes/82465-a-friendly-mkdir/ -- Regards, Peng From breamoreboy at yahoo.co.uk Sun Jul 18 01:45:09 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 18 Jul 2010 00:45:09 +0100 Subject: [Python-Dev] IDLE contributors and committers In-Reply-To: References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> <4C41991A.9040103@voidspace.org.uk> Message-ID: On 17/07/2010 22:57, Terry Reedy wrote: > On 7/17/2010 8:41 AM, Mark Lawrence wrote: > >> IIRC Terry Reedy is also interested in moving IDLE forward. > > Interested, yes. But until either a) I can commit patches, or b) there > is someone who will respond to commit review recommendations with "No, > here is why not" or "Yes, committed", I will work on other issues, such > as doc issues, for which I can get responses. > > I am certainly reluctant to recruit others to help, as I did for #9222, > if there will be no action indefinitely. > This is standard Python behavour. The worst case I've come across is a patch that dated back to 2001 that had not been dealt with. But I'm staggered as to how a third party supplies a patch for (say) 2.3, it doesn't get applied, then the issue tracker simply keeps updating the version targeted until we're now at 3.2. That of course doesn't mean that anything will get done, better wait until py4.7? My approach is very simple, maybe even ruthless, but in the long term I believe it's better for everybody. Does this patch stay open, yes or no? At least it gets the mind focused. Some people have complained at me about my approach. Others have said great job. Obviously there's no correct or incorrect way, there's nowt as queer as folk. Reminds me of Canned Heat, "Let's stick together". Kindest regards. Mark Lawrence. From Petre.Galan at gmail.com Sun Jul 18 07:12:41 2010 From: Petre.Galan at gmail.com (Petre Galan) Date: Sun, 18 Jul 2010 05:12:41 +0000 (UTC) Subject: [Python-Dev] More C API abstraction for user defined types References: Message-ID: Nick Coghlan gmail.com> writes: > >On Wed, Jul 14, 2010 at 11:50 PM, Petre Galan gmail.com wrote: >> No. The right interface is PyNumber_Long. The purpose of the PyNumber_Index >> (and nb_index slot) is as index in slicing. > > Allowing other objects to say "I'm a real integer, treat me as one" is > exactly what the nb_index slot is for (see PEP 357). The use as an > index in slicing was just the primary initial use case and the one > that was chosen as the name for the new slot because nb_int was > already taken for the lossy transformation. Indexing is not (and never > has been) intended to be the only use case. > PyNumber_Long is the right interface as it's the right way to do it. PyNumber_Index allows me to compute a value as index in slicing, value that may be different that the integer behaviour of object. PyNumber_Index is serving it's purpose as index in slicing, beyond that it's getting abused. As for related discussions, float should not implement nb_int slot but be resolved in int's constructor. Petre From ncoghlan at gmail.com Sun Jul 18 08:59:11 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 18 Jul 2010 16:59:11 +1000 Subject: [Python-Dev] More C API abstraction for user defined types In-Reply-To: References: Message-ID: On Sun, Jul 18, 2010 at 3:12 PM, Petre Galan wrote: > PyNumber_Long is the right interface as it's the right way to do it. > PyNumber_Index allows me to compute a value as index in slicing, value that > may be different that the integer behaviour of object. PyNumber_Index is serving > it's purpose as index in slicing, beyond that it's getting abused. > As for related discussions, float should not implement nb_int slot but be > resolved in int's constructor. What's your basis for making that assertion? As one of the devs involved in adding PyNumber_Index and getting it to work properly, I like to think I have some idea as to the purpose of the slot. nb_int = can be coerced to int, but may lose numerical precision in the process nb_index = can be coerced to int without losing numerical precision (used for indexing, slicing and sequence repetition) It's somewhat unintuitive, but that's backwards compatibility for you. To quote the relevant part of PEP 357: Why not use nb_int which is already there? The nb_int method is used for coercion and so means something fundamentally different than what is requested here. This PEP proposes a method for something that *can* already be thought of as an integer communicate that information to Python when it needs an integer. The biggest example of why using nb_int would be a bad thing is that float objects already define the nb_int method, but float objects *should not* be used as indexes in a sequence. Why the name __index__? Some questions were raised regarding the name __index__ when other interpretations of the slot are possible. For example, the slot can be used any time Python requires an integer internally (such as in "mystring" * 3). The name was suggested by Guido because slicing syntax is the biggest reason for having such a slot and in the end no better name emerged. See the discussion thread: http://mail.python.org/pipermail/python-dev/2006-February/thread.html#60594 for examples of names that were suggested such as "__discrete__" and "__ordinal__". So, what spec are you using to say that you're right and I'm wrong? (Now, I'd agree the documentation is a little lacking in this area, as the docs for operator.index, PyNumber_Index and __index__ don't explain very well how they differ semantically from int, PyNumber_Int and __int__, but that's a separate issue. PEP 357 explains the intent quite clearly, even if the published documentation doesn't do so) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ipatrol6010 at yahoo.com Sun Jul 18 01:25:48 2010 From: ipatrol6010 at yahoo.com (ipatrol6010 at yahoo.com) Date: Sat, 17 Jul 2010 16:25:48 -0700 (PDT) Subject: [Python-Dev] Windows process creation flags Message-ID: <255833.76080.qm@web45405.mail.sp1.yahoo.com> I was reading http://msdn.microsoft.com/en-us/library/ms684863%28v=VS.85%29.aspx and http://stackoverflow.com/questions/89228/how-to-call-external-command-in-python#2251026 when I realized that the process creation flags for subprocess.Popen on Windows are not specified anywhere in the standard library. I was thinking of putting them in os or subprocess by prepending a P_ to their MSVC++ names as listed above. It seems like a fairly minor, uncontroversial idea. Can we patch it in? Please CC all replies to this message back to ipatrol6010 at yahoo.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Sun Jul 18 11:28:23 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 18 Jul 2010 10:28:23 +0100 Subject: [Python-Dev] Windows process creation flags In-Reply-To: <255833.76080.qm@web45405.mail.sp1.yahoo.com> References: <255833.76080.qm@web45405.mail.sp1.yahoo.com> Message-ID: <4C42C937.9030008@voidspace.org.uk> On 18/07/2010 00:25, ipatrol6010 at yahoo.com wrote: > I was reading > http://msdn.microsoft.com/en-us/library/ms684863%28v=VS.85%29.aspx and > http://stackoverflow.com/questions/89228/how-to-call-external-command-in-python#2251026 > when I realized that the process creation flags for subprocess.Popen > on Windows are not specified anywhere in the standard library. I was > thinking of putting them in os or subprocess by prepending a P_ to > their MSVC++ names as listed above. It seems like a fairly minor, > uncontroversial idea. Can we patch it in? > > Please CC all replies to this message back to ipatrol6010 at yahoo.com > > Could you create an issue on the Python bug tracker for this, preferably with a patch. Thanks Michael Foord > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Sun Jul 18 11:30:27 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 18 Jul 2010 10:30:27 +0100 Subject: [Python-Dev] mkdir -p in python In-Reply-To: References: Message-ID: <4C42C9B3.7000902@voidspace.org.uk> On 17/07/2010 23:03, Peng Yu wrote: > I don't see that there is a function in the library that mimic the > behavior of 'mkdir -p'. If 'makedirs' is used, it will generate an > error if the file already exists. There are some functions available > on the website to close the gap. But I'd prefer this is in the > library. Is there any plan to add a function that mimic the behavior > of 'mkdir -p'? > > http://code.activestate.com/recipes/82465-a-friendly-mkdir/ > > Presumably a new keyword argument could be added to os.makedirs. Could you raise a feature request on the python bug tracker, preferably with patch and tests of course! It would be worth searching to see if there is an existing feature request for this, and if so you can add a comment to it. All the best, Michael Foord -- http://www.ironpythoninaction.com/ From mail at timgolden.me.uk Sun Jul 18 11:30:51 2010 From: mail at timgolden.me.uk (Tim Golden) Date: Sun, 18 Jul 2010 10:30:51 +0100 Subject: [Python-Dev] mkdir -p in python In-Reply-To: References: Message-ID: <4C42C9CB.1090003@timgolden.me.uk> On 17/07/2010 11:03 PM, Peng Yu wrote: > I don't see that there is a function in the library that mimic the > behavior of 'mkdir -p'. If 'makedirs' is used, it will generate an > error if the file already exists. There are some functions available > on the website to close the gap. But I'd prefer this is in the > library. Is there any plan to add a function that mimic the behavior > of 'mkdir -p'? If I were you I'd ping Tarek who's taken ownership of the shutil module. It would go there if it went anywhere. That said, it's not clear just how far the stdlib should go to mimic every switch and option of shell commands... TJG From fuzzyman at voidspace.org.uk Sun Jul 18 11:42:51 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 18 Jul 2010 10:42:51 +0100 Subject: [Python-Dev] Markup of command-line options in Python's .rst documentation In-Reply-To: References: <4C41AF9F.6090408@voidspace.org.uk> Message-ID: <4C42CC9B.2010303@voidspace.org.uk> On 17/07/2010 14:44, Eli Bendersky wrote: > > > On Sat, Jul 17, 2010 at 16:26, Michael Foord > > wrote: > > On 17/07/2010 14:23, Eli Bendersky wrote: >> Hello, >> >> I'm currently working, together with Terry Reedy, on improving >> the documentation of the trace module, and I ran into a peculiar >> convention of marking command-line options which seems to be >> widespread. >> >> Consider the documentation of timeit, for instance: >> http://docs.python.org/dev/py3k/library/timeit.html >> >> The "--help" option appears as a hyperlink leading to >> http://docs.python.org/dev/py3k/using/cmdline.html#cmdoption--help, >> which is hardly relevant or useful. >> >> The same applies for several command-line options documented for >> the trace module (for example -m and -s). This is a result of the >> following markup (again, taking the timeit module as an example) >> in the relevant .rst file (Doc/library/timeit.rst): >> >> -h/:option:`--help` >> print a short usage message and exit >> >> The :option: markup seems to be translated by Sphinx into a link >> to the Python executable's own command line arguments. This >> creates the aforementioned problem in other modules as well, for >> example unittest. Is there really any merit in marking >> command-line options for modules with :option:, if it's only >> useful for Python's own options? >> > > If it links to the wrong thing then the markup is incorrect > (unless it is due to a regression in Sphinx but I think that is > unlikely). > > Michael > > > > Michael, > What *should* it link to in case of modules, however? Is there some > streamlined policy as to how module command line options should look > and where they should be listed? From a cursory look on some > documentation files, it's unlikely. > > Perhaps the answer is not to markup module options with :option: at > all, because it's reserved for Python's own command-line options. :option: is "reserved" for Python command line options so *shouldn't* be used for module options. We don't have specific markup for module options, so just ``code`` markup I guess. Michael > Eli > > > > > > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Sun Jul 18 11:53:42 2010 From: guido at python.org (Guido van Rossum) Date: Sun, 18 Jul 2010 11:53:42 +0200 Subject: [Python-Dev] mkdir -p in python In-Reply-To: <4C42C9CB.1090003@timgolden.me.uk> References: <4C42C9CB.1090003@timgolden.me.uk> Message-ID: On Sun, Jul 18, 2010 at 11:30 AM, Tim Golden wrote: > On 17/07/2010 11:03 PM, Peng Yu wrote: >> >> I don't see that there is a function in the library that mimic the >> behavior of 'mkdir -p'. If 'makedirs' is used, it will generate an >> error if the file already exists. There are some functions available >> on the website to close the gap. But I'd prefer this is in the >> library. Is there any plan to add a function that mimic the behavior >> of ?'mkdir -p'? > > If I were you I'd ping Tarek who's taken ownership of the shutil > module. It would go there if it went anywhere. > > That said, it's not clear just how far the stdlib should go to > mimic every switch and option of shell commands... Well, it really should have behaved like mkdir -p in the first place. Eric Raymond submitted the code ages ago, and at the time I wasn't familiar with mkdir -p, otherwise, I would have made him change it. I believe every single use of os.mkdirs() I've made goes something like this: if not os.path.exists(x): os.mkdirs(x) or perhaps like this: try: os.mkdirs(x) except os.error: pass (Though the latter masks a bunch of other rare but possible failure modes -- often this is in throw-away code though where it doesn't matter or the next line will raise an exception that's good enough to diagnose the situation anyway.) -- --Guido van Rossum (python.org/~guido) From solipsis at pitrou.net Sun Jul 18 16:34:43 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 18 Jul 2010 16:34:43 +0200 Subject: [Python-Dev] tracker contribution References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> <4C41991A.9040103@voidspace.org.uk> Message-ID: <20100718163443.41ce2538@pitrou.net> Hello Mark, On Sun, 18 Jul 2010 00:45:09 +0100 Mark Lawrence wrote: > On 17/07/2010 22:57, Terry Reedy wrote: > > On 7/17/2010 8:41 AM, Mark Lawrence wrote: > > > >> IIRC Terry Reedy is also interested in moving IDLE forward. > > > > Interested, yes. But until either a) I can commit patches, or b) there > > is someone who will respond to commit review recommendations with "No, > > here is why not" or "Yes, committed", I will work on other issues, such > > as doc issues, for which I can get responses. > > > > I am certainly reluctant to recruit others to help, as I did for #9222, > > if there will be no action indefinitely. > > > > This is standard Python behavour. The worst case I've come across is a > patch that dated back to 2001 that had not been dealt with. But I'm > staggered as to how a third party supplies a patch for (say) 2.3, it > doesn't get applied, then the issue tracker simply keeps updating the > version targeted until we're now at 3.2. That of course doesn't mean > that anything will get done, better wait until py4.7? > > My approach is very simple, maybe even ruthless, but in the long term I > believe it's better for everybody. Does this patch stay open, yes or > no? At least it gets the mind focused. > > Some people have complained at me about my approach. Others have said > great job. Obviously there's no correct or incorrect way, there's nowt > as queer as folk. Well, I would still encourage you to adopt a different approach. Simply bumping up bugs without adding any useful content to the discussion does not help much. Especially if you do so several times in a day, because it increases a lot the bandwidth of messages we have to deal to. And, given we have a limited amount of time and cognitive capacity to devote to Python (we're all volunteers), that means we either spend a lot of time reading your messages and context switching between several issues per day, or start filtering out your messages in order to stay focussed. And if we start doing the latter, it means you are just wasting your own time. It would be IMHO much more useful if you concentrated on a couple of issues which interest you, or which you feel competent in, and where you either: - produce a review of a posted patch - post a patch yourself - give suggestions as to how the issue could be solved (or, alternatively, explain why it is detrimental, or useless, or too late (etc.) to solve the issue) - ask questions to the original submitter if things seem fuzzy It is especially useful if you manage to produce a negative review, that is point out something wrong with the patch/issue. Things that can get wrong with a patch (non-exhaustive list follows): - the patch doesn't apply cleanly anymore (on the py3k or 2.7 SVN branch, depending on whether the issue regards 3.x or only 2.x) - the proposed solution breaks compatibility in an obvious or subtle way - the patch lacks unit tests - unit tests don't adequately test behaviour (too strict, too laxist, too fragile, etc.) - portability problems, i.e. patch wouldn't have guaranteed or proper behaviour on all platforms - inconsistent or improper coding style - performance problems on arguably performance-critical code - undesireable side effects - etc. I would stress that this kind of involvment would also be able to get you much higher on the learning curve than your current involvment does: i.e., non-trivial contributions will build the competences to improve both yourself (assuming that's something you're interested in) and the complexity, correctness and completeness of your future contributions. Which in the middle-term is generally quite gratifying. Regards Antoine. From christopherolah.co at gmail.com Sun Jul 18 17:34:46 2010 From: christopherolah.co at gmail.com (Christopher Olah) Date: Sun, 18 Jul 2010 11:34:46 -0400 Subject: [Python-Dev] Function Operators Message-ID: Dear python-dev, In mathematical notation, f*g = z->f(g(z)) and f^n = f*f*f... (n times). I often run into situations in python where such operators could result in cleaner code. Eventually, I decided to implement it myself and see how it worked in practice. However, my intuitive implementation [1] doesn't seem to work. In particular, despite what it says in function's documentation, function does not seem to be in __builtin__. Furthermore, when I try to implement this through type(f) (where f is a function) I get invalid syntax errors. I hope I haven't made some trivial error; I'm rather inexperienced as a pythonist. Christopher Olah [1] Sketch: def __builtin__.function.__mul__(self, f): return lambda x: self(f(x)) def __builtin__.function.__pow__(self, n): return lambda x: reduce(lambda a,b: [f for i in range(n)]+[x]) From reid.kleckner at gmail.com Sun Jul 18 17:52:07 2010 From: reid.kleckner at gmail.com (Reid Kleckner) Date: Sun, 18 Jul 2010 08:52:07 -0700 Subject: [Python-Dev] Function Operators In-Reply-To: References: Message-ID: Usual disclaimer: python-dev is for the development *of* python, not *with*. See python-list, etc. That said, def declares new functions or methods, so you can't put arbitrary expressions in there like type(f).__mul__ . You can usually assign to things like that though, but in this case you run into trouble, as shown below: >>> def func(): pass ... >>> type(func) >>> def compose(f, g): ... return lambda x: f(g(x)) ... >>> type(func).__mul__ = compose Traceback (most recent call last): File "", line 1, in TypeError: can't set attributes of built-in/extension type 'function' As the interpreter says, it doesn't like people mucking with operator slots on built in types. Finally, if you like coding in that very functional style, I'd recommend Haskell or other ML derived languages. Python doesn't support that programming style very well by choice. Reid On Sun, Jul 18, 2010 at 8:34 AM, Christopher Olah wrote: > Dear python-dev, > > In mathematical notation, f*g = z->f(g(z)) and f^n = f*f*f... (n > times). I often run into situations in python where such operators > could result in cleaner code. Eventually, I decided to implement it > myself and see how it worked in practice. > > However, my intuitive implementation [1] doesn't seem to work. In > particular, despite what it says in function's documentation, function > does not seem to be in __builtin__. Furthermore, when I try to > implement this through type(f) (where f is a function) I get invalid > syntax errors. > > I hope I haven't made some trivial error; I'm rather inexperienced as > a pythonist. > > Christopher Olah > > > [1] Sketch: > > def __builtin__.function.__mul__(self, f): > ? ?return lambda x: self(f(x)) > > def __builtin__.function.__pow__(self, n): > ? ?return lambda x: reduce(lambda a,b: [f for i in range(n)]+[x]) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/reid.kleckner%40gmail.com > From alexander.belopolsky at gmail.com Sun Jul 18 18:01:35 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 18 Jul 2010 12:01:35 -0400 Subject: [Python-Dev] tracker contribution In-Reply-To: <20100718163443.41ce2538@pitrou.net> References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> <4C41991A.9040103@voidspace.org.uk> <20100718163443.41ce2538@pitrou.net> Message-ID: Antoine, You've just saved me from composing essentially the same message. I am top-posting because I have very little to add. Mark, I actually reviewed the issues that got closed thanks to your "bumping them up". That was 30+ issues over the last week or two. Quite impressive. However, I see you "nosy" on about 180 open issues. This raises a question about signal to noise ratio that Antoine mentioned. I would like to join Antoine recommending that you concentrate on issues that you are competent in. In addition, I think your breadth first approach is also valuable, but don't comment of every issue that you read. Unless you feel that the issue was simply forgotten (last message is from OP and no response from assignee or issue is not assigned), it may be better to move on when you don't have something meaningful to contribute. On Sun, Jul 18, 2010 at 10:34 AM, Antoine Pitrou wrote: .. > On Sun, 18 Jul 2010 00:45:09 +0100 > Mark Lawrence wrote: .. >> Some people have complained at me about my approach. ?Others have said >> great job. ?Obviously there's no correct or incorrect way, there's nowt >> as queer as folk. > > Well, I would still encourage you to adopt a different approach. > Simply bumping up bugs without adding any useful content to the > discussion does not help much. Especially if you do so several times in > a day, because it increases a lot the bandwidth of messages we have to > deal to. And, given we have a limited amount of time and cognitive > capacity to devote to Python (we're all volunteers), that means we > either spend a lot of time reading your messages and context switching > between several issues per day, or start filtering out your messages in > order to stay focussed. And if we start doing the latter, it means you > are just wasting your own time. > > It would be IMHO much more useful if you concentrated on a > couple of issues which interest you, or which you feel competent in, > and where you either: > > - produce a review of a posted patch > - post a patch yourself > - give suggestions as to how the issue could be solved (or, > ?alternatively, explain why it is detrimental, or useless, or too late > ?(etc.) to solve the issue) > - ask questions to the original submitter if things seem fuzzy > > It is especially useful if you manage to produce a negative review, > that is point out something wrong with the patch/issue. Things that can > get wrong with a patch (non-exhaustive list follows): > > - the patch doesn't apply cleanly anymore (on the py3k or 2.7 SVN > ?branch, depending on whether the issue regards 3.x or only 2.x) > - the proposed solution breaks compatibility in an obvious or subtle way > - the patch lacks unit tests > - unit tests don't adequately test behaviour (too strict, too laxist, > ?too fragile, etc.) > - portability problems, i.e. patch wouldn't have guaranteed or proper > ?behaviour on all platforms > - inconsistent or improper coding style > - performance problems on arguably performance-critical code > - undesireable side effects > - etc. > > I would stress that this kind of involvment would also be able to get > you much higher on the learning curve than your current involvment does: > i.e., non-trivial contributions will build the competences to improve > both yourself (assuming that's something you're interested in) and the > complexity, correctness and completeness of your future contributions. > Which in the middle-term is generally quite gratifying. > > Regards > > Antoine. From alexander.belopolsky at gmail.com Sun Jul 18 19:46:45 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 18 Jul 2010 13:46:45 -0400 Subject: [Python-Dev] What to do with languishing patches? Message-ID: On Sat, Jul 17, 2010 at 7:45 PM, Mark Lawrence wrote: > On 17/07/2010 22:57, Terry Reedy wrote: .. >> I am certainly reluctant to recruit others to help, as I did for #9222, >> if there will be no action indefinitely. >> > > This is standard Python behavour. ?The worst case I've come across is a > patch that dated back to 2001 that had not been dealt with. ?But I'm > staggered as to how a third party supplies a patch for (say) 2.3, it doesn't > get applied, then the issue tracker simply keeps updating the version > targeted until we're now at 3.2. ?That of course doesn't mean that anything > will get done, better wait until py4.7? > If this is the reputation that Python gets as a project, I think it is alarming. I was going to dismiss Mark's comment as over dramatizing the situation, but unfortunately it is not. Let me focus on a specific example not because it is important, but because it illustrates many different problems in an easy to understand case. The issue is http://bugs.python.org/issue1699259, which is a very simple patch submitted back in 2007. I came across that patch a year an a half later and posted a review. OP responded the next day and posted an updated patch. I gave the resulting patch a positive review, but the timing was unfortunate because it was during RC phase for 3.0. At the time, I made a prediction that 'deferring [the patch] to 2.7 and 3.1 [was] virtually equivalent to closing with "won't fix"'. That was quite right: the next comment came a year later with "Moving the 3.1 target to 3.2, since 3.1 is out" only to be forgotten for another year to be rediscovered in Mark's recent crusade. As I said, the issue is not an important one, but I noticed that OP did not submit any other patch or commented on any other issue since then. It is impossible to tell if he was turned off by the lack of attention to his patch, but I think it is likely that Python is loosing contributors when they submit a patch, respond to reviews and see their patch languish for no specified reason. I think there is at least one improvement that we can make. We can set up a mechanism that will assure that acceptable patches that don't get applied because they come at the wrong time in development cycle, get applied early in the development of the next release. We already have "posponed" and "remind" resolutions, but these are exclusive of "accepted". I think there should be a clear way to mark the issue "accepted and would be applied if X.Y was out already." Chances are one of the resolution labels already has such meaning, but in this case it should be more prominently documented as such. From thomas at jollans.com Sun Jul 18 19:43:44 2010 From: thomas at jollans.com (Thomas Jollans) Date: Sun, 18 Jul 2010 19:43:44 +0200 Subject: [Python-Dev] Function Operators In-Reply-To: References: Message-ID: <4C433D50.70004@jollans.com> On 07/18/2010 05:52 PM, Reid Kleckner wrote: > Usual disclaimer: python-dev is for the development *of* python, not > *with*. See python-list, etc. Moving to python-list. Please keep discussion there. > > That said, def declares new functions or methods, so you can't put > arbitrary expressions in there like type(f).__mul__ . > > You can usually assign to things like that though, but in this case > you run into trouble, as shown below: > >>>> def func(): pass > ... >>>> type(func) > >>>> def compose(f, g): > ... return lambda x: f(g(x)) > ... >>>> type(func).__mul__ = compose > Traceback (most recent call last): > File "", line 1, in > TypeError: can't set attributes of built-in/extension type 'function' > > As the interpreter says, it doesn't like people mucking with operator > slots on built in types. > > Finally, if you like coding in that very functional style, I'd > recommend Haskell or other ML derived languages. Python doesn't > support that programming style very well by choice. > > Reid > > On Sun, Jul 18, 2010 at 8:34 AM, Christopher Olah > wrote: >> Dear python-dev, >> >> In mathematical notation, f*g = z->f(g(z)) and f^n = f*f*f... (n >> times). I often run into situations in python where such operators >> could result in cleaner code. Eventually, I decided to implement it >> myself and see how it worked in practice. >> >> However, my intuitive implementation [1] doesn't seem to work. In >> particular, despite what it says in function's documentation, function >> does not seem to be in __builtin__. Furthermore, when I try to >> implement this through type(f) (where f is a function) I get invalid >> syntax errors. >> >> I hope I haven't made some trivial error; I'm rather inexperienced as >> a pythonist. >> >> Christopher Olah >> >> >> [1] Sketch: >> >> def __builtin__.function.__mul__(self, f): >> return lambda x: self(f(x)) >> >> def __builtin__.function.__pow__(self, n): >> return lambda x: reduce(lambda a,b: [f for i in range(n)]+[x]) As Reid explained, you can't just muck around with built-in types like that. However, you can "use a different type". If you're not familiar with Python decorators, look them up, and then have a look at this simple implementation of what you were looking for: >>> class mfunc: ... def __init__(self, func): ... self.func = func ... self.__doc__ = func.__doc__ ... self.__name__ = func.__name__ ... def __call__(self, *args, **kwargs): ... return self.func(*args, **kwargs) ... def __mul__(self, f2): ... @mfunc ... def composite(*a, **kwa): ... return self.func(f2(*a, **kwa)) ... return composite ... def __pow__(self, n): ... if n < 1: ... raise ValueError(n) ... elif n == 1: ... return self.func ... else: ... return self * (self ** (n-1)) ... >>> @mfunc ... def square(x): return x*x ... >>> @mfunc ... def twice(x): return 2*x ... >>> (square*twice)(1.5) 9.0 >>> addthree = mfunc(lambda x: x+3) >>> addfifteen = (addthree ** 5) >>> addfifteen(0) 15 >>> From breamoreboy at yahoo.co.uk Sun Jul 18 20:43:53 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 18 Jul 2010 19:43:53 +0100 Subject: [Python-Dev] tracker contribution In-Reply-To: <20100718163443.41ce2538@pitrou.net> References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> <4C41991A.9040103@voidspace.org.uk> <20100718163443.41ce2538@pitrou.net> Message-ID: On 18/07/2010 15:34, Antoine Pitrou wrote: > > Hello Mark, > > On Sun, 18 Jul 2010 00:45:09 +0100 > Mark Lawrence wrote: >> On 17/07/2010 22:57, Terry Reedy wrote: >>> On 7/17/2010 8:41 AM, Mark Lawrence wrote: >>> >>>> IIRC Terry Reedy is also interested in moving IDLE forward. >>> >>> Interested, yes. But until either a) I can commit patches, or b) there >>> is someone who will respond to commit review recommendations with "No, >>> here is why not" or "Yes, committed", I will work on other issues, such >>> as doc issues, for which I can get responses. >>> >>> I am certainly reluctant to recruit others to help, as I did for #9222, >>> if there will be no action indefinitely. >>> >> >> This is standard Python behavour. The worst case I've come across is a >> patch that dated back to 2001 that had not been dealt with. But I'm >> staggered as to how a third party supplies a patch for (say) 2.3, it >> doesn't get applied, then the issue tracker simply keeps updating the >> version targeted until we're now at 3.2. That of course doesn't mean >> that anything will get done, better wait until py4.7? >> >> My approach is very simple, maybe even ruthless, but in the long term I >> believe it's better for everybody. Does this patch stay open, yes or >> no? At least it gets the mind focused. >> >> Some people have complained at me about my approach. Others have said >> great job. Obviously there's no correct or incorrect way, there's nowt >> as queer as folk. > > Well, I would still encourage you to adopt a different approach. > Simply bumping up bugs without adding any useful content to the > discussion does not help much. Especially if you do so several times in > a day, because it increases a lot the bandwidth of messages we have to > deal to. And, given we have a limited amount of time and cognitive > capacity to devote to Python (we're all volunteers), that means we > either spend a lot of time reading your messages and context switching > between several issues per day, or start filtering out your messages in > order to stay focussed. And if we start doing the latter, it means you > are just wasting your own time. > > It would be IMHO much more useful if you concentrated on a > couple of issues which interest you, or which you feel competent in, > and where you either: > > - produce a review of a posted patch > - post a patch yourself > - give suggestions as to how the issue could be solved (or, > alternatively, explain why it is detrimental, or useless, or too late > (etc.) to solve the issue) > - ask questions to the original submitter if things seem fuzzy > > It is especially useful if you manage to produce a negative review, > that is point out something wrong with the patch/issue. Things that can > get wrong with a patch (non-exhaustive list follows): > > - the patch doesn't apply cleanly anymore (on the py3k or 2.7 SVN > branch, depending on whether the issue regards 3.x or only 2.x) > - the proposed solution breaks compatibility in an obvious or subtle way > - the patch lacks unit tests > - unit tests don't adequately test behaviour (too strict, too laxist, > too fragile, etc.) > - portability problems, i.e. patch wouldn't have guaranteed or proper > behaviour on all platforms > - inconsistent or improper coding style > - performance problems on arguably performance-critical code > - undesireable side effects > - etc. > > I would stress that this kind of involvment would also be able to get > you much higher on the learning curve than your current involvment does: > i.e., non-trivial contributions will build the competences to improve > both yourself (assuming that's something you're interested in) and the > complexity, correctness and completeness of your future contributions. > Which in the middle-term is generally quite gratifying. > > Regards > > Antoine. > > I'm extremely offended by your comments. I'll just back off and let the number of outstanding bugs grow and grow and grow, until such time as people get fed up with Python and go to (say) Ruby. Kindest regards. Mark Lawrence From glyph at twistedmatrix.com Sun Jul 18 21:57:57 2010 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Sun, 18 Jul 2010 15:57:57 -0400 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: References: Message-ID: <7B78A282-EADA-48BD-93DD-26719168E7B9@twistedmatrix.com> On Jul 18, 2010, at 1:46 PM, Alexander Belopolsky wrote: > We already have "posponed" and "remind" resolutions, but these are > exclusive of "accepted". I think there should be a clear way to mark > the issue "accepted and would be applied if X.Y was out already." > Chances are one of the resolution labels already has such meaning, but > in this case it should be more prominently documented as such. This is what branches are for. When the X.Y release cycle starts, there should be a branch for X.Y. Any "would be applied" patches can simply be applied to trunk without interrupting anything; the X.Y release branch can be merged back into trunk as necessary. -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Sun Jul 18 22:32:34 2010 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 18 Jul 2010 21:32:34 +0100 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: <7B78A282-EADA-48BD-93DD-26719168E7B9@twistedmatrix.com> References: <7B78A282-EADA-48BD-93DD-26719168E7B9@twistedmatrix.com> Message-ID: On 18 July 2010 20:57, Glyph Lefkowitz wrote: > > On Jul 18, 2010, at 1:46 PM, Alexander Belopolsky wrote: > > We already have "posponed" and "remind" resolutions, but these are > exclusive of "accepted". ??I think there should be a clear way to mark > the issue "accepted and would be applied if X.Y was out already." > Chances are one of the resolution labels already has such meaning, but > in this case it should be more prominently documented as such. > > This is what branches are for. > When the X.Y release cycle starts, there should be a branch for X.Y. ?Any > "would be applied" patches can simply be applied to trunk without > interrupting anything; the X.Y release branch can be merged back into trunk > as necessary. Agreed. If that isn't already the recommended workflow under Mercurial, I'd suggest making it so. (I can imagine that under Subversion, where branching and merging is more awkward, it might have been deemed not worth doing). Paul. From breamoreboy at yahoo.co.uk Sun Jul 18 23:02:38 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 18 Jul 2010 22:02:38 +0100 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: References: Message-ID: On 18/07/2010 18:46, Alexander Belopolsky wrote: > On Sat, Jul 17, 2010 at 7:45 PM, Mark Lawrence wrote: >> On 17/07/2010 22:57, Terry Reedy wrote: > .. >>> I am certainly reluctant to recruit others to help, as I did for #9222, >>> if there will be no action indefinitely. >>> >> >> This is standard Python behavour. The worst case I've come across is a >> patch that dated back to 2001 that had not been dealt with. But I'm >> staggered as to how a third party supplies a patch for (say) 2.3, it doesn't >> get applied, then the issue tracker simply keeps updating the version >> targeted until we're now at 3.2. That of course doesn't mean that anything >> will get done, better wait until py4.7? >> > > If this is the reputation that Python gets as a project, I think it is > alarming. I was going to dismiss Mark's comment as over dramatizing > the situation, but unfortunately it is not. > > Let me focus on a specific example not because it is important, but > because it illustrates many different problems in an easy to > understand case. The issue is http://bugs.python.org/issue1699259, > which is a very simple patch submitted back in 2007. I came across > that patch a year an a half later and posted a review. OP responded > the next day and posted an updated patch. I gave the resulting patch > a positive review, but the timing was unfortunate because it was > during RC phase for 3.0. > > At the time, I made a prediction that 'deferring [the patch] to 2.7 > and 3.1 [was] virtually equivalent to closing with "won't fix"'. > That was quite right: the next comment came a year later with "Moving > the 3.1 target to 3.2, since 3.1 is out" only to be forgotten for > another year to be rediscovered in Mark's recent crusade. > > As I said, the issue is not an important one, but I noticed that OP > did not submit any other patch or commented on any other issue since > then. It is impossible to tell if he was turned off by the lack of > attention to his patch, but I think it is likely that Python is > loosing contributors when they submit a patch, respond to reviews and > see their patch languish for no specified reason. > > I think there is at least one improvement that we can make. We can > set up a mechanism that will assure that acceptable patches that don't > get applied because they come at the wrong time in development cycle, > get applied early in the development of the next release. > > We already have "posponed" and "remind" resolutions, but these are > exclusive of "accepted". I think there should be a clear way to mark > the issue "accepted and would be applied if X.Y was out already." > Chances are one of the resolution labels already has such meaning, but > in this case it should be more prominently documented as such. Alexander, This strikes me as being simple, straight forward common sense, well put. Until such time as this happens, I'll carry on ploughing through the outstanding issues flagging up things to try and get them moved. Apologies to all and sundry if I tread on toes. Maybe going off on a tangent, but I find it frustrating because you (plural) can't find a given module on the issue tracker. Say I'm looking for issues relating to smtplib, all I can do is look in the title of the issue. Have I missed something, or is there a need to have subsections so that if Lib is selected, you can then find the appropriate module? Kindest regards. Mark Lawrence. From alexander.belopolsky at gmail.com Sun Jul 18 23:05:20 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 18 Jul 2010 17:05:20 -0400 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: References: <7B78A282-EADA-48BD-93DD-26719168E7B9@twistedmatrix.com> Message-ID: On Sun, Jul 18, 2010 at 4:32 PM, Paul Moore wrote: > On 18 July 2010 20:57, Glyph Lefkowitz wrote: .. >> This is what branches are for. >> When the X.Y release cycle starts, there should be a branch for X.Y. ?Any >> "would be applied" patches can simply be applied to trunk without >> interrupting anything; the X.Y release branch can be merged back into trunk >> as necessary. > > Agreed. If that isn't already the recommended workflow under > Mercurial, I'd suggest making it so. (I can imagine that under > Subversion, where branching and merging is more awkward, it might have > been deemed not worth doing). Do I understand it correctly that the proposal is to create an X.Y-maint branch at the time when alpha (or beta) release goes out rather than the current practice of creating the branch at the time of the final release? In this case issue1699259 patch could be committed to py3k branch without affecting what goes into 3.0. I still don't understand why "Python 2.6 is already out" was a reason not to commit it to trunk at the time. From alexander.belopolsky at gmail.com Sun Jul 18 23:17:16 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 18 Jul 2010 17:17:16 -0400 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: References: Message-ID: [Removing idle-dev from CC] On Sun, Jul 18, 2010 at 5:02 PM, Mark Lawrence wrote: .. > Maybe going off on a tangent, but I find it frustrating because you (plural) > can't find a given module on the issue tracker. ?Say I'm looking for issues > relating to smtplib, all I can do is look in the title of the issue. ?Have I > missed something, or is there a need to have subsections so that if Lib is > selected, you can then find the appropriate module? I don't think you missed anything and I would also find this feature useful. For example, I recently took interest in datetime and pickle modules, but other than full text search, there was no way to find all issues related to either of these two modules. The "Components" list is somewhat arbitrary with narrow components such as ctypes and broad components such a Lib. If you could make a specific proposal on how this can be reorganized without adding much clutter to the interface and creating submitter confusion, I think it will be very helpful. Another feature that comes to mind is to allow searing for issues with patches that touch a particular file. I have not looked at the tracker code, but I think this should not be hard to implement. From ncoghlan at gmail.com Sun Jul 18 23:22:14 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 19 Jul 2010 07:22:14 +1000 Subject: [Python-Dev] tracker contribution In-Reply-To: References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> <4C41991A.9040103@voidspace.org.uk> <20100718163443.41ce2538@pitrou.net> Message-ID: On Mon, Jul 19, 2010 at 4:43 AM, Mark Lawrence wrote: > ?I'm extremely offended by your comments. ?I'll just back off and let the > number of outstanding bugs grow and grow and grow, until such time as people > get fed up with Python and go to (say) Ruby. Please don't take it that way - Antoine and Alexander are just trying to help you make the most effective use of the time you spend contributing (which *is* appreciated!). In my case, I don't spend much time trawling the tracker for issues, so I'm reliant on other people kicking import related issues in my direction by adding me to the nosy list or bringing them up here on python-dev. I think a couple of the items you have commented on ended up on my plate and I should be able to do something about them. Cheers, Nick. P.S. 30 closures for 180 comments actually sounds like a reasonable SNR to me. Maybe my perspective is skewed by the fact that I'm not involved in most of them :) -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From jnoller at gmail.com Sun Jul 18 23:24:07 2010 From: jnoller at gmail.com (Jesse Noller) Date: Sun, 18 Jul 2010 17:24:07 -0400 Subject: [Python-Dev] tracker contribution In-Reply-To: References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> <4C41991A.9040103@voidspace.org.uk> <20100718163443.41ce2538@pitrou.net> Message-ID: On Sun, Jul 18, 2010 at 5:22 PM, Nick Coghlan wrote: > On Mon, Jul 19, 2010 at 4:43 AM, Mark Lawrence wrote: >> ?I'm extremely offended by your comments. ?I'll just back off and let the >> number of outstanding bugs grow and grow and grow, until such time as people >> get fed up with Python and go to (say) Ruby. > > Please don't take it that way - Antoine and Alexander are just trying > to help you make the most effective use of the time you spend > contributing (which *is* appreciated!). > > In my case, I don't spend much time trawling the tracker for issues, > so I'm reliant on other people kicking import related issues in my > direction by adding me to the nosy list or bringing them up here on > python-dev. I think a couple of the items you have commented on ended > up on my plate and I should be able to do something about them. > > Cheers, > Nick. > > P.S. 30 closures for 180 comments actually sounds like a reasonable > SNR to me. Maybe my perspective is skewed by the fact that I'm not > involved in most of them :) And I'm with Nick here. I don't have the time or bandwidth to trawl the tracker :( So you bumping things to/for me helps. From pez at apocalyptech.com Sun Jul 18 23:28:18 2010 From: pez at apocalyptech.com (CJ Kucera) Date: Sun, 18 Jul 2010 16:28:18 -0500 Subject: [Python-Dev] Fast Implementation for ZIP decryption Message-ID: <20100718212818.GL20775@apocalyptech.com> Hello list, resurrecting a rather old thread from here: http://mail.python.org/pipermail/python-dev/2009-August/091450.html I've actually come across a use case where faster zipfile decryption would be very helpful to me (though one could certainly argue that it's a frivolous case). I'm writing a map editor for a game whose graphics datafiles are stored inside an encrypted zipfile. I've received the game developer's blessing for my editor to be able to read inside the zipfile, so legality isn't an issue here, but the pure-Python decryption currently available in the 'zipfile' module really is dog-slow. When I have my app load directly from already-uncompressed files on the HD, it's rare for a map to take more than 1 second to load; when the app has to use the pure-Python decryption routines to load all the graphics images, it can take over a minute (and this is on a 3GHz Core 2 Duo). Bundling the necessary graphics with my app isn't an option, nor is asking users to crack the zipfile encryption themselves to provide an uncompressed directory to read from. So anyway, I'd at least love to have a peek at the C code mentioned in the thread; if nothing more, perhaps I could bundle it into something that could be installed optionally by someone who wants it. Does anyone still have this code available somewhere? Thanks a bunch, CJ -- WOW: Flemmy | "Happiness isn't good enough for me! I pez at apocalyptech.com | demand euphoria!" 24.24.2.3171 | - Calvin From martin at v.loewis.de Mon Jul 19 00:10:23 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 18 Jul 2010 23:10:23 +0100 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: References: Message-ID: <4C437BCF.5000409@v.loewis.de> > Maybe going off on a tangent, but I find it frustrating because you > (plural) can't find a given module on the issue tracker. Say I'm looking > for issues relating to smtplib, all I can do is look in the title of the > issue. Have I missed something, or is there a need to have subsections > so that if Lib is selected, you can then find the appropriate module? If you think any specific feature is missing in the tracker, please post a report to the meta-tracker. Regards, Martin From alexander.belopolsky at gmail.com Mon Jul 19 00:38:09 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 18 Jul 2010 18:38:09 -0400 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: <4C437BCF.5000409@v.loewis.de> References: <4C437BCF.5000409@v.loewis.de> Message-ID: On Sun, Jul 18, 2010 at 6:10 PM, "Martin v. L?wis" wrote: >> Maybe going off on a tangent, but I find it frustrating because you >> (plural) can't find a given module on the issue tracker. Say I'm looking >> for issues relating to smtplib, all I can do is look in the title of the >> issue. Have I missed something, or is there a need to have subsections >> so that if Lib is selected, you can then find the appropriate module? > > If you think any specific feature is missing in the tracker, please post > a report to the meta-tracker. This is an excellent point and for those who don't know what "meta-tracker" is, see the link to "Report Tracker Problem" at the bottom of left column on the main tracker page. The direct link is http://psf.upfronthosting.co.za/roundup/meta/ . (I did not know about the "meta-tracker" myself until a couple of months ago.) From breamoreboy at yahoo.co.uk Mon Jul 19 01:06:32 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Mon, 19 Jul 2010 00:06:32 +0100 Subject: [Python-Dev] tracker contribution In-Reply-To: References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> <4C41991A.9040103@voidspace.org.uk> <20100718163443.41ce2538@pitrou.net> Message-ID: On 18/07/2010 22:24, Jesse Noller wrote: > On Sun, Jul 18, 2010 at 5:22 PM, Nick Coghlan wrote: >> On Mon, Jul 19, 2010 at 4:43 AM, Mark Lawrence wrote: >>> I'm extremely offended by your comments. I'll just back off and let the >>> number of outstanding bugs grow and grow and grow, until such time as people >>> get fed up with Python and go to (say) Ruby. >> >> Please don't take it that way - Antoine and Alexander are just trying >> to help you make the most effective use of the time you spend >> contributing (which *is* appreciated!). >> >> In my case, I don't spend much time trawling the tracker for issues, >> so I'm reliant on other people kicking import related issues in my >> direction by adding me to the nosy list or bringing them up here on >> python-dev. I think a couple of the items you have commented on ended >> up on my plate and I should be able to do something about them. >> >> Cheers, >> Nick. >> >> P.S. 30 closures for 180 comments actually sounds like a reasonable >> SNR to me. Maybe my perspective is skewed by the fact that I'm not >> involved in most of them :) > > > And I'm with Nick here. I don't have the time or bandwidth to trawl > the tracker :( So you bumping things to/for me helps. Nick, Jesse, Thank you for your kind comments, they're much appreciated. Now the fighter bombers dive in to finish of the last pockets of resistance. (from a Monty Python album in case you didn't know) What am I meant to do when as happened earlier today, I see an issue that was first raised two years ago, then a year later the OP has asked what if anything is happening? Leave it? That's a great advert for Python. How do I apply a patch that was raised *SEVEN* years ago to modern versions of Python? The issue is still open. I do my best to run the unit tests if they're available on an issue, but so often they're so old that it's all but impossible to apply them without a hell of a lot of manual work. If this had been done in the first place it wouldn't have been an issue. I believe that by now you get my drift. But to me the most infuriating thing on the bug tracker is when someone else has simply bumped the Python versions and that's it. No attempt at contacting the OP as to whether the issue is still applicable. No attempt to try the patches, all in all zilch. So I come along and attempt to backtrack years and get slagged off for it. Hell, I just wish I was fully healthy and had my MBCS/CEng status back, then I'd really feel capable of letting fly. Having worked on massive UK MOD projects (can't say much, Official Secrets Acts and all that stuff) and knowing a hell of a lot about change control, configuration management, call it what you like, quite frankly Python is years behind. But there I go again, can't rock the boat because someone might get upset, to hell with the poor sods putting in their patches years ago and being completely ignored. You all might have gathered that I'm very dispirited by the negative attitudes that I get from a relatively small minority of Python people. I might as well quit because it doesn't do my mental health a great deal of good. That minority will of course be very happy to see me go because they'll be able to sit back on their loathsome, spotty behinds picking blackheads and resort to the usual "we're far too busy to do anything" routine. You might understand that I don't completely agree with this. But to hell with it, I'm as usual feeling absolutely shattered so I'm going to bed, please don't try to close too many issues whilst I'm sleeping, I won't have anything to do tomorrow afternoon. Kindest regards. Mark Lawrence. From alexander.belopolsky at gmail.com Mon Jul 19 01:21:10 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 18 Jul 2010 19:21:10 -0400 Subject: [Python-Dev] Use of coding cookie in 3.x stdlib Message-ID: I was looking at the inspect module and noticed that it's source starts with "# -*- coding: iso-8859-1 -*-". I have checked and there are no non-ascii characters in the file. There are several other modules that still use the cookie: Lib/ast.py:# -*- coding: utf-8 -*- Lib/getopt.py:# -*- coding: utf-8 -*- Lib/inspect.py:# -*- coding: iso-8859-1 -*- Lib/pydoc.py:# -*- coding: latin-1 -*- Lib/shlex.py:# -*- coding: iso-8859-1 -*- Lib/encodings/punycode.py:# -*- coding: utf-8 -*- Lib/msilib/__init__.py:# -*- coding: utf-8 -*- Lib/sqlite3/__init__.py:#-*- coding: ISO-8859-1 -*- Lib/sqlite3/dbapi2.py:#-*- coding: ISO-8859-1 -*- Lib/test/bad_coding.py:# -*- coding: uft-8 -*- Lib/test/badsyntax_3131.py:# -*- coding: utf-8 -*- I understand that coding: utf-8 is strictly redundant in 3.x. There are cases such as Lib/shlex.py where using encoding other than utf-8 is justified. (See http://svn.python.org/view?view=rev&revision=82560). What are the guidelines for other cases? Should redundant cookies be removed? Since not all editors respect the -*- cookie, I think the answer should be "yes" particularly when the cookie is setting encoding other than utf-8. From g.brandl at gmx.net Mon Jul 19 01:42:31 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 19 Jul 2010 00:42:31 +0100 Subject: [Python-Dev] IDLE contributors and committers In-Reply-To: References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> <4C41991A.9040103@voidspace.org.uk> Message-ID: Am 18.07.2010 00:45, schrieb Mark Lawrence: > On 17/07/2010 22:57, Terry Reedy wrote: >> On 7/17/2010 8:41 AM, Mark Lawrence wrote: >> >>> IIRC Terry Reedy is also interested in moving IDLE forward. >> >> Interested, yes. But until either a) I can commit patches, or b) there >> is someone who will respond to commit review recommendations with "No, >> here is why not" or "Yes, committed", I will work on other issues, such >> as doc issues, for which I can get responses. >> >> I am certainly reluctant to recruit others to help, as I did for #9222, >> if there will be no action indefinitely. >> > > This is standard Python behavour. That phrasing implies that there is purpose behind letting issues rot. Believe me that this is not the case. > The worst case I've come across is a > patch that dated back to 2001 that had not been dealt with. But I'm > staggered as to how a third party supplies a patch for (say) 2.3, it > doesn't get applied, then the issue tracker simply keeps updating the > version targeted until we're now at 3.2. That of course doesn't mean > that anything will get done, better wait until py4.7? If no developer can invest the time necessary before, yes. Better not to apply a patch than to rush in something that might have a negative impact in the long run. For code such as Python's standard library, we have to think very carefully about everything that could affect compatibility or introduce new bugs or regressions. I don't want to say that all patches are seriously flawed, but for most patches I applied I had to make some changes before committing, and I needed time enough to think about them. > My approach is very simple, maybe even ruthless, but in the long term I > believe it's better for everybody. Does this patch stay open, yes or > no? At least it gets the mind focused. And this is what is great about what you're doing, because many issues can be closed immediately, and some after minor work. > Some people have complained at me about my approach. Others have said > great job. Obviously there's no correct or incorrect way, there's nowt > as queer as folk. Of course committers are different. I guess that most who complained did what I did, saying "I think you're doing a great job, but try to ...". Why not take the advice? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From alexander.belopolsky at gmail.com Mon Jul 19 01:55:35 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 18 Jul 2010 19:55:35 -0400 Subject: [Python-Dev] tracker contribution In-Reply-To: References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> <4C41991A.9040103@voidspace.org.uk> <20100718163443.41ce2538@pitrou.net> Message-ID: On Sun, Jul 18, 2010 at 7:06 PM, Mark Lawrence wrote: .. > What am I meant to do when as happened earlier today, I see an issue that > was first raised two years ago, then a year later the OP has asked what if > anything is happening? ?Leave it? That's a great advert for Python. > > How do I apply a patch that was raised *SEVEN* years ago to modern versions > of Python? ?The issue is still open. ?I do my best to run the unit tests if > they're available on an issue, but so often they're so old that it's all but > impossible to apply them without a hell of a lot of manual work. ?If this > had been done in the first place it wouldn't have been an issue. > Don't get frustrated and don't give up. Python is valued for its stability and most patches are delayed for good reasons. Oftentimes the problem is simply nontrivial and the proposed patch is not obviously correct. In other cases the patch addresses a rare use case and is deemed not important enough to take priority over other things that are on developer's todo list. What to do with old patches? I would suggest the following steps: 1. Verify that the issue is still present in the latest releases. 2. Add a comment confirming that you were able to reproduce the bug and explain how. 3. Try to apply the patch. With a 7-year old patch your chances that it will apply cleanly are slim. Try both 2.x and 3.x branches. If patch contains tests, tests may apply but not code or the other way around. If you get any success, post your results. 4. If you find really egregious cases where you think that a valuable patch fell through the cracks, - bring it up here. By this time you probably have at least in your mind the list of issues that should have been closed long ago. Pick top three by value you think they would add and post here. I know first hand that posting a patch and see it linger until it does not apply anymore can be frustrating. One of the issues you flagged recently (http://bugs.python.org/issue4335) has a patch that I posted 1.5 years ago. It does not apply cleanly anymore and it would get a low priority on my list these days. Nevertheless, I appreciate the thorough approach that python developers take when it comes to accepting contributions or even committing their own changes. You may have noticed that developers with commit privileges often post their patches on the tracker first and get them reviewed before checking in. This is why python is such a high quality product. Once again, don't get frustrated. Your work is appreciated. There is room for all kinds of approaches to improving python and as long as you have time and desire to contribute, your contributions will make a difference. From greg.ewing at canterbury.ac.nz Mon Jul 19 03:05:58 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 19 Jul 2010 13:05:58 +1200 Subject: [Python-Dev] mkdir -p in python In-Reply-To: <4C42C9CB.1090003@timgolden.me.uk> References: <4C42C9CB.1090003@timgolden.me.uk> Message-ID: <4C43A4F6.4090800@canterbury.ac.nz> Tim Golden wrote: > That said, it's not clear just how far the stdlib should go to > mimic every switch and option of shell commands... I don't think it's a matter of mimicking switches just because they're there. The operation of "make sure this directory and all its parents exist" is very commonly required, e.g. when installing software. It makes sense to have it easily available. -- Greg From tjreedy at udel.edu Mon Jul 19 04:22:07 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 18 Jul 2010 22:22:07 -0400 Subject: [Python-Dev] IDLE contributors and committers In-Reply-To: References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> <4C41991A.9040103@voidspace.org.uk> Message-ID: On 7/18/2010 7:42 PM, Georg Brandl wrote: > That phrasing implies that there is purpose behind letting issues rot. > Believe me that this is not the case. This seems like a good place to mention that doc issues have become a bright spot in the last few years, with a couple of days turnaround not unusual for an issue ready for commit review. This is one reason I have focused a bit on doc issues. It is rewarding. As I remember, a reason to switch from latex to sphinx/rst was so that more people could/would work on doc issues. That seems to have worked. And I do not think it bad is the docs catch up to the code by getting extra attention. -- Terry Jan Reedy From eliben at gmail.com Mon Jul 19 04:57:34 2010 From: eliben at gmail.com (Eli Bendersky) Date: Mon, 19 Jul 2010 05:57:34 +0300 Subject: [Python-Dev] Markup of command-line options in Python's .rst documentation In-Reply-To: <4C41B3A4.2000501@netwok.org> References: <4C41B3A4.2000501@netwok.org> Message-ID: On Sat, Jul 17, 2010 at 16:44, ?ric Araujo wrote: > > The "--help" option appears as a hyperlink leading to > > http://docs.python.org/dev/py3k/using/cmdline.html#cmdoption--help, > which is > > hardly relevant or useful. [...] > > > > -h/:option:`--help` > > print a short usage message and exit > > I think this is a doc bug in Doc/documenting/markup.rst > :cmdoption: and :option: are not clearly distinguished; the latter > creates references to using/cmdline, the former is what you?re looking > for for documenting trace.py. > How would you guys recommend to proceed from here? The simplest approach for me is just use :cmdoption: instead of :option: in my work on trace.py . However, a few more things can be done if this is indeed "official policy": 1. Fix other modules that use :option: to use :cmdoption: instead (timeit, unittest and others) 2. Fix Doc/documenting/markup.rst to clarify which option kind goes where If these steps get approved I'll be happy to create an issue and submit patches to the relevant files. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Mon Jul 19 05:02:57 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 18 Jul 2010 23:02:57 -0400 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: References: <7B78A282-EADA-48BD-93DD-26719168E7B9@twistedmatrix.com> Message-ID: On 7/18/2010 5:05 PM, Alexander Belopolsky wrote: > On Sun, Jul 18, 2010 at 4:32 PM, Paul Moore wrote: >> On 18 July 2010 20:57, Glyph Lefkowitz wrote: > .. >>> This is what branches are for. >>> When the X.Y release cycle starts, there should be a branch for X.Y. Any >>> "would be applied" patches can simply be applied to trunk without >>> interrupting anything; the X.Y release branch can be merged back into trunk >>> as necessary. >> >> Agreed. If that isn't already the recommended workflow under >> Mercurial, I'd suggest making it so. (I can imagine that under >> Subversion, where branching and merging is more awkward, it might have >> been deemed not worth doing). > > Do I understand it correctly that the proposal is to create an > X.Y-maint branch at the time when alpha (or beta) release goes out > rather than the current practice of creating the branch at the time of > the final release? I would understand it to be at the point when some commits would otherwise be deferred and possibly forgotten. Whick is to say, first beta. I would expect that partial and then nearly complete closure of 'trunk' would work best if there were only a few committers who nearly all turn to bugs and then critical bugs during the beta and rc phases and who would not look at anything else during those phases. Of course, it would be best if pending patches for known bugs were applied before the split, so that the only patches applied to the new branch were for newly discovered bugs. -- Terry Jan Reedy From eliben at gmail.com Mon Jul 19 05:28:11 2010 From: eliben at gmail.com (Eli Bendersky) Date: Mon, 19 Jul 2010 06:28:11 +0300 Subject: [Python-Dev] Markup of command-line options in Python's .rst documentation In-Reply-To: References: <4C41B3A4.2000501@netwok.org> Message-ID: On Mon, Jul 19, 2010 at 05:57, Eli Bendersky wrote: > On Sat, Jul 17, 2010 at 16:44, ?ric Araujo wrote: > >> > The "--help" option appears as a hyperlink leading to >> > http://docs.python.org/dev/py3k/using/cmdline.html#cmdoption--help, >> which is >> > hardly relevant or useful. [...] >> > >> > -h/:option:`--help` >> > print a short usage message and exit >> >> I think this is a doc bug in Doc/documenting/markup.rst >> :cmdoption: and :option: are not clearly distinguished; the latter >> creates references to using/cmdline, the former is what you?re looking >> for for documenting trace.py. >> > > How would you guys recommend to proceed from here? > > The simplest approach for me is just use :cmdoption: instead of :option: in > my work on trace.py . However, a few more things can be done if this is > indeed "official policy": > > 1. Fix other modules that use :option: to use :cmdoption: instead (timeit, > unittest and others) > 2. Fix Doc/documenting/markup.rst to clarify which option kind goes where > > If these steps get approved I'll be happy to create an issue and submit > patches to the relevant files. > > Eli > More input on this issue: 'cmdoption' is a directive, while 'option' is inline markup. Therefore I wouldn't say they're completely similar, just meant for different purposes. Both a directive and inline markup is useful for describing "official python executable options/flags". Regarding per-module options, I'm not sure special markup is needed at all. So a policy has to be define regarding the correct usage of these directives/markups, and probably documented in Doc/documenting/markup.rst Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.belopolsky at gmail.com Mon Jul 19 05:40:14 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 18 Jul 2010 23:40:14 -0400 Subject: [Python-Dev] Markup of command-line options in Python's .rst documentation In-Reply-To: References: <4C41B3A4.2000501@netwok.org> Message-ID: On Sun, Jul 18, 2010 at 11:28 PM, Eli Bendersky wrote: > .. So a policy has to be define regarding the > correct usage of these directives/markups, and probably documented in > Doc/documenting/markup.rst I wonder if in addition to documenting proper markup you could add an option to argparse to generate help in ReST format. I understand that this will not be immediately useful for trace.py which uses getopt and getopt does not have enough structured data to generate good help. On the other hand, upgrading trace.py to use argparse may not be a bad idea. From eliben at gmail.com Mon Jul 19 05:46:16 2010 From: eliben at gmail.com (Eli Bendersky) Date: Mon, 19 Jul 2010 06:46:16 +0300 Subject: [Python-Dev] Markup of command-line options in Python's .rst documentation In-Reply-To: References: <4C41B3A4.2000501@netwok.org> Message-ID: On Mon, Jul 19, 2010 at 06:40, Alexander Belopolsky < alexander.belopolsky at gmail.com> wrote: > On Sun, Jul 18, 2010 at 11:28 PM, Eli Bendersky wrote: > > .. So a policy has to be define regarding the > > correct usage of these directives/markups, and probably documented in > > Doc/documenting/markup.rst > > I wonder if in addition to documenting proper markup you could add an > option to argparse to generate help in ReST format. I understand that > this will not be immediately useful for trace.py which uses getopt and > getopt does not have enough structured data to generate good help. On > the other hand, upgrading trace.py to use argparse may not be a bad > idea. > I actually considered switching to argparse when I noticed that trace.py still lives with getopt (personally I've long been using optparse). However, I wonder what this means for backwards compatibility. Is it valid to switch trace.py to use the newer command-line argument parsing module that's only available in the newest versions of Python? I guess it could be since trace.py is stdlib which always arrives with its relevant version of Python. This *would* mean, however, that trace.py would be incompatible between 2.7 (argparse) and 2.6 (which doesn't have argparse), but I'm not sure how much of a problem this is. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.belopolsky at gmail.com Mon Jul 19 05:52:42 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 18 Jul 2010 23:52:42 -0400 Subject: [Python-Dev] Markup of command-line options in Python's .rst documentation In-Reply-To: References: <4C41B3A4.2000501@netwok.org> Message-ID: On Sun, Jul 18, 2010 at 11:46 PM, Eli Bendersky wrote: .. > However, I wonder what this means for backwards compatibility. Is it valid > to switch trace.py to use the newer command-line argument parsing module > that's only available in the newest versions of Python? I guess it could be > since trace.py is stdlib which always arrives with its relevant version of > Python. This *would* mean, however, that trace.py would be incompatible > between 2.7 (argparse) and 2.6 (which doesn't have argparse), but I'm not > sure how much of a problem this is. As long as 3.x trace supports all command line options and switches that 2.x does, I don't see any compatibility problems. Adding more options or using new language or library features in implementation should be fine. From tjreedy at udel.edu Mon Jul 19 06:02:31 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 19 Jul 2010 00:02:31 -0400 Subject: [Python-Dev] Does trace modules have a unit test? Message-ID: In reviewing http://bugs.python.org/issue9282 the issue came up, where is the unit test for trace.py? test/test_trace.py is actually a test of the line trace facility of sys.settrace (and should have been called test_linetrace or test_settrace). The only trace import Eli could find in Lib/test is the one in regrtest.py for the -T option. Have we missed something? If you say 'make one', what should be its name? Can the existing file be renamed (I gather this can be a bit of a problem)? -- Terry Jan Reedy From eliben at gmail.com Mon Jul 19 06:12:04 2010 From: eliben at gmail.com (Eli Bendersky) Date: Mon, 19 Jul 2010 07:12:04 +0300 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: On Mon, Jul 19, 2010 at 07:02, Terry Reedy wrote: > In reviewing > http://bugs.python.org/issue9282 > the issue came up, where is the unit test for trace.py? > > test/test_trace.py is actually a test of the line trace facility of > sys.settrace (and should have been called test_linetrace or test_settrace). > The only trace import Eli could find in Lib/test is the one in regrtest.py > for the -T option. Have we missed something? > > If you say 'make one', what should be its name? Can the existing file be > renamed (I gather this can be a bit of a problem)? > I'd like to ask pydev another question on this issue. When thinking about adding unit tests for trace.py, I quickly realized it's not a usual module. Most of its output is either tracing reports to stdout, or even worse, to special files named .cover stdout output can be captured, but what about the .cover files? Can a Python unit test create temporary files in tmp/ (or somewhere else) as part of its testing, or is this forbidden? Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.belopolsky at gmail.com Mon Jul 19 06:21:36 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 19 Jul 2010 00:21:36 -0400 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: On Mon, Jul 19, 2010 at 12:12 AM, Eli Bendersky wrote: .. > stdout output can be captured, but what about the .cover files? Can a Python > unit test create temporary files in tmp/ (or somewhere else) as part of its > testing, or is this forbidden? > That's perfectly fine. Grep in the Lib/test directory for 'tempfile.mkdtemp()' to see examples. From alexander.belopolsky at gmail.com Mon Jul 19 06:23:50 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 19 Jul 2010 00:23:50 -0400 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: On Mon, Jul 19, 2010 at 12:21 AM, Alexander Belopolsky wrote: > On Mon, Jul 19, 2010 at 12:12 AM, Eli Bendersky wrote: > .. >> stdout output can be captured, but what about the .cover files? Can a Python >> unit test create temporary files in tmp/ (or somewhere else) as part of its >> testing, or is this forbidden? >> > > That's perfectly fine. ?Grep in the Lib/test directory for > 'tempfile.mkdtemp()' to see examples. > Actually the first hit is Lib/test/script_helper.py which contains several utilities that you may find useful. From guido at python.org Mon Jul 19 08:45:17 2010 From: guido at python.org (Guido van Rossum) Date: Mon, 19 Jul 2010 08:45:17 +0200 Subject: [Python-Dev] Use of coding cookie in 3.x stdlib In-Reply-To: References: Message-ID: Sounds like a good idea to try to remove redundant cookies *and* to remove most occasional use of non-ASCII characters outside comments (except for unittests specifically trying to test Unicode features). Personally I would use \xXX escapes instead of spelling out the characters in shlex.py, for example. Both with or without the coding cookies, many ways of displaying text files garble characters outside the ASCII range, so it's better to stick to ASCII as much as possible. --Guido On Mon, Jul 19, 2010 at 1:21 AM, Alexander Belopolsky wrote: > I was looking at the inspect module and noticed that it's source > starts with "# -*- coding: iso-8859-1 -*-". ? I have checked and there > are no non-ascii characters in the file. ? There are several other > modules that still use the cookie: > > Lib/ast.py:# -*- coding: utf-8 -*- > Lib/getopt.py:# -*- coding: utf-8 -*- > Lib/inspect.py:# -*- coding: iso-8859-1 -*- > Lib/pydoc.py:# -*- coding: latin-1 -*- > Lib/shlex.py:# -*- coding: iso-8859-1 -*- > Lib/encodings/punycode.py:# -*- coding: utf-8 -*- > Lib/msilib/__init__.py:# -*- coding: utf-8 -*- > Lib/sqlite3/__init__.py:#-*- coding: ISO-8859-1 -*- > Lib/sqlite3/dbapi2.py:#-*- coding: ISO-8859-1 -*- > Lib/test/bad_coding.py:# -*- coding: uft-8 -*- > Lib/test/badsyntax_3131.py:# -*- coding: utf-8 -*- > > I understand that coding: utf-8 is strictly redundant in 3.x. ?There > are cases such as Lib/shlex.py where using encoding other than utf-8 > is justified. ?(See > http://svn.python.org/view?view=rev&revision=82560). ?What are the > guidelines for other cases? ?Should redundant cookies be removed? > Since not all editors respect the ?-*- cookie, I think the answer > should be "yes" particularly when the cookie is setting encoding other > than utf-8. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) From g.brandl at gmx.net Mon Jul 19 08:58:12 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 19 Jul 2010 07:58:12 +0100 Subject: [Python-Dev] Markup of command-line options in Python's .rst documentation In-Reply-To: References: <4C41B3A4.2000501@netwok.org> Message-ID: Am 19.07.2010 04:28, schrieb Eli Bendersky: > On Mon, Jul 19, 2010 at 05:57, Eli Bendersky > wrote: > > On Sat, Jul 17, 2010 at 16:44, ?ric Araujo > wrote: > > > The "--help" option appears as a hyperlink leading to > > http://docs.python.org/dev/py3k/using/cmdline.html#cmdoption--help, > which is > > hardly relevant or useful. [...] > > > > -h/:option:`--help` > > print a short usage message and exit > > I think this is a doc bug in Doc/documenting/markup.rst > :cmdoption: and :option: are not clearly distinguished; the latter > creates references to using/cmdline, the former is what you?re looking > for for documenting trace.py. > > > How would you guys recommend to proceed from here? > > The simplest approach for me is just use :cmdoption: instead of :option: in > my work on trace.py . However, a few more things can be done if this is > indeed "official policy": > > 1. Fix other modules that use :option: to use :cmdoption: instead (timeit, > unittest and others) > 2. Fix Doc/documenting/markup.rst to clarify which option kind goes where > > If these steps get approved I'll be happy to create an issue and submit > patches to the relevant files. > > Eli > > > More input on this issue: > > 'cmdoption' is a directive, while 'option' is inline markup. Therefore I > wouldn't say they're completely similar, just meant for different purposes. Both > a directive and inline markup is useful for describing "official python > executable options/flags". Regarding per-module options, I'm not sure special > markup is needed at all. So a policy has to be define regarding the correct > usage of these directives/markups, and probably documented in > Doc/documenting/markup.rst I've done that now in r82961. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ncoghlan at gmail.com Mon Jul 19 12:41:04 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 19 Jul 2010 20:41:04 +1000 Subject: [Python-Dev] tracker contribution In-Reply-To: References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> <4C41991A.9040103@voidspace.org.uk> <20100718163443.41ce2538@pitrou.net> Message-ID: On Mon, Jul 19, 2010 at 9:06 AM, Mark Lawrence wrote: > Hell, I just wish I was fully healthy and had my MBCS/CEng status back, then > I'd really feel capable of letting fly. ?Having worked on massive UK MOD > projects (can't say much, Official Secrets Acts and all that stuff) and > knowing a hell of a lot about change control, configuration management, call > it what you like, quite frankly Python is years behind. ?But there I go > again, can't rock the boat because someone might get upset, to hell with the > poor sods putting in their patches years ago and being completely ignored. I have worked (and do work) on similar projects and in many ways Python is well ahead of what some large corporations do. In terms of test automation, baseline control, public auditing of the source repository, public communications, the very nature of open source development avoids a lot of the pitfalls private enterprise can fall into. We *have* to document and automate stuff, because you can't just yell over at Bob in the next cubicle to ask "hey, what do I need to install to get [fnord] to build?". We *know* that everything we commit is going to land in the email inbox of a large number of people, so we better give it a reasonable checkin message and include a meaningful comment explaining any apparently-stupid-but-actually-correct code. The fundamental constraint, though, is that the core developers aren't paid specifically to hack on Python. We may use it in our day jobs, and I think some of the folks may get a bit of time to spend on it during their work week, but making the new versions better isn't the primary task for any of us. Large corporations, on the other hand, have a lot more money to throw at things and can pay people to work on the boring stuff rather than relying purely on volunteers. If you get to a feature request, find that it doesn't apply cleanly anymore (or even come close), post a comment to say that. If nobody steps up to modernise the patch, but it isn't a fundamentally bad idea, then it's OK for the tracker issue to remain open indefinitely (e.g. one of mine you commented on recently I had deliberately left open for a long time because I hadn't made up my mind whether I agreed with it or not. There were some comments from earlier this year that I missed at the time, but reading them all now means that I'm inclined to accept the change for 3.2). For bug reports, the basic thing is to see if the issue can be reproduced on currently maintained versions. If it can, update the version applicability accordingly, if it can't, suggest closure as out of date. This isn't a company where the metrics mavens will see a growing count of low priority feature requests and issue reports and demand that they either be accepted or rejected and people will be taken from other tasks and given the responsibility to work through the list. Is it *good* that things are this way? No, not at all. But it isn't likely to change anytime soon, either. Cheers, Nick. P.S. Regarding the version bumps with no other comment: I believe there are some scripts kicking around to bump feature requests up to the new development version after a new release goes into maintenance-only mode. It may be good if any such scripts are updated to also add a comment to that effect, but I don't believe those scripts are centrally controlled anywhere. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Mon Jul 19 12:58:54 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 19 Jul 2010 20:58:54 +1000 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: On Mon, Jul 19, 2010 at 2:23 PM, Alexander Belopolsky wrote: > On Mon, Jul 19, 2010 at 12:21 AM, Alexander Belopolsky > wrote: >> On Mon, Jul 19, 2010 at 12:12 AM, Eli Bendersky wrote: >> .. >>> stdout output can be captured, but what about the .cover files? Can a Python >>> unit test create temporary files in tmp/ (or somewhere else) as part of its >>> testing, or is this forbidden? >>> >> >> That's perfectly fine. ?Grep in the Lib/test directory for >> 'tempfile.mkdtemp()' to see examples. >> > > Actually the first hit is Lib/test/script_helper.py which contains > several utilities that you may find useful. Yeah, the script_helper.temp_dir CM is especially handy for mucking about on the filesystem. test_cmd_line_script.py and test_runpy.py are other places to look for inspiration on ways to make heavy use of the filesystem for testing without leaving junk behind when the test is over (script_helper was actually born by refactoring the helpers from those two test suites out to a common module). I really should get back to the issue about a temp_dir equivalent worthy of inclusion in the tempfile module at some point, though (there's an unfortunate corner case relating to interpreter shutdown and generators or explicit invocation of __enter__ that causes problems for the script_helper.temp_dir incarnation. The test suite avoids that corner case naturally, but a tempfile.tempdir CM needs to cope with it) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From merwok at netwok.org Mon Jul 19 13:31:12 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Mon, 19 Jul 2010 13:31:12 +0200 Subject: [Python-Dev] Markup of command-line options in Python's .rst documentation In-Reply-To: References: <4C41B3A4.2000501@netwok.org> Message-ID: <4C443780.5070005@netwok.org> Le 19/07/2010 04:57, Eli Bendersky a ?crit : > On Sat, Jul 17, 2010 at 16:44, ?ric Araujo wrote: >> I think this is a doc bug in Doc/documenting/markup.rst >> :cmdoption: and :option: are not clearly distinguished; the latter >> creates references to using/cmdline, the former is what you?re looking >> for for documenting trace.py. I had actually misunderstood the documentation here. Georg corrected me in http://bugs.python.org/issue9288 and made the right fix (thanks!) Kind regards From eric at netwok.org Mon Jul 19 13:21:46 2010 From: eric at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Mon, 19 Jul 2010 13:21:46 +0200 Subject: [Python-Dev] Markup of command-line options in Python's .rst documentation In-Reply-To: References: <4C41B3A4.2000501@netwok.org> Message-ID: <4C44354A.8030303@netwok.org> Le 19/07/2010 04:57, Eli Bendersky a ?crit : > On Sat, Jul 17, 2010 at 16:44, ?ric Araujo wrote: >> I think this is a doc bug in Doc/documenting/markup.rst >> :cmdoption: and :option: are not clearly distinguished; the latter >> creates references to using/cmdline, the former is what you?re looking >> for for documenting trace.py. I had actually misunderstood the documentation here. Georg corrected me in http://bugs.python.org/issue9288 and made the right fix (thanks!) Kind regards From ysj.ray at gmail.com Mon Jul 19 07:16:28 2010 From: ysj.ray at gmail.com (Ray Allen) Date: Mon, 19 Jul 2010 13:16:28 +0800 Subject: [Python-Dev] mkdir -p in python In-Reply-To: <4C43A4F6.4090800@canterbury.ac.nz> References: <4C42C9CB.1090003@timgolden.me.uk> <4C43A4F6.4090800@canterbury.ac.nz> Message-ID: Agree. Through searching for existing issues, I found there isn't any such request. And I will submit one. The '-p' option of shell's mkdir has tow functions: 1, create parents directories if not exists, 2, suppress "File exists" error if the target has already exists. What we need is the second function. So I think both os.mkdir() and os.makedirs() should add a keyword argument to suppress the "OSError: [Errno 17] File exists". So we can remove many "try...except..." codes surround os.mkdir() and os.makedirs() as Greg said, "The operation of 'make sure this directory and all its parents exist' is very commonly required" -- Ray Allen Best wishes! -------------- next part -------------- An HTML attachment was scrubbed... URL: From ysj.ray at gmail.com Mon Jul 19 10:49:26 2010 From: ysj.ray at gmail.com (Ray Allen) Date: Mon, 19 Jul 2010 16:49:26 +0800 Subject: [Python-Dev] mkdir -p in python In-Reply-To: References: <4C42C9CB.1090003@timgolden.me.uk> <4C43A4F6.4090800@canterbury.ac.nz> Message-ID: See http://bugs.python.org/issue9299 On Mon, Jul 19, 2010 at 1:16 PM, Ray Allen wrote: > > Agree. Through searching for existing issues, I found there isn't any such > request. And I will submit one. > > The '-p' option of shell's mkdir has tow functions: > 1, create parents directories if not exists, > 2, suppress "File exists" error if the target has already exists. > What we need is the second function. So I think both os.mkdir() and > os.makedirs() should add a keyword argument to suppress the "OSError: [Errno > 17] File exists". So we can remove many "try...except..." codes surround > os.mkdir() and os.makedirs() as Greg said, "The operation of 'make sure > this directory and all its parents exist' is very commonly required" > > -- > Ray Allen > Best wishes! > -- Ray Allen Best wishes! -------------- next part -------------- An HTML attachment was scrubbed... URL: From scott.mccarty at gmail.com Mon Jul 19 18:11:10 2010 From: scott.mccarty at gmail.com (Scott McCarty) Date: Mon, 19 Jul 2010 12:11:10 -0400 Subject: [Python-Dev] Python signal processing question Message-ID: All, I have searched everywhere (mostly the code and a little google) and I cannot understand where the SIGKILL signal gets checked when it is set as a handler. I have scoured the Modules/signalmodule.c only to find two instances of the RuntimeError exception, but I cannot understand how python knows when a handler is set for SIGKILL. I understand that this changed in 2.4 and I am not trying to change it, I just really want to understand where this happens. I used grep to find SIGKILL and SIGTERM to see if I could determine where the critical difference is, but I cannot figure it out. I have about 2 hours of searching around and I can't figure it out, I assume it has to rely on some default behaviour in Unix, but I have no idea. I don't see a difference between SIGKILL and SIGTERM in the python code, but obviously there is some difference. I understand what the difference is in Unix/Linux, I just want to see it in the python code. Since python is checking at run time to see what signals handlers are added, I know there must be a difference. Please can someone just point me in the right direction. Thank You Scott M -------------- next part -------------- An HTML attachment was scrubbed... URL: From phd at phd.pp.ru Mon Jul 19 18:27:29 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Mon, 19 Jul 2010 20:27:29 +0400 Subject: [Python-Dev] Python signal processing question In-Reply-To: References: Message-ID: <20100719162729.GF21891@phd.pp.ru> Hello. We'are sorry but we cannot help you. This mailing list is to work on developing Python (fixing bugs and adding new features to Python itself); if you're having problems using Python, please find another forum. Probably python-list (comp.lang.python) news group/mailing list is the best place. See http://www.python.org/community/ for other lists/news groups/fora. Thank you for understanding. Regarding SIGKILL - read "man kill" carefully. Or better yet, a Unix programmer guide. On Mon, Jul 19, 2010 at 12:11:10PM -0400, Scott McCarty wrote: > All, I have searched everywhere (mostly the code and a little google) and I > cannot understand where the SIGKILL signal gets checked when it is set as a > handler. I have scoured the Modules/signalmodule.c only to find two > instances of the RuntimeError exception, but I cannot understand how python > knows when a handler is set for SIGKILL. I understand that this changed in > 2.4 and I am not trying to change it, I just really want to understand where > this happens. I used grep to find SIGKILL and SIGTERM to see if I could > determine where the critical difference is, but I cannot figure it out. > > I have about 2 hours of searching around and I can't figure it out, I assume > it has to rely on some default behaviour in Unix, but I have no idea. I > don't see a difference between SIGKILL and SIGTERM in the python code, but > obviously there is some difference. I understand what the difference is in > Unix/Linux, I just want to see it in the python code. Since python is > checking at run time to see what signals handlers are added, I know there > must be a difference. > > Please can someone just point me in the right direction. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From scott.mccarty at gmail.com Mon Jul 19 19:09:36 2010 From: scott.mccarty at gmail.com (Scott McCarty) Date: Mon, 19 Jul 2010 13:09:36 -0400 Subject: [Python-Dev] Python signal processing question In-Reply-To: <20100719162729.GF21891@phd.pp.ru> References: <20100719162729.GF21891@phd.pp.ru> Message-ID: I apologize if this is the wrong place to ask questions about Python C code, but I didn't know where else to turn. I don't think the regular user group will be technical enough to find this, am I wrong? There is no forum for something this in depth. This is not a usage problem, I want to know where in the code this is happening, I understand how to use these signals perfectly and why they are different. Clearly SIGKILL cannot be caught and would never get used even if you did register that handler in Python, while SIGTERM is able to be caught, this is standard Unix signals. The part I am asking about is how python checks ahead of time. If it didn't check (case before python 2.4), then it would get silently ignored because the OS would kill the process without ever letting python even try and handle that signal. I just want to understand the C/Python piece better because I am writing a tutorial on signals and I am using python to demonstrate. I thought it would be fun to show that the SIGKILL is never processed, but instead python errors out. There is something in Python checking the SIGKILL signal handler, while not checking SIGTERM and I can't find the Python C code that handles it. When I am writing something like this, I like to point out the C code, not just cite the documentation (I did find this change in behaviour noted in the Python change log). I will try and find a different list Scott M On Mon, Jul 19, 2010 at 12:27 PM, Oleg Broytman wrote: > Hello. > > We'are sorry but we cannot help you. This mailing list is to work on > developing Python (fixing bugs and adding new features to Python itself); > if > you're having problems using Python, please find another forum. Probably > python-list (comp.lang.python) news group/mailing list is the best place. > See http://www.python.org/community/ for other lists/news groups/fora. > Thank you for understanding. > > Regarding SIGKILL - read "man kill" carefully. Or better yet, a Unix > programmer guide. > > On Mon, Jul 19, 2010 at 12:11:10PM -0400, Scott McCarty wrote: > > All, I have searched everywhere (mostly the code and a little google) and > I > > cannot understand where the SIGKILL signal gets checked when it is set as > a > > handler. I have scoured the Modules/signalmodule.c only to find two > > instances of the RuntimeError exception, but I cannot understand how > python > > knows when a handler is set for SIGKILL. I understand that this changed > in > > 2.4 and I am not trying to change it, I just really want to understand > where > > this happens. I used grep to find SIGKILL and SIGTERM to see if I could > > determine where the critical difference is, but I cannot figure it out. > > > > I have about 2 hours of searching around and I can't figure it out, I > assume > > it has to rely on some default behaviour in Unix, but I have no idea. I > > don't see a difference between SIGKILL and SIGTERM in the python code, > but > > obviously there is some difference. I understand what the difference is > in > > Unix/Linux, I just want to see it in the python code. Since python is > > checking at run time to see what signals handlers are added, I know there > > must be a difference. > > > > Please can someone just point me in the right direction. > > Oleg. > -- > Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru > Programmers don't die, they just GOSUB without RETURN. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From phd at phd.pp.ru Mon Jul 19 19:37:55 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Mon, 19 Jul 2010 21:37:55 +0400 Subject: [Python-Dev] Python signal processing question In-Reply-To: References: <20100719162729.GF21891@phd.pp.ru> Message-ID: <20100719173755.GA5127@phd.pp.ru> On Mon, Jul 19, 2010 at 01:09:36PM -0400, Scott McCarty wrote: > I apologize if this is the wrong place to ask questions about Python C code, > but I didn't know where else to turn. I don't think the regular user group > will be technical enough to find this, am I wrong? I am sure there are quite a lot of skilled professionals in the python-list (comp.lang.python). > On Mon, Jul 19, 2010 at 12:27 PM, Oleg Broytman wrote: > > python-list (comp.lang.python) news group/mailing list is the best place. > > See http://www.python.org/community/ for other lists/news groups/fora. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From alexander.belopolsky at gmail.com Mon Jul 19 20:58:31 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 19 Jul 2010 14:58:31 -0400 Subject: [Python-Dev] Use of coding cookie in 3.x stdlib In-Reply-To: References: Message-ID: On Mon, Jul 19, 2010 at 2:45 AM, Guido van Rossum wrote: > Sounds like a good idea to try to remove redundant cookies *and* to > remove most occasional use of non-ASCII characters outside comments. ... Please see http://bugs.python.org/issue9308 . I am going to post a patch shortly. From dreamingforward at gmail.com Tue Jul 20 00:08:00 2010 From: dreamingforward at gmail.com (average) Date: Mon, 19 Jul 2010 16:08:00 -0600 Subject: [Python-Dev] Python signal processing question In-Reply-To: <20100719162729.GF21891@phd.pp.ru> References: <20100719162729.GF21891@phd.pp.ru> Message-ID: > ? We'are sorry but we cannot help you. This mailing list is to work on > developing Python (fixing bugs and adding new features to Python itself); if > you're having problems using Python, please find another forum. Probably > python-list (comp.lang.python) news group/mailing list is the best place. > See http://www.python.org/community/ for other lists/news groups/fora. > Thank you for understanding. > > ? Regarding SIGKILL - read "man kill" carefully. Or better yet, a Unix > programmer guide. Wha? How could this not be the right place? He's not asking about USING python, but asking: WHERE in the PYTHON CODE BASE does the signal get checked? A-bit-miffed-at-the-cold-shoulderly yours, Marcos (wink wink) From steve at holdenweb.com Tue Jul 20 00:29:32 2010 From: steve at holdenweb.com (Steve Holden) Date: Mon, 19 Jul 2010 23:29:32 +0100 Subject: [Python-Dev] Python signal processing question In-Reply-To: References: <20100719162729.GF21891@phd.pp.ru> Message-ID: On 7/19/2010 11:08 PM, average wrote: >> We'are sorry but we cannot help you. This mailing list is to work on >> developing Python (fixing bugs and adding new features to Python itself); if >> you're having problems using Python, please find another forum. Probably >> python-list (comp.lang.python) news group/mailing list is the best place. >> See http://www.python.org/community/ for other lists/news groups/fora. >> Thank you for understanding. >> >> Regarding SIGKILL - read "man kill" carefully. Or better yet, a Unix >> programmer guide. > > Wha? How could this not be the right place? He's not asking about > USING python, but asking: WHERE in the PYTHON CODE BASE does the > signal get checked? > > A-bit-miffed-at-the-cold-shoulderly yours, > > Marcos (wink wink) +1 Another potential developer walks away feeling unwanted? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From hodgestar+pythondev at gmail.com Tue Jul 20 00:37:10 2010 From: hodgestar+pythondev at gmail.com (Simon Cross) Date: Tue, 20 Jul 2010 00:37:10 +0200 Subject: [Python-Dev] Python signal processing question In-Reply-To: References: <20100719162729.GF21891@phd.pp.ru> Message-ID: On Tue, Jul 20, 2010 at 12:29 AM, Steve Holden wrote: > Another potential developer walks away feeling unwanted? Let just kill this thread here. SIGKILL can't be caught. :) (I had sort of assumed that Marcos was being ironic). Schiavo Simon From greg.ewing at canterbury.ac.nz Tue Jul 20 01:11:19 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 20 Jul 2010 11:11:19 +1200 Subject: [Python-Dev] mkdir -p in python In-Reply-To: References: <4C42C9CB.1090003@timgolden.me.uk> <4C43A4F6.4090800@canterbury.ac.nz> Message-ID: <4C44DB97.8030108@canterbury.ac.nz> Ray Allen wrote: > I think both os.mkdir() and > os.makedirs() should add a keyword argument to suppress the "OSError: > [Errno 17] File exists". This could be seen as violating the "no constant arguments" guideline. Maybe separate function would be better? -- Greg From greg.ewing at canterbury.ac.nz Tue Jul 20 01:53:05 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 20 Jul 2010 11:53:05 +1200 Subject: [Python-Dev] Python signal processing question In-Reply-To: References: Message-ID: <4C44E561.9060501@canterbury.ac.nz> Scott McCarty wrote: > All, I have searched everywhere (mostly the code and a little google) > and I cannot understand where the SIGKILL signal gets checked when it is > set as a handler. Possibly it's not being checked at all by Python, but is being rejected by the system call. The Darwin man page says this about signal(3) in the ERRORS section: [EINVAL] An attempt is made to ignore or supply a handler for SIGKILL or SIGSTOP. Not sure why it gets reported as a RuntimeError rather than an OSError, though. -- Greg From mcrute at gmail.com Tue Jul 20 03:22:39 2010 From: mcrute at gmail.com (Michael Crute) Date: Mon, 19 Jul 2010 21:22:39 -0400 Subject: [Python-Dev] Python signal processing question In-Reply-To: References: Message-ID: On Mon, Jul 19, 2010 at 12:11 PM, Scott McCarty wrote: > All, I have searched everywhere (mostly the code and a little google) and I > cannot understand where the SIGKILL signal gets checked when it is set as a > handler. I have scoured the Modules/signalmodule.c only to find two > instances of the RuntimeError exception, but I cannot understand how python > knows when a handler is set for SIGKILL. I understand that this changed in > 2.4 and I am not trying to change it, I just really want to understand where > this happens. I used grep to find SIGKILL and SIGTERM to see if I could > determine where the critical difference is, but I cannot figure it out. > > I have about 2 hours of searching around and I can't figure it out, I assume > it has to rely on some default behaviour in Unix, but I have no idea. I > don't see a difference between SIGKILL and SIGTERM in the python code, but > obviously there is some difference. I understand what the difference is in > Unix/Linux, I just want to see it in the python code. Since python is > checking at run time to see what signals handlers are added, I know there > must be a difference. The signals get initialized at interpreter startup within the initsigs function (Python/pythonrun.c). By default the interpreter ignores SIGPIPE, SIGXFZ and SIGXFSZ. Eventually initsignal (Modules/signalmodule.c) which will loop over an array of all handlers and set their handlers to the default handler, ignore handler or None, depending on the default state of the handler as reported by the OS. After all of that python will register its own signal handler for SIGINT which raises a KeyboardInterrupt error (you can change this). By default on Linux SIGKILL gets a None handler since it can not be handled or ignored (as dictated by the OS) and SIGTERM gets the default handler which at least under Linux is to terminate the program without doing any kind of cleanup. It is worth noting that you can handle SIGTERM but python chooses not to do this by default. Doug Hellmann wrote a post about the signal module that is a good complement to the C code for understanding how signals are handled in python. http://www.doughellmann.com/PyMOTW/signal/ HTH. -mike -- Michael E. Crute http://mike.crute.org It is a mistake to think you can solve any major problem just with potatoes. --Douglas Adams From eliben at gmail.com Tue Jul 20 05:42:04 2010 From: eliben at gmail.com (Eli Bendersky) Date: Tue, 20 Jul 2010 06:42:04 +0300 Subject: [Python-Dev] Markup of command-line options in Python's .rst documentation In-Reply-To: References: <4C41B3A4.2000501@netwok.org> Message-ID: > > > > More input on this issue: > > > > 'cmdoption' is a directive, while 'option' is inline markup. Therefore I > > wouldn't say they're completely similar, just meant for different > purposes. Both > > a directive and inline markup is useful for describing "official python > > executable options/flags". Regarding per-module options, I'm not sure > special > > markup is needed at all. So a policy has to be define regarding the > correct > > usage of these directives/markups, and probably documented in > > Doc/documenting/markup.rst > > I've done that now in r82961. > > Georg Georg, Your fix indeed clarifies :option:. However, I think that cmdoption could also be made clearer by re-iterating that it is to refer to Python's own options only, and not to be used for command line options of modules. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From breamoreboy at yahoo.co.uk Tue Jul 20 05:55:29 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 20 Jul 2010 04:55:29 +0100 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: References: <4C437BCF.5000409@v.loewis.de> Message-ID: On 18/07/2010 23:38, Alexander Belopolsky wrote: > On Sun, Jul 18, 2010 at 6:10 PM, "Martin v. L?wis" wrote: >>> Maybe going off on a tangent, but I find it frustrating because you >>> (plural) can't find a given module on the issue tracker. Say I'm looking >>> for issues relating to smtplib, all I can do is look in the title of the >>> issue. Have I missed something, or is there a need to have subsections >>> so that if Lib is selected, you can then find the appropriate module? >> >> If you think any specific feature is missing in the tracker, please post >> a report to the meta-tracker. > > This is an excellent point and for those who don't know what > "meta-tracker" is, see the link to "Report Tracker Problem" at the > bottom of left column on the main tracker page. The direct link is > http://psf.upfronthosting.co.za/roundup/meta/ . (I did not know about > the "meta-tracker" myself until a couple of months ago.) Is this the same login as for the issue tracker or is a new one needed? I also suspect that subsections for Extension Modules would be extremely useful for our C developers, thoughts anybody? Cheers. Mark Lawrence. From alexander.belopolsky at gmail.com Tue Jul 20 06:19:36 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 20 Jul 2010 00:19:36 -0400 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: References: <4C437BCF.5000409@v.loewis.de> Message-ID: On Mon, Jul 19, 2010 at 11:55 PM, Mark Lawrence wrote: .. > Is this the same login as for the issue tracker or is a new one needed? > AFAIK, it is separate. > I also suspect that subsections for Extension Modules would be extremely > useful for our C developers, thoughts anybody? Well, with the current trend of making extensions an optional fast reimplementation of library modules, the distinction between Lib and Extension Modules gets blurred. For example, http://bugs.python.org/issue1062277, "Pickle breakage with reduction of recursive structures" is a bug that affects both python and C implementation of pickle module. Should this be Lib or Extension issue or both? (The issue is actually misclassified as "Interpreter Core".) I think it is both. Here is a less clear example. Suppose someone reports a bug is time.strptime. The time module is a C extension, but strptime is implemented in a "hidden" python module, _strptime. How would you classify this issue? If your head does not spin yet, consider http://bugs.python.org/issue1677872, "Efficient reverse line iterator". When the patch was submitted, io module was implemented in python, but since then it was reimplemented in C. If we allow tagging issues with the module name, I would rather not make distinction between C and python modules. just make it one flat list similar to the one in maintainers.txt. From eliben at gmail.com Tue Jul 20 06:22:14 2010 From: eliben at gmail.com (Eli Bendersky) Date: Tue, 20 Jul 2010 07:22:14 +0300 Subject: [Python-Dev] Markup of command-line options in Python's .rst documentation In-Reply-To: References: <4C41B3A4.2000501@netwok.org> Message-ID: > > 'cmdoption' is a directive, while 'option' is inline markup. Therefore I > > wouldn't say they're completely similar, just meant for different > purposes. Both > > a directive and inline markup is useful for describing "official python > > executable options/flags". Regarding per-module options, I'm not sure > special > > markup is needed at all. So a policy has to be define regarding the > correct > > usage of these directives/markups, and probably documented in > > Doc/documenting/markup.rst > > I've done that now in r82961. > As per your recommendation in the updated documentation guide, I have fixed the ReST docs of stdlib modules that used :option:, to use code markup (``) instead. I've opened issue 9312 for this and submitted a patch. Eli -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Tue Jul 20 07:09:27 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 20 Jul 2010 01:09:27 -0400 Subject: [Python-Dev] Python signal processing question In-Reply-To: References: <20100719162729.GF21891@phd.pp.ru> Message-ID: On 7/19/2010 6:08 PM, average wrote: >> We'are sorry but we cannot help you. This mailing list is to work on >> developing Python (fixing bugs and adding new features to Python itself); if >> you're having problems using Python, please find another forum. Probably >> python-list (comp.lang.python) news group/mailing list is the best place. >> See http://www.python.org/community/ for other lists/news groups/fora. >> Thank you for understanding. This is a polite boilerplate response for off-topic posts. I suppose 'using' should be expanded to 'using or understanding'. >> Regarding SIGKILL - read "man kill" carefully. Or better yet, a Unix >> programmer guide. > > Wha? How could this not be the right place? He's not asking about > USING python, but asking: WHERE in the PYTHON CODE BASE does the > signal get checked? Right, he asked about how Python operates, which is a constant type of question on python-list and which has nothing to do with developing the next version(s) of Python. > A-bit-miffed-at-the-cold-shoulderly yours, Py-dev was started when python-list because unusable for development. If this list were to become, in turn, overwhelmed with general questions, as it would be it non-development questions were not directed elsewhere, then the devs would have to start a new python-really-development-only list, and perhaps not make it so public. If someone tries an advanced question on python-list first and gets no answer, then there might be an excuse to turn to py-dev to grab the attention of those developers who do not ever read python-list. But in this case, the question was readily answered there, in part by devs who do frequent both. The OP acknowledged there that he acted on prejudice and should have posted there in the first place. -- Terry Jan Reedy From fijall at gmail.com Tue Jul 20 07:35:31 2010 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 20 Jul 2010 07:35:31 +0200 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: References: <7B78A282-EADA-48BD-93DD-26719168E7B9@twistedmatrix.com> Message-ID: On Sun, Jul 18, 2010 at 10:32 PM, Paul Moore wrote: > On 18 July 2010 20:57, Glyph Lefkowitz wrote: >> >> On Jul 18, 2010, at 1:46 PM, Alexander Belopolsky wrote: >> >> We already have "posponed" and "remind" resolutions, but these are >> exclusive of "accepted". ??I think there should be a clear way to mark >> the issue "accepted and would be applied if X.Y was out already." >> Chances are one of the resolution labels already has such meaning, but >> in this case it should be more prominently documented as such. >> >> This is what branches are for. >> When the X.Y release cycle starts, there should be a branch for X.Y. ?Any >> "would be applied" patches can simply be applied to trunk without >> interrupting anything; the X.Y release branch can be merged back into trunk >> as necessary. > > Agreed. If that isn't already the recommended workflow under > Mercurial, I'd suggest making it so. (I can imagine that under > Subversion, where branching and merging is more awkward, it might have > been deemed not worth doing). > > Paul. Contrary to a widespread popular belief subversion supports branching and I don't think anyone suggested merging release branches back (even though you can do it in subversion as well). Cheers, fijal From stephen at xemacs.org Tue Jul 20 07:37:17 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 20 Jul 2010 14:37:17 +0900 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: References: <4C437BCF.5000409@v.loewis.de> Message-ID: <877hkqzpbm.fsf@uwakimon.sk.tsukuba.ac.jp> Mark Lawrence writes: > Is this the same login as for the issue tracker or is a new one needed? It's different. Both trackers are supposed to support OpenID logins, I believe. (However, there are somewhat frequent reports of difficulties with it; I don't know if the system is fully debugged. I think most people are OK, but making a new login is the sure path.) > I also suspect that subsections for Extension Modules would be extremely > useful for our C developers, thoughts anybody? Do you mean an entry per module? The XEmacs Tracker (http://tracker.xemacs.org/), which also is based on Roundup, has an extensive list of add-on packages (somewhat like the stdlib). We're currently stuck in the doldrums, and it is sadly mostly unused, so I can't really comment on the utility. I find that there are some UI issues with such long lists in Roundup. I don't want to discourage adding such information to the tracker; I think it's a good idea. But it will need attention to UI to be most useful. From g.brandl at gmx.net Tue Jul 20 12:39:05 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 20 Jul 2010 11:39:05 +0100 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: <877hkqzpbm.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4C437BCF.5000409@v.loewis.de> <877hkqzpbm.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: Am 20.07.2010 06:37, schrieb Stephen J. Turnbull: > Mark Lawrence writes: > > > Is this the same login as for the issue tracker or is a new one needed? > > It's different. Both trackers are supposed to support OpenID logins, > I believe. (However, there are somewhat frequent reports of > difficulties with it; I don't know if the system is fully debugged. I > think most people are OK, but making a new login is the sure path.) The meta tracker doesn't support OpenID (it's a very plain roundup without customization). The OpenID support of the Python tracker is quite mature now, or so I've heard. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ben+python at benfinney.id.au Tue Jul 20 12:23:45 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 20 Jul 2010 20:23:45 +1000 Subject: [Python-Dev] OpenID login for Roundup bug trackers (was: What to do with languishing patches?) References: <4C437BCF.5000409@v.loewis.de> Message-ID: <87r5iy8n9q.fsf_-_@benfinney.id.au> "Stephen J. Turnbull" writes: > Mark Lawrence writes: > > > On 18/07/2010 23:38, Alexander Belopolsky wrote: > > > The direct link [to the ?meta-tracker?] is > > > http://psf.upfronthosting.co.za/roundup/meta/ > > > > Is this the same login as for the issue tracker or is a new one > > needed? > > It's different. Both trackers are supposed to support OpenID logins, I > believe. The tracker at doesn't mention OpenID on its login form, so I must assume it doesn't currently support it. The Python bug tracker does support arbitrary OpenID for authentication. It didn't in the recent past, so I'm very glad to see that it does now. I can finally get an account there without the hassle of managing a site-specific auth set. > (However, there are somewhat frequent reports of difficulties with it; > I don't know if the system is fully debugged. I think most people are > OK, but making a new login is the sure path.) It worked for me so far on , logging in with my OpenID and then going through the registration process. Thanks for drawing my attention to that; if the people who made OpenID auth happen are observing this, then thank you all very much! Any hope of feeding those changes back upstream so it's available to all Roundup users at some point? -- \ ?Pinky, are you pondering what I'm pondering?? ?Well, I think | `\ so, Brain, but first you'd have to take that whole bridge | _o__) apart, wouldn't you?? ?_Pinky and The Brain_ | Ben Finney From phd at phd.pp.ru Tue Jul 20 12:59:28 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Tue, 20 Jul 2010 14:59:28 +0400 Subject: [Python-Dev] Python-dev signal-to-noise processing question In-Reply-To: References: <20100719162729.GF21891@phd.pp.ru> Message-ID: <20100720105928.GA4289@phd.pp.ru> On Mon, Jul 19, 2010 at 04:08:00PM -0600, average wrote: > Wha? How could this not be the right place? He's not asking about > USING python, but asking: WHERE in the PYTHON CODE BASE does the > signal get checked? > > A-bit-miffed-at-the-cold-shoulderly yours, > > Marcos (wink wink) I know, the task of sending answers like I've sent is quite unappreciated. I know, the meaning of my answer is rude because, in short, it's simply "Please, go away", and however I stress the "please" part it's still "go away". If I were a help seeker it'd be quite a hard blow for me to receive such an answer. Yes, I know. Still, two other alternatives are even worse. The first alternative is to not answer "using" questions at all; quite rude. The second is answer all questions and make the developers quit the list and found a new quiet one. I don't see any other alternative, do you? Of those three - which one do you prefer? The original question was about using python, that's my understanding. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From martin at v.loewis.de Tue Jul 20 14:25:25 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Tue, 20 Jul 2010 13:25:25 +0100 Subject: [Python-Dev] OpenID login for Roundup bug trackers In-Reply-To: <87r5iy8n9q.fsf_-_@benfinney.id.au> References: <4C437BCF.5000409@v.loewis.de> <87r5iy8n9q.fsf_-_@benfinney.id.au> Message-ID: <4C4595B5.8090409@v.loewis.de> > Thanks for drawing my attention to that; if the people who made OpenID > auth happen are observing this, then thank you all very much! You're welcome! > Any hope of feeding those changes back upstream so it's available to all > Roundup users at some point? Hope dies last. It's main foundation is an openid library that I wrote (http://pypi.python.org/pypi/openid2rp). As for the roundup integration, I likely won't have much time to contribute it back. Regards, Martin From ysj.ray+python at gmail.com Tue Jul 20 10:49:44 2010 From: ysj.ray+python at gmail.com (=?UTF-8?B?5bKz5biF5p2w?=) Date: Tue, 20 Jul 2010 16:49:44 +0800 Subject: [Python-Dev] mkdir -p in python In-Reply-To: <4C44DB97.8030108@canterbury.ac.nz> References: <4C42C9CB.1090003@timgolden.me.uk> <4C43A4F6.4090800@canterbury.ac.nz> <4C44DB97.8030108@canterbury.ac.nz> Message-ID: Sorry, I don't know what is the "no constant arguments" guideline refers to. Could you give me some more explanation? By the way, I feel adding separate functions is not quiet worthy for such a function. On Tue, Jul 20, 2010 at 7:11 AM, Greg Ewing wrote: > Ray Allen wrote: > > I think both os.mkdir() and os.makedirs() should add a keyword argument to >> suppress the "OSError: [Errno 17] File exists". >> > > This could be seen as violating the "no constant arguments" > guideline. Maybe separate function would be better? > > > -- > Greg > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ysj.ray%2Bpython-dev%40gmail.com > -- Ray Allen Best wishes! -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Jul 20 15:09:59 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 20 Jul 2010 23:09:59 +1000 Subject: [Python-Dev] mkdir -p in python In-Reply-To: References: <4C44DB97.8030108@canterbury.ac.nz> Message-ID: <201007202309.59704.steve@pearwood.info> On Tue, 20 Jul 2010 06:49:44 pm ??? wrote: > Sorry, I don't know what is the "no constant arguments" guideline > refers to. Could you give me some more explanation? It refers to the guideline that you shouldn't have a single function with two (or more) different behaviour and an argument that does nothing but select between them. Here is a toy example: def change_case(s, make_upper=True): if make_upper: return s.upper() else: return s.lower() > By the way, I feel adding separate functions is not quiet worthy for > such a function. I tend to agree. Perhaps all we need is a recipe in the docs: try: os.makedirs(path) except OSError, e: if e.errno != 17: raise I think that's all it takes to get the behaviour wanted without any false negatives, and four lines is short enough to use in-place, or if you prefer it's easy enough to wrap it in a function in your own module. Not everything needs to be a built-in. I'm -0 on adding an argument to os.makedirs, +0 on adding a variant function to os, and +0.5 on adding the variant to the shutil module. -- Steven D'Aprano From rdmurray at bitdance.com Tue Jul 20 15:32:26 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 20 Jul 2010 09:32:26 -0400 Subject: [Python-Dev] tracker contribution In-Reply-To: References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> <4C41991A.9040103@voidspace.org.uk> <20100718163443.41ce2538@pitrou.net> Message-ID: <20100720133226.A2AC5200CC0@kimball.webabinitio.net> On Mon, 19 Jul 2010 00:06:32 +0100, Mark Lawrence wrote: > You all might have gathered that I'm very dispirited by the negative > attitudes that I get from a relatively small minority of Python people. > I might as well quit because it doesn't do my mental health a great > deal of good. That minority will of course be very happy to see me go > because they'll be able to sit back on their loathsome, spotty behinds > picking blackheads and resort to the usual "we're far too busy to do > anything" routine. You might understand that I don't completely agree > with this. Mark, please understand that (as far as I can tell) the "small minority of people" of whom you are speaking here, who have offered advice on how to make your triage work more helpful to the Python community, are some of the most *active* Python committers and community participants. I think the best thing for your mental health would be to see if you can just ignore things that seem negative, and see if there's any useful advice in what is said. And if you don't find the opinions useful, then you don't. The essence of community is cooperation, and cooperation can't happen without communication; and yes, some of it is going to sound negative (and some of it will even *be* negative...that's human nature, as your message demonstrates). Which I think you have done, since I've seen you put into practice some of the suggestions that were made. I regret that I haven't been able to follow your prolific work closely (in fact, I have had to stop following the bugs list, I now just watch the new bugs list...not just due to the volume, but because of my other commitments, but the volume certainly contributed, unfortunately[*]). I especially regret that because I was the one who gave you developer privs on the tracker, and so I have something of a responsibility to so. I'm relying on the positive feedback that you are getting here to assuage my guilt for not paying closer attention to your work, and also on the fact that I have seen the quality of your work improve over time from the bugs I have reviewed that you've commented on. -- R. David Murray www.bitdance.com [*] I think this may be a source of some of the discomfort you have sensed. I had been used to being able to keep up with the bugs list, but even before you started your work I was already thinking that that wasn't going to work for much longer...as Python becomes more popular, the number of bug reports increases, and as we add more committers and gather more people who are actively helping out, the message traffic increases. This is *good*: it means we are getting more done (well, at least if we can continue to add committers it does), but it also means that it is no longer possible for a part time volunteer to keep an overview of the bug/patch activity. The danger, of course, is that more things will slip through the cracks....but this is not, as you have observed, a new problem. From ncoghlan at gmail.com Tue Jul 20 15:43:53 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 20 Jul 2010 23:43:53 +1000 Subject: [Python-Dev] mkdir -p in python In-Reply-To: <201007202309.59704.steve@pearwood.info> References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> Message-ID: On Tue, Jul 20, 2010 at 11:09 PM, Steven D'Aprano wrote: > I'm -0 on adding an argument to os.makedirs, +0 on adding a variant > function to os, and +0.5 on adding the variant to the shutil module. shutil seems like the place for it to me. The subtlety of getting the error suppression correct makes it worth implementing this once and for all I think. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From anders.sandvig at gmail.com Tue Jul 20 15:57:30 2010 From: anders.sandvig at gmail.com (Anders Sandvig) Date: Tue, 20 Jul 2010 15:57:30 +0200 Subject: [Python-Dev] Curious datetime method In-Reply-To: References: Message-ID: > I wonder why would anyone want to use datetime.today() instead of > datetime.now()? Because this method is also present in datetime.date. Thus, you can reference stuff like d.today().day without caring whether d is a date or a datetime object. Anders From ijmorlan at uwaterloo.ca Tue Jul 20 15:51:42 2010 From: ijmorlan at uwaterloo.ca (Isaac Morland) Date: Tue, 20 Jul 2010 09:51:42 -0400 (EDT) Subject: [Python-Dev] mkdir -p in python In-Reply-To: <201007202309.59704.steve@pearwood.info> References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> Message-ID: On Tue, 20 Jul 2010, Steven D'Aprano wrote: > I tend to agree. Perhaps all we need is a recipe in the docs: > > try: > os.makedirs(path) > except OSError, e: > if e.errno != 17: > raise What if the path or a parent of it already exists as a file? If one has requested -p I believe one typically wants this to be an error. And I assume that one would not use the literal 17 in production code. Isaac Morland CSCF Web Guru DC 2554C, x36650 WWW Software Specialist From rdmurray at bitdance.com Tue Jul 20 16:20:05 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 20 Jul 2010 10:20:05 -0400 Subject: [Python-Dev] mkdir -p in python In-Reply-To: <201007202309.59704.steve@pearwood.info> References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> Message-ID: <20100720142005.70AFE2007B4@kimball.webabinitio.net> I'd go with putting it in shutil. We could also add a function there that wraps up the recipe in issue 9311 to work around the quirks of os.access on FreeBSD (and possibly other platforms). -- R. David Murray www.bitdance.com From steve at holdenweb.com Tue Jul 20 16:42:41 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 20 Jul 2010 15:42:41 +0100 Subject: [Python-Dev] Python-dev signal-to-noise processing question In-Reply-To: <20100720105928.GA4289@phd.pp.ru> References: <20100719162729.GF21891@phd.pp.ru> <20100720105928.GA4289@phd.pp.ru> Message-ID: On 7/20/2010 11:59 AM, Oleg Broytman wrote: > On Mon, Jul 19, 2010 at 04:08:00PM -0600, average wrote: >> Wha? How could this not be the right place? He's not asking about >> USING python, but asking: WHERE in the PYTHON CODE BASE does the >> signal get checked? >> >> A-bit-miffed-at-the-cold-shoulderly yours, >> >> Marcos (wink wink) > > I know, the task of sending answers like I've sent is quite > unappreciated. I know, the meaning of my answer is rude because, in short, > it's simply "Please, go away", and however I stress the "please" part it's > still "go away". If I were a help seeker it'd be quite a hard blow for me > to receive such an answer. Yes, I know. No, the reply is fine as far as it goes, and I am sure the poster did get a reply from c.l.py, but his question revealed a thirst for knowledge not usually evidenced in non-dev inquiries. > Still, two other alternatives are even worse. The first alternative is > to not answer "using" questions at all; quite rude. The second is answer > all questions and make the developers quit the list and found a new quiet > one. > I don't see any other alternative, do you? Of those three - which one > do you prefer? > The original question was about using python, that's my understanding. Could we perhaps include a link to the material for Python devs, just so people realise that although they are being sent away for good reason, if they are interested in joining the dev community there is material to help and guide them. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From alexander.belopolsky at gmail.com Tue Jul 20 17:15:12 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 20 Jul 2010 11:15:12 -0400 Subject: [Python-Dev] Python-dev signal-to-noise processing question In-Reply-To: <20100720105928.GA4289@phd.pp.ru> References: <20100719162729.GF21891@phd.pp.ru> <20100720105928.GA4289@phd.pp.ru> Message-ID: On Tue, Jul 20, 2010 at 6:59 AM, Oleg Broytman wrote: .. > ? I know, the task of sending answers like I've sent is quite > unappreciated. I know, the meaning of my answer is rude because, in short, > it's simply "Please, go away", and however I stress the "please" part it's > still "go away". If I were a help seeker it'd be quite a hard blow for me > to receive such an answer. Yes, I know. > ? Still, two other alternatives are even worse. The first alternative is > to not answer "using" questions at all; quite rude. The second is answer > all questions and make the developers quit the list and found a new quiet > one. > ? I don't see any other alternative, do you? Of those three - which one > do you prefer? People dislike canned responses no matter how friendly and polite they sound. I believe python-dev tradition is to redirect the poster to python-list but still include at least a hint to what the answer to his or her question is. I believe this works fairly well. From fdrake at acm.org Tue Jul 20 17:43:27 2010 From: fdrake at acm.org (Fred Drake) Date: Tue, 20 Jul 2010 11:43:27 -0400 Subject: [Python-Dev] mkdir -p in python In-Reply-To: <201007202309.59704.steve@pearwood.info> References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> Message-ID: On Tue, Jul 20, 2010 at 9:09 AM, Steven D'Aprano wrote: > It refers to the guideline that you shouldn't have a single function > with two (or more) different behaviour and an argument that does > nothing but select between them. In particular, when that argument is almost never given a variable value, but is specified using a constant at the call site. ? -Fred -- Fred L. Drake, Jr.? ? "A storm broke loose in my mind."? --Albert Einstein From merwok at netwok.org Tue Jul 20 17:45:42 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Tue, 20 Jul 2010 17:45:42 +0200 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: References: Message-ID: <4C45C4A6.5020208@netwok.org> Le 18/07/2010 23:17, Alexander Belopolsky a ?crit : > On Sun, Jul 18, 2010 at 5:02 PM, Mark Lawrence wrote: >> Maybe going off on a tangent, but I find it frustrating because you (plural) >> can't find a given module on the issue tracker. Say I'm looking for issues >> relating to smtplib, all I can do is look in the title of the issue. Have I >> missed something, or is there a need to have subsections so that if Lib is >> selected, you can then find the appropriate module? > > I don't think you missed anything and I would also find this feature > useful. For example, I recently took interest in datetime and pickle > modules, but other than full text search, there was no way to find all > issues related to either of these two modules. The "Components" list > is somewhat arbitrary with narrow components such as ctypes and broad > components such a Lib. When I suggested such a modules list on , R. David Murray replied ?This has been suggested and rejected a number of times on python-dev? but I have been unable to find those discussions. Regards From alexander.belopolsky at gmail.com Tue Jul 20 17:56:47 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 20 Jul 2010 11:56:47 -0400 Subject: [Python-Dev] Curious datetime method In-Reply-To: References: Message-ID: On Tue, Jul 20, 2010 at 9:57 AM, Anders Sandvig wrote: >> I wonder why would anyone want to use datetime.today() instead of >> datetime.now()? > > Because this method is also present in datetime.date. Thus, you can > reference stuff like ?d.today().day without caring whether d is a date > or a datetime object. For this particular use it is not necessary for today() method to have covariant returns. In other words, if datetime.today() returned a date object in the same way as date.today() does, you could still write ?d.today().day. Moreover, since today() is a class method, the result does not depend on d, so you can simply do date.today().day. A more interesting use case would be a function that tells whether d is in the future: def isfuture(d): return d > d.today() Here covariant return is necessary because you cannot compare datetime to date and even if you could, this is probably not what you want in this case. This implementation of isfuture() has a problem: it does not work with aware datetime objects. To make that work, we would need today(tz) method that can account for a timezone, or better have now(tz=None) available as a date class method: def isfuture(d): return d > d.now(d.tzinfo) Furthermore, this use case does not explain the need to have datetime.today() respect overriding of fromtimestamp() in datetime subclasses or having to reproduce datetime.fromtimestamp(time.time()) with all its undefined behavior that comes with floating point timestamps. In the ideal world, I would like today() to accept tz and always return date object while adding "covariant" now([tz]) method to date and possibly even time class. (The later is the subject of http://bugs.python.org/issue8902). In the real world where we have to take backward compatibility into account, I would like to make today() and now() to be the same: both taking optional tz argument, both available as either date or datetime methods and both covariant. the justification for having two methods doing exactly the same will be just readability: date.today() and datetime.now() are more readable than date.now() and datetime.today(). From rrr at ronadam.com Tue Jul 20 18:47:11 2010 From: rrr at ronadam.com (Ron Adam) Date: Tue, 20 Jul 2010 11:47:11 -0500 Subject: [Python-Dev] mkdir -p in python In-Reply-To: References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> Message-ID: On 07/20/2010 10:43 AM, Fred Drake wrote: > On Tue, Jul 20, 2010 at 9:09 AM, Steven D'Aprano wrote: >> It refers to the guideline that you shouldn't have a single function >> with two (or more) different behaviour and an argument that does >> nothing but select between them. > > In particular, when that argument is almost never given a variable > value, but is specified using a constant at the call site. I was thinking it should have been two functions, but I realized there is more subtleties involved than simply just reusing a directory that already exists. One possibility might be... mkdir(path [, allow=None, mode=0777]) Where None can be replaced with one or more of the following. 'exists' dir can already exist (with same permissions as mode) 'case' dir case can be different. (Windows) 'files' dir can have files in it. ... or a string containing the initials. It doesn't fall under the single constant rule if done this way. Ron From fuzzyman at voidspace.org.uk Tue Jul 20 18:49:58 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 20 Jul 2010 17:49:58 +0100 Subject: [Python-Dev] mkdir -p in python In-Reply-To: References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> Message-ID: <4C45D3B6.1020701@voidspace.org.uk> On 20/07/2010 14:43, Nick Coghlan wrote: > On Tue, Jul 20, 2010 at 11:09 PM, Steven D'Aprano wrote: > >> I'm -0 on adding an argument to os.makedirs, +0 on adding a variant >> function to os, and +0.5 on adding the variant to the shutil module. >> > shutil seems like the place for it to me. The subtlety of getting the > error suppression correct makes it worth implementing this once and > for all I think. > Really - so we have two almost identical functions with slightly different behaviour in different standard library modules? Aren't the Python APIs for working with files and paths confused enough already. That seems crazy to me and a switch on os.makedirs much more sensible. Michael > Cheers, > Nick. > > > -- http://www.ironpythoninaction.com/ From rrr at ronadam.com Tue Jul 20 18:58:52 2010 From: rrr at ronadam.com (Ron Adam) Date: Tue, 20 Jul 2010 11:58:52 -0500 Subject: [Python-Dev] mkdir -p in python In-Reply-To: References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> Message-ID: On 07/20/2010 11:47 AM, Ron Adam wrote: > > > On 07/20/2010 10:43 AM, Fred Drake wrote: >> On Tue, Jul 20, 2010 at 9:09 AM, Steven D'Aprano >> wrote: >>> It refers to the guideline that you shouldn't have a single function >>> with two (or more) different behaviour and an argument that does >>> nothing but select between them. >> >> In particular, when that argument is almost never given a variable >> value, but is specified using a constant at the call site. > > I was thinking it should have been two functions, but I realized there > is more subtleties involved than simply just reusing a directory that > already exists. > > > One possibility might be... > > mkdir(path [, allow=None, mode=0777]) > > Where None can be replaced with one or more of the following. > > 'exists' dir can already exist (with same permissions as mode) > 'case' dir case can be different. (Windows) > 'files' dir can have files in it. Add 'path', Complete path as -p option does. > ... or a string containing the initials. > > > It doesn't fall under the single constant rule if done this way. > > > Ron From fdrake at acm.org Tue Jul 20 19:00:18 2010 From: fdrake at acm.org (Fred Drake) Date: Tue, 20 Jul 2010 13:00:18 -0400 Subject: [Python-Dev] mkdir -p in python In-Reply-To: References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> Message-ID: On Tue, Jul 20, 2010 at 12:47 PM, Ron Adam wrote: > It doesn't fall under the single constant rule if done this way. If the value for 'allow' were almost always given as a constant, this would be an argument for three functions instead of one. The guideline has little to do with the type of the value, but the number of possible values ("small") and whether they're normally given as constants in the code. If there's more than one, and combinations then to vary, then keeping them as args makes sense. Also, if we don't know what we want the functionality to be, as you suggest, then worry about that is premature. :-) Let's decide on the required functionality first. From eliben at gmail.com Tue Jul 20 19:16:34 2010 From: eliben at gmail.com (Eli Bendersky) Date: Tue, 20 Jul 2010 20:16:34 +0300 Subject: [Python-Dev] :program: markup in .rst documentation Message-ID: While fixing the :option: markup in the library docs (http://bugs.python.org/issue9312), another question came up: In a few places, documenting a program execution with options was made with: :program:`python regrtest.py` :option:`test_spam.py` Since my patch had the :option: markup removed, this became: :program:`python regrtest.py test_spam.py` However, according to ?ric Araujo's comment, due to the intention of the :program: markup this should actually just say: ``python regrtest.py test_spam.py`` Which way to choose? I will update my patch to reflect this. Eli From alexander.belopolsky at gmail.com Tue Jul 20 19:24:02 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 20 Jul 2010 13:24:02 -0400 Subject: [Python-Dev] :program: markup in .rst documentation In-Reply-To: References: Message-ID: On Tue, Jul 20, 2010 at 1:16 PM, Eli Bendersky wrote: .. > > ``python regrtest.py test_spam.py`` > > Which way to choose? I will update my patch to reflect this. Sorry to add the third way to the mix, but shouldn't the recommended way to run a module as a script be python -m modname? As in $ python -m test.regrtest test_spam From fdrake at acm.org Tue Jul 20 19:32:47 2010 From: fdrake at acm.org (Fred Drake) Date: Tue, 20 Jul 2010 13:32:47 -0400 Subject: [Python-Dev] :program: markup in .rst documentation In-Reply-To: References: Message-ID: Switching to "python -m" is generally good where it applies (as in this case). The original intent for :option: and :program: were in their use as references rather than in sample command lines. Georg should be the final arbiter, but I'd be in favor of ``...`` for command lines. From merwok at netwok.org Tue Jul 20 19:34:02 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Tue, 20 Jul 2010 19:34:02 +0200 Subject: [Python-Dev] :program: markup in .rst documentation In-Reply-To: References: Message-ID: <4C45DE0A.9080806@netwok.org> > Sorry to add the third way to the mix, but shouldn't the recommended > way to run a module as a script be python -m modname? As in > > $ python -m test.regrtest test_spam This is true but orthogonal to our problem, which is that :program:`python -m thing` is wrong if I understood the doc rightly and should be replaced by ``python -m thing``. Regards From eliben at gmail.com Tue Jul 20 19:51:05 2010 From: eliben at gmail.com (Eli Bendersky) Date: Tue, 20 Jul 2010 20:51:05 +0300 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: I've opened issue 9315 (http://bugs.python.org/issue9315) to address the lack of unit tests for the trace.py module. I hope to get to it in a few days and add some tests. As Terry wrote in the beginning of this thread, Lib/test/test_trace.py currently tests the sys.settrace module, so the tests of trace.py should find a new home. Does Lib/test/test_trace_module.py make sense or is something else preferable? Eli From tjreedy at udel.edu Tue Jul 20 19:51:07 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 20 Jul 2010 13:51:07 -0400 Subject: [Python-Dev] Python-dev signal-to-noise processing question In-Reply-To: <20100720105928.GA4289@phd.pp.ru> References: <20100719162729.GF21891@phd.pp.ru> <20100720105928.GA4289@phd.pp.ru> Message-ID: On 7/20/2010 6:59 AM, Oleg Broytman wrote: > I know, the task of sending answers like I've sent is quite > unappreciated. *I* appreciate it. I mostly do not respond to such because I expect you or Aahz will. > I know, the meaning of my answer is rude because, in short, > it's simply "Please, go away", and however I stress the "please" part it's > still "go away". If I were a help seeker it'd be quite a hard blow for me > to receive such an answer. Yes, I know. 1. I suggested one improvement to the canned response in my previous post: expand 'using' to 'using or understanding'. 2. Here is another: mention that Python developers who are willing to answer non-development questions already participate on python-list/gmane.comp.python.general to answer such questions there. 3. That brings up: also mention gmane.comp.python.general, for those like me who prefer the newsgroup interface. 4. Alexander Belopolsky wrote: "People dislike canned responses no matter how friendly and polite they sound. I believe python-dev tradition is to redirect the poster to python-list but still include at least a hint to what the answer to his or her question is." I believe you *did* do that, but it was easily missed. So: end the boilerplate with ------------------------------------------------------------- and put hints here so they stand out. > Still, two other alternatives are even worse. The first alternative is > to not answer "using" questions at all; quite rude. The second is answer > all questions and make the developers quit the list and found a new quiet > one. I brought up that same possibility. > I don't see any other alternative, do you? Of those three - which one > do you prefer? > The original question was about using python, that's my understanding. I agree that it appeared to be, but it was phrased in terms of understanding its operation, hence the claim that it was not, and hence suggestion 1) above. -- Terry Jan Reedy From tjreedy at udel.edu Tue Jul 20 20:08:52 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 20 Jul 2010 14:08:52 -0400 Subject: [Python-Dev] mkdir -p in python In-Reply-To: <4C45D3B6.1020701@voidspace.org.uk> References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> <4C45D3B6.1020701@voidspace.org.uk> Message-ID: On 7/20/2010 12:49 PM, Michael Foord wrote: > On 20/07/2010 14:43, Nick Coghlan wrote: >> On Tue, Jul 20, 2010 at 11:09 PM, Steven D'Aprano >> wrote: >>> I'm -0 on adding an argument to os.makedirs, +0 on adding a variant >>> function to os, and +0.5 on adding the variant to the shutil module. >> shutil seems like the place for it to me. The subtlety of getting the >> error suppression correct makes it worth implementing this once and >> for all I think. > > Really - so we have two almost identical functions with slightly > different behaviour in different standard library modules? Aren't the > Python APIs for working with files and paths confused enough already. > That seems crazy to me and a switch on os.makedirs much more sensible. Guido has twice approved changing os.makedirs. First 2 1/2 years ago in http://bugs.python.org/msg58924 from http://bugs.python.org/issue1675 and recently in this thread http://mail.python.org/pipermail/python-dev/2010-July/102027.html which lead to http://bugs.python.org/issue9299 Both issues have a patch and I reviewed their major similarities and minor differences in http://bugs.python.org/msg110770 -- Terry Jan Reedy From reid.kleckner at gmail.com Tue Jul 20 20:12:25 2010 From: reid.kleckner at gmail.com (Reid Kleckner) Date: Tue, 20 Jul 2010 11:12:25 -0700 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: On Tue, Jul 20, 2010 at 10:51 AM, Eli Bendersky wrote: > As Terry wrote in the beginning of this thread, Lib/test/test_trace.py > currently tests the sys.settrace module, so the tests of trace.py > should find a new home. Does Lib/test/test_trace_module.py make sense > or is something else preferable? IMO you should just rename test_trace.py to test_settrace.py, and put the trace.py tests in test_trace.py. Reid From janssen at parc.com Tue Jul 20 20:15:49 2010 From: janssen at parc.com (Bill Janssen) Date: Tue, 20 Jul 2010 11:15:49 PDT Subject: [Python-Dev] mkdir -p in python In-Reply-To: <4C45D3B6.1020701@voidspace.org.uk> References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> <4C45D3B6.1020701@voidspace.org.uk> Message-ID: <41027.1279649749@parc.com> Michael Foord wrote: > On 20/07/2010 14:43, Nick Coghlan wrote: > > On Tue, Jul 20, 2010 at 11:09 PM, Steven D'Aprano wrote: > > > >> I'm -0 on adding an argument to os.makedirs, +0 on adding a variant > >> function to os, and +0.5 on adding the variant to the shutil module. > >> > > shutil seems like the place for it to me. The subtlety of getting the > > error suppression correct makes it worth implementing this once and > > for all I think. > > > > Really - so we have two almost identical functions with slightly > different behaviour in different standard library modules? Aren't the > Python APIs for working with files and paths confused enough > already. That seems crazy to me and a switch on os.makedirs much more > sensible. Yep, me too. Bill > > Michael > > Cheers, > > Nick. > > > > > > > > > -- > http://www.ironpythoninaction.com/ > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com From rdmurray at bitdance.com Tue Jul 20 20:16:40 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 20 Jul 2010 14:16:40 -0400 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: <4C45C4A6.5020208@netwok.org> References: <4C45C4A6.5020208@netwok.org> Message-ID: <20100720181640.D89CA2007A2@kimball.webabinitio.net> On Tue, 20 Jul 2010 17:45:42 +0200, wrote: On 18/07/2010 23:17, Alexander Belopolsky wrote: > On Sun, Jul 18, 2010 at 5:02 PM, Mark Lawrence wrote: >> Maybe going off on a tangent, but I find it frustrating because you (plural) >> can't find a given module on the issue tracker. Say I'm looking for issues >> relating to smtplib, all I can do is look in the title of the issue. Have I >> missed something, or is there a need to have subsections so that if Lib is >> selected, you can then find the appropriate module? > > I don't think you missed anything and I would also find this feature > useful. For example, I recently took interest in datetime and pickle > modules, but other than full text search, there was no way to find all > issues related to either of these two modules. The "Components" list > is somewhat arbitrary with narrow components such as ctypes and broad > components such a Lib. When I suggested such a modules list on , R. David Murray replied ???This has been suggested and rejected a number of times on python-dev??? but I have been unable to find those discussions. During the most recent discussion I can remember, I thought I remembered Stephen Trumble saying that they'd tried that in xemacs and it really hadn't worked very well. Since he now says he thinks it's a good idea (or more likely I misremembered what he said the first time), it sounds like we should consider it again. -- R. David Murray www.bitdance.com From alexander.belopolsky at gmail.com Tue Jul 20 20:18:46 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 20 Jul 2010 14:18:46 -0400 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: On Tue, Jul 20, 2010 at 2:12 PM, Reid Kleckner wrote: .. > IMO you should just rename test_trace.py to test_settrace.py, and put > the trace.py tests in test_trace.py. +1 From rrr at ronadam.com Tue Jul 20 20:38:27 2010 From: rrr at ronadam.com (Ron Adam) Date: Tue, 20 Jul 2010 13:38:27 -0500 Subject: [Python-Dev] mkdir -p in python In-Reply-To: References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> Message-ID: On 07/20/2010 12:00 PM, Fred Drake wrote: > On Tue, Jul 20, 2010 at 12:47 PM, Ron Adam wrote: >> It doesn't fall under the single constant rule if done this way. > > If the value for 'allow' were almost always given as a constant, this > would be an argument for three functions instead of one. > > The guideline has little to do with the type of the value, but the > number of possible values ("small") and whether they're normally given > as constants in the code. > > If there's more than one, and combinations then to vary, then keeping > them as args makes sense. > > Also, if we don't know what we want the functionality to be, as you > suggest, then worry about that is premature. :-) Let's decide on the > required functionality first. That makes sense. :-) Another things that comes to mind, is it may make sense to choose either strict, and have args to *allow* different cases, or to choose lenient, and have args to *restrict* different cases. That keeps it somewhat less confusing, and doesn't require memorization to remember what the mixed mode default might be. (Unless we use a single combination 99% of the time, then that probably *should* be the default.) Ron From alexander.belopolsky at gmail.com Tue Jul 20 20:53:53 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 20 Jul 2010 14:53:53 -0400 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: <20100720181640.D89CA2007A2@kimball.webabinitio.net> References: <4C45C4A6.5020208@netwok.org> <20100720181640.D89CA2007A2@kimball.webabinitio.net> Message-ID: On Tue, Jul 20, 2010 at 2:16 PM, R. David Murray wrote: .. > During the most recent discussion I can remember, I thought I remembered > Stephen Trumble saying that they'd tried that in xemacs and it really > hadn't worked very well. ?Since he now says he thinks it's a good idea > (or more likely I misremembered what he said the first time), it sounds > like we should consider it again. The only problem I can see with this feature is how to design a usable UI. Have you ever cursed these extra clever address forms that make you select a state from a drop-down list of 50 if not a country from the list of 100+? A text field with auto-completion/correction and validation would be the best, but may get cluttered if multiple modules are affected. We actually have similar usability issue with the nosy list. The + button was a great addition, but I would also like to see a user list integration that would allow me to add other by real name or by approximate username spelling. From phd at phd.pp.ru Tue Jul 20 20:59:44 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Tue, 20 Jul 2010 22:59:44 +0400 Subject: [Python-Dev] Python-dev signal-to-noise processing question In-Reply-To: References: <20100719162729.GF21891@phd.pp.ru> <20100720105928.GA4289@phd.pp.ru> Message-ID: <20100720185944.GA10496@phd.pp.ru> On Tue, Jul 20, 2010 at 01:51:07PM -0400, Terry Reedy wrote: > On 7/20/2010 6:59 AM, Oleg Broytman wrote: > 1. I suggested one improvement to the canned response in my previous > post: expand 'using' to 'using or understanding'. I changed wording to "if you're having problems learning, understanding or using Python" > 2. Here is another: mention that Python developers who are willing to > answer non-development questions already participate on > python-list/gmane.comp.python.general to answer such questions there. Good addition, thank you! I'll add something like "there are Python developers who participate on python-list/comp.lang.python". > 3. That brings up: also mention gmane.comp.python.general, for those > like me who prefer the newsgroup interface. It is referenced at http://www.python.org/community/lists/ and my text points to http://www.python.org/community/; I think it's enough. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From phd at phd.pp.ru Tue Jul 20 21:05:27 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Tue, 20 Jul 2010 23:05:27 +0400 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: References: <4C45C4A6.5020208@netwok.org> <20100720181640.D89CA2007A2@kimball.webabinitio.net> Message-ID: <20100720190527.GB10496@phd.pp.ru> On Tue, Jul 20, 2010 at 02:53:53PM -0400, Alexander Belopolsky wrote: > Have you ever cursed these extra clever address forms that make > you select a state from a drop-down list of 50 if not a country from > the list of 100+? Well, I have never, because in any of these drop-down lists I can press a few first letters of the name and the cursor jumps to the country. I often select countries in such lists in web browsers. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From alexander.belopolsky at gmail.com Tue Jul 20 21:13:41 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 20 Jul 2010 15:13:41 -0400 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: <20100720190527.GB10496@phd.pp.ru> References: <4C45C4A6.5020208@netwok.org> <20100720181640.D89CA2007A2@kimball.webabinitio.net> <20100720190527.GB10496@phd.pp.ru> Message-ID: On Tue, Jul 20, 2010 at 3:05 PM, Oleg Broytman wrote: .. > ? Well, I have never, because in any of these drop-down lists I can press > a few first letters of the name and the cursor jumps to the country. I > often select countries in such lists in web browsers. Really? What smartphone are you using? :-) From phd at phd.pp.ru Tue Jul 20 21:38:17 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Tue, 20 Jul 2010 23:38:17 +0400 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: References: <4C45C4A6.5020208@netwok.org> <20100720181640.D89CA2007A2@kimball.webabinitio.net> <20100720190527.GB10496@phd.pp.ru> Message-ID: <20100720193817.GC10496@phd.pp.ru> On Tue, Jul 20, 2010 at 03:13:41PM -0400, Alexander Belopolsky wrote: > On Tue, Jul 20, 2010 at 3:05 PM, Oleg Broytman wrote: > .. > > ? Well, I have never, because in any of these drop-down lists I can press > > a few first letters of the name and the cursor jumps to the country. I > > often select countries in such lists in web browsers. > > Really? What smartphone are you using? :-) Are you developing an interface for smartphones? Wouldn't it hurt usability for desktops/notebooks? Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From alexander.belopolsky at gmail.com Tue Jul 20 22:27:45 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 20 Jul 2010 16:27:45 -0400 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: <20100720193817.GC10496@phd.pp.ru> References: <4C45C4A6.5020208@netwok.org> <20100720181640.D89CA2007A2@kimball.webabinitio.net> <20100720190527.GB10496@phd.pp.ru> <20100720193817.GC10496@phd.pp.ru> Message-ID: On Tue, Jul 20, 2010 at 3:38 PM, Oleg Broytman wrote: .. >> Really? ?What smartphone are you using? ?:-) > > ? Are you developing an interface for smartphones? Wouldn't it hurt > usability for desktops/notebooks? You missed the smiley in my response. But seriously, I do find the interfaces that work well on smartphones to improve usability for desktops/notebooks. Back on the topic, I don't think a drop-down list of all modules is workable even in browsers that display them as combo boxes. How many modules do we have in std lib? About 100? Maybe more. What if the bug affects several modules? What if the patch modifies several modules? Do we want to allow selection of multiple modules for the given issue? The components window is already hard to use if you want to select say both Extension Modules and Windows. This is with just about 20 entries. Imagine over 100 entries there. From phd at phd.pp.ru Tue Jul 20 22:43:19 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Wed, 21 Jul 2010 00:43:19 +0400 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: References: <4C45C4A6.5020208@netwok.org> <20100720181640.D89CA2007A2@kimball.webabinitio.net> <20100720190527.GB10496@phd.pp.ru> <20100720193817.GC10496@phd.pp.ru> Message-ID: <20100720204319.GA16407@phd.pp.ru> On Tue, Jul 20, 2010 at 04:27:45PM -0400, Alexander Belopolsky wrote: > On Tue, Jul 20, 2010 at 3:38 PM, Oleg Broytman wrote: > .. > >> Really? ?What smartphone are you using? ?:-) > > > > ? Are you developing an interface for smartphones? Wouldn't it hurt > > usability for desktops/notebooks? > > You missed the smiley in my response. I am a well-known wet blanket. (-: > But seriously, I do find the > interfaces that work well on smartphones to improve usability for > desktops/notebooks. Sometimes they do and sometimes they don't. Big screen(s) and a wheeled multi-button mouse allow quite a different interface compared to small screen and single-finger taps. > Back on the topic, I don't think a drop-down list of all modules is > workable even in browsers that display them as combo boxes. How many > modules do we have in std lib? About 100? Maybe more. What if the > bug affects several modules? What if the patch modifies several > modules? Do we want to allow selection of multiple modules for the > given issue? The components window is already hard to use if you want > to select say both Extension Modules and Windows. This is with just > about 20 entries. Imagine over 100 entries there. In this particular case I'd rather tend to agree - an editable single-line box to enter space-*and*-comma-separated modules list would be the best interface. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From db3l.net at gmail.com Tue Jul 20 23:53:53 2010 From: db3l.net at gmail.com (David Bolen) Date: Tue, 20 Jul 2010 17:53:53 -0400 Subject: [Python-Dev] MSDN subscription extensions? Message-ID: As one of the beneficiaries of the efforts (much appreciated) last year to obtain Microsoft MSDN subscriptions for developers/testers (in my case, primarily buildbot operation), I was wondering if anyone might know if those subscriptions will be able to be renewed this year? -- David From tjreedy at udel.edu Wed Jul 21 00:09:50 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 20 Jul 2010 18:09:50 -0400 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: <20100720204319.GA16407@phd.pp.ru> References: <4C45C4A6.5020208@netwok.org> <20100720181640.D89CA2007A2@kimball.webabinitio.net> <20100720190527.GB10496@phd.pp.ru> <20100720193817.GC10496@phd.pp.ru> <20100720204319.GA16407@phd.pp.ru> Message-ID: In any case, a module list should be separate from components. On 7/20/2010 4:43 PM, Oleg Broytman wrote: > In this particular case I'd rather tend to agree - an editable > single-line box to enter space-*and*-comma-separated modules list would be > the best interface. An interesting idea. In any case, a module list should be separate from components. A custom app would only allow entry there if Lib is a selected component. Such could also use a selecton screen with multiple columns rather than a single column. But we are limited to what can be done with a web interface. If Lib is not selected and modules are entered, Lib could be autoselected as it is unlikely that one would type in module names if it should not be. A back-end check against would be nice, but would suffer from the update problem, as well as module name changes. -- Terry Jan Reedy From brian.curtin at gmail.com Wed Jul 21 00:16:25 2010 From: brian.curtin at gmail.com (Brian Curtin) Date: Tue, 20 Jul 2010 17:16:25 -0500 Subject: [Python-Dev] MSDN subscription extensions? In-Reply-To: References: Message-ID: On Tue, Jul 20, 2010 at 16:53, David Bolen wrote: > As one of the beneficiaries of the efforts (much appreciated) last > year to obtain Microsoft MSDN subscriptions for developers/testers (in > my case, primarily buildbot operation), I was wondering if anyone > might know if those subscriptions will be able to be renewed this > year? > > -- David > I'll add your name to the list, wait a few days for others to respond, then contact Microsoft together as a group. For anyone else out there with a current subscription which will be expiring soon, email me privately and I'll get you going with an extension. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Wed Jul 21 00:19:53 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 21 Jul 2010 08:19:53 +1000 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: On Wed, Jul 21, 2010 at 4:18 AM, Alexander Belopolsky wrote: > On Tue, Jul 20, 2010 at 2:12 PM, Reid Kleckner wrote: > .. >> IMO you should just rename test_trace.py to test_settrace.py, and put >> the trace.py tests in test_trace.py. > > +1 Yep, we're pretty free to rename stuff as required inside the test suite. The only real exception is test.support, as that's a documented standard library module. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Wed Jul 21 00:25:27 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 21 Jul 2010 08:25:27 +1000 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: References: <4C45C4A6.5020208@netwok.org> <20100720181640.D89CA2007A2@kimball.webabinitio.net> Message-ID: On Wed, Jul 21, 2010 at 4:53 AM, Alexander Belopolsky wrote: > The only problem I can see with this feature is how to design a usable > UI. ?Have you ever cursed these extra clever address forms that make > you select a state from a drop-down list of 50 if not a country from > the list of 100+? ?A text field with auto-completion/correction and > validation would be the best, but may get cluttered if multiple > modules are affected. ?We actually have similar usability issue with > the nosy list. ?The + button was a great addition, but I would also > like to see a user list integration that would allow me to add other > by real name or by approximate username spelling. Despite the nosy list's issues, I think it is actually a better model here than the components list. That is: - comma separated list in main form that you can just type into - separate search form that pops up to allow multiple selections Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From cs at zip.com.au Wed Jul 21 00:35:45 2010 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 21 Jul 2010 08:35:45 +1000 Subject: [Python-Dev] mkdir -p in python In-Reply-To: <4C45D3B6.1020701@voidspace.org.uk> References: <4C45D3B6.1020701@voidspace.org.uk> Message-ID: <20100720223545.GA16998@cskk.homeip.net> On 20Jul2010 17:49, Michael Foord wrote: | On 20/07/2010 14:43, Nick Coghlan wrote: | >On Tue, Jul 20, 2010 at 11:09 PM, Steven D'Aprano wrote: | >>I'm -0 on adding an argument to os.makedirs, +0 on adding a variant | >>function to os, and +0.5 on adding the variant to the shutil module. | >shutil seems like the place for it to me. The subtlety of getting the | >error suppression correct makes it worth implementing this once and | >for all I think. | | Really - so we have two almost identical functions with slightly | different behaviour in different standard library modules? Three if you count os.mkdir itself as well. | Aren't | the Python APIs for working with files and paths confused enough | already. That seems crazy to me and a switch on os.makedirs much | more sensible. +1 from me. Hasn't the BDFL blessed this particular suggestion too? Disclaimer: I am _not_ a dev. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ I am not a free man, I am a number. - Beirne Konarski From guido at python.org Wed Jul 21 01:10:25 2010 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Jul 2010 00:10:25 +0100 Subject: [Python-Dev] mkdir -p in python In-Reply-To: <20100720223545.GA16998@cskk.homeip.net> References: <4C45D3B6.1020701@voidspace.org.uk> <20100720223545.GA16998@cskk.homeip.net> Message-ID: On Tue, Jul 20, 2010 at 11:35 PM, Cameron Simpson wrote: > On 20Jul2010 17:49, Michael Foord wrote: > | On 20/07/2010 14:43, Nick Coghlan wrote: > | >On Tue, Jul 20, 2010 at 11:09 PM, Steven D'Aprano ?wrote: > | >>I'm -0 on adding an argument to os.makedirs, +0 on adding a variant > | >>function to os, and +0.5 on adding the variant to the shutil module. > | >shutil seems like the place for it to me. The subtlety of getting the > | >error suppression correct makes it worth implementing this once and > | >for all I think. > | > | Really - so we have two almost identical functions with slightly > | different behaviour in different standard library modules? > > Three if you count os.mkdir itself as well. > > | Aren't > | the Python APIs for working with files and paths confused enough > | already. That seems crazy to me and a switch on os.makedirs much > | more sensible. > > +1 from me. Hasn't the BDFL blessed this particular suggestion too? Now I have. :-) -1 on a new function (despite the constant-argument guideline) and +1 on a flag. If it weren't for backwards compatibility I'd just change os.makedirs() to act like mkdir -p period, but the last opportunity we had for that was Python 3.0. -- --Guido van Rossum (python.org/~guido) From g.brandl at gmx.net Wed Jul 21 03:02:33 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 21 Jul 2010 02:02:33 +0100 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: References: <4C45C4A6.5020208@netwok.org> <20100720181640.D89CA2007A2@kimball.webabinitio.net> <20100720190527.GB10496@phd.pp.ru> <20100720193817.GC10496@phd.pp.ru> <20100720204319.GA16407@phd.pp.ru> Message-ID: Am 20.07.2010 23:09, schrieb Terry Reedy: > In any case, a module list should be separate from components. > > On 7/20/2010 4:43 PM, Oleg Broytman wrote: > >> In this particular case I'd rather tend to agree - an editable >> single-line box to enter space-*and*-comma-separated modules list would be >> the best interface. > > An interesting idea. And hardly a new idea :) I don't know how often I've now proposed tags for tracker issues... Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From greg.ewing at canterbury.ac.nz Wed Jul 21 02:27:07 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 21 Jul 2010 12:27:07 +1200 Subject: [Python-Dev] mkdir -p in python In-Reply-To: References: <4C42C9CB.1090003@timgolden.me.uk> <4C43A4F6.4090800@canterbury.ac.nz> <4C44DB97.8030108@canterbury.ac.nz> Message-ID: <4C463EDB.7090403@canterbury.ac.nz> ??? wrote: > Sorry, I don't know what is the "no constant arguments" guideline refers > to. Could you give me some more explanation? It's a rule of thumb that Guido says he uses when designing an API. If in the majority of use cases for a proposed function, one of its arguments would always be a constant that chooses between two or more different behaviours, then it would probably be better to have separate functions for each of the behaviours. -- Greg From greg.ewing at canterbury.ac.nz Wed Jul 21 02:37:20 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 21 Jul 2010 12:37:20 +1200 Subject: [Python-Dev] mkdir -p in python In-Reply-To: <201007202309.59704.steve@pearwood.info> References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> Message-ID: <4C464140.3040509@canterbury.ac.nz> Steven D'Aprano wrote: > Perhaps all we need is a recipe in the docs: > > try: > os.makedirs(path) > except OSError, e: > if e.errno != 17: > raise I don't like writing code that depends on particular errno values, because I don't trust it to work cross- platform. Also it seems suboptimal to require people to use a recipe like this to get the behaviour that is wanted in the overwhelming majority of cases. I don't see what would be so bad about adding a new function for this. Think of it as correcting the mistake of not making makedirs() behave this way from the beginning. -- Greg From ysj.ray at gmail.com Wed Jul 21 05:18:45 2010 From: ysj.ray at gmail.com (Ray Allen) Date: Wed, 21 Jul 2010 11:18:45 +0800 Subject: [Python-Dev] mkdir -p in python In-Reply-To: <4C464140.3040509@canterbury.ac.nz> References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> <4C464140.3040509@canterbury.ac.nz> Message-ID: > > I don't see what would be so bad about adding a new function for this. Think of it as correcting the mistake of not making makedirs() behave this way from the beginning. If you want to add a new function, then what its name should be? I guess it should be too similar as existing ones. It's confusing for using because they have similar names and do the same major things except exception. For users it's easier to remember one function than tow. If we want to correct a mistake, it's better not to import another mistake. I don't like writing code that depends on particular errno values, because I don't trust it to work cross- platform. I think in this case, the errno is generate by c standard library, which can be seen as cross-platform. > > > -- > Greg > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ysj.ray%2Bpython-dev%40gmail.com > -- Ray Allen Best wishes! -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Wed Jul 21 07:46:03 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 21 Jul 2010 14:46:03 +0900 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: <20100720181640.D89CA2007A2@kimball.webabinitio.net> References: <4C45C4A6.5020208@netwok.org> <20100720181640.D89CA2007A2@kimball.webabinitio.net> Message-ID: <87y6d5xu90.fsf@uwakimon.sk.tsukuba.ac.jp> R. David Murray writes: > During the most recent discussion I can remember, I thought I remembered > Stephen Trumble saying that they'd tried that in xemacs and it really > hadn't worked very well. Since he now says he thinks it's a good idea > (or more likely I misremembered what he said the first time), it sounds > like we should consider it again. No, you didn't misremember. I changed my mind. Now I know more about *why* it didn't appeal to me at the time: the UI for links and multilinks with large numbers of selectable value, and we had a poor configuration of nosy lists, so people weren't getting on the right issues. So, I don't recommend it with the stock Roundup UI if there are going to be more modules than would fit on a page (50), and the UI is somewhat annoying for as few as 20 checkboxes. Unfortunately, I can't offer a patch, but the basic things that need to be done are (1) make the "multilink input" window multicolumn (you could get 100 module names to fit in the window, but 50 should be plenty for this purpose), and (2) bump the limit on entries per page so they all fit on one page. Also, it would be nice if there were a UI to make oneself "auto-nosy" on certain modules. From stephen at xemacs.org Wed Jul 21 08:09:53 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 21 Jul 2010 15:09:53 +0900 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: <20100720204319.GA16407@phd.pp.ru> References: <4C45C4A6.5020208@netwok.org> <20100720181640.D89CA2007A2@kimball.webabinitio.net> <20100720190527.GB10496@phd.pp.ru> <20100720193817.GC10496@phd.pp.ru> <20100720204319.GA16407@phd.pp.ru> Message-ID: <87wrspxt5a.fsf@uwakimon.sk.tsukuba.ac.jp> Oleg Broytman writes: > > Back on the topic, I don't think a drop-down list of all modules is > > workable even in browsers that display them as combo boxes. How many > > modules do we have in std lib? About 100? Maybe more. What if the > > bug affects several modules? What if the patch modifies several > > modules? Do we want to allow selection of multiple modules for the > > given issue? Roundup has a multilink type which allows selection of multiple items. The input interface is a list of checkboxes formatted as a two-column HTML table (checkbox and module name), not a combo box (which doesn't allow a list of selections). XEmacs uses such a multilink for its 147 Modules (in Python's tracker, that would be Components). For me, with my most frequently affected modules living at the top of the first page of the module list I have no problem with the multilink interface; it's when I need to find a library on the second or third page that things get frustrating. About 50 items with checkboxes should fit easily in a popup window in table form if there were multiple columns (say four or five); it just hasn't been coded yet. And there's no reason the table has to fit in the window. A scrolling table would be fine for this, as long as the total height wasn't more than about 3 times that of the window. I would guess that a table of checkboxes could easily handle a multilink with about 200 options, and would be usable both by newbies and triagers. Possibly active developers would find the one-line text- entry more efficient, but I think it would be a close race: > In this particular case I'd rather tend to agree - an editable > single-line box to enter space-*and*-comma-separated modules list > would be the best interface. For active developers, yes. But this is unhelpful for people who aren't hyper-familiar with the component list or who can't spell. The checkboxes are also faster when doing triage, at least if the relevant component is in the first screen. Maybe with enough Javascript to give autocompletion and validation, a space/comma-separated list would be OK, but I'm dubious. Also, some of our users are able to pick out which modules seem to be involved from backtraces in error messages, but not without a list. From clinton.roy at gmail.com Wed Jul 21 10:11:44 2010 From: clinton.roy at gmail.com (Clinton Roy) Date: Wed, 21 Jul 2010 18:11:44 +1000 Subject: [Python-Dev] mkdir -p in python In-Reply-To: <4C464140.3040509@canterbury.ac.nz> References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> <4C464140.3040509@canterbury.ac.nz> Message-ID: Hey folks, On 21 July 2010 10:37, Greg Ewing wrote: > Steven D'Aprano wrote: >> >> Perhaps all we need is a recipe in the docs: >> >> try: >> ? ?os.makedirs(path) >> except OSError, e: >> ? ?if e.errno != 17: >> ? ? ? ?raise > > I don't like writing code that depends on particular > errno values, because I don't trust it to work cross- > platform. I use errno.EEXIST instead of a magic number. later, -- Clinton Roy Software Engineer Global Change Institute University of Queensland humbug.org.au? - Brisbane Unix Group clintonroy.wordpress.com - Blog flickr.com/photos/croy/ - Photos From anders.sandvig at gmail.com Wed Jul 21 13:03:20 2010 From: anders.sandvig at gmail.com (Anders Sandvig) Date: Wed, 21 Jul 2010 13:03:20 +0200 Subject: [Python-Dev] Curious datetime method In-Reply-To: References: Message-ID: On Tue, Jul 20, 2010 at 5:56 PM, Alexander Belopolsky wrote: > In the real world where we have to take backward compatibility into > account, I would like to make today() and now() to be the same: both > taking optional tz argument, both available as either date or datetime > methods and both covariant. ?the justification for having two methods > doing exactly the same will be just readability: date.today() and > datetime.now() are more readable than date.now() and datetime.today(). I agree. Unless, of course, someone has a good explanation/reason for why today() and now() are implemented differently (i.e. a use case where the difference is significant). Anders From martin at v.loewis.de Wed Jul 21 13:43:59 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 21 Jul 2010 12:43:59 +0100 Subject: [Python-Dev] Python-dev signal-to-noise processing question In-Reply-To: References: <20100719162729.GF21891@phd.pp.ru> <20100720105928.GA4289@phd.pp.ru> Message-ID: <4C46DD7F.2090102@v.loewis.de> > No, the reply is fine as far as it goes, and I am sure the poster did > get a reply from c.l.py, but his question revealed a thirst for > knowledge not usually evidenced in non-dev inquiries. Unfortunately (?) the question also revealed a lack of understanding of a fairly basic concept. IIUC, he wanted to know how Python handles SIGKILL, when the hole point of SIGKILL is that you cannot handle it. So he shouldn't have been surprised that he couldn't find a place in Python where it's handled. Regards, Martin From barry at python.org Wed Jul 21 15:08:28 2010 From: barry at python.org (Barry Warsaw) Date: Wed, 21 Jul 2010 15:08:28 +0200 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> Message-ID: <20100721150828.0f671cbd@snowdog> On Jul 16, 2010, at 12:16 AM, Amaury Forgeot d'Arc wrote: >2010/7/15 Barry Warsaw : >> The first draft of PEP 3149 is ready for review. > >I like it! Cool! >I think it could mention the case where packages are not installed >in the canonical directory, but placed elsewhere along the PYTHONPATH. >This is how I deploy applications, for example, and the differences >between python versions makes this a pain. Because of the way the results of ./configure are embodied in the Makefile's $SO variable, and that variable is used unconditionally from sysconfig (and thus distutils), once you've configured --with-so-abi-tag, that tag will be used globally everywhere. I've added a note about this. >The specific case of Windows should be mentioned: each foo.pyd >contains the name of the python library (Python27.dll) it has been >linked with; It must be rebuilt for a major version change. >IMO the Windows installers provided by python.org should be built with >a tag that contains this major number. The current version of the PEP and my implementation do not change the Windows builds at all. I don't feel qualified to integrate the ideas expressed in PEP 3149 for Windows builds, but I would be happy to accept patches to either the PEP or implementation to export the same tagging feature for Windows. >> Thus for example, an initial implementation of PEP 384, compiled with >> `--with-so-abi-tag=cpython-xy` would search for the following file >> names when extension module `foo` is imported (in this order):: >> >> ? ?foo.abi3.so >> ? ?foo.cpython-xy.so >> ? ?foo.so > >Is this the correct order? IMO the so-abi-tag is more precise and the >two first items should be swapped. Good point, fixed. >PyPy would also benefit from this patch: it can now use extension >modules, but the ABI is slightly different. By default, PyPy would >load only modules containing the ABI tag, and refuse foo.so which is >incompatible for sure. But the installations could still be shared >between Python implementations. Interesting. I've added a note about this to the PEP. Thanks for the feedback. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From alexander.belopolsky at gmail.com Wed Jul 21 16:43:47 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 21 Jul 2010 10:43:47 -0400 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: <87wrspxt5a.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4C45C4A6.5020208@netwok.org> <20100720181640.D89CA2007A2@kimball.webabinitio.net> <20100720190527.GB10496@phd.pp.ru> <20100720193817.GC10496@phd.pp.ru> <20100720204319.GA16407@phd.pp.ru> <87wrspxt5a.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Wed, Jul 21, 2010 at 2:09 AM, Stephen J. Turnbull wrote: .. > ?> ? ?In this particular case I'd rather tend to agree - an editable > ?> single-line box to enter space-*and*-comma-separated modules list > ?> would be the best interface. > > For active developers, yes. ?But this is unhelpful for people who > aren't hyper-familiar with the component list or who can't spell. It does not have to be either or. We can have a text box and a link/button next to it which will pop up a window with a selectable list of modules. Once modules are selected and pop-up closed, the text box should contain comma separated list of selected modules. From alexander.belopolsky at gmail.com Wed Jul 21 16:59:50 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 21 Jul 2010 10:59:50 -0400 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: On Tue, Jul 20, 2010 at 6:19 PM, Nick Coghlan wrote: .. > Yep, we're pretty free to rename stuff as required inside the test > suite. The only real exception is test.support, as that's a documented > standard library module. Would anyone object to moving Lib/test/test_trace.py to Lib/test/test_line_tracing.py in 2.7 maint branch as Benjamin did in py3k? As I suggested in issue 9315 [1], the tests should be first written for 2.7 and either designed to run unchanged in 3.x, or ported in a controlled fashion to assure that we don't have regressions. (And in fact we do! [2, 3]) PS: The bikeshedding period is probably closed on this, but I would rather call the new test file test_sys_settrace rather than test_line_tracing. The sys.settrace function does more than line tracing and the test naming convention seems to be component rather than functionality oriented in the rest of the test suite. [1] http://bugs.python.org/issue9315 [2] http://bugs.python.org/issue9317 [3] http://bugs.python.org/issue9323 From mail at timgolden.me.uk Wed Jul 21 17:11:39 2010 From: mail at timgolden.me.uk (Tim Golden) Date: Wed, 21 Jul 2010 16:11:39 +0100 Subject: [Python-Dev] Python Language Summit EuroPython 2010 Message-ID: <4C470E2B.5060406@timgolden.me.uk> Before the main events of EuroPython 2010 a Python Language Summit took place at the Conference venue in Birmingham. Present were (in the order they sat around the table): * Brett Cannon * Guido van Rossum * Holger Krekel * Amaury Forgeot D'Arc * Georg Brandl * P?ter Szab? * Ezio Melotti * Michael Foord * Mark Dickinson * Martin von Loewis * Ronald Oussoren * Tim Golden * Marc Andre Lemburg * Richard Jones Michael initiated a round-up of current and prospective Python versions for various implementations. CPython and IronPython have both just released v2.7 with IronPython offering some Py3 compatibility via a command-line switch. The recent/current migration of Numpy and SciPy to Py3 should give a boost to uptake. Amaury confirmed that PyPy currently supports 2.5.2 but is looking to target 2.7. The PyPy guys also announced a C API bridging layer which should enable a range of Python extension modules to work directly with PyPy. This is only a stepping stone by way of broadening support. Brett suggested that the Unladen Swallow merge to trunk was waiting for some work to complete on the JIT compiler and Georg, as release manager for 3.2, confirmed that Unladen Swallow would not be merged before 3.3. The email module needs some work in Py3. David Murray has been given some money by the PSF but needs more from other sources to complete the work. This is hampered by the legalities around commercial organisations making donations to not-for-profits when those donations are earmarked. Various suggestions were put forward with no-one sure of the legal issues. Guido suggested that we should move forward rather than stall for want of legal advice. A broad discussion arose concerning the issues debated on web-sig concerning the WSGI protocol and the bytes vs string issues. Marc Andre brought up the cgi module which has similar issues under Py3 and other examples were given, including ftplib, urllib and some os functions. Various solutions were put forward including a hybrid bytes-with-encoding object. This proposal was widely unpopular, but two proposals met with broad approval: that certain stdlib functions might be polymorphic, returning the type of their input as output; and that the encoding string should include its error-handling. An example of the first would be that os.getenv ("HOME") would return "/home/tjg" while os.getenv (b"HOME") would return b"/home/tjg". An example of the latter would be "utf8:strict". Something of the sort already works for PYTHONIOENCODING. The issue of a __format__ equivalent for bytes was also raised as was the idea of object methods to render an object as string or bytes, which could be used in the polymorphic functions above. Martin spoke about the state of the stable ABI PEP, indicating that he was targetting 3.2. This work would reduce the need to recompile extension modules separately on Windows for every version of Python -- something especially pertinent when code has been orphaned but is still useful. The versioned .so files PEP being worked out by Barry Warsaw overlaps with this PEP and would only be useful for extensions which don't target the stable ABI. A messy discussion turned on the question of garbage collection of module objects, and the order in which finalisers are called if at all, especially when reference cycles exist. Marc Andre was proposing a __cleanup__ magic function for Python modules, which would enable the implementer to define the order in which resources are released / closed down. This is quite a subtle area and raised the issue of unfinalised objects in a reference cycle whose memory has been freed out from under them but which still exist. Martin described the Java approach where finalisers are called once and then flagged so they are not called again even if their object is resurrected. This sounded like a useful approach for Python but would break code which expected to be able to resurrect an object during its __del__ method... which is not expected to account for much code. Guido pointed out that no-one can be expected to hold enough of the complexities of this area of Python's implementation in their head, and that an implementation of some sort would need to be written so that the corner-cases could emerge. Ronald described the issues around the version and architecture differences on Mac OS X and especially around Tkinter (and therefore IDLE). It was agreed that two installers could be provided: one targetting OS 10.3 on 32-bit Intel/PPC; the other targetting 10.6 on 32 and 64-bit Intel. This latter would then be able to use the system's Tk 8.5. The 10.6 binary would also work for 10.5, which would be indicated in the install docs. The Mercurial migration should move forward once Dirkjan has finished work on his thesis. Martin insisted that a for-real repository would have to be set up so that people can really see how it would work. An outstanding issue in hg-svn prevents the Python history from being imported, but it should be fixable. Martin & Tim brought up the issue of externals which the buildbots use on Windows to bring in and build slightly patched versions of external libraries such as OpenSSL and sqlite3. Brett confirmed that he would like to see the stdlib broken out into its own repository which could then be shared between the different Python implementations. A discussion on the Cheeseshop / Package Index highlighted the fact that the packaging infrastructure has become increasingly important especially since setuptools, buildout and pip all download from it. Richard produced graphs showing the increase in package downloads over time, and attributed the recent slight tail-off to the fact that the toolchains are now becoming more canny with respect to cacheing and mirroring. Martin & Richard confirmed that mirrors are now in place and Marc Andre confirmed that he would be putting together a proposal to have PyPI hosted in the cloud. Guido pointed out that if an AppEngine implementation were desirable, he was sure that AppEngine team would support it with resources as needed. Martin didn't feel that there was a problem with loading on the box in question; it's the uptime that's behind people's concern as it's now so essential to installing and deploying Python applications. Several people outlined the recent heated discussion over the addition of a checkbox to the PyPI user-registration pages. Tarek has already undertaken to patch PyPI to move the checkbox back one step, allowing existing distutils users to register from the command line. At the same time, Brett advised removing that functionality from distutils2 as signing up on a web page is no great hardship. From reid.kleckner at gmail.com Wed Jul 21 17:14:42 2010 From: reid.kleckner at gmail.com (Reid Kleckner) Date: Wed, 21 Jul 2010 08:14:42 -0700 Subject: [Python-Dev] Python-dev signal-to-noise processing question In-Reply-To: <4C46DD7F.2090102@v.loewis.de> References: <20100719162729.GF21891@phd.pp.ru> <20100720105928.GA4289@phd.pp.ru> <4C46DD7F.2090102@v.loewis.de> Message-ID: On Wed, Jul 21, 2010 at 4:43 AM, "Martin v. L?wis" wrote: > Unfortunately (?) the question also revealed a lack of understanding > of a fairly basic concept. IIUC, he wanted to know how Python > handles SIGKILL, when the hole point of SIGKILL is that you cannot > handle it. So he shouldn't have been surprised that he couldn't find > a place in Python where it's handled. No, you misunderstood. He knew that one cannot set a SIGKILL signal handler. He just wanted to find the code in CPython responsible for turning that error into an exception for the purposes of giving a tutorial on signals. Reid From jnoller at gmail.com Wed Jul 21 17:42:00 2010 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 21 Jul 2010 11:42:00 -0400 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: <4C470E2B.5060406@timgolden.me.uk> References: <4C470E2B.5060406@timgolden.me.uk> Message-ID: On Wed, Jul 21, 2010 at 11:11 AM, Tim Golden wrote: [...snip...] > A messy discussion turned on the question of garbage collection of module > objects, and the order in which finalisers are called if at all, especially > when reference cycles exist. Marc Andre was proposing a __cleanup__ magic > function > for Python modules, which would enable the implementer to define the order > in which resources are released / closed down. This is quite a subtle area > and raised the issue of unfinalised objects in a reference cycle whose > memory has been freed out from under them but which still exist. Martin > described > the Java approach where finalisers are called once and then flagged so > they are not called again even if their object is resurrected. This sounded > like a useful approach for Python but would break code which expected to > be able to resurrect an object during its __del__ method... which is not > expected to account for much code. > > Guido pointed out that no-one can be expected to hold enough of the > complexities > of this area of Python's implementation in their head, and that an > implementation > of some sort would need to be written so that the corner-cases could emerge. FWIW; I'm currently dealing with a bug in this area w.r.t multiprocessing and threads and modules we have imported vanishing due to this issue. I'm interested in hearing more. The problem affects 2.7 final, as well. jesse From solipsis at pitrou.net Wed Jul 21 17:58:19 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 21 Jul 2010 17:58:19 +0200 Subject: [Python-Dev] Python Language Summit EuroPython 2010 References: <4C470E2B.5060406@timgolden.me.uk> Message-ID: <20100721175819.3b037ad9@pitrou.net> On Wed, 21 Jul 2010 11:42:00 -0400 Jesse Noller wrote: > On Wed, Jul 21, 2010 at 11:11 AM, Tim Golden wrote: > [...snip...] > > A messy discussion turned on the question of garbage collection of module > > objects, and the order in which finalisers are called if at all, especially > > when reference cycles exist. Marc Andre was proposing a __cleanup__ magic > > function > > for Python modules, which would enable the implementer to define the order > > in which resources are released / closed down. This is quite a subtle area > > and raised the issue of unfinalised objects in a reference cycle whose > > memory has been freed out from under them but which still exist. Martin > > described > > the Java approach where finalisers are called once and then flagged so > > they are not called again even if their object is resurrected. This sounded > > like a useful approach for Python but would break code which expected to > > be able to resurrect an object during its __del__ method... which is not > > expected to account for much code. > > > > Guido pointed out that no-one can be expected to hold enough of the > > complexities > > of this area of Python's implementation in their head, and that an > > implementation > > of some sort would need to be written so that the corner-cases could emerge. > > FWIW; I'm currently dealing with a bug in this area w.r.t > multiprocessing and threads and modules we have imported vanishing due > to this issue. I'm interested in hearing more. One common resolution is to not use a __del__ method, but instead a weakref callback which will do the necessary cleanup of a certain set of resources. This is of course not applicable in all situations. From dirkjan at ochtman.nl Wed Jul 21 18:47:49 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Wed, 21 Jul 2010 18:47:49 +0200 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: <4C470E2B.5060406@timgolden.me.uk> References: <4C470E2B.5060406@timgolden.me.uk> Message-ID: On Wed, Jul 21, 2010 at 17:11, Tim Golden wrote: > The Mercurial migration should move forward once Dirkjan has finished work > on his thesis. Martin insisted that a for-real repository would have to be > set up so that people can really see how it would work. An outstanding issue > in hg-svn prevents the Python history from being imported, but it should be > fixable. Martin & Tim brought up the issue of externals which the buildbots > use on Windows to bring in and build slightly patched versions of external > libraries such as OpenSSL and sqlite3. The issue in hgsubversion (which is different from hgsvn) has been fixed. The current problem is some oddity in the SVN bindings where it keeps complaining about having too many files open... I've been troubleshooting this in my thesis downtime, but nothing definitive yet. I'd like to hear more about where these externals are, so we can look into fixing that. Recent Mercurial includes support for svn subrepos, which might be usable for this. I wonder if it's sensible to discuss the splitting of the interpreter from the stdlib in separate repositories, because it might be better to convert to separate repositories in one go (instead of splitting out the stdlib later on). Not sure about this, but it would be useful to discuss at least. Cheers, Dirkjan From reid.kleckner at gmail.com Wed Jul 21 18:50:23 2010 From: reid.kleckner at gmail.com (Reid Kleckner) Date: Wed, 21 Jul 2010 09:50:23 -0700 Subject: [Python-Dev] Unladen swallow status Message-ID: On Wed, Jul 21, 2010 at 8:11 AM, Tim Golden wrote: > Brett suggested that > the Unladen Swallow merge to trunk was waiting for some work to complete > on the JIT compiler and Georg, as release manager for 3.2, confirmed that > Unladen Swallow would not be merged before 3.3. Yeah, this has slipped. I have patches that need review, and Jeff and Collin have been distracted with other work. Hopefully when one of them gets around to that, I can proceed with the merge without blocking on them. Reid From janssen at parc.com Wed Jul 21 18:53:19 2010 From: janssen at parc.com (Bill Janssen) Date: Wed, 21 Jul 2010 09:53:19 PDT Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: <4C470E2B.5060406@timgolden.me.uk> References: <4C470E2B.5060406@timgolden.me.uk> Message-ID: <68779.1279731199@parc.com> Tim, thanks for this write-up! Bill From ziade.tarek at gmail.com Wed Jul 21 19:21:17 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 21 Jul 2010 19:21:17 +0200 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: <4C470E2B.5060406@timgolden.me.uk> References: <4C470E2B.5060406@timgolden.me.uk> Message-ID: Thanks Tim ! On Wed, Jul 21, 2010 at 5:11 PM, Tim Golden wrote: [..] > > Several people outlined the recent heated discussion over the addition > of a checkbox to the PyPI user-registration pages. Tarek has already > undertaken to patch PyPI to move the checkbox back one step, allowing > existing distutils users to register from the command line. Yes, so the problem goes away and we are conforming to the legal requirements. > At the same > time, Brett advised removing that functionality from distutils2 as > signing up on a web page is no great hardship. We will remove/simplify the register command, and get rid of that part. In the meantime, to try to avoid this problem in the future, since quite a few projects depend on PyPI now, I would like to suggest that a PEP of the various protocols PyPI has, is created. We do have some documentation here and there about the XML-RPC interface, the HTTP interface, etc., but we should have a single document describing the PyPI APIs, and maybe adding an API version number, so it can evolve smoothly. This would also be helpful for the people that are writing their own PyPI software, and ensure that distutils2 can work out of the box with any PyPI server out there. Tarek -- Tarek Ziad? | http://ziade.org From martin at v.loewis.de Wed Jul 21 19:26:27 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 21 Jul 2010 18:26:27 +0100 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: References: <4C470E2B.5060406@timgolden.me.uk> Message-ID: <4C472DC3.1070705@v.loewis.de> Am 21.07.10 17:47, schrieb Dirkjan Ochtman: > Martin& Tim brought up the issue of externals which the buildbots >> use on Windows to bring in and build slightly patched versions of external >> libraries such as OpenSSL and sqlite3. > > The issue in hgsubversion (which is different from hgsvn) has been > fixed. The current problem is some oddity in the SVN bindings where it > keeps complaining about having too many files open... I've been > troubleshooting this in my thesis downtime, but nothing definitive > yet. I'd like to hear more about where these externals are, so we can > look into fixing that. Recent Mercurial includes support for svn > subrepos, which might be usable for this. This isn't really about svn:externals, but about http://svn.python.org/projects/external/ which gets used in http://svn.python.org/projects/python/trunk/Tools/buildbot/external-common.bat There may not be a problem - all is needed is a recommendation how to replace it. The requirements are these: - multiple versions of external libraries - each Python version only needs a single copy of the library - we make changes to the code after its imported from upstream, and want to remember what these changes are Regards, Martin From ianb at colorstudy.com Wed Jul 21 19:38:32 2010 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 21 Jul 2010 12:38:32 -0500 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: <4C470E2B.5060406@timgolden.me.uk> References: <4C470E2B.5060406@timgolden.me.uk> Message-ID: On Wed, Jul 21, 2010 at 10:11 AM, Tim Golden wrote: > A discussion on the Cheeseshop / Package Index highlighted the fact that > the packaging infrastructure has become increasingly important especially > since setuptools, buildout and pip all download from it. Richard produced > graphs showing the increase in package downloads over time, and attributed > the recent slight tail-off to the fact that the toolchains are now becoming > more canny with respect to cacheing and mirroring. > > Martin & Richard confirmed that mirrors are now in place and Marc Andre > confirmed > that he would be putting together a proposal to have PyPI hosted in the > cloud. Guido pointed out that if an AppEngine implementation were > desirable, > he was sure that AppEngine team would support it with resources as needed. > Martin didn't feel that there was a problem with loading on the box in > question; it's the uptime that's behind people's concern as it's now so > essential to installing and deploying Python applications. > >From what I've been able to tell from afar, I strongly suspect PyPI's downtimes would be greatly reduced with a move to mod_wsgi (currently it is using mod_fcgi, and most downtime is solved with an Apache restart -- mod_wsgi generally recovers from these problems without intervention). Martin attempted this at one time but ran into some installation problems. It seems like the team of people managing PyPI could benefit from the addition of someone with more of a sysadmin background (e.g., to help with installing a monitor on the server). -- Ian Bicking | http://blog.ianbicking.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Wed Jul 21 19:42:27 2010 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 21 Jul 2010 12:42:27 -0500 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: 2010/7/21 Alexander Belopolsky : > On Tue, Jul 20, 2010 at 6:19 PM, Nick Coghlan wrote: > .. >> Yep, we're pretty free to rename stuff as required inside the test >> suite. The only real exception is test.support, as that's a documented >> standard library module. > > Would anyone object to moving Lib/test/test_trace.py to > Lib/test/test_line_tracing.py in 2.7 maint branch as ?Benjamin did in > py3k? ? As I suggested in issue 9315 [1], the tests should be first > written for 2.7 and either designed to run unchanged in 3.x, or ported > in a controlled fashion to assure that we don't have regressions. > (And in fact we do! [2, 3]) > > PS: The bikeshedding period is probably closed on this, but I would > rather call the new test file test_sys_settrace rather than > test_line_tracing. ?The sys.settrace function does more than line > tracing and the test naming convention seems to be component rather > than functionality oriented in the rest of the test suite. True, but the tests in that file are (mostly?) all about line tracing. -- Regards, Benjamin From alexander.belopolsky at gmail.com Wed Jul 21 19:45:24 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 21 Jul 2010 13:45:24 -0400 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: On Wed, Jul 21, 2010 at 1:42 PM, Benjamin Peterson wrote: .. > True, but the tests in that file are (mostly?) all about line tracing. > Hopefully this will change soon. :-) From dirkjan at ochtman.nl Wed Jul 21 23:40:23 2010 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Wed, 21 Jul 2010 23:40:23 +0200 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: References: <4C470E2B.5060406@timgolden.me.uk> Message-ID: On Wed, Jul 21, 2010 at 19:38, Ian Bicking wrote: > From what I've been able to tell from afar, I strongly suspect PyPI's > downtimes would be greatly reduced with a move to mod_wsgi (currently it is > using mod_fcgi, and most downtime is solved with an Apache restart -- > mod_wsgi generally recovers from these problems without intervention). > Martin attempted this at one time but ran into some installation problems. > It seems like the team of people managing PyPI could benefit from the > addition of someone with more of a sysadmin background (e.g., to help with > installing a monitor on the server). I have some experience with mod_wsgi (and this is what we're using for the hg.python.org setup, I think). I'd be happy to help out more with that, if it helps (though most of my experience is not with the Debian-based distributions). Cheers, Dirkjan From fijall at gmail.com Wed Jul 21 23:43:15 2010 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 21 Jul 2010 23:43:15 +0200 Subject: [Python-Dev] Unladen swallow status In-Reply-To: References: Message-ID: On Wed, Jul 21, 2010 at 6:50 PM, Reid Kleckner wrote: > On Wed, Jul 21, 2010 at 8:11 AM, Tim Golden wrote: >> Brett suggested that >> the Unladen Swallow merge to trunk was waiting for some work to complete >> on the JIT compiler and Georg, as release manager for 3.2, confirmed that >> Unladen Swallow would not be merged before 3.3. > > Yeah, this has slipped. ?I have patches that need review, and Jeff and > Collin have been distracted with other work. ?Hopefully when one of > them gets around to that, I can proceed with the merge without > blocking on them. > > Reid The merge py3k-jit to trunk? Cheers, fijal From collinwinter at google.com Wed Jul 21 23:45:10 2010 From: collinwinter at google.com (Collin Winter) Date: Wed, 21 Jul 2010 14:45:10 -0700 Subject: [Python-Dev] Unladen swallow status In-Reply-To: References: Message-ID: On Wed, Jul 21, 2010 at 2:43 PM, Maciej Fijalkowski wrote: > On Wed, Jul 21, 2010 at 6:50 PM, Reid Kleckner wrote: >> On Wed, Jul 21, 2010 at 8:11 AM, Tim Golden wrote: >>> Brett suggested that >>> the Unladen Swallow merge to trunk was waiting for some work to complete >>> on the JIT compiler and Georg, as release manager for 3.2, confirmed that >>> Unladen Swallow would not be merged before 3.3. >> >> Yeah, this has slipped. ?I have patches that need review, and Jeff and >> Collin have been distracted with other work. ?Hopefully when one of >> them gets around to that, I can proceed with the merge without >> blocking on them. >> >> Reid > > The merge py3k-jit to trunk? I believe he's talking about the merger of the Unladen tree into the py3k-jit branch. Collin From fijall at gmail.com Wed Jul 21 23:48:27 2010 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 21 Jul 2010 23:48:27 +0200 Subject: [Python-Dev] Unladen swallow status In-Reply-To: References: Message-ID: On Wed, Jul 21, 2010 at 11:45 PM, Collin Winter wrote: > On Wed, Jul 21, 2010 at 2:43 PM, Maciej Fijalkowski wrote: >> On Wed, Jul 21, 2010 at 6:50 PM, Reid Kleckner wrote: >>> On Wed, Jul 21, 2010 at 8:11 AM, Tim Golden wrote: >>>> Brett suggested that >>>> the Unladen Swallow merge to trunk was waiting for some work to complete >>>> on the JIT compiler and Georg, as release manager for 3.2, confirmed that >>>> Unladen Swallow would not be merged before 3.3. >>> >>> Yeah, this has slipped. ?I have patches that need review, and Jeff and >>> Collin have been distracted with other work. ?Hopefully when one of >>> them gets around to that, I can proceed with the merge without >>> blocking on them. >>> >>> Reid >> >> The merge py3k-jit to trunk? > > I believe he's talking about the merger of the Unladen tree into the > py3k-jit branch. > > Collin > If so, how does this relate to 3.2 and 3.3 release schedule? Cheers, fijal From greg.ewing at canterbury.ac.nz Thu Jul 22 02:23:45 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 22 Jul 2010 12:23:45 +1200 Subject: [Python-Dev] mkdir -p in python In-Reply-To: References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> <4C464140.3040509@canterbury.ac.nz> Message-ID: <4C478F91.7060607@canterbury.ac.nz> Ray Allen wrote: > I think in this case, the errno is generate by c standard library, which > can be seen as cross-platform. But I'm never sure how standard the actual error numbers are, though. I tend to think of them as coming from Unix-land, and thus fair game for getting screwed around with on Windows. Am I worrying too much? -- Greg From steve at holdenweb.com Thu Jul 22 03:26:24 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 22 Jul 2010 02:26:24 +0100 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: On 7/21/2010 6:45 PM, Alexander Belopolsky wrote: > On Wed, Jul 21, 2010 at 1:42 PM, Benjamin Peterson wrote: > .. >> True, but the tests in that file are (mostly?) all about line tracing. >> > > Hopefully this will change soon. :-) At which point the renaming would be justified. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From dreamingforward at gmail.com Thu Jul 22 03:28:24 2010 From: dreamingforward at gmail.com (average) Date: Wed, 21 Jul 2010 19:28:24 -0600 Subject: [Python-Dev] Python-dev signal-to-noise processing question In-Reply-To: <20100720185944.GA10496@phd.pp.ru> References: <20100719162729.GF21891@phd.pp.ru> <20100720105928.GA4289@phd.pp.ru> <20100720185944.GA10496@phd.pp.ru> Message-ID: >> 1. I suggested one improvement to the canned response in my previous >> post: expand 'using' to 'using or understanding'. > > ? I changed wording to "if you're having problems learning, understanding > or using Python" I think it's critical to disambiguate between questions about "using and programming with the Python language" versus working with the actual code base that makes Python. Your boilerplate is appropriate for the former, but completely confusing for the latter. As to your question of how best to handle inquiries from the blue or "noisy questions", I personally prefer the following (only slightly tongue-in-cheek): ...After a sufficient period of waiting, say a day or two with no response: "busy busy busy, the gnomes must be hard at work. You've no answer. That means that your question(s) either isn't interesting enough in an obvious way, or it's simply off-topic. If you feel you been unfairly ignored, you might try rewording the question, doing some research to show that you've checked the obvious, or try the following venues: comp.lang.python, for blah blah blah python-ideas,for pitching your new lambda form, to etc, etc" In any case, Mr Crute gave a quite perfect answer, not that such detail should be exptected. Cheers, Marcos From steve at holdenweb.com Thu Jul 22 03:31:18 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 22 Jul 2010 02:31:18 +0100 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: References: <4C470E2B.5060406@timgolden.me.uk> Message-ID: <4C479F66.7000007@holdenweb.com> On 7/21/2010 6:21 PM, Tarek Ziad? wrote: > Thanks Tim ! > > On Wed, Jul 21, 2010 at 5:11 PM, Tim Golden wrote: > > [..] >> >> Several people outlined the recent heated discussion over the addition >> of a checkbox to the PyPI user-registration pages. Tarek has already >> undertaken to patch PyPI to move the checkbox back one step, allowing >> existing distutils users to register from the command line. > > Yes, so the problem goes away and we are conforming to the legal requirements. > >> At the same >> time, Brett advised removing that functionality from distutils2 as >> signing up on a web page is no great hardship. > > We will remove/simplify the register command, and get rid of that part. > > In the meantime, to try to avoid this problem in the future, since > quite a few projects > depend on PyPI now, I would like to suggest that a PEP of the various > protocols PyPI has, is created. > > We do have some documentation here and there about the XML-RPC > interface, the HTTP interface, etc., but we should have a single > document describing the PyPI APIs, and maybe adding an API version > number, so it can evolve smoothly. > > This would also be helpful for the people that are writing their own > PyPI software, and ensure that distutils2 can work out of the box with > any PyPI server out there. > > Tarek +1 Readers may not be aware that there is a current attempt to try to catalog and to some extent unify the different web presences that the Foundation maintains. This is why I am copying this message to Ethan Schoonover, who leads that effort. PyPi will doubtless be an increasingly important part of that presence, so I hope it can be integrated with the rest somehow. I do not imagine that this will be an easy issue to resolve, but a journey of a thousand miles begins with a single step. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Thu Jul 22 03:33:33 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 22 Jul 2010 02:33:33 +0100 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: References: <4C470E2B.5060406@timgolden.me.uk> Message-ID: <4C479FED.1030807@holdenweb.com> On 7/21/2010 10:40 PM, Dirkjan Ochtman wrote: > On Wed, Jul 21, 2010 at 19:38, Ian Bicking wrote: >> From what I've been able to tell from afar, I strongly suspect PyPI's >> downtimes would be greatly reduced with a move to mod_wsgi (currently it is >> using mod_fcgi, and most downtime is solved with an Apache restart -- >> mod_wsgi generally recovers from these problems without intervention). >> Martin attempted this at one time but ran into some installation problems. >> It seems like the team of people managing PyPI could benefit from the >> addition of someone with more of a sysadmin background (e.g., to help with >> installing a monitor on the server). > > I have some experience with mod_wsgi (and this is what we're using for > the hg.python.org setup, I think). I'd be happy to help out more with > that, if it helps (though most of my experience is not with the > Debian-based distributions). > > Cheers, > > Dirkjan Copying this message to the mod_wsgi author/maintainer, who should be able to give us sound advice on this (as should Ian, but he's a secondary server for the data :-). regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Thu Jul 22 03:31:18 2010 From: steve at holdenweb.com (Steve Holden) Date: Thu, 22 Jul 2010 02:31:18 +0100 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: References: <4C470E2B.5060406@timgolden.me.uk> Message-ID: <4C479F66.7000007@holdenweb.com> On 7/21/2010 6:21 PM, Tarek Ziad? wrote: > Thanks Tim ! > > On Wed, Jul 21, 2010 at 5:11 PM, Tim Golden wrote: > > [..] >> >> Several people outlined the recent heated discussion over the addition >> of a checkbox to the PyPI user-registration pages. Tarek has already >> undertaken to patch PyPI to move the checkbox back one step, allowing >> existing distutils users to register from the command line. > > Yes, so the problem goes away and we are conforming to the legal requirements. > >> At the same >> time, Brett advised removing that functionality from distutils2 as >> signing up on a web page is no great hardship. > > We will remove/simplify the register command, and get rid of that part. > > In the meantime, to try to avoid this problem in the future, since > quite a few projects > depend on PyPI now, I would like to suggest that a PEP of the various > protocols PyPI has, is created. > > We do have some documentation here and there about the XML-RPC > interface, the HTTP interface, etc., but we should have a single > document describing the PyPI APIs, and maybe adding an API version > number, so it can evolve smoothly. > > This would also be helpful for the people that are writing their own > PyPI software, and ensure that distutils2 can work out of the box with > any PyPI server out there. > > Tarek +1 Readers may not be aware that there is a current attempt to try to catalog and to some extent unify the different web presences that the Foundation maintains. This is why I am copying this message to Ethan Schoonover, who leads that effort. PyPi will doubtless be an increasingly important part of that presence, so I hope it can be integrated with the rest somehow. I do not imagine that this will be an easy issue to resolve, but a journey of a thousand miles begins with a single step. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From stephen at xemacs.org Thu Jul 22 03:56:00 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 22 Jul 2010 10:56:00 +0900 Subject: [Python-Dev] What to do with languishing patches? In-Reply-To: References: <4C45C4A6.5020208@netwok.org> <20100720181640.D89CA2007A2@kimball.webabinitio.net> <20100720190527.GB10496@phd.pp.ru> <20100720193817.GC10496@phd.pp.ru> <20100720204319.GA16407@phd.pp.ru> <87wrspxt5a.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87mxtkxosv.fsf@uwakimon.sk.tsukuba.ac.jp> Alexander Belopolsky writes: > On Wed, Jul 21, 2010 at 2:09 AM, Stephen J. Turnbull wrote: > .. > > ?> ? ?In this particular case I'd rather tend to agree - an editable > > ?> single-line box to enter space-*and*-comma-separated modules list > > ?> would be the best interface. > > > > For active developers, yes. ?But this is unhelpful for people who > > aren't hyper-familiar with the component list or who can't spell. > > It does not have to be either or. We can have a text box and a > link/button next to it which will pop up a window with a selectable This is what Roundup multilinks do anyway. I just don't want the "best" (cf the post I'm replying to) to get in the way of the "good". From greg.ewing at canterbury.ac.nz Thu Jul 22 04:52:37 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 22 Jul 2010 14:52:37 +1200 Subject: [Python-Dev] Python-dev signal-to-noise processing question In-Reply-To: <4C46DD7F.2090102@v.loewis.de> References: <20100719162729.GF21891@phd.pp.ru> <20100720105928.GA4289@phd.pp.ru> <4C46DD7F.2090102@v.loewis.de> Message-ID: <4C47B275.6040905@canterbury.ac.nz> On 21/07/10 23:43, "Martin v. L?wis" wrote: > IIUC, he wanted to know how Python > handles SIGKILL, when the hole point of SIGKILL is that you cannot > handle it. No, I think he wanted to know how Python disallows attempting to set a handler for SIGKILL, when he couldn't find any code that special-cases SIGKILL in the implementation of signal(). Although he didn't express the question all that clearly, so it's hard to tell. -- Greg From stephen at xemacs.org Thu Jul 22 06:00:16 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 22 Jul 2010 13:00:16 +0900 Subject: [Python-Dev] Python signal processing question In-Reply-To: <4C44E561.9060501@canterbury.ac.nz> References: <4C44E561.9060501@canterbury.ac.nz> Message-ID: <87iq48xj1r.fsf@uwakimon.sk.tsukuba.ac.jp> Greg Ewing writes: > Scott McCarty wrote: > > All, I have searched everywhere (mostly the code and a little google) > > and I cannot understand where the SIGKILL signal gets checked when it is > > set as a handler. > > Possibly it's not being checked at all by Python, but > is being rejected by the system call. The Darwin man > page says this about signal(3) in the ERRORS section: > > [EINVAL] An attempt is made to ignore or supply a handler for > SIGKILL or SIGSTOP. The Linux signal(2) manpage just says EINVAL signum is invalid. But I suppose that could be stretched to cover "SIGKILL, SIGSTOP, or not a signal number known to the kernel". > Not sure why it gets reported as a RuntimeError rather > than an OSError, though. My understanding of OSError is that the OS is saying "sorry, what you tried to do is perfectly reasonable under some circumstances, but you can't do that now." ENOMEM, EPERM, ENOENT etc fit this model. RuntimeError OTOH is basically saying "You should know better than to try that!" EINVAL fits this model. From reid.kleckner at gmail.com Thu Jul 22 07:09:49 2010 From: reid.kleckner at gmail.com (Reid Kleckner) Date: Wed, 21 Jul 2010 22:09:49 -0700 Subject: [Python-Dev] Timeouts for subprocess module Message-ID: Hi python-dev, I've been working through a patch to add timeouts to the subprocess module: http://bugs.python.org/issue5673 It's gotten a fair amount of review, and I'm planning to commit it. Since it's my first contribution, I'm taking Georg's suggestion to send mail to python-dev to see if anyone objects. If not, I'll commit it to the py3k branch tomorrow night. Reid From ben+python at benfinney.id.au Thu Jul 22 07:52:45 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 22 Jul 2010 15:52:45 +1000 Subject: [Python-Dev] mkdir -p in python References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> <4C464140.3040509@canterbury.ac.nz> <4C478F91.7060607@canterbury.ac.nz> Message-ID: <87d3ug6p1u.fsf@benfinney.id.au> Greg Ewing writes: > Ray Allen wrote: > > > I think in this case, the errno is generate by c standard library, > > which can be seen as cross-platform. > > But I'm never sure how standard the actual error numbers are, though. You can use them by name, and in fact I strongly recommend it: import os import errno try: os.makedirs(path) except OSError, exc: if exc.errno != errno.EEXIST: raise > I tend to think of them as coming from Unix-land, and thus fair game > for getting screwed around with on Windows. Am I worrying too much? There are some that differ between different OSen, true. Using them by name avoids dealing with *different* numbers (since the ?errno? module's attributes will use the same name for semantically the same error), leaving only the problem of errors that are *missing* on some platforms. EEXIST is common to all of them though, AFAIK. -- \ ?Sittin' on the fence, that's a dangerous course / You can even | `\ catch a bullet from the peace-keeping force? ?Dire Straits, | _o__) _Once Upon A Time In The West_ | Ben Finney From glyph at twistedmatrix.com Thu Jul 22 08:07:18 2010 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Thu, 22 Jul 2010 02:07:18 -0400 Subject: [Python-Dev] Python signal processing question In-Reply-To: <87iq48xj1r.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4C44E561.9060501@canterbury.ac.nz> <87iq48xj1r.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <1B6131B2-1152-418E-8DBA-1E4EC6F24A82@twistedmatrix.com> On Jul 22, 2010, at 12:00 AM, Stephen J. Turnbull wrote: > My understanding of OSError is that the OS is saying "sorry, what you > tried to do is perfectly reasonable under some circumstances, but you > can't do that now." ENOMEM, EPERM, ENOENT etc fit this model. > > RuntimeError OTOH is basically saying "You should know better than to > try that!" EINVAL fits this model. That is not my understanding of OSError at all, especially given that I have seen plenty of OSErrors that have EINVAL set by various things. OSError's docstring specifically says "OS system call failed.", and that's the way I've already understood it: you made a syscall and got some kind of error. Python _mostly_ avoids classifying OSErrors into different exception types in other APIs. The selection of RuntimeError in this particular case seems somewhat random and ad-hoc, given that out-of-range signal values give ValueError while SIGKILL and SIGSTOP give RuntimeError. The RuntimeError's args start with "22" (which I assume is supposed to mean "EINVAL") but it doesn't have an 'errno' attribute as an OSError would. The ValueError doesn't relate to an errno at all. Nowhere does the documentation say "raises OSError or ValueError or TypeError or RuntimeError whose args[0] may be an errno". To be clear, this particular area doesn't bother me. I've been dealing with weird and puzzling signal-handling issues in Python for years and years and this dusty corner of the code has never come up. I did want to reply to this particular message, though, because I *would* eventually like the exception hierarchy raised by certain stdlib functions to be more thoroughly documented and coherent, but a prerequisite to that is to avoid rationalizing the random potpourri of exception types that certain parts of the stdlib emit. I think signal.signal is one such part. From phd at phd.pp.ru Thu Jul 22 08:09:09 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Thu, 22 Jul 2010 10:09:09 +0400 Subject: [Python-Dev] Python-dev signal-to-noise processing question In-Reply-To: References: <20100719162729.GF21891@phd.pp.ru> <20100720105928.GA4289@phd.pp.ru> <20100720185944.GA10496@phd.pp.ru> Message-ID: <20100722060909.GB14498@phd.pp.ru> On Wed, Jul 21, 2010 at 07:28:24PM -0600, average wrote: > As to your question of how best to handle inquiries from the blue or > "noisy questions", I personally prefer the following (only slightly > tongue-in-cheek): > > ...After a sufficient period of waiting, say a day or two with no response: Ok, I'll wait a bit longer. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From greg.ewing at canterbury.ac.nz Thu Jul 22 09:55:19 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 22 Jul 2010 19:55:19 +1200 Subject: [Python-Dev] Python signal processing question In-Reply-To: <1B6131B2-1152-418E-8DBA-1E4EC6F24A82@twistedmatrix.com> References: <4C44E561.9060501@canterbury.ac.nz> <87iq48xj1r.fsf@uwakimon.sk.tsukuba.ac.jp> <1B6131B2-1152-418E-8DBA-1E4EC6F24A82@twistedmatrix.com> Message-ID: <4C47F967.7000002@canterbury.ac.nz> Glyph Lefkowitz wrote: > The selection of RuntimeError in this particular case seems somewhat random and ad-hoc, Indeed -- usually a RuntimeError indicates that something concerning the internals of Python itself is screwed up, e.g. attempting to execute invalid bytecode. The fact that it turns up here may be a historical accident. I can imagine that the signal module dates back to the very early days of Python, when the conventions for error reporting were not so well formed. Maybe this is something that could be considered in the exception hierarchy revamp proposal that was posted recently? -- Greg From brett at python.org Thu Jul 22 09:51:57 2010 From: brett at python.org (Brett Cannon) Date: Thu, 22 Jul 2010 08:51:57 +0100 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: <20100721175819.3b037ad9@pitrou.net> References: <4C470E2B.5060406@timgolden.me.uk> <20100721175819.3b037ad9@pitrou.net> Message-ID: On Wed, Jul 21, 2010 at 16:58, Antoine Pitrou wrote: > On Wed, 21 Jul 2010 11:42:00 -0400 > Jesse Noller wrote: > > On Wed, Jul 21, 2010 at 11:11 AM, Tim Golden > wrote: > > [...snip...] > > > A messy discussion turned on the question of garbage collection of > module > > > objects, and the order in which finalisers are called if at all, > especially > > > when reference cycles exist. Marc Andre was proposing a __cleanup__ > magic > > > function > > > for Python modules, which would enable the implementer to define the > order > > > in which resources are released / closed down. This is quite a subtle > area > > > and raised the issue of unfinalised objects in a reference cycle whose > > > memory has been freed out from under them but which still exist. Martin > > > described > > > the Java approach where finalisers are called once and then flagged so > > > they are not called again even if their object is resurrected. This > sounded > > > like a useful approach for Python but would break code which expected > to > > > be able to resurrect an object during its __del__ method... which is > not > > > expected to account for much code. > > > > > > Guido pointed out that no-one can be expected to hold enough of the > > > complexities > > > of this area of Python's implementation in their head, and that an > > > implementation > > > of some sort would need to be written so that the corner-cases could > emerge. > > > > FWIW; I'm currently dealing with a bug in this area w.r.t > > multiprocessing and threads and modules we have imported vanishing due > > to this issue. I'm interested in hearing more. > > One common resolution is to not use a __del__ method, but instead a > weakref callback which will do the necessary cleanup of a certain set > of resources. This is of course not applicable in all situations. That's an option. I just remember Tim bringing up something about that approach that didn't quite work as a complete replacement for __del__. Basically the whole setting a module's globals to None was done before gc came into the language. Now that it's there it seems that it might work to simply let gc clean up the module itself. But this brings in the question of how to order calling finalizers. It seemed the room had no specific answer other than it might just have to be near random as figuring out a reasonable order is hard. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Thu Jul 22 10:43:28 2010 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 22 Jul 2010 10:43:28 +0200 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: References: <4C470E2B.5060406@timgolden.me.uk> <20100721175819.3b037ad9@pitrou.net> Message-ID: On Thu, Jul 22, 2010 at 9:51 AM, Brett Cannon wrote: > > > On Wed, Jul 21, 2010 at 16:58, Antoine Pitrou wrote: >> >> On Wed, 21 Jul 2010 11:42:00 -0400 >> Jesse Noller wrote: >> > On Wed, Jul 21, 2010 at 11:11 AM, Tim Golden >> > wrote: >> > [...snip...] >> > > A messy discussion turned on the question of garbage collection of >> > > module >> > > objects, and the order in which finalisers are called if at all, >> > > especially >> > > when reference cycles exist. Marc Andre was proposing a __cleanup__ >> > > magic >> > > function >> > > for Python modules, which would enable the implementer to define the >> > > order >> > > in which resources are released / closed down. This is quite a subtle >> > > area >> > > and raised the issue of unfinalised objects in a reference cycle whose >> > > memory has been freed out from under them but which still exist. >> > > Martin >> > > described >> > > the Java approach where finalisers are called once and then flagged so >> > > they are not called again even if their object is resurrected. This >> > > sounded >> > > like a useful approach for Python but would break code which expected >> > > to >> > > be able to resurrect an object during its __del__ method... which is >> > > not >> > > expected to account for much code. >> > > >> > > Guido pointed out that no-one can be expected to hold enough of the >> > > complexities >> > > of this area of Python's implementation in their head, and that an >> > > implementation >> > > of some sort would need to be written so that the corner-cases could >> > > emerge. >> > >> > FWIW; I'm currently dealing with a bug in this area w.r.t >> > multiprocessing and threads and modules we have imported vanishing due >> > to this issue. I'm interested in hearing more. >> >> One common resolution is to not use a __del__ method, but instead a >> weakref callback which will do the necessary cleanup of a certain set >> of resources. This is of course not applicable in all situations. > > That's an option. I just remember Tim bringing up something about that > approach that didn't quite work as a complete replacement for __del__. > Basically the whole setting a module's globals to None was done before gc > came into the language. Now that it's there it seems that it might work to > simply let gc clean up the module itself. But this brings in the question of > how to order calling finalizers. It seemed the room had no specific answer > other than it might just have to be near random as figuring out a reasonable > order is hard. If you have a cycle, there is no other way than breaking it at random, right? Cheers, fijal From stephen at xemacs.org Thu Jul 22 10:50:00 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 22 Jul 2010 17:50:00 +0900 Subject: [Python-Dev] Python signal processing question In-Reply-To: <4C47F967.7000002@canterbury.ac.nz> References: <4C44E561.9060501@canterbury.ac.nz> <87iq48xj1r.fsf@uwakimon.sk.tsukuba.ac.jp> <1B6131B2-1152-418E-8DBA-1E4EC6F24A82@twistedmatrix.com> <4C47F967.7000002@canterbury.ac.nz> Message-ID: <87hbjryk7b.fsf@uwakimon.sk.tsukuba.ac.jp> Greg Ewing writes: > Glyph Lefkowitz wrote: > > > The selection of RuntimeError in this particular case seems > > somewhat random and ad-hoc, Well, I guess we'd have to catch the person who wrote the code and ask. > Maybe this is something that could be considered in the > exception hierarchy revamp proposal that was posted recently? I think that's Antoine's PEP 3151. Interestingly, he doesn't mention EINVAL at all. http://www.python.org/dev/peps/pep-3151/ From barry at python.org Thu Jul 22 10:58:09 2010 From: barry at python.org (Barry Warsaw) Date: Thu, 22 Jul 2010 10:58:09 +0200 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: <4C470E2B.5060406@timgolden.me.uk> References: <4C470E2B.5060406@timgolden.me.uk> Message-ID: <20100722105809.317bbf1f@snowdog> On Jul 21, 2010, at 04:11 PM, Tim Golden wrote: >The email module needs some work in Py3. David Murray has been given >some money by the PSF but needs more from other sources to complete >the work. This is hampered by the legalities around commercial >organisations making donations to not-for-profits when those donations >are earmarked. Various suggestions were put forward with no-one sure >of the legal issues. Guido suggested that we should move forward >rather than stall for want of legal advice. +1 please! Or contact the folks at the FSF for their experiences. I know that they accept tax exempt donations targeted for specific projects. Of course, when they disburse funds to developers, those developers may incur tax liabilities, but that's a different matter. >Martin spoke about the state of the stable ABI PEP, indicating that he >was targetting 3.2. This work would reduce the need to recompile >extension modules separately on Windows for every version of Python -- >something especially pertinent when code has been orphaned but is >still useful. The versioned .so files PEP being worked out by Barry >Warsaw overlaps with this PEP and would only be useful for extensions >which don't target the stable ABI. Right. Expect an update to PEP 3149 later today, along with more followups to that thread, and an updated patch. I'd also like to see the stable ABI make it into Python 3.2. It's a longer term project in the sense that it will take time for extensions to be written against the ABI. Getting it into Python 3.2 will allow that process to start, but I think even that doesn't eliminate the need for versioned .so files because there will always be extensions accessing API outside PEP 384. I'd still like to help work on the implementation of PEP 384. It would be helpful for folks who care about it to comment on the substance of the PEP. -Barr -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From stephen at xemacs.org Thu Jul 22 11:02:33 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 22 Jul 2010 18:02:33 +0900 Subject: [Python-Dev] Python-dev signal-to-noise processing question In-Reply-To: <20100722060909.GB14498@phd.pp.ru> References: <20100719162729.GF21891@phd.pp.ru> <20100720105928.GA4289@phd.pp.ru> <20100720185944.GA10496@phd.pp.ru> <20100722060909.GB14498@phd.pp.ru> Message-ID: <87fwzbyjme.fsf@uwakimon.sk.tsukuba.ac.jp> Oleg Broytman writes: > On Wed, Jul 21, 2010 at 07:28:24PM -0600, average wrote: > > As to your question of how best to handle inquiries from the blue or > > "noisy questions", I personally prefer the following (only slightly > > tongue-in-cheek): > > > > ...After a sufficient period of waiting, say a day or two with no response: > > Ok, I'll wait a bit longer. I don't think that's a good idea. It just encourages people to give a response on python-dev plus follow-ups, and if they give an answer without saying "you'd better ask this on comp.lang.python", when you do, you look like a netcop rather than being helpful. The mail you originally sent was sufficiently polite and clear IMO, but anything can be improved and I'm glad you took up the suggestions on the wording. OTOH I think as quick as possible an answer is a good idea here. It saves the intended audience the thought about whether to reply or not, and an instant, constructive answer says that somebody cares. The message should be "The people who will answer your question on python-dev are also on comp.lang.python, as well as many more (you may get a faster, and probably more complete, answer on comp.lang.python). The people on python-dev don't need to see the answer (they already know it), but the people on comp.lang.python are likely to be happy to learn it." From solipsis at pitrou.net Thu Jul 22 12:33:02 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 22 Jul 2010 12:33:02 +0200 Subject: [Python-Dev] EINVAL References: <4C44E561.9060501@canterbury.ac.nz> <87iq48xj1r.fsf@uwakimon.sk.tsukuba.ac.jp> <1B6131B2-1152-418E-8DBA-1E4EC6F24A82@twistedmatrix.com> <4C47F967.7000002@canterbury.ac.nz> <87hbjryk7b.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20100722123302.066605f6@pitrou.net> On Thu, 22 Jul 2010 17:50:00 +0900 "Stephen J. Turnbull" wrote: > Greg Ewing writes: > > Glyph Lefkowitz wrote: > > > > > The selection of RuntimeError in this particular case seems > > > somewhat random and ad-hoc, > > Well, I guess we'd have to catch the person who wrote the code and > ask. > > > Maybe this is something that could be considered in the > > exception hierarchy revamp proposal that was posted recently? > > I think that's Antoine's PEP 3151. Interestingly, he doesn't mention > EINVAL at all. > > http://www.python.org/dev/peps/pep-3151/ That's right. It is based on a survey of existing exception-catching code in the stdlib. There's only one match in the whole Lib/ subtree: $ grep -r EINVAL Lib/ Lib/plat-sunos5/STROPTS.py:968:EINVAL = 22 I guess EINVAL would most often indicate a programming error, which is why it doesn't get handled specifically in except clauses. Semantically, EINVAL seems close to ValueError. If I search inside the man pages here, I get excerpts such as: - wait(2): EINVAL The options argument was invalid. - gettimeofday(2): EINVAL Timezone (or something else) is invalid. - socket(2): EINVAL Unknown protocol, or protocol family not available. EINVAL Invalid flags in type. - rename(2): EINVAL The new pathname contained a path prefix of the old, or, more generally, an attempt was made to make a directory a subdirectory of itself. Regards Antoine. From g.brandl at gmx.net Thu Jul 22 13:34:35 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 22 Jul 2010 12:34:35 +0100 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: <20100713163544.5f3a60d2@pitrou.net> References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> <20100713163544.5f3a60d2@pitrou.net> Message-ID: Am 13.07.2010 15:35, schrieb Antoine Pitrou: > On Tue, 13 Jul 2010 15:20:23 +0100 > Michael Foord wrote: >> On 13/07/2010 15:17, Reid Kleckner wrote: >> > On Mon, Jul 12, 2010 at 2:07 PM, Nick Coghlan wrote: >> > >> >> MRAB's module offers a superset of re's features rather than a subset >> >> though, so once it has had more of a chance to bake on PyPI it may be >> >> worth another look. >> >> >> > I feel like the new module is designed to replace the current re >> > module, and shouldn't need to spend time in PyPI. A faster regex >> > library isn't going to motivate users to add external dependencies to >> > their projects. >> > >> > >> If the backwards compatibility issues can be addressed and MRAB is >> willing to remain as maintainer then the advantages seem well worth it >> to me. > > To me as well. The code needs a full review before integrating, though. FWIW, I've now run the Pygments test suite (Pygments has about 2500 regular expressions that are exercised there) and only had two problems: * Scoped flags: A few lexers use (?s) and similar flags at the end of the expression, which has no effect in regex currently. * POSIX character classes: One regex used a class '[][:xyz]', so the [: was seen as the start of a character class. I'm not sure how common this is, as most people seem to escape brackets in character classes. Also, it gives a clear error on regex.compile(), not "mysterious" failures. Timings (seconds to run the test suite): re 26.689 26.015 26.008 regex 26.066 25.797 25.865 So, I thought there wasn't a difference in performance for this use case (which is compiling a lot of regexes and matching most of them only a few times in comparison). However, I found that looking at the regex caching is very important in this case: re._MAXCACHE is by default set to 100, and regex._MAXCACHE to 1024. When I set re._MAXCACHE to 1024 before running the test suite, I get times around 18 (!) seconds for re. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From phd at phd.pp.ru Thu Jul 22 12:50:59 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Thu, 22 Jul 2010 14:50:59 +0400 Subject: [Python-Dev] Python-dev signal-to-noise processing question In-Reply-To: <87fwzbyjme.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20100719162729.GF21891@phd.pp.ru> <20100720105928.GA4289@phd.pp.ru> <20100720185944.GA10496@phd.pp.ru> <20100722060909.GB14498@phd.pp.ru> <87fwzbyjme.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20100722105059.GE27843@phd.pp.ru> On Thu, Jul 22, 2010 at 06:02:33PM +0900, Stephen J. Turnbull wrote: > Oleg Broytman writes: > > On Wed, Jul 21, 2010 at 07:28:24PM -0600, average wrote: > > > As to your question of how best to handle inquiries from the blue or > > > "noisy questions", I personally prefer the following (only slightly > > > tongue-in-cheek): > > > > > > ...After a sufficient period of waiting, say a day or two with no response: > > > > Ok, I'll wait a bit longer. > > I don't think that's a good idea. It just encourages people to give a > response on python-dev plus follow-ups, and if they give an answer > without saying "you'd better ask this on comp.lang.python", when you > do, you look like a netcop rather than being helpful. Yes, that's a kind of a problem. Not a big one - I live in the "wrong" time zone and can afford to wait a few hours. Certainly not a few days. > The mail you originally sent was sufficiently polite and clear IMO, > but anything can be improved and I'm glad you took up the suggestions > on the wording. > > OTOH I think as quick as possible an answer is a good idea here. It > saves the intended audience the thought about whether to reply or not, > and an instant, constructive answer says that somebody cares. The > message should be "The people who will answer your question on > python-dev are also on comp.lang.python, as well as many more (you may > get a faster, and probably more complete, answer on comp.lang.python). > The people on python-dev don't need to see the answer (they already > know it), but the people on comp.lang.python are likely to be happy to > learn it." Thank you. I'll think how to add something like this so that my boilerplate wouldn't become too big, Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From guido at python.org Thu Jul 22 12:53:12 2010 From: guido at python.org (Guido van Rossum) Date: Thu, 22 Jul 2010 11:53:12 +0100 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C36491C.2050300@mrabarnett.plus.com> <4C366EC2.3000203@mrabarnett.plus.com> Message-ID: On Fri, Jul 16, 2010 at 6:08 PM, Georg Brandl wrote: > Nevertheless, the authoritative reference for our regex engine is its > docs, i.e. http://docs.python.org/library/re.html -- and that states > clearly that inline flags apply to the whole regex. > >> I think with a new regex implementation, not all of this "historical" >> semantics must be copied, unless there are major real usecases, which >> would be affected by this. > > As I already said, I *have* seen this in real code. ?As MRAB indicated, > this was the only silent change in semantics as compared to the old > regex engine. ?If we replace re by regex, which I think is the only > way to get the new features in the stdlib, changing this one aspect is > a) not backwards compatible and b) in a subtle way that forces everyone > to review his/her regular expressions. ?That's definitely not > acceptable. I wonder if the solution could be a new flag that you have to specify if you want the flags to apply locally only. Let's say add 'L' to the flags. Then for existing code (not using 'L') the flags should apply globally, while someone wanting flags to apply to only part of a regex can add 'L' to the flags syntax. Is that acceptable? -- --Guido van Rossum (python.org/~guido) From sbrunthaler at gmail.com Thu Jul 22 13:22:48 2010 From: sbrunthaler at gmail.com (stefan brunthaler) Date: Thu, 22 Jul 2010 13:22:48 +0200 Subject: [Python-Dev] Python 3 optimizations... Message-ID: Hello, during the last year, I have developed a couple of quickening-based optimizations for the Python 3.1 interpreter. As part of my PhD programme, I have published a first technique that combines quickening with inline caching at this year's ECOOP, and subsequently extended this technique to optimize several load instructions as well as eliminate redundant reference counting operations from instructions, which has been accepted for publication for an upcoming conference. I have a working prototype combining all of these optimizations that achieves a maximum speedup of 2.18 on the spectralnorm benchmark of the computer language benchmarks game. Early measurements on the Unladen Swallow django benchmark (with Martin von Loewis' patch for Python 3) achieve a speedup of about 1.3. Both speedups were measured on an i7 920 when combined with the threaded code/computed goto optimization enabled, and normalized by the standard Python 3.1 interpreter with all optimizations disabled. Since all of these optimizations are purely interpretative, they have next-to-no additional memory requirements and do not incur extensive warm-up costs. I wonder whether you would be interested in integrating these optimizations with the Python 3 distribution, hence this mail. I could send copies of the papers, as well as provide my prototype source code to interested members of the python development community. Have a nice day, --stefan From solipsis at pitrou.net Thu Jul 22 14:00:46 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 22 Jul 2010 14:00:46 +0200 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: References: <4C470E2B.5060406@timgolden.me.uk> <20100721175819.3b037ad9@pitrou.net> Message-ID: <20100722140046.5e7551a1@pitrou.net> On Thu, 22 Jul 2010 08:51:57 +0100 Brett Cannon wrote: > > That's an option. I just remember Tim bringing up something about that > approach that didn't quite work as a complete replacement for __del__. > > Basically the whole setting a module's globals to None was done before gc > came into the language. Now that it's there it seems that it might work to > simply let gc clean up the module itself. There is a patch at http://bugs.python.org/issue812369 for GC-based module shutdown, but it doesn't actually remove the setting of module globals to None. I think further testing and experimentation would be required to validate it. Regards Antoine. From steve at pearwood.info Thu Jul 22 14:22:52 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 22 Jul 2010 22:22:52 +1000 Subject: [Python-Dev] Python-dev signal-to-noise processing question In-Reply-To: <87fwzbyjme.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20100722060909.GB14498@phd.pp.ru> <87fwzbyjme.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <201007222222.52796.steve@pearwood.info> On Thu, 22 Jul 2010 07:02:33 pm Stephen J. Turnbull wrote: > OTOH I think as quick as possible an answer is a good idea here. It > saves the intended audience the thought about whether to reply or > not, and an instant, constructive answer says that somebody cares. +1 I think that waiting "a day or two" as suggested by Average does nobody any good. It doesn't help the poster, who will almost certainly have given up by then, and may have concluded that we're stuck-up and rude for not answering a polite question. It doesn't help those who choose to reply, because we're sitting around wondering if we've waiting long enough. If you consider posting a question to a list without making any effort to determine whether it is on-topic or not to be a social faux pas, and therefore worthy of a mild rebuke, the longer you wait the less effective the negative reinforcement is. Even if you don't see it like that, and just want to point the poster in the right direction, there's still no advantage in waiting. But frankly, I think we're worrying too much. What exactly are we worried about? As I see it, the worst that can happen is a false negative, somebody will say "You've posted to the wrong list" and will be corrected by somebody else who says "No, you've misunderstood the question, this is the right list". No harm done. > The message should be "The people who will answer your question on > python-dev are also on comp.lang.python, as well as many more (you > may get a faster, and probably more complete, answer on > comp.lang.python). The people on python-dev don't need to see the > answer (they already know it), but the people on comp.lang.python are > likely to be happy to learn it." We don't need to make excuses for why we don't give the answer here. It's enough to give the reason -- it's off-topic for this list, which is about the development of Python. That and a pointer to the right list is, in my opinion, all we need to say. We don't need to sugar coat it -- we're all adults here. -- Steven D'Aprano From benjamin at python.org Thu Jul 22 14:23:35 2010 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 22 Jul 2010 07:23:35 -0500 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: <20100722140046.5e7551a1@pitrou.net> References: <4C470E2B.5060406@timgolden.me.uk> <20100721175819.3b037ad9@pitrou.net> <20100722140046.5e7551a1@pitrou.net> Message-ID: 2010/7/22 Antoine Pitrou : > On Thu, 22 Jul 2010 08:51:57 +0100 > Brett Cannon wrote: >> >> That's an option. I just remember Tim bringing up something about that >> approach that didn't quite work as a complete replacement for __del__. >> >> Basically the whole setting a module's globals to None was done before gc >> came into the language. Now that it's there it seems that it might work to >> simply let gc clean up the module itself. > > There is a patch at http://bugs.python.org/issue812369 for GC-based > module shutdown, but it doesn't actually remove the setting of module > globals to None. I think further testing and experimentation would be > required to validate it. Also, it seems to have been stalled by static globals in extension modules that the gc doesn't know about. -- Regards, Benjamin From solipsis at pitrou.net Thu Jul 22 14:29:32 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 22 Jul 2010 14:29:32 +0200 Subject: [Python-Dev] module shutdown procedure In-Reply-To: References: <4C470E2B.5060406@timgolden.me.uk> <20100721175819.3b037ad9@pitrou.net> <20100722140046.5e7551a1@pitrou.net> Message-ID: <1279801772.3201.3.camel@localhost.localdomain> Le jeudi 22 juillet 2010 ? 07:23 -0500, Benjamin Peterson a ?crit : > 2010/7/22 Antoine Pitrou : > > On Thu, 22 Jul 2010 08:51:57 +0100 > > Brett Cannon wrote: > >> > >> That's an option. I just remember Tim bringing up something about that > >> approach that didn't quite work as a complete replacement for __del__. > >> > >> Basically the whole setting a module's globals to None was done before gc > >> came into the language. Now that it's there it seems that it might work to > >> simply let gc clean up the module itself. > > > > There is a patch at http://bugs.python.org/issue812369 for GC-based > > module shutdown, but it doesn't actually remove the setting of module > > globals to None. I think further testing and experimentation would be > > required to validate it. > > Also, it seems to have been stalled by static globals in extension > modules that the gc doesn't know about. Is it the reason why? With the new module creation API in 3.x, extension modules should be able to handle deletion of their own internal resources. Regards Antoine. From bartosz-tarnowski at zlotniki.pl Thu Jul 22 16:04:00 2010 From: bartosz-tarnowski at zlotniki.pl (Bartosz Tarnowski) Date: Thu, 22 Jul 2010 16:04:00 +0200 Subject: [Python-Dev] Set the namespace free! Message-ID: <4C484FD0.2080803@zlotniki.pl> Hello, guys. Python has more and more reserved words over time. It becomes quite annoying, since you can not use variables and attributes of such names. Suppose I want to make an XML parser that reads a document and returns an object with attributes corresponding to XML element attributes: > elem = parse_xml("") > print elem.param boo What should I do then, when the attribute is a reserver word? I could use trailing underscore, but this is quite ugly and introduces ambiguity. > elem = parse_xml("") > print elem.for_ #????? > elem = parse_xml("") > print elem.for__ #????? My proposal: let's make a syntax change. Let all reserved words be preceded with some symbol, i.e. "!" (exclamation mark). This goes also for standard library global identifiers. !for boo in foo: !if boo is !None: !print(hoo) !else: !return !sorted(woo) This would allow the user to declare any identifier with any name: for = with(return) + try What do you think of it? It is a major change, but I think Python needs it. -- haael From solipsis at pitrou.net Thu Jul 22 14:45:36 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 22 Jul 2010 14:45:36 +0200 Subject: [Python-Dev] Python 3 optimizations... References: Message-ID: <20100722144536.7236b80e@pitrou.net> On Thu, 22 Jul 2010 13:22:48 +0200 stefan brunthaler wrote: > > I wonder whether you would be interested in integrating these > optimizations with the Python 3 distribution, hence this mail. I could > send copies of the papers, as well as provide my prototype source code > to interested members of the python development community. Is the source code under an open source non-copyleft license? Have you checked that the whole regression test suite passes? Can you publish all this online (papers and source code)? Regards Antoine. From simon at brunningonline.net Thu Jul 22 14:45:49 2010 From: simon at brunningonline.net (Simon Brunning) Date: Thu, 22 Jul 2010 13:45:49 +0100 Subject: [Python-Dev] Set the namespace free! In-Reply-To: <4C484FD0.2080803@zlotniki.pl> References: <4C484FD0.2080803@zlotniki.pl> Message-ID: On 22 July 2010 15:04, Bartosz Tarnowski wrote: > What should I do then, when the attribute is a reserver word? You would use elem.getattr('param'). That's what it's for. > Let all reserved words be preceded with some symbol, i.e. "!" (exclamation > mark). Oh, God, no. This would be better off on the Python Ideas list. Well, it would be better of stone dead IMHO, but better ideas than here. -- Cheers, Simon B. From ijmorlan at uwaterloo.ca Thu Jul 22 14:46:27 2010 From: ijmorlan at uwaterloo.ca (Isaac Morland) Date: Thu, 22 Jul 2010 08:46:27 -0400 (EDT) Subject: [Python-Dev] Set the namespace free! In-Reply-To: <4C484FD0.2080803@zlotniki.pl> References: <4C484FD0.2080803@zlotniki.pl> Message-ID: On Thu, 22 Jul 2010, Bartosz Tarnowski wrote: [....] > My proposal: let's make a syntax change. I'm pretty sure this belongs on python-ideas. > Let all reserved words be preceded with some symbol, i.e. "!" (exclamation > mark). This goes also for standard library global identifiers. > > !for boo in foo: > !if boo is !None: > !print(hoo) > !else: > !return !sorted(woo) Is today April 1st? Seriously, an identifier-quoting capability like PostgreSQL has wouldn't necessarily be a bad idea, but would be a topic for python-ideas, not here on python-dev. Isaac Morland CSCF Web Guru DC 2554C, x36650 WWW Software Specialist From ziade.tarek at gmail.com Thu Jul 22 14:56:40 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 22 Jul 2010 14:56:40 +0200 Subject: [Python-Dev] Distutils reverted in py3k Message-ID: Hello As decided during the summit, I've reverted Distutils in the py3k branch, to its release3.1-maint state. This was already done in 2.7. I will only work on bugfixes for now on for distutils. Everything new is done in distutils2. So if you have a feature request, use the distutils2 component in the tracker. Let me know if you have/see any problem this revert created. Regards Tarek -- Tarek Ziad? | http://ziade.org From ziade.tarek at gmail.com Thu Jul 22 15:01:36 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 22 Jul 2010 15:01:36 +0200 Subject: [Python-Dev] Distutils reverted in py3k In-Reply-To: References: Message-ID: Note that I'll revert Doc/distutils as well, but I need to check first with Ronald a few Mac OS X points. From benjamin at python.org Thu Jul 22 15:03:19 2010 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 22 Jul 2010 08:03:19 -0500 Subject: [Python-Dev] module shutdown procedure In-Reply-To: <1279801772.3201.3.camel@localhost.localdomain> References: <4C470E2B.5060406@timgolden.me.uk> <20100721175819.3b037ad9@pitrou.net> <20100722140046.5e7551a1@pitrou.net> <1279801772.3201.3.camel@localhost.localdomain> Message-ID: 2010/7/22 Antoine Pitrou : > Le jeudi 22 juillet 2010 ? 07:23 -0500, Benjamin Peterson a ?crit : >> 2010/7/22 Antoine Pitrou : >> > On Thu, 22 Jul 2010 08:51:57 +0100 >> > Brett Cannon wrote: >> >> >> >> That's an option. I just remember Tim bringing up something about that >> >> approach that didn't quite work as a complete replacement for __del__. >> >> >> >> Basically the whole setting a module's globals to None was done before gc >> >> came into the language. Now that it's there it seems that it might work to >> >> simply let gc clean up the module itself. >> > >> > There is a patch at http://bugs.python.org/issue812369 for GC-based >> > module shutdown, but it doesn't actually remove the setting of module >> > globals to None. I think further testing and experimentation would be >> > required to validate it. >> >> Also, it seems to have been stalled by static globals in extension >> modules that the gc doesn't know about. > > Is it the reason why? With the new module creation API in 3.x, extension > modules should be able to handle deletion of their own internal > resources. Well, then the reason is that no modules use it. I also believe the new API is dangerous because modules can be deallocated before objects in them resulting in segfaults when those objects require globals stored in the module state. -- Regards, Benjamin From sbrunthaler at gmail.com Thu Jul 22 15:08:45 2010 From: sbrunthaler at gmail.com (stefan brunthaler) Date: Thu, 22 Jul 2010 15:08:45 +0200 Subject: [Python-Dev] Python 3 optimizations... In-Reply-To: References: Message-ID: > Is the source code under an open source non-copyleft license? > I am (unfortunately) not employed or funded by anybody, so I think that I can license/release the code as I see fit. > Have you checked that the whole regression test suite passes? > Currently, I am sure my prototype will not pass the regression test suite, because I used it for benchmarking only. However, I think it is probably not a too much work to get this done. > Can you publish all this online (papers and source code)? > I am not sure whether I can publish the ECOOP paper without violating any Springer copyrights, but will check if that is any problem. I could send you the second paper in private; until it is going to be published in the corresponding proceedings, I fear that I might not be able to put this online, though. Publishing the source code should not be a problem at all, though I think some polishing might not be a bad idea... Best, --stefan From ncoghlan at gmail.com Thu Jul 22 15:12:16 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 22 Jul 2010 23:12:16 +1000 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> <20100713163544.5f3a60d2@pitrou.net> Message-ID: On Thu, Jul 22, 2010 at 9:34 PM, Georg Brandl wrote: > So, I thought there wasn't a difference in performance for this use case > (which is compiling a lot of regexes and matching most of them only a > few times in comparison). ?However, I found that looking at the regex > caching is very important in this case: re._MAXCACHE is by default set to > 100, and regex._MAXCACHE to 1024. ?When I set re._MAXCACHE to 1024 before > running the test suite, I get times around 18 (!) seconds for re. That still fits with the compile/match performance trade-off changes between re and regex though. It does make it clear this isn't going to be a win across the board though - things like test suites are going to have more one-off regex operations than a long-running web server or a filesystem or database scanning operation. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From python-dev at masklinn.net Thu Jul 22 15:14:52 2010 From: python-dev at masklinn.net (Xavier Morel) Date: Thu, 22 Jul 2010 15:14:52 +0200 Subject: [Python-Dev] Set the namespace free! In-Reply-To: References: <4C484FD0.2080803@zlotniki.pl> Message-ID: On 2010-07-22, at 14:45 , Simon Brunning wrote: > On 22 July 2010 15:04, Bartosz Tarnowski wrote: >> What should I do then, when the attribute is a reserver word? > > You would use elem.getattr('param'). That's what it's for. getattr(elem, 'param') I believe, rather than elem.getattr('param') From g.brandl at gmx.net Thu Jul 22 16:13:49 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 22 Jul 2010 15:13:49 +0100 Subject: [Python-Dev] module shutdown procedure In-Reply-To: <1279801772.3201.3.camel@localhost.localdomain> References: <4C470E2B.5060406@timgolden.me.uk> <20100721175819.3b037ad9@pitrou.net> <20100722140046.5e7551a1@pitrou.net> <1279801772.3201.3.camel@localhost.localdomain> Message-ID: Am 22.07.2010 13:29, schrieb Antoine Pitrou: > Le jeudi 22 juillet 2010 ? 07:23 -0500, Benjamin Peterson a ?crit : >> 2010/7/22 Antoine Pitrou : >> > On Thu, 22 Jul 2010 08:51:57 +0100 >> > Brett Cannon wrote: >> >> >> >> That's an option. I just remember Tim bringing up something about that >> >> approach that didn't quite work as a complete replacement for __del__. >> >> >> >> Basically the whole setting a module's globals to None was done before gc >> >> came into the language. Now that it's there it seems that it might work to >> >> simply let gc clean up the module itself. >> > >> > There is a patch at http://bugs.python.org/issue812369 for GC-based >> > module shutdown, but it doesn't actually remove the setting of module >> > globals to None. I think further testing and experimentation would be >> > required to validate it. >> >> Also, it seems to have been stalled by static globals in extension >> modules that the gc doesn't know about. > > Is it the reason why? With the new module creation API in 3.x, extension > modules should be able to handle deletion of their own internal > resources. Yes, but as Martin noted at the summit, nobody since went through all the extension modules and changed them to use a struct instead of globals. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From simon at brunningonline.net Thu Jul 22 15:20:34 2010 From: simon at brunningonline.net (Simon Brunning) Date: Thu, 22 Jul 2010 14:20:34 +0100 Subject: [Python-Dev] Set the namespace free! In-Reply-To: References: <4C484FD0.2080803@zlotniki.pl> Message-ID: On 22 July 2010 14:14, Xavier Morel wrote: > getattr(elem, 'param') I believe, rather than elem.getattr('param') Doh! You're absolutely right, of course. -- Cheers, Simon B. From ncoghlan at gmail.com Thu Jul 22 15:26:49 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 22 Jul 2010 23:26:49 +1000 Subject: [Python-Dev] Python 3 optimizations... In-Reply-To: References: Message-ID: On Thu, Jul 22, 2010 at 9:22 PM, stefan brunthaler wrote: > I wonder whether you would be interested in integrating these > optimizations with the Python 3 distribution, hence this mail. I could > send copies of the papers, as well as provide my prototype source code > to interested members of the python development community. The Springer link [1] at least shows the front page to give more of an idea as to what this is about. The idea does sound potentially interesting, although I'm not sure how applicable it will be with a full-blown LLVM-based JIT on the way for 3.3 (via Unladen Swallow). Depending on the size and complexity of the patches, it may still be worth exploring for 3.2. Regards, Nick. [1] http://www.springerlink.com/content/p4u0851w34180n74/ -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From exarkun at twistedmatrix.com Thu Jul 22 15:30:24 2010 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Thu, 22 Jul 2010 13:30:24 -0000 Subject: [Python-Dev] EINVAL In-Reply-To: <20100722123302.066605f6@pitrou.net> References: <4C44E561.9060501@canterbury.ac.nz> <87iq48xj1r.fsf@uwakimon.sk.tsukuba.ac.jp> <1B6131B2-1152-418E-8DBA-1E4EC6F24A82@twistedmatrix.com> <4C47F967.7000002@canterbury.ac.nz> <87hbjryk7b.fsf@uwakimon.sk.tsukuba.ac.jp> <20100722123302.066605f6@pitrou.net> Message-ID: <20100722133024.27251.293090378.divmod.xquotient.140@localhost.localdomain> On 10:33 am, solipsis at pitrou.net wrote: >On Thu, 22 Jul 2010 17:50:00 +0900 >"Stephen J. Turnbull" wrote: >> >>I think that's Antoine's PEP 3151. Interestingly, he doesn't mention >>EINVAL at all. >> >>http://www.python.org/dev/peps/pep-3151/ > >That's right. It is based on a survey of existing exception-catching >code in the stdlib. There's only one match in the whole Lib/ subtree: > >$ grep -r EINVAL Lib/ >Lib/plat-sunos5/STROPTS.py:968:EINVAL = 22 > >I guess EINVAL would most often indicate a programming error, which is >why it doesn't get handled specifically in except clauses. For setgroups it means you exceeded a platform-specific limit. On Windows, for non-blocking connect, it means wait a little longer. Jean-Paul From sbrunthaler at gmail.com Thu Jul 22 15:36:56 2010 From: sbrunthaler at gmail.com (stefan brunthaler) Date: Thu, 22 Jul 2010 15:36:56 +0200 Subject: [Python-Dev] Python 3 optimizations... In-Reply-To: References: Message-ID: > The Springer link [1] at least shows the front page to give more of an > idea as to what this is about. > Thanks, I forgot to mention the link. > The idea does sound potentially interesting, although I'm not sure how > applicable it will be with a full-blown LLVM-based JIT on the way for > 3.3 (via Unladen Swallow). > Yeah, I see that. However, I think that a JIT compiler could very well re-use the information gathered by using my techniques to generate better code in the first run. There certainly are a couple of other benefits for combining both approaches, i.e., they are not mutually exclusive. > Depending on the size and complexity of the > patches, it may still be worth exploring for 3.2. > I am currently not aware of the planned release schedule, but I think that some of the techniques could very well be released in 3.2 -- with a conditional compilation target similar to the computed-goto feature. Regards, --stefan From g.brandl at gmx.net Thu Jul 22 16:42:47 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 22 Jul 2010 15:42:47 +0100 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> <20100713163544.5f3a60d2@pitrou.net> Message-ID: Am 22.07.2010 14:12, schrieb Nick Coghlan: > On Thu, Jul 22, 2010 at 9:34 PM, Georg Brandl wrote: >> So, I thought there wasn't a difference in performance for this use case >> (which is compiling a lot of regexes and matching most of them only a >> few times in comparison). However, I found that looking at the regex >> caching is very important in this case: re._MAXCACHE is by default set to >> 100, and regex._MAXCACHE to 1024. When I set re._MAXCACHE to 1024 before >> running the test suite, I get times around 18 (!) seconds for re. > > That still fits with the compile/match performance trade-off changes > between re and regex though. It does make it clear this isn't going to > be a win across the board though - things like test suites are going > to have more one-off regex operations than a long-running web server > or a filesystem or database scanning operation. Sure -- I don't think this is a showstopper for regex. However if we don't include regex in a future version, we might think about increasing MAXCACHE a bit, and maybe not clear the cache when it reaches its max length, but rather remove another element. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From solipsis at pitrou.net Thu Jul 22 15:48:48 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 22 Jul 2010 15:48:48 +0200 Subject: [Python-Dev] PEP 360 has outdated contents Message-ID: <20100722154848.75996b4b@pitrou.net> Hello all, PEP 360 - ?Externally Maintained Packages? seems to have outdated contents. First of all, I don't think Optik and wsgiref are externally maintained anymore (both seem unmaintained by their original authors). Second, the version numbers mentioned there could be out of date too (especially for ElementTree whose stdlib copy has recently been bumped by Florent). Third, it seems json should be added to the list, despite the prominent warning at the beginning that: ?No new modules are to be added to this PEP. It has been deemed dangerous to codify external maintenance of any code checked into Python's code repository. Code contributers should expect Python's development methodology to be used for any and all code checked into Python's code repository.? Regards Antoine. From barry at python.org Thu Jul 22 16:40:36 2010 From: barry at python.org (Barry Warsaw) Date: Thu, 22 Jul 2010 16:40:36 +0200 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <4C40372C.9030101@ubuntu.com> References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> Message-ID: <20100722164036.7a80d27c@snowdog> On Jul 16, 2010, at 12:40 PM, Matthias Klose wrote: >I like the proposal, but IMO it is too unspecific about the abi tag. >Assume that an extension is built with such a configured python and >then tried to run with an abi compatible configured python, but with a >slightly different version tag, the extension won't be found. >Differing file names per configuration should be avoided. Proposing > > 1) Remove the configure option and use the new naming using the tag > for all configurations unconditionally. Everybody can expect the same > name on every installation, and PEP 384 most likely will require > using a tag too. > As Amaury did point out, there is a use case in that PyPy can > use this tag too to build extensions only usable by PyPy. > > 2) As PEP 3147 defines a non-configurable name for .pyc files, this > PEP should define a non-configurable way for the tag. The tag should > include all information which currently makes an extension ABI > incompatible: > - the python implementation (cpython, PyPy, ...) > - the python version (3,2, 3.3, ...) > - unicode configure option (--with-wide-unicode, 16 or 32) > - platform information (necessary?) > If this list changes for coming python versions, then it can be > extended. > Barry pointed out on irc chat that people might want to build > experimental ABI incompatible versions, which should have its > own tag. If this is wanted, please provide a configure option > which lets extend/append to the tag. Okay, I'm convinced we need to hard code this tag, and I think it's okay not to provide a configure switch to control this. I've pushed a new version of the diff and PEP, the latter attached here for your convenience. Thanks for all the feedback. -Barry PEP: 3149 Title: ABI version tagged .so files Version: $Revision: 81577 $ Last-Modified: $Date: 2010-05-27 19:54:25 -0400 (Thu, 27 May 2010) $ Author: Barry Warsaw Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 2010-07-09 Python-Version: 3.2 Post-History: 2010-07-14, 2010-07-22 Resolution: TBD Abstract ======== PEP 3147 [1]_ described an extension to Python's import machinery that improved the sharing of Python source code, by allowing more than one byte compilation file (.pyc) to be co-located with each source file. This PEP defines an adjunct feature which allows the co-location of extension module files (.so) in a similar manner. This optional, build-time feature will enable downstream distributions of Python to more easily provide more than one Python major version at a time. Background ========== PEP 3147 defined the file system layout for a pure-Python package, where multiple versions of Python are available on the system. For example, where the `alpha` package containing source modules `one.py` and `two.py` exist on a system with Python 3.2 and 3.3, the post-byte compilation file system layout would be:: alpha/ __pycache__/ __init__.cpython-32.pyc __init__.cpython-33.pyc one.cpython-32.pyc one.cpython-33.pyc two.cpython-32.pyc two.cpython-33.pyc __init__.py one.py two.py For packages with extension modules, a similar differentiation is needed for the module's .so files. Extension modules compiled for different Python major versions are incompatible with each other due to changes in the ABI. Different configuration/compilation options for the same Python version can result in different ABIs (e.g. --with-wide-unicode). While PEP 384 [2]_ defines a stable ABI, it will minimize, but not eliminate extension module incompatibilities between Python builds or major versions. Thus a mechanism for discriminating extension module file names is proposed. Rationale ========= Linux distributions such as Ubuntu [3]_ and Debian [4]_ provide more than one Python version at the same time to their users. For example, Ubuntu 9.10 Karmic Koala users can install Python 2.5, 2.6, and 3.1, with Python 2.6 being the default. In order to share as much as possible between the available Python versions, these distributions install third party (i.e. non-standard library) packages into `/usr/share/pyshared` and symlink to them from `/usr/lib/pythonX.Y/dist-packages`. The symlinks exist because in a pre-PEP 3147 world (i.e < Python 3.2), the `.pyc` files resulting from byte compilation by the various installed Pythons will name collide with each other. For Python versions >= 3.2, all pure-Python packages can be shared, because the `.pyc` files will no longer cause file system naming conflicts. Eliminating these symlinks makes for a simpler, more robust Python distribution. A similar situation arises with shared library extensions. Because extension modules are typically named `foo.so` for a `foo` extension module, these would also name collide if `foo` was provided for more than one Python version. In addition, because different configuration/compilation options for the same Python version can cause different ABIs to be presented to extension modules. On POSIX systems for example, the configure options ``--with-pydebug``, ``--with-pymalloc``, and ``--with-wide-unicode`` all change the ABI. This PEP proposes to encode build-time options in the file name of the ``.so`` extension module files. PyPy [5]_ can also benefit from this PEP, allowing it to avoid name collisions in extension modules built for its API, but with a different `.so` tag. Proposal ======== The configure/compilation options chosen at Python interpreter build-time will be encoded in the shared library file name for extension modules. This "tag" will appear between the module base name and the operation file system extension for shared libraries. The following information *MUST* be included in the shared library file name: * The Python implementation (e.g. cpython, pypy, jython, etc.) * The interpreter's major and minor version numbers These two fields are separated by a hyphen and no dots are to appear between the major and minor version numbers. E.g. ``cpython-32``. Python implementations *MAY* include additional flags in the file name tag as appropriate. For example, on POSIX systems these flags will also contribute to the file name: * ``--with-pydebug`` (flag: ``d``) * ``--with-pymalloc`` (flag: ``m``) * ``--with-wide-unicode`` (flag: ``u``) By default in Python 3.2, ``configure`` enables ``--with-pymalloc`` so shared library file names would appear as ``foo.cpython-32m.so``. When the other two flags are also enabled, the file names would be ``foo.cpython-32dmu.so``. (This PEP only addresses build issues on POSIX systems that use the ``configure`` script. While Windows or other platform support is not explicitly disallowed under this PEP, platform expertise is needed in order to evaluate, describe, and implement support on such platforms.) The shared library file name tag is used unconditionally; it cannot be changed. The tag and extension module suffix are available through the ``sysconfig`` modules via the following variables:: >>> sysconfig.get_config_var('SO') '.cpython-32mu.so' >>> sysconfig.get_config_var('SOABI') 'cpython-32mu' Note that ``$SOABI`` contains just the tag, while ``$SO`` includes the platform extension for shared library files, and is the exact suffix added to the extension module name. For an arbitrary package `foo`, you would see these files when the distribution package was installed:: /usr/share/pyshared/foo.cpython-32m.so /usr/share/pyshared/foo.cpython-33m.so Python's dynamic module loader will recognize and import shared library extension modules with a tag that matches its build-time options. For backward compatibility, Python will also continue to import untagged extension modules, e.g. ``foo.so``. This shared library tag would be used globally for all distutils-based extension modules, regardless of where on the file system they are built. Extension modules built by means other than distutils would either have to calculate the tag manually, or fallback to the non-tagged `.so` file name. Proven approach =============== The approach described here is already proven, in a sense, on Debian and Ubuntu system where different extensions are used for debug builds of Python and extension modules. Debug builds on Windows also already use a different file extension for dynamic libraries, and in fact encoded (in a different way than proposed in this PEP) the Python major and minor version in the `.dll` file name. PEP 384 ======= PEP 384 defines a stable ABI for extension modules. In theory, universal adoption of PEP 384 would eliminate the need for this PEP because all extension modules could be compatible with any Python version. In practice of course, it will be impossible to achieve universal adoption, and as described above, different built-time flags still affect the ABI. Thus even with a stable ABI, this PEP may still be necessary. While a complete specification is reserved for PEP 384, here is a discussion of the relevant issues. PEP 384 describes a change to ``PyModule_Create()`` where ``3`` is passed as the API version if the extension was complied with ``Py_LIMITED_API``. This should be formalized into an official macro called ``PYTHON_ABI_VERSION`` to mirror ``PYTHON_API_VERSION``. If and when the ABI changes in an incompatible way, this version number would be bumped. To facilitate sharing, Python would be extended to search for extension modules with the ``PYTHON_ABI_VERSION`` number in its name. The prefix ``abi`` is reserved for Python's use. Thus, an initial implementation of PEP 384, when Python is configured with the default set of flags, would search for the following file names when extension module `foo` is imported (in this order):: foo.cpython-XYm.so foo.abi3.so foo.so The distutils [6]_ ``build_ext`` command would also have to be extended to compile to shared library files with the ``abi3`` tag, when the module author indicates that their extension supports that version of the ABI. This could be done in a backward compatible way by adding a keyword argument to the ``Extension`` class, such as:: Extension('foo', ['foo.c'], abi=3) Alternatives ============ In the initial python-dev thread [7]_ where this idea was first introduced, several alternatives were suggested. For completeness they are listed here, along with the reasons for not adopting them. Independent directories or symlinks ----------------------------------- Debian and Ubuntu could simply add a version-specific directory to ``sys.path`` that would contain just the extension modules for that version of Python. Or the symlink trick eliminated in PEP 3147 could be retained for just shared libraries. This approach is rejected because it propagates the essential complexity that PEP 3147 tries to avoid, and adds potentially several additional directories to search for all modules, even when the number of extension modules is much fewer than the total number of Python packages. For example, builds were made available both with and without wide unicode, with and without pydebug, and with and without pymalloc, the total number of directories search increases substantially. Don't share packages with extension modules ------------------------------------------- It has been suggested that Python packages with extension modules not be shared among all supported Python versions on a distribution. Even with adoption of PEP 3149, extension modules will have to be compiled for every supported Python version, so perhaps sharing of such packages isn't useful anyway. Not sharing packages with extensions though is infeasible for several reasons. If a pure-Python package is shared in one version, should it suddenly be not-shared if the next release adds an extension module for speed? Also, even though all extension shared libraries will be compiled and distributed once for every supported Python, there's a big difference between duplicating the `.so` files and duplicating all `.py` files. The extra size increases the download time for such packages, and more immediately, increases the space pressures on already constrained distribution CD-ROMs. Reference implementation ======================== Work on this code is tracked in a Bazaar branch on Launchpad [8]_ until it's ready for merge into Python 3.2. The work-in-progress diff can also be viewed [9]_ and is updated automatically as new changes are uploaded. References ========== .. [1] PEP 3147 .. [2] PEP 384 .. [3] Ubuntu: .. [4] Debian: .. [5] http://codespeak.net/pypy/dist/pypy/doc/ .. [6] http://docs.python.org/py3k/distutils/index.html .. [7] http://mail.python.org/pipermail/python-dev/2010-June/100998.html .. [8] https://code.edge.launchpad.net/~barry/python/sovers .. [9] https://code.edge.launchpad.net/~barry/python/sovers/+merge/29411 Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From jnoller at gmail.com Thu Jul 22 16:41:39 2010 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 22 Jul 2010 10:41:39 -0400 Subject: [Python-Dev] Set the namespace free! In-Reply-To: <4C484FD0.2080803@zlotniki.pl> References: <4C484FD0.2080803@zlotniki.pl> Message-ID: On Thu, Jul 22, 2010 at 10:04 AM, Bartosz Tarnowski wrote: > > Hello, guys. > > Python has more and more reserved words over time. It becomes quite > annoying, since you can not use variables and attributes of such names. > Suppose I want to make an XML parser that reads a document and returns an > object with attributes corresponding to XML element attributes: > >> elem = parse_xml("") >> print elem.param > boo > > What should I do then, when the attribute is a reserver word? I could use > trailing underscore, but this is quite ugly and introduces ambiguity. > >> elem = parse_xml("") >> print elem.for_ #????? >> elem = parse_xml("") >> print elem.for__ #????? > > My proposal: let's make a syntax change. > > Let all reserved words be preceded with some symbol, i.e. "!" (exclamation > mark). This goes also for standard library global identifiers. > > !for boo in foo: > ? ?!if boo is !None: > ? ? ? ?!print(hoo) > ? ?!else: > ? ? ? ?!return !sorted(woo) > > > This would allow the user to declare any identifier with any name: > > for = with(return) + try > > What do you think of it? It is a major change, but I think Python needs it. > > -- > haael > I'm not a fan of this - I'd much prefer[1] that we use the exclamation point to determine scope: foobar - local !foobar - one up !!foobar - higher than the last one !!!foobar - even higher in scope We could do the inverse as well; if you append ! you can push variable down in scope. Jesse [1] I am not serious. From g.brandl at gmx.net Thu Jul 22 17:54:58 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 22 Jul 2010 16:54:58 +0100 Subject: [Python-Dev] Set the namespace free! In-Reply-To: <4C484FD0.2080803@zlotniki.pl> References: <4C484FD0.2080803@zlotniki.pl> Message-ID: Am 22.07.2010 15:04, schrieb Bartosz Tarnowski: > > Hello, guys. > > Python has more and more reserved words over time. It becomes quite annoying, > since you can not use variables and attributes of such names. Suppose I want to > make an XML parser that reads a document and returns an object with attributes > corresponding to XML element attributes: > > > elem = parse_xml("") > > print elem.param > boo > > What should I do then, when the attribute is a reserver word? I could use > trailing underscore, but this is quite ugly and introduces ambiguity. > > > elem = parse_xml("") > > print elem.for_ #????? > > elem = parse_xml("") > > print elem.for__ #????? > > My proposal: let's make a syntax change. > > Let all reserved words be preceded with some symbol, i.e. "!" (exclamation > mark). This goes also for standard library global identifiers. > > !for boo in foo: > !if boo is !None: > !print(hoo) > !else: > !return !sorted(woo) You raise a good point. However, I'd rather explicitly signify names instead of keywords: for $boo in $foo: if $boo is $None: print($hoo) else: return sorted($woo) That also has the advantage of introducing a measure of much needed compatibility with industry-leading web programming languages. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From scialexlight at gmail.com Thu Jul 22 16:57:15 2010 From: scialexlight at gmail.com (Alex Light) Date: Thu, 22 Jul 2010 10:57:15 -0400 Subject: [Python-Dev] Set the namespace free! In-Reply-To: <4C484FD0.2080803@zlotniki.pl> References: <4C484FD0.2080803@zlotniki.pl> Message-ID: On Thu, Jul 22, 2010 at 10:04 AM, Bartosz Tarnowski < bartosz-tarnowski at zlotniki.pl> wrote: > > Let all reserved words be preceded with some symbol, i.e. "!" (exclamation > mark). This goes also for standard library global identifiers. > > !for boo in foo: > !if boo is !None: > !print(hoo) > !else: > !return !sorted(woo) > > A) this should be in the 'ideas' list B) it will never ever happen. not only does it have very few benefits it makes every single piece of python code ever written invalid and it is a bad idea to arbitrarily add punctuation to the language. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.belopolsky at gmail.com Thu Jul 22 16:59:51 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 22 Jul 2010 10:59:51 -0400 Subject: [Python-Dev] Set the namespace free! In-Reply-To: References: <4C484FD0.2080803@zlotniki.pl> Message-ID: On Thu, Jul 22, 2010 at 10:41 AM, Jesse Noller wrote: .. > I'm not a fan of this - I'd much prefer[1] that we use the exclamation > point to determine scope: > > foobar - local > !foobar - one up > !!foobar - higher than the last one > !!!foobar - even higher in scope > > We could do the inverse as well; if you append ! you can push variable > down in scope. Replace '!' with '$' and we have a winner! [IANSE] > [1] I am not serious. [IANSE] I am not serious either. From cournape at gmail.com Thu Jul 22 17:00:15 2010 From: cournape at gmail.com (David Cournapeau) Date: Fri, 23 Jul 2010 00:00:15 +0900 Subject: [Python-Dev] Python 3 optimizations... In-Reply-To: References: Message-ID: On Thu, Jul 22, 2010 at 10:08 PM, stefan brunthaler wrote: >> Is the source code under an open source non-copyleft license? >> > I am (unfortunately) not employed or funded by anybody, so I think > that I can license/release the code as I see fit. If you did this work under your PhD program, you may be more restricted than you think. You may want to check with your adviser first, cheers, David From alexander.belopolsky at gmail.com Thu Jul 22 17:03:21 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 22 Jul 2010 11:03:21 -0400 Subject: [Python-Dev] Set the namespace free! In-Reply-To: References: <4C484FD0.2080803@zlotniki.pl> Message-ID: On Thu, Jul 22, 2010 at 11:54 AM, Georg Brandl wrote: .. > That also has the advantage of introducing a measure of much needed > compatibility with industry-leading web programming languages. Looks like our messages crossed in flight. pathologically-eclecticly-yours From reid.kleckner at gmail.com Thu Jul 22 17:13:53 2010 From: reid.kleckner at gmail.com (Reid Kleckner) Date: Thu, 22 Jul 2010 08:13:53 -0700 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> <20100713163544.5f3a60d2@pitrou.net> Message-ID: On Thu, Jul 22, 2010 at 7:42 AM, Georg Brandl wrote: > Am 22.07.2010 14:12, schrieb Nick Coghlan: >> On Thu, Jul 22, 2010 at 9:34 PM, Georg Brandl wrote: >>> So, I thought there wasn't a difference in performance for this use case >>> (which is compiling a lot of regexes and matching most of them only a >>> few times in comparison). ?However, I found that looking at the regex >>> caching is very important in this case: re._MAXCACHE is by default set to >>> 100, and regex._MAXCACHE to 1024. ?When I set re._MAXCACHE to 1024 before >>> running the test suite, I get times around 18 (!) seconds for re. It might be fun to do a pygments based macro benchmark. Just have it syntax highlight itself and time it. > Sure -- I don't think this is a showstopper for regex. ?However if we don't > include regex in a future version, we might think about increasing MAXCACHE > a bit, and maybe not clear the cache when it reaches its max length, but > rather remove another element. +50 for the last idea. Collin encountered a problem two summers ago in Mondrian where we were relying on the regex cache and were surprised to find that it cleared itself after filling up, rather than using LRU or random eviction. Reid From eliben at gmail.com Thu Jul 22 17:25:15 2010 From: eliben at gmail.com (Eli Bendersky) Date: Thu, 22 Jul 2010 18:25:15 +0300 Subject: [Python-Dev] :program: markup in .rst documentation In-Reply-To: <4C45DE0A.9080806@netwok.org> References: <4C45DE0A.9080806@netwok.org> Message-ID: On Tue, Jul 20, 2010 at 20:34, ?ric Araujo wrote: >> Sorry to add the third way to the mix, but shouldn't the recommended >> way to run a module as a script be python -m modname? ?As in >> >> $ python -m test.regrtest test_spam > > This is true but orthogonal to our problem, which is that > :program:`python -m thing` is wrong if I understood the doc rightly and > should be replaced by ``python -m thing``. > So, how can a decision be reached on this issue? I'd like to fix the relevant docs because currently there's a lot of confusion. I will fix the :program: markup together with all the :option: and :cmdoption: markup. Eli From solipsis at pitrou.net Thu Jul 22 17:37:19 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 22 Jul 2010 17:37:19 +0200 Subject: [Python-Dev] Set the namespace free! References: <4C484FD0.2080803@zlotniki.pl> Message-ID: <20100722173719.5889b2bb@pitrou.net> On Thu, 22 Jul 2010 16:54:58 +0100 Georg Brandl wrote: > > You raise a good point. However, I'd rather explicitly signify names instead > of keywords: > > for $boo in $foo: > if $boo is $None: > print($hoo) > else: > return sorted($woo) > > That also has the advantage of introducing a measure of much needed > compatibility with industry-leading web programming languages. Also, Python would gain much needed flexibility if we allowed indirect name lookup using `$$foo`. Current abstractions are too poor compared to best-of-breed OO languages such as PHP or Perl 5. From chris at subtlety.com Thu Jul 22 17:49:16 2010 From: chris at subtlety.com (Chris Bergstresser) Date: Thu, 22 Jul 2010 10:49:16 -0500 Subject: [Python-Dev] Set the namespace free! In-Reply-To: <20100722173719.5889b2bb@pitrou.net> References: <4C484FD0.2080803@zlotniki.pl> <20100722173719.5889b2bb@pitrou.net> Message-ID: On Thu, Jul 22, 2010 at 10:37 AM, Antoine Pitrou wrote: > On Thu, 22 Jul 2010 16:54:58 +0100 > Georg Brandl wrote: >> >> That also has the advantage of introducing a measure of much needed >> compatibility with industry-leading web programming languages. > > Also, Python would gain much needed flexibility if we allowed indirect > name lookup using `$$foo`. Current abstractions are too poor compared > to best-of-breed OO languages such as PHP or Perl 5. Let's not forget additional lookup operators, like %foo, to specify the kind of lookup we're interested in (whether we want the result as a dict vs. list vs. whatever). We could even allow overloading (something like object.__$__) to allow objects to customize the results of their lookup operations. Really, I think with this and a world-class regex implementation we'll be well-positioned when the Internet finally hits it big. -- Chris From breamoreboy at yahoo.co.uk Thu Jul 22 18:24:31 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Thu, 22 Jul 2010 17:24:31 +0100 Subject: [Python-Dev] Issues #7717 and #7718 ask for Stuart Shelton contributor agreement Message-ID: I believe that Pat Campbell is responsible for handling contributor agreements. I'm trying to find out if we have one from Stuart Sheldon. Could someone in the know please forward this to Pat, then we'll be able to move these issues. TIA. Mark Lawrence. From alexander.belopolsky at gmail.com Thu Jul 22 18:25:39 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 22 Jul 2010 12:25:39 -0400 Subject: [Python-Dev] :program: markup in .rst documentation In-Reply-To: References: <4C45DE0A.9080806@netwok.org> Message-ID: On Thu, Jul 22, 2010 at 11:25 AM, Eli Bendersky wrote: >>> .. shouldn't the recommended >>> way to run a module as a script be python -m modname? ?As in >>> >>> $ python -m test.regrtest test_spam .. > So, how can a decision be reached on this issue? I'd like to fix the > relevant docs because currently there's a lot of confusion. I will fix > the :program: markup together with all the :option: and :cmdoption: > markup. Note also that argparse/optparse does not know about -m way either: $ python -m profile -h Usage: profile.py [-o output_file_path] [-s sort] scriptfile [arg] ... I am not sure if it is possible for argparse to distinguish between -m and file name invocations, so maybe it should always use python -m modname if sys.argv[0] ends with .py. Again, this is quite important if the module being run is inside a package. From martin at v.loewis.de Thu Jul 22 14:51:43 2010 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 22 Jul 2010 13:51:43 +0100 Subject: [Python-Dev] PEP 382 progress: import hooks Message-ID: <4C483EDF.7050002@v.loewis.de> At EuroPython, I sat down with Brett and we propose an approach how namespace packages get along with import hooks. I reshuffled the order in which things get done a little bit, and added a section that elaborates on the hooks. Basically, a finder will need to support a find_path method, return all .pth files, and a loader will need to support a load_module_with_path method, to initialize __path__. Please comment if you think that this needs further changes; I'll hope to start implementing it soon. Regards, Martin From ysj.ray+python at gmail.com Thu Jul 22 15:17:56 2010 From: ysj.ray+python at gmail.com (=?UTF-8?B?5bKz5biF5p2w?=) Date: Thu, 22 Jul 2010 21:17:56 +0800 Subject: [Python-Dev] Set the namespace free! In-Reply-To: <4C484FD0.2080803@zlotniki.pl> References: <4C484FD0.2080803@zlotniki.pl> Message-ID: > > !for boo in foo: !if boo is !None: !print(hoo) !else: !return !sorted(woo) I feel most people could not bear such a difficult syntax. Why have I to type so much '!'s ? On Thu, Jul 22, 2010 at 10:04 PM, Bartosz Tarnowski < bartosz-tarnowski at zlotniki.pl> wrote: > > Hello, guys. > > Python has more and more reserved words over time. It becomes quite > annoying, since you can not use variables and attributes of such names. > Suppose I want to make an XML parser that reads a document and returns an > object with attributes corresponding to XML element attributes: > > > elem = parse_xml("") > > print elem.param > boo > > What should I do then, when the attribute is a reserver word? I could use > trailing underscore, but this is quite ugly and introduces ambiguity. > > > elem = parse_xml("") > > print elem.for_ #????? > > elem = parse_xml("") > > print elem.for__ #????? > > My proposal: let's make a syntax change. > > Let all reserved words be preceded with some symbol, i.e. "!" (exclamation > mark). This goes also for standard library global identifiers. > > !for boo in foo: > !if boo is !None: > !print(hoo) > !else: > !return !sorted(woo) > > > This would allow the user to declare any identifier with any name: > > for = with(return) + try > > What do you think of it? It is a major change, but I think Python needs it. > > -- > haael > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ysj.ray%2Bpython-dev%40gmail.com > -- Ray Allen Best wishes! -------------- next part -------------- An HTML attachment was scrubbed... URL: From gregory.smith3 at sympatico.ca Thu Jul 22 18:24:27 2010 From: gregory.smith3 at sympatico.ca (gregory.smith3 at sympatico.ca) Date: Thu, 22 Jul 2010 16:24:27 +0000 Subject: [Python-Dev] Set the namespace free! In-Reply-To: References: <4C484FD0.2080803@zlotniki.pl>, Message-ID: I agree with the idea, but a far less radical change is needed to get the desired result. The basic idea is this: it should be possible to use any name as an identifier in the syntax, including names like 'while' and 'import'. But there is no need to mess up the entire language to allow this (either by quoting all the identifiers, perl-style, or by marking the keywords). All that is needed is something like this: foo = 7 :foo = 7 # exactly like foo=7 :while= 3 # assigns 3 to variable 'while' globals()['while']=3 # current way to do this print element.:for # from example below # # keyword parameters to a function call: # BuildUrlQuery( lang='en', item='monsoon', :class='normal') # -> "?lang=en&query=monsoon&class=normal" The generic keyword function call is a really nice language feature, but it's rather impaired by the need to avoid those names which happen to be keywords. The lack of this is most painful when you are auto-generating python code which forms a bridge to another language with its own namespace (as in XML example). It's a pain when some of the names you might generate could conflict with python keywords. So, you end up using dicts and getattrs for everything and the code gets much less readable. With a simple escape like :name available, it's worthwhile to do everything with identifiers and generate the escape only as needed for these. One of the great strengths of python is the ability to form cleans and comprehensive bridges to other languages and environments (thus, in many cases, avoiding the need to write programs in those other environments :-) ). This feature would fill a gap there. The python tcl/tk interface is a bit messed up since tcl/tk uses some names for options which conflict with python keywords, and so you need to add '_' to those names. There is a feature like this in VHDL: \name\ and \while\ are identifiers, the backslashes are not part of the name, but just quote it. In VHDL you can write identifiers like \22\, and \!This?is=Strange\ as well; since VHDL creates modules that have named ports, and those modules can interface to things generated by other environments, they needed a way to assign any name to a port. For python, I'm not sure it makes sense to allow identifiers that doesn't follow the basic rule "[A-Za-z_][A-Za-z_0-9]*" -- that could break some debugging tools which expect variable names to be well-formed -- but it would be useful to be able to use any well-formed name as an identifier, including those which happen to be keywords. I've suggested :name, which doesn't break old code, and doesn't require using any new punctuation. Syntax would not change, just the lexical definition of 'identifier'. If the intent is to allow arbitrary names (not just well-formed ones), then n'name' would work better (and is consistent with existing stuff). > Date: Thu, 22 Jul 2010 10:41:39 -0400 > From: jnoller at gmail.com > To: bartosz-tarnowski at zlotniki.pl > CC: python-dev at python.org > Subject: Re: [Python-Dev] Set the namespace free! > > On Thu, Jul 22, 2010 at 10:04 AM, Bartosz Tarnowski > wrote: > > > > Hello, guys. > > > > Python has more and more reserved words over time. It becomes quite > > annoying, since you can not use variables and attributes of such names. > > Suppose I want to make an XML parser that reads a document and returns an > > object with attributes corresponding to XML element attributes: > > > >> elem = parse_xml("") > >> print elem.param > > boo > > > > What should I do then, when the attribute is a reserver word? I could use > > trailing underscore, but this is quite ugly and introduces ambiguity. > > > >> elem = parse_xml("") > >> print elem.for_ #????? > >> elem = parse_xml("") > >> print elem.for__ #????? > > > > My proposal: let's make a syntax change. > > > > Let all reserved words be preceded with some symbol, i.e. "!" (exclamation > > mark). This goes also for standard library global identifiers. > > > > !for boo in foo: > > !if boo is !None: > > !print(hoo) > > !else: > > !return !sorted(woo) > > > > > > This would allow the user to declare any identifier with any name: > > > > for = with(return) + try > > > > What do you think of it? It is a major change, but I think Python needs it. > > > > -- > > haael > > > > I'm not a fan of this - I'd much prefer[1] that we use the exclamation > point to determine scope: > > foobar - local > !foobar - one up > !!foobar - higher than the last one > !!!foobar - even higher in scope > > We could do the inverse as well; if you append ! you can push variable > down in scope. > > Jesse > > > [1] I am not serious. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/gsmith%40alumni.uwaterloo.ca -------------- next part -------------- An HTML attachment was scrubbed... URL: From gregory.smith3 at sympatico.ca Thu Jul 22 18:44:07 2010 From: gregory.smith3 at sympatico.ca (gregory.smith3 at sympatico.ca) Date: Thu, 22 Jul 2010 16:44:07 +0000 Subject: [Python-Dev] Set the namespace free! In-Reply-To: References: <4C484FD0.2080803@zlotniki.pl>, , Message-ID: oops, :name does break things, e.g if x :return So, it could be ::name or |name or &name or !name or whatever. From: gregory.smith3 at sympatico.ca To: python-dev at python.org Subject: RE: [Python-Dev] Set the namespace free! Date: Thu, 22 Jul 2010 16:24:27 +0000 I agree with the idea, but a far less radical change is needed to get the desired result. The basic idea is this: it should be possible to use any name as an identifier in the syntax, including names like 'while' and 'import'. But there is no need to mess up the entire language to allow this (either by quoting all the identifiers, perl-style, or by marking the keywords). All that is needed is something like this: foo = 7 :foo = 7 # exactly like foo=7 ... -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.belopolsky at gmail.com Thu Jul 22 19:30:03 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 22 Jul 2010 13:30:03 -0400 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: On Wed, Jul 21, 2010 at 9:26 PM, Steve Holden wrote: > On 7/21/2010 6:45 PM, Alexander Belopolsky wrote: >> On Wed, Jul 21, 2010 at 1:42 PM, Benjamin Peterson wrote: >> .. >>> True, but the tests in that file are (mostly?) all about line tracing. >>> >> >> Hopefully this will change soon. :-) > > At which point the renaming would be justified. True, but at that point it may be in three or more branches while right now it is in one. It turns out, however, that the tests in the file formerly known as test_trace cover more than line tracing, for example: # Test the handling of exceptions raised by each kind of trace event. def test_call(self): self.run_test_for_event('call') def test_line(self): self.run_test_for_event('line') def test_return(self): self.run_test_for_event('return') def test_exception(self): self.run_test_for_event('exception') Furthermore, the now out of date comment in test_sys says: # testing sys.settrace() is done in test_trace.py which makes it clear that the original intent of test_trace was to test everything settrace does, not just line tracing. In any case, it looks like my original question got shadowed by the choice of name discussion. Is it ok to rename test_trace to test_whatever in the maintenance branches? The reason is that the trace module is severely broken in 3.x and there are no regression tests in 2.7 to test the 3.x port against. Another alternative is to simply add trace module tests into test_trace. After all, the test_sys comment I mentioned is followed by # testing sys.setprofile() is done in test_profile.py The profile situation is different, though: test_profile tests the profile module, but not sys.setprofile() . Since there is some movement in this area, I don't think it would hurt to make a decision now that will help writing tests in the future. I see three solutions: 1. Minimal: do not rename test_trace in 2.7 and add trace module tests to the existing file. Whether to revert test_trace to test_line_tracing renaming in 3.2 can be decided after we have 2.7 tests written. 2. Rename test_trace in 2.7/3.1 and test_line_tracing in 3.2 to test_sys_settrace. 3. Rename test_trace in 2.7/3.1 to test_line_tracing. My preference in #2, but I can live with #1 (assuming Eli is the one who will write the tests:-). From eric at trueblade.com Thu Jul 22 19:50:02 2010 From: eric at trueblade.com (Eric Smith) Date: Thu, 22 Jul 2010 13:50:02 -0400 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: <4C470E2B.5060406@timgolden.me.uk> References: <4C470E2B.5060406@timgolden.me.uk> Message-ID: <4C4884CA.9080603@trueblade.com> Thanks for writing this, Tim. On 7/21/10 11:11 AM, Tim Golden wrote: > The issue of a __format__ equivalent for bytes was also raised as was the > idea of object methods to render an object as string or bytes, which could > be used in the polymorphic functions above. Does this mean bytes.__format__, or does it mean bytes.format? bytes.__format__ already exists, but it just calls object.__format__. I'm not sure what the desired functionality for bytes.__format__ would be. If it means bytes.format, this would be relatively easy to do in CPython, if there's agreement to add it. -- Eric. From nagle at animats.com Thu Jul 22 20:04:26 2010 From: nagle at animats.com (John Nagle) Date: Thu, 22 Jul 2010 11:04:26 -0700 Subject: [Python-Dev] Set the namespace free! In-Reply-To: References: Message-ID: <4C48882A.70204@animats.com> On 7/22/2010 5:45 AM, python-dev-request at python.org wrote: > Message: 10 > Date: Thu, 22 Jul 2010 16:04:00 +0200 > From: Bartosz Tarnowski > To:python-dev at python.org > Subject: [Python-Dev] Set the namespace free! > Message-ID:<4C484FD0.2080803 at zlotniki.pl> > Content-Type: text/plain; charset=UTF-8; format=flowed > > > Hello, guys. > > Python has more and more reserved words over time. It becomes quite annoying, > since you can not use variables and attributes of such names. Suppose I want to > make an XML parser that reads a document and returns an object with attributes > corresponding to XML element attributes: > > > elem = parse_xml("") > > print elem.param > > What should I do then, when the attribute is a reserver word? That's a misuse of attributes. When you need objects with unconstrained fields, inherit them from "dict", and write print(elem['param']) This protects you not only from name clashes, but from difficulties with names that don't fit Python attribute syntax. (BeautifulSoup occasionally crashes due to this problem when parsing malformed HTML). You can still provide a "__getattr__" function, if desired, for convenient access to commonly used attributes. Using "setattr" to set attributes, where the attribute string comes from an external source, can create a security hole. Remember that you can override functions on an object, for that object only, by setting an attribute. This offers the opportunity for an attack similar to SQL injection. Think about what this can do to a parser that has and calls a method "display" for each element: You are pwned. John Nagle From pje at telecommunity.com Thu Jul 22 20:19:50 2010 From: pje at telecommunity.com (P.J. Eby) Date: Thu, 22 Jul 2010 14:19:50 -0400 Subject: [Python-Dev] PEP 382 progress: import hooks In-Reply-To: <4C483EDF.7050002@v.loewis.de> References: <4C483EDF.7050002@v.loewis.de> Message-ID: <20100722181939.1FA273A40D9@sparrow.telecommunity.com> At 01:51 PM 7/22/2010 +0100, Martin v. L?wis wrote: >At EuroPython, I sat down with Brett and we propose an approach >how namespace packages get along with import hooks. I reshuffled >the order in which things get done a little bit, and added a >section that elaborates on the hooks. > >Basically, a finder will need to support a find_path method, >return all .pth files, and a loader will need to support a >load_module_with_path method, to initialize __path__. > >Please comment if you think that this needs further changes; I'm not certain I understand it precisely. There seem to be some ambiguities in the spec, e.g.: "If fullname is not found, is not a package, or does not have any *.pth files, None must be returned." What does "is not a package" actually mean in that context? What happens if an empty list is returned - does that mean the importer is saying, "this is a package, whether it has an __init__.py or not?" As for the "list of strings" returned, is each string the entire contents of the .pth file? Is it to be \n-separated, or is any universal-newlines-compatible string accepted? Is there a particular order in which .pth file contents are to be returned? Regarding load_module_with_path(), how does its specification differ from simply creating a module in sys.modules, setting its __path__, and then invoking the standard load_module()? (i.e., is this method actually needed, since a correct PEP 302 loader *must* reuse an existing module object in sys.modules) >I'll hope to start implementing it soon. Am I correct in understanding that, as written, one would have to redefine __import__ to implement this in a library for older Python versions? Or is it implementable as a meta_path importer? >Regards, >Martin Thanks for your work on this, I was just thinking about pinging to see how it was going. ;-) (I want setuptools 0.7 to be able to supply an add-on module for supporting this PEP in older Pythons, so that its current .pth hacks for implementing namespace packages can be dropped.) From gregory.smith3 at sympatico.ca Thu Jul 22 18:53:51 2010 From: gregory.smith3 at sympatico.ca (gregory.smith3 at sympatico.ca) Date: Thu, 22 Jul 2010 16:53:51 +0000 Subject: [Python-Dev] Set the namespace free! In-Reply-To: References: <4C484FD0.2080803@zlotniki.pl>, Message-ID: I have no idea why my last post was a copy of the previous one. Webmail choking on a hairball. It was supposed to say: === Oops, :name does break things, e.g. if x :return So, ::name or &name or |name or whatever. I'm very amused by all the jokes about turning python into perl, but there's a good idea here that doesn't actually require that... Regards (and hoping it works this time) - Greg -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.belopolsky at gmail.com Thu Jul 22 20:49:17 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 22 Jul 2010 14:49:17 -0400 Subject: [Python-Dev] Set the namespace free! In-Reply-To: References: <4C484FD0.2080803@zlotniki.pl> Message-ID: On Thu, Jul 22, 2010 at 12:53 PM, wrote: .. > So, ::name or &name or |name or whatever. > > I'm very amused by all the jokes about turning python into perl, but there's > a good idea here that doesn't actually require that... No, there isn't. And both '&' and '|' are valid python operators that cannot be used this way. If you want ::, I think you can find a language or two to your liking. :-) From merwok at netwok.org Thu Jul 22 21:01:17 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Thu, 22 Jul 2010 21:01:17 +0200 Subject: [Python-Dev] Set the namespace free! In-Reply-To: <4C48882A.70204@animats.com> References: <4C48882A.70204@animats.com> Message-ID: <4C48957D.6090208@netwok.org> > Using "setattr" to set attributes, where the attribute string > comes from an external source, can create a security hole. Remember > that you can override functions on an object, for that object only, > by setting an attribute. This offers the opportunity for an attack > similar to SQL injection. Think about what this can do to a parser > that has and calls a method "display" for each element: > > > > You are pwned. Nope. You?d have to give setattr a function object, not a string. Regards From eric at netwok.org Thu Jul 22 20:59:08 2010 From: eric at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Thu, 22 Jul 2010 20:59:08 +0200 Subject: [Python-Dev] Set the namespace free! In-Reply-To: <4C48882A.70204@animats.com> References: <4C48882A.70204@animats.com> Message-ID: <4C4894FC.8080809@netwok.org> > Using "setattr" to set attributes, where the attribute string > comes from an external source, can create a security hole. Remember > that you can override functions on an object, for that object only, > by setting an attribute. This offers the opportunity for an attack > similar to SQL injection. Think about what this can do to a parser > that has and calls a method "display" for each element: > > > > You are pwned. Nope. You?d have to give setattr a function object, not a string. Regards From dreamingforward at gmail.com Thu Jul 22 21:29:46 2010 From: dreamingforward at gmail.com (average) Date: Thu, 22 Jul 2010 13:29:46 -0600 Subject: [Python-Dev] Python-dev signal-to-noise processing question In-Reply-To: <87fwzbyjme.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20100719162729.GF21891@phd.pp.ru> <20100720105928.GA4289@phd.pp.ru> <20100720185944.GA10496@phd.pp.ru> <20100722060909.GB14498@phd.pp.ru> <87fwzbyjme.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: > ?> > ...After a sufficient period of waiting, say a day or two with no response: > ?> > ?> ? ?Ok, I'll wait a bit longer. > > I don't think that's a good idea. My bad, I really only meant a "sufficient delay to allow the possibility of an interested party replying". I actually figured about a day. > OTOH I think as quick as possible an answer is a good idea here. ?It > saves the intended audience the thought about whether to reply or not, The intended audience (the python developers or other interested lurkers) hardly need to give much thought on whether or not to reply. No one on a -dev list is expected to be a tutor. To have any reply whatsoever should be considered a bestowal of time without equal merit, or gratis. Ideally, those who aren't subscribed should get an immediate, automatic reply about resources available and how to engage the community without raising the ire of same. If there's no such automatic resource, but a designated list manager, this should be sent in private so as not to engage or bother the rest of the list with canned responses, otherwise publically so as to avoid duplication of effort. To those who *are* subscribed, they probably don't need to be told. A reference or link to ESR's "How to Ask Questions The Smart Way" (http://catb.org/esr/faqs/smart-questions.html) is a pretty good inclusion also. For those who may have never heard of it, it's pretty much the authority on developer list etiquette. $0.02, Marcos From reid.kleckner at gmail.com Thu Jul 22 22:49:20 2010 From: reid.kleckner at gmail.com (Reid Kleckner) Date: Thu, 22 Jul 2010 13:49:20 -0700 Subject: [Python-Dev] Set the namespace free! In-Reply-To: References: <4C484FD0.2080803@zlotniki.pl> Message-ID: On Thu, Jul 22, 2010 at 11:49 AM, Alexander Belopolsky wrote: > On Thu, Jul 22, 2010 at 12:53 PM, ? wrote: >> I'm very amused by all the jokes about turning python into perl, but there's >> a good idea here that doesn't actually require that... > > No, there isn't. ?And both '&' and '|' are valid python operators that > cannot be used this way. > > If you want ::, I think you can find a language or two to your liking. :-) A syntax for escaping reserved words used as identifiers is a worthy and interesting idea, but I don't think it's worth adding to Python. Appending '_' in the kinds of cases described by the OP is what's suggested by the style guide, and seems acceptable to me. Prefixing all reserved words with punctuation as suggested by the OP is, of course, completely ludicrous. He might just be trolling. Reid P.S. "I'm not trolling!" http://www.youtube.com/watch?v=6bMLrA_0O5I P.P.S. Sorry, I couldn't help it. From tjreedy at udel.edu Thu Jul 22 23:35:00 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 22 Jul 2010 17:35:00 -0400 Subject: [Python-Dev] Python 3 optimizations... In-Reply-To: References: Message-ID: On 7/22/2010 9:36 AM, stefan brunthaler wrote: >> Depending on the size and complexity of the >> patches, it may still be worth exploring for 3.2. If your work speeds CPython, U.S. would have to be even better to knock it out. > I am currently not aware of the planned release schedule, but I think > that some of the techniques could very well be released in 3.2 -- with > a conditional compilation target similar to the computed-goto feature. 2.6.6 is scheduled for August. 3.2 releases should start thereafter with a December target. On licenses. Look for the contributor agreement on the site. It says that you license your code on a non-exclusive basis to the PSF under one of two licenses. Both allow PSF to relicense and redistribute your code as part of a Python release. I recommend you print it out and, as someone else suggested, check with your advisor/university as to whether there is any problem with you contributing the software under one of*those particular licenses*. -- Terry Jan Reedy From tjreedy at udel.edu Thu Jul 22 23:51:08 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 22 Jul 2010 17:51:08 -0400 Subject: [Python-Dev] Set the namespace free! In-Reply-To: <4C48882A.70204@animats.com> References: <4C48882A.70204@animats.com> Message-ID: On 7/22/2010 2:04 PM, John Nagle wrote: >> From: Bartosz Tarnowski >> Python has more and more reserved words over time ... >> What should I do then, when the attribute is a reserver word? I am going to be a grinch and note that this is strictly a usage question with no development implications. It therefore is off topic here and should have been asked and answered on python-list or the mirrored newsgroup. > That's a misuse of attributes. When you need objects with > unconstrained fields, inherit them from "dict", and write > > print(elem['param']) Right ;-) -- Terry Jan Reedy From tjreedy at udel.edu Fri Jul 23 00:02:07 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 22 Jul 2010 18:02:07 -0400 Subject: [Python-Dev] Python-dev signal-to-noise processing question In-Reply-To: References: <20100719162729.GF21891@phd.pp.ru> <20100720105928.GA4289@phd.pp.ru> <20100720185944.GA10496@phd.pp.ru> <20100722060909.GB14498@phd.pp.ru> <87fwzbyjme.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On 7/22/2010 3:29 PM, average wrote: Speacking of etiquette, it is traditional to use real names in the from field on pydev. It will get you more attention and respect. > A reference or link to ESR's "How to Ask Questions The Smart Way" > (http://catb.org/esr/faqs/smart-questions.html) is a pretty good > inclusion also. For those who may have never heard of it, it's pretty > much the authority on developer list etiquette. I agree > Marcos Well, that is a start ;-) -- Terry Jan Reedy From ncoghlan at gmail.com Fri Jul 23 00:26:34 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 23 Jul 2010 08:26:34 +1000 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> <20100713163544.5f3a60d2@pitrou.net> Message-ID: On Fri, Jul 23, 2010 at 12:42 AM, Georg Brandl wrote: > Sure -- I don't think this is a showstopper for regex. ?However if we don't > include regex in a future version, we might think about increasing MAXCACHE > a bit, and maybe not clear the cache when it reaches its max length, but > rather remove another element. Yikes, I didn't know it did that. That certainly sounds like it should be an RFE in its own right - some basic form of Least Recently Used accounting should be beneficial (although the extra bookkeeping might hurt scripts that aren't hitting the cache limit). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From gregory.smith3 at sympatico.ca Thu Jul 22 23:01:37 2010 From: gregory.smith3 at sympatico.ca (gregory.smith3 at sympatico.ca) Date: Thu, 22 Jul 2010 21:01:37 +0000 Subject: [Python-Dev] Set the namespace free! In-Reply-To: References: <4C484FD0.2080803@zlotniki.pl>, , , Message-ID: > Date: Thu, 22 Jul 2010 14:49:17 -0400 > Subject: Re: [Python-Dev] Set the namespace free! > From: alexander.belopolsky at gmail.com > To: gregory.smith3 at sympatico.ca > CC: python-dev at python.org > > On Thu, Jul 22, 2010 at 12:53 PM, wrote: > .. > > So, ::name or &name or |name or whatever. > > > > I'm very amused by all the jokes about turning python into perl, but there's > > a good idea here that doesn't actually require that... > > No, there isn't. And both '&' and '|' are valid python operators that > cannot be used this way. > Um, of course. Serious brain freeze today, using too many languages at once. Yeah, that's it. Despite my knuckleheadedness, I say there's still a hole here that can be easily plugged. it's clumsy that you can't call, e.g. GenerateURL( reqtype='basic', class='local') other than by GenerateURL( **{'reqtype': 'basic', 'class': 'local'}) ... just because 'class' is a keyword. That's letting a parser issue degrade the value of a really good feature. Likewise for attributes; python allows you to have named parameters or attributes called 'class' and 'import' if you like; it just doesn't let you write them directly; this restriction doesn't seem to be necessary except for the parse issue, which is fixable. I.e. nothing would break by allowing GenerateURL(::class = 'local') or Request.::class. See my previous longer post for the full discussion (not the short stupid one). -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Fri Jul 23 00:25:21 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 22 Jul 2010 18:25:21 -0400 Subject: [Python-Dev] Python-dev signal-to-noise processing question In-Reply-To: <201007222222.52796.steve@pearwood.info> References: <20100722060909.GB14498@phd.pp.ru> <87fwzbyjme.fsf@uwakimon.sk.tsukuba.ac.jp> <201007222222.52796.steve@pearwood.info> Message-ID: On 7/22/2010 8:22 AM, Steven D'Aprano wrote: > On Thu, 22 Jul 2010 07:02:33 pm Stephen J. Turnbull wrote: > >> OTOH I think as quick as possible an answer is a good idea here. It >> saves the intended audience the thought about whether to reply or >> not, and an instant, constructive answer says that somebody cares. > > +1 Ditto. The sooner a person redirects a question to python-list, the sooner I will answer if I have an answer. And I do answer a lot of questions on Py-list, especially those that have sat a few hours without an answer. > I think that waiting "a day or two" as suggested by Average does nobody > any good. The only 'waiting' by anyone should be a few days by someone who posts on pylist *before* they try here for other expers. -- Terry Jan Reedy From ncoghlan at gmail.com Fri Jul 23 00:31:33 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 23 Jul 2010 08:31:33 +1000 Subject: [Python-Dev] :program: markup in .rst documentation In-Reply-To: References: <4C45DE0A.9080806@netwok.org> Message-ID: On Fri, Jul 23, 2010 at 2:25 AM, Alexander Belopolsky wrote: > Note also that argparse/optparse does not know about -m way either: > > $ python -m profile -h > Usage: profile.py [-o output_file_path] [-s sort] scriptfile [arg] ... > > I am not sure if it is possible for argparse to distinguish between -m > and file name invocations, so maybe it should always use python -m > modname if sys.argv[0] ends with .py. ?Again, this is quite important > if the module being run is inside a package. If __main__.__package__ is set to anything other than None, odds are fairly good that runpy did the import. If it's set to a non-empty string, you definitely *shouldn't* be invoking the relevant main script by filename. Sounds like one for the tracker. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Fri Jul 23 00:33:59 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 23 Jul 2010 08:33:59 +1000 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: On Fri, Jul 23, 2010 at 3:30 AM, Alexander Belopolsky wrote: > I see three solutions: > > 1. ?Minimal: ?do not rename test_trace in 2.7 and add trace module > tests to the existing file. ?Whether to revert test_trace to > test_line_tracing renaming in 3.2 can be decided after we have 2.7 > tests written. > 2. Rename test_trace in 2.7/3.1 and test_line_tracing in 3.2 to > test_sys_settrace. > 3. Rename test_trace in 2.7/3.1 to test_line_tracing. > > My preference in #2, but I can live with #1 ?(assuming Eli is the one > who will write the tests:-). I'd say just fix it and go with #2 (just update the comment in test_sys as well!) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From breamoreboy at yahoo.co.uk Fri Jul 23 01:11:25 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 23 Jul 2010 00:11:25 +0100 Subject: [Python-Dev] Python-dev signal-to-noise processing question In-Reply-To: References: <20100722060909.GB14498@phd.pp.ru> <87fwzbyjme.fsf@uwakimon.sk.tsukuba.ac.jp> <201007222222.52796.steve@pearwood.info> Message-ID: On 22/07/2010 23:25, Terry Reedy wrote: > On 7/22/2010 8:22 AM, Steven D'Aprano wrote: >> On Thu, 22 Jul 2010 07:02:33 pm Stephen J. Turnbull wrote: >> >>> OTOH I think as quick as possible an answer is a good idea here. It >>> saves the intended audience the thought about whether to reply or >>> not, and an instant, constructive answer says that somebody cares. >> >> +1 > > Ditto. The sooner a person redirects a question to python-list, the > sooner I will answer if I have an answer. And I do answer a lot of > questions on Py-list, especially those that have sat a few hours without > an answer. > >> I think that waiting "a day or two" as suggested by Average does nobody >> any good. > > The only 'waiting' by anyone should be a few days by someone who posts > on pylist *before* they try here for other expers. > If only the same principal applied on the issue tracker. Anyone fancy taking on a few orphans, allowing for the fact that you might get told off for creating noise or for offending devs who don't like being reminded of outstanding issues? Still, what's nine years to someone who's waiting for a bug fix? Kindest regards. Mark Lawrence. From g.brandl at gmx.net Fri Jul 23 03:46:01 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 23 Jul 2010 02:46:01 +0100 Subject: [Python-Dev] Set the namespace free! In-Reply-To: References: <4C484FD0.2080803@zlotniki.pl> Message-ID: Am 22.07.2010 21:49, schrieb Reid Kleckner: > On Thu, Jul 22, 2010 at 11:49 AM, Alexander Belopolsky > wrote: >> On Thu, Jul 22, 2010 at 12:53 PM, wrote: >>> I'm very amused by all the jokes about turning python into perl, but there's >>> a good idea here that doesn't actually require that... >> >> No, there isn't. And both '&' and '|' are valid python operators that >> cannot be used this way. >> >> If you want ::, I think you can find a language or two to your liking. :-) > > A syntax for escaping reserved words used as identifiers is a worthy > and interesting idea, but I don't think it's worth adding to Python. Exactly. Such a feature is much more needed for languages that don't feature globals(), getattr() and __dict__s. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ben+python at benfinney.id.au Fri Jul 23 02:59:32 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 23 Jul 2010 10:59:32 +1000 Subject: [Python-Dev] Python-dev signal-to-noise processing question References: <20100722060909.GB14498@phd.pp.ru> <87fwzbyjme.fsf@uwakimon.sk.tsukuba.ac.jp> <201007222222.52796.steve@pearwood.info> Message-ID: <87r5iv57yj.fsf@benfinney.id.au> Steven D'Aprano writes: > We don't need to make excuses for why we don't give the answer here. > It's enough to give the reason -- it's off-topic for this list, which > is about the development of Python. That and a pointer to the right > list is, in my opinion, all we need to say. We don't need to sugar > coat it -- we're all adults here. That's fine for people who already feel they are part of this particular community. The concern, as I understand it from earlier in the thread, is the regular only-one-opportunity-at-a-first-impression response given to someone *new to* this community, who clearly doesn't yet know the mores and expectations. In that case, it's not enough to say ?we're all adults?, because we're also all human: the perceived tone of a first response we receive will strongly influence our impression of the wider community, despite any appeals one might make to the irrationality of that reaction. Asking about how Python works is a likely first step to becoming a valuable contributor later on (even if most such steps never go beyond the first), and a poor impression will surely kill most of that potential. So I do think it's worth this discussion about how to achieve the conflicting goals already mentioned ? efficient communication and welcome to newcomers ? and that ?we don't need to sugar coat it? is incorrect for this purpose. -- \ ?Working out the social politics of who you can trust and why | `\ is, quite literally, what a very large part of our brain has | _o__) evolved to do.? ?Douglas Adams | Ben Finney From greg.ewing at canterbury.ac.nz Fri Jul 23 05:10:11 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 23 Jul 2010 15:10:11 +1200 Subject: [Python-Dev] Set the namespace free! In-Reply-To: References: <4C484FD0.2080803@zlotniki.pl> Message-ID: <4C490813.30903@canterbury.ac.nz> On 23/07/10 04:24, gregory.smith3 at sympatico.ca wrote: > I've suggested :name, which doesn't break old code, I'm not so sure about that. Consider foo[a::b] Do you parse that as a 3-element slice, or as a 2-element slice with :b as the second element? -- Greg From nas at arctrix.com Fri Jul 23 05:15:08 2010 From: nas at arctrix.com (Neil Schemenauer) Date: Fri, 23 Jul 2010 03:15:08 +0000 (UTC) Subject: [Python-Dev] module shutdown procedure References: <4C470E2B.5060406@timgolden.me.uk> <20100721175819.3b037ad9@pitrou.net> <20100722140046.5e7551a1@pitrou.net> <1279801772.3201.3.camel@localhost.localdomain> Message-ID: Georg Brandl wrote: > Am 22.07.2010 13:29, schrieb Antoine Pitrou: >> Is it the reason why? With the new module creation API in 3.x, extension >> modules should be able to handle deletion of their own internal >> resources. > > Yes, but as Martin noted at the summit, nobody since went through all the > extension modules and changed them to use a struct instead of globals. And that's a necessary but likely not sufficient condition for the GC based module shutdown scheme to work. It really would take a huge amount of effort to make the idea fly. That doesn't mean it is not worth doing. The current scheme is pretty ugly. Neil From steve at pearwood.info Fri Jul 23 06:11:21 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 23 Jul 2010 14:11:21 +1000 Subject: [Python-Dev] Python-dev signal-to-noise processing question In-Reply-To: <87r5iv57yj.fsf@benfinney.id.au> References: <201007222222.52796.steve@pearwood.info> <87r5iv57yj.fsf@benfinney.id.au> Message-ID: <201007231411.22871.steve@pearwood.info> On Fri, 23 Jul 2010 10:59:32 am Ben Finney wrote: > Steven D'Aprano writes: > > We don't need to make excuses for why we don't give the answer > > here. It's enough to give the reason -- it's off-topic for this > > list, which is about the development of Python. That and a pointer > > to the right list is, in my opinion, all we need to say. We don't > > need to sugar coat it -- we're all adults here. > > That's fine for people who already feel they are part of this > particular community. The concern, as I understand it from earlier in > the thread, is the regular only-one-opportunity-at-a-first-impression > response given to someone *new to* this community, who clearly > doesn't yet know the mores and expectations. I'm not suggesting that we need to be rude. But we can give them the respect of starting off by treating them according to the mores and expectations of our community, rather than by assuming that they will immediately take offence or be frightened off, or by immediately getting defensive about the fact that this isn't a beginner's list. We have nothing to be ashamed of for wanting to keep this list for dev related issues, why make excuses for it? In other words, keep doing what we've been doing for as long as I've been subscribed to this list: answer politely but firmly that this is the wrong list, point them to the correct place, and, if the mood strikes you, give a brief answer to the question that doesn't invite a long discussion. I think that's fair and respectful. -- Steven D'Aprano From stefan_ml at behnel.de Fri Jul 23 07:19:40 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 23 Jul 2010 07:19:40 +0200 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: References: <4C470E2B.5060406@timgolden.me.uk> <20100721175819.3b037ad9@pitrou.net> Message-ID: Maciej Fijalkowski, 22.07.2010 10:43: > On Thu, Jul 22, 2010 at 9:51 AM, Brett Cannon wrote: >> Basically the whole setting a module's globals to None was done before gc >> came into the language. Now that it's there it seems that it might work to >> simply let gc clean up the module itself. But this brings in the question of >> how to order calling finalizers. It seemed the room had no specific answer >> other than it might just have to be near random as figuring out a reasonable >> order is hard. > > If you have a cycle, there is no other way than breaking it at random, right? This is where reference counting is pretty handy. It allows users to deliberately increase the reference count of certain objects that they want to stay alive longer than their own object, so that the GC cannot just collect them before they call the opposite decref in your finaliser. This obviously requires taking care that the objects you manipulate here cannot create reference-cycles themselves, but given that, it helped me fix a lot of hard to track down crashes in lxml. Stefan From stefan_ml at behnel.de Fri Jul 23 07:48:54 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 23 Jul 2010 07:48:54 +0200 Subject: [Python-Dev] module shutdown procedure In-Reply-To: References: <4C470E2B.5060406@timgolden.me.uk> <20100721175819.3b037ad9@pitrou.net> <20100722140046.5e7551a1@pitrou.net> <1279801772.3201.3.camel@localhost.localdomain> Message-ID: Georg Brandl, 22.07.2010 16:13: > Am 22.07.2010 13:29, schrieb Antoine Pitrou: >> Le jeudi 22 juillet 2010 ? 07:23 -0500, Benjamin Peterson a ?crit : >>> 2010/7/22 Antoine Pitrou: >>>> Brett Cannon wrote: >>>>> Basically the whole setting a module's globals to None was done before gc >>>>> came into the language. Now that it's there it seems that it might work to >>>>> simply let gc clean up the module itself. >>>> >>>> There is a patch at http://bugs.python.org/issue812369 for GC-based >>>> module shutdown, but it doesn't actually remove the setting of module >>>> globals to None. I think further testing and experimentation would be >>>> required to validate it. >>> >>> Also, it seems to have been stalled by static globals in extension >>> modules that the gc doesn't know about. >> >> Is it the reason why? With the new module creation API in 3.x, extension >> modules should be able to handle deletion of their own internal >> resources. > > Yes, but as Martin noted at the summit, nobody since went through all the > extension modules and changed them to use a struct instead of globals. The Cython project has had this on the agenda ever since the early days of 3.0, but we never got around to investing the time. http://trac.cython.org/cython_trac/ticket/173 http://trac.cython.org/cython_trac/ticket/218 We already generate optional module cleanup code as an atexit function to help with valgrind, so much of the code required for m_clear() is already there, but getting it right (i.e. making all module globals local to an instance) would require some effort on our side that is not currently considered worth it (just ask yourself what you want first: blazingly fast generator functions? Or Py3-only module cleanup code?). It wouldn't be hard to do, though, and adapting Cython here would immediately migrate a whole bunch of extension modules to the new module handling API. Note that this may not mean that all of these modules will work out of the box. The current cleanup code is disabled by default because module globals can be used in finalisers, which means that the module cleanup can crash if the globals are cleaned up in the wrong order. This could be worked around, though, as we could potentially detect required globals at compile time and either change the order of their cleanup or emit a warning that there is no such order in the face of cycles. So, as always, it's all just a matter of investing the time to get this implemented. Stefan From stefan_ml at behnel.de Fri Jul 23 08:10:28 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 23 Jul 2010 08:10:28 +0200 Subject: [Python-Dev] Inline caching for method calls Message-ID: Hi, Stefan Brunthaler started a thread on python-dev about a couple of patches he has written for CPython as part of his PhD. It seems that he has implemented inline caching for CPython. http://comments.gmane.org/gmane.comp.python.devel/115362 http://en.wikipedia.org/wiki/Inline_caching Something like this might be interesting for Cython, too. He hasn't published the patches yet, but they might well be worth a look when he does. Stefan From stefan_ml at behnel.de Fri Jul 23 08:13:17 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 23 Jul 2010 08:13:17 +0200 Subject: [Python-Dev] Sorry, wrong list, please ignore - Re: Inline caching for method calls In-Reply-To: References: Message-ID: Sorry, this should have gone to cython-dev, not python-dev ... From stefan_ml at behnel.de Fri Jul 23 08:23:16 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 23 Jul 2010 08:23:16 +0200 Subject: [Python-Dev] Python 3 optimizations... In-Reply-To: References: Message-ID: Hi, stefan brunthaler, 22.07.2010 13:22: > during the last year, I have developed a couple of quickening-based > optimizations for the Python 3.1 interpreter. As part of my PhD > programme, I have published a first technique that combines quickening > with inline caching at this year's ECOOP, and subsequently extended > this technique to optimize several load instructions as well as > eliminate redundant reference counting operations from instructions, > which has been accepted for publication for an upcoming conference. > [...] > I wonder whether you would be interested in integrating these > optimizations with the Python 3 distribution, hence this mail. I could > send copies of the papers, as well as provide my prototype source code > to interested members of the python development community. I'm absolutely interested, although not for the CPython project but for Cython. I wonder how you do inline caching in Python if the methods of a type can be replaced by whatever at runtime. Could you elaborate on that? Based on what information do you switch between inlining states? Or do you restrict yourself to builtin types? That might be worth it already, just think of list.append(). We have an optimistic optimisation for object.append() in Cython that gives us massive speed-ups in loops that build lists, even if we don't know at compile time that we are dealing with lists. Stefan From guido at python.org Fri Jul 23 08:44:42 2010 From: guido at python.org (Guido van Rossum) Date: Fri, 23 Jul 2010 07:44:42 +0100 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: <4C4884CA.9080603@trueblade.com> References: <4C470E2B.5060406@timgolden.me.uk> <4C4884CA.9080603@trueblade.com> Message-ID: On Thu, Jul 22, 2010 at 6:50 PM, Eric Smith wrote: > Thanks for writing this, Tim. > > On 7/21/10 11:11 AM, Tim Golden wrote: > >> The issue of a __format__ equivalent for bytes was also raised as was the >> idea of object methods to render an object as string or bytes, which could >> be used in the polymorphic functions above. > > Does this mean bytes.__format__, or does it mean bytes.format? > bytes.__format__ already exists, but it just calls object.__format__. I'm > not sure what the desired functionality for bytes.__format__ would be. > > If it means bytes.format, this would be relatively easy to do in CPython, if > there's agreement to add it. Indeed, we meant b'...{}...{}...'.format(x, y). The problem is that it can't invoke x.__format__() or y.__format__() since those will return text strings instead of bytes. A proposed solution was to try x.__bformat__() etc. Another proposed solution was to limit x and y to a small set of common types and hard-code their meanings (e.g. numbers, bytes, and dates). -- --Guido van Rossum (python.org/~guido) From stefan at brunthaler.net Fri Jul 23 08:48:26 2010 From: stefan at brunthaler.net (stefan brunthaler) Date: Fri, 23 Jul 2010 08:48:26 +0200 Subject: [Python-Dev] Python 3 optimizations... In-Reply-To: References: Message-ID: Hi, I guess it would be a good idea to quickly outline my inline caching approach, so that we all have a basic understanding of how it works. If we take for instance the BINARY_ADD instruction, the interpreter evaluates the actual operand types and chooses the matching operation implementation at runtime, i.e., operands that are unicode strings will be concatenated via unicode_concatenate, for float operands on the other hand, the interpreter would end up invoking float_add via binary_op1. Now, a very efficient way to achieve purely interpretative inline caching is to quicken the type-generic BINARY_ADD instruction to a type-dependent FLOAT_ADD instruction (this technique, i.e., inline caching via quickening, is the primary contribution of my ECOOP paper). Hence, I have a very simple code generator, that generates type-dependent interpreter instructions in a pre-compile step of the interpreter, and uses runtime type information to quicken/rewrite instructions. Aside of the operators, I have implemented this quickening technique for FOR_ITER, COMPARE_OP and CALL_FUNCTION instructions. > I'm absolutely interested, although not for the CPython project but for > Cython. I wonder how you do inline caching in Python if the methods of a > type can be replaced by whatever at runtime. Could you elaborate on that? > Currently, I only provide optimized derivatives for several separate call targets, i.e., whether a call target is a C function with varargs, or a Python function/method--this already eliminates a lot of overhead from invoking call_function. Based on further quantitative analysis, I can easily provide inline cached derivatives of frequently called functions, such as some builtin primitives. > Based on what information do you switch between inlining states? > I have instrumented versions of some functions that allow me to make quickening decisions, such as binary_op1, do_richcompare, or call_function, where I can quicken instructions to an optimized, inline cached, instruction derivative. > Or do you restrict yourself to builtin types? > Currently, my approach provides optimized derivative instructions for the standard library, e.g., unicode strings, numerical objects, containers, and iterators. > That might be worth it > already, just think of list.append(). We have an optimistic optimisation for > object.append() in Cython that gives us massive speed-ups in loops that > build lists, even if we don't know at compile time that we are dealing with > lists. > Yes, that sounds like a reasonable thing to do. I could provide much more optimized derivatives based on application profiles, too. Since I use a simple code generator for generating the derivatives, it would also be possible to provide end-users with the means to analyze their apps and generate optimized instruction derivatives matching their profile. Regards, --stefan From stefan_ml at behnel.de Fri Jul 23 10:38:32 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 23 Jul 2010 10:38:32 +0200 Subject: [Python-Dev] Python 3 optimizations... In-Reply-To: References: Message-ID: stefan brunthaler, 23.07.2010 08:48: > I guess it would be a good idea to quickly outline my inline caching > approach, so that we all have a basic understanding of how it works. Yes, that certainly makes it easier to discuss. > If we take for instance the BINARY_ADD instruction, the interpreter > evaluates the actual operand types and chooses the matching operation > implementation at runtime, i.e., operands that are unicode strings > will be concatenated via unicode_concatenate, for float operands on > the other hand, the interpreter would end up invoking float_add via > binary_op1. Now, a very efficient way to achieve purely interpretative > inline caching is to quicken the type-generic BINARY_ADD instruction > to a type-dependent FLOAT_ADD instruction (this technique, i.e., > inline caching via quickening, is the primary contribution of my ECOOP > paper). Hence, I have a very simple code generator, that generates > type-dependent interpreter instructions in a pre-compile step of the > interpreter, and uses runtime type information to quicken/rewrite > instructions. > Aside of the operators, I have implemented this quickening technique > for FOR_ITER, COMPARE_OP and CALL_FUNCTION instructions. This sounds like wpython (a CPython derivative with a wider set of byte code commands) could benefit from it. Do I understand correctly that you modify the byte code of modules/functions at runtime? >> I'm absolutely interested, although not for the CPython project but for >> Cython. I wonder how you do inline caching in Python if the methods of a >> type can be replaced by whatever at runtime. Could you elaborate on that? > > Currently, I only provide optimized derivatives for several separate > call targets, i.e., whether a call target is a C function with > varargs, or a Python function/method--this already eliminates a lot of > overhead from invoking call_function. Ah, yes, that makes good sense. So you basically add an intermediate step to calls that provides faster dispatch for known C functions. >> Or do you restrict yourself to builtin types? > > Currently, my approach provides optimized derivative instructions for > the standard library, e.g., unicode strings, numerical objects, > containers, and iterators. I'm interested in the code that determines what can be optimised in what way. I read that Jython recently received a contribution that provides type information for lots of modules and builtins, but having something like that for CPython would be cool. >> That might be worth it >> already, just think of list.append(). We have an optimistic optimisation for >> object.append() in Cython that gives us massive speed-ups in loops that >> build lists, even if we don't know at compile time that we are dealing with >> lists. >> > Yes, that sounds like a reasonable thing to do. I could provide much > more optimized derivatives based on application profiles, too. Since I > use a simple code generator for generating the derivatives, it would > also be possible to provide end-users with the means to analyze their > apps and generate optimized instruction derivatives matching their > profile. Such an approach would also be very useful for Cython. Think of a profiler that runs a program in CPython and tells you exactly what static type annotations to put where in your Python code to make it compile to a fast binary with Cython. Or, even better, it could just spit out a .pxd file that you drop next to your .py file and that provides the static type information for you. Stefan From ronaldoussoren at mac.com Thu Jul 22 16:58:06 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Thu, 22 Jul 2010 15:58:06 +0100 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <20100722164036.7a80d27c@snowdog> References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> Message-ID: <63B36042-C4F7-4BCA-8623-5182405D8852@mac.com> On 22 Jul, 2010, at 15:40, Barry Warsaw wrote: > > > Abstract > ======== > > PEP 3147 [1]_ described an extension to Python's import machinery that > improved the sharing of Python source code, by allowing more than one > byte compilation file (.pyc) to be co-located with each source file. > > This PEP defines an adjunct feature which allows the co-location of > extension module files (.so) in a similar manner. This optional, > build-time feature will enable downstream distributions of Python to > more easily provide more than one Python major version at a time. I guess this is not an explicit goal of this PEP, but the structure is very close to supporting multiple system architectures at the same time. I regularly develop code that needs to run on Windows, Linux and OSX and it is very convenient to do so in a shared directory tree (locally on one machine and accessed using remote mounts on the other ones). This works fine for pure python code, but I currently have to resort to tricks for extension modules. > > Proposal > ======== > > The configure/compilation options chosen at Python interpreter > build-time will be encoded in the shared library file name for > extension modules. This "tag" will appear between the module base > name and the operation file system extension for shared libraries. > > The following information *MUST* be included in the shared library > file name: > > * The Python implementation (e.g. cpython, pypy, jython, etc.) > * The interpreter's major and minor version numbers > > These two fields are separated by a hyphen and no dots are to appear > between the major and minor version numbers. E.g. ``cpython-32``. > > Python implementations *MAY* include additional flags in the file name > tag as appropriate. For example, on POSIX systems these flags will > also contribute to the file name: > > * ``--with-pydebug`` (flag: ``d``) > * ``--with-pymalloc`` (flag: ``m``) > * ``--with-wide-unicode`` (flag: ``u``) > > By default in Python 3.2, ``configure`` enables ``--with-pymalloc`` so > shared library file names would appear as ``foo.cpython-32m.so``. > When the other two flags are also enabled, the file names would be > ``foo.cpython-32dmu.so``. A way to generically solve my problem is to add the platform name as well, such as "foo.cpython-32m-darwin.so" or "foo.cpython-32mu-linux2.so". Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From stefan at brunthaler.net Fri Jul 23 10:58:41 2010 From: stefan at brunthaler.net (stefan brunthaler) Date: Fri, 23 Jul 2010 10:58:41 +0200 Subject: [Python-Dev] Python 3 optimizations... In-Reply-To: References: Message-ID: > This sounds like wpython (a CPython derivative with a wider set of byte code > commands) could benefit from it. > I am aware of the wpython project of Cesare di Mauro. I change the instruction format from bytecode to wordcode, too (because it allows for more efficient instruction decoding). Contrary to his approach, however, I do not change the instruction encoding to pack in additional optimizations. (I hope to have put that correctly; I have seen his slides about a year ago.) > Do I understand correctly that you modify the byte code of modules/functions > at runtime? > Yes. Quickening is runtime only optimization technique that rewrites instructions from a generic instruction to an optimized derivative (orignally for the Java virtual machine). It is completely hidden from the compiler and has no other dependencies than the interpreter dispatch routine itself. > Ah, yes, that makes good sense. So you basically add an intermediate step to > calls that provides faster dispatch for known C functions. > Exactly. I also contemplated to provide optimized derivatives for all builtin functions, but never implemented it (lack of time). Based on quantitative analysis of usage frequency one could very well decide to, e.g., provide an optimized CALL_FUNCTION derivative for the "len" function. Another benefit of using my technique is that a compiler could decide to inline all of the functions of the optimized derivatives (e.g., the float_add function call inside my FLOAT_ADD interpreter instruction). Unfortunately, however, gcc currently does not allow for cross-module inlining (AFAIR). (Preliminary tests with manually changing the default inlining size for ceval.c resulted in speedups of up to 1.3 on my machine, so I think inlinling of function bodies for the optimized derivatives would boost performance noticeably.) > I'm interested in the code that determines what can be optimised in what > way. I read that Jython recently received a contribution that provides type > information for lots of modules and builtins, but having something like that > for CPython would be cool. > Ok. For this year's PPPJ I wanted to submit a paper realizing my optimization in Jython. Because of bytecode-rewriting tools, the interpreter could decide at runtime which optimized derivatives to generate and add rewriting code that supports the changing instruction set. Either way (static pre-compiling or dynamic bytecode rewriting that is), I think that Jython and IronPython would greatly benefit from applying this optimization technique, because their JIT compilers would inline the function calls with a high likelihood. > Such an approach would also be very useful for Cython. Think of a profiler > that runs a program in CPython and tells you exactly what static type > annotations to put where in your Python code to make it compile to a fast > binary with Cython. Or, even better, it could just spit out a .pxd file that > you drop next to your .py file and that provides the static type information > for you. > Hm, I think you could very easily save away the optimized bytecode sequence for function calls that would allow you to do that (e.g., you could save something similar to: LOAD_FAST LOAD_CONST LONG_ADD or LOAD_GLOBAL CALL_C_ZERO ) Cheers, --stefan From cesare.di.mauro at gmail.com Fri Jul 23 10:59:18 2010 From: cesare.di.mauro at gmail.com (Cesare Di Mauro) Date: Fri, 23 Jul 2010 10:59:18 +0200 Subject: [Python-Dev] Python 3 optimizations... In-Reply-To: References: Message-ID: 2010/7/23 Stefan Behnel > stefan brunthaler, 23.07.2010 08:48: > > If we take for instance the BINARY_ADD instruction, the interpreter >> evaluates the actual operand types and chooses the matching operation >> implementation at runtime, i.e., operands that are unicode strings >> will be concatenated via unicode_concatenate, for float operands on >> the other hand, the interpreter would end up invoking float_add via >> binary_op1. Now, a very efficient way to achieve purely interpretative >> inline caching is to quicken the type-generic BINARY_ADD instruction >> to a type-dependent FLOAT_ADD instruction (this technique, i.e., >> inline caching via quickening, is the primary contribution of my ECOOP >> paper). Hence, I have a very simple code generator, that generates >> type-dependent interpreter instructions in a pre-compile step of the >> interpreter, and uses runtime type information to quicken/rewrite >> instructions. >> Aside of the operators, I have implemented this quickening technique >> for FOR_ITER, COMPARE_OP and CALL_FUNCTION instructions. >> >> This sounds like wpython (a CPython derivative with a wider set of byte > code commands) could benefit from it. > WPython 1.1 does it at compile time, if you enable the new "experimental integer opcodes" flag. Similar optimizations were introduced with new opcodes for specialized string interpolation and joins, which are common operations in Python. It also added a new opcode GET_GENERATOR which internally uses a faster function call, which is used also by (the modified) BUILD_CLASS for the same reason (cut some unnecessary checks and code). Cesare -------------- next part -------------- An HTML attachment was scrubbed... URL: From ronaldoussoren at mac.com Fri Jul 23 11:12:45 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Fri, 23 Jul 2010 10:12:45 +0100 Subject: [Python-Dev] Distutils reverted in py3k In-Reply-To: References: Message-ID: <2487F1C9-8697-4FB5-9D48-2588DF73FC81@mac.com> On 22 Jul, 2010, at 14:01, Tarek Ziad? wrote: > Note that I'll revert Doc/distutils as well, but I need to check first > with Ronald a few Mac OS X points. Not just a few point, the revert completely broke universal builds on OSX. I'm working on a fix. Ronald > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From ronaldoussoren at mac.com Fri Jul 23 11:44:34 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Fri, 23 Jul 2010 10:44:34 +0100 Subject: [Python-Dev] Distutils reverted in py3k In-Reply-To: <2487F1C9-8697-4FB5-9D48-2588DF73FC81@mac.com> References: <2487F1C9-8697-4FB5-9D48-2588DF73FC81@mac.com> Message-ID: <83B60F89-688A-492A-A708-484B11EBC9A7@mac.com> On 23 Jul, 2010, at 10:12, Ronald Oussoren wrote: > > On 22 Jul, 2010, at 14:01, Tarek Ziad? wrote: > >> Note that I'll revert Doc/distutils as well, but I need to check first >> with Ronald a few Mac OS X points. > > Not just a few point, the revert completely broke universal builds on OSX. I'm working on a fix. The build seems to work again, my fix is in r83066. There are a couple of test failures left, but those don't seem to be related to distutils. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From ziade.tarek at gmail.com Fri Jul 23 11:47:59 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 23 Jul 2010 11:47:59 +0200 Subject: [Python-Dev] Distutils reverted in py3k In-Reply-To: <83B60F89-688A-492A-A708-484B11EBC9A7@mac.com> References: <2487F1C9-8697-4FB5-9D48-2588DF73FC81@mac.com> <83B60F89-688A-492A-A708-484B11EBC9A7@mac.com> Message-ID: 2010/7/23 Ronald Oussoren : > > On 23 Jul, 2010, at 10:12, Ronald Oussoren wrote: > >> >> On 22 Jul, 2010, at 14:01, Tarek Ziad? wrote: >> >>> Note that I'll revert Doc/distutils as well, but I need to check first >>> with Ronald a few Mac OS X points. >> >> Not just a few point, the revert completely broke universal builds on OSX. I'm working on a fix. mmm sorry about that... I though you backported everything in 3.1 > > The build seems to work again, my fix is in r83066. There are a couple of test failures left, but those don't seem to be related to distutils. Thanks. The next task zould be to check what 3,2 doc should be kept before Docs/distutils is reverted Regards Tarek -- Tarek Ziad? | http://ziade.org From ronaldoussoren at mac.com Fri Jul 23 11:55:08 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Fri, 23 Jul 2010 10:55:08 +0100 Subject: [Python-Dev] Distutils reverted in py3k In-Reply-To: References: <2487F1C9-8697-4FB5-9D48-2588DF73FC81@mac.com> <83B60F89-688A-492A-A708-484B11EBC9A7@mac.com> Message-ID: On 23 Jul, 2010, at 10:47, Tarek Ziad? wrote: > 2010/7/23 Ronald Oussoren : >> >> On 23 Jul, 2010, at 10:12, Ronald Oussoren wrote: >> >>> >>> On 22 Jul, 2010, at 14:01, Tarek Ziad? wrote: >>> >>>> Note that I'll revert Doc/distutils as well, but I need to check first >>>> with Ronald a few Mac OS X points. >>> >>> Not just a few point, the revert completely broke universal builds on OSX. I'm working on a fix. > > mmm sorry about that... I though you backported everything in 3.1 3.1 is probably broken as well due to the recent move to setting PY_CFLAGS instead of CFLAGS. That is next on my list. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From barry at python.org Fri Jul 23 12:02:52 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 23 Jul 2010 12:02:52 +0200 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <63B36042-C4F7-4BCA-8623-5182405D8852@mac.com> References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> <63B36042-C4F7-4BCA-8623-5182405D8852@mac.com> Message-ID: <20100723120252.2e0690cc@snowdog> On Jul 22, 2010, at 03:58 PM, Ronald Oussoren wrote: >I guess this is not an explicit goal of this PEP, but the structure is >very close to supporting multiple system architectures at the same >time. I regularly develop code that needs to run on Windows, Linux >and OSX and it is very convenient to do so in a shared directory tree >(locally on one machine and accessed using remote mounts on the other >ones). This works fine for pure python code, but I currently have to >resort to tricks for extension modules. [...] >A way to generically solve my problem is to add the platform name as >well, such as "foo.cpython-32m-darwin.so" or >"foo.cpython-32mu-linux2.so". This could certainly be done in the Windows build, but that wouldn't help bridge the gap among different POSIX systems. I'd be open to adding the platform name to the tag, but I'd probably define it as part of the implementation field, e.g. foo.cpython-linux2-32m.so. Or maybe start with the platform name, e.g. foo.linux2-cpython-32m. This isn't a strong preference though. Thoughts? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From brett at python.org Fri Jul 23 12:13:22 2010 From: brett at python.org (Brett Cannon) Date: Fri, 23 Jul 2010 11:13:22 +0100 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: <20100722140046.5e7551a1@pitrou.net> References: <4C470E2B.5060406@timgolden.me.uk> <20100721175819.3b037ad9@pitrou.net> <20100722140046.5e7551a1@pitrou.net> Message-ID: On Thu, Jul 22, 2010 at 13:00, Antoine Pitrou wrote: > On Thu, 22 Jul 2010 08:51:57 +0100 > Brett Cannon wrote: > > > > That's an option. I just remember Tim bringing up something about that > > approach that didn't quite work as a complete replacement for __del__. > > > > Basically the whole setting a module's globals to None was done before gc > > came into the language. Now that it's there it seems that it might work > to > > simply let gc clean up the module itself. > > There is a patch at http://bugs.python.org/issue812369 for GC-based > module shutdown, but it doesn't actually remove the setting of module > globals to None. I think further testing and experimentation would be > required to validate it. > Yep, it was agreed that for this to happen someone would have to write the patch and see what broke. -Brett > > Regards > > Antoine. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hrvoje.niksic at avl.com Fri Jul 23 12:16:04 2010 From: hrvoje.niksic at avl.com (Hrvoje Niksic) Date: Fri, 23 Jul 2010 12:16:04 +0200 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> <20100713163544.5f3a60d2@pitrou.net> Message-ID: <4C496BE4.1000608@avl.com> On 07/22/2010 01:34 PM, Georg Brandl wrote: > Timings (seconds to run the test suite): > > re 26.689 26.015 26.008 > regex 26.066 25.797 25.865 > > So, I thought there wasn't a difference in performance for this use case > (which is compiling a lot of regexes and matching most of them only a > few times in comparison). However, I found that looking at the regex > caching is very important in this case: re._MAXCACHE is by default set to > 100, and regex._MAXCACHE to 1024. When I set re._MAXCACHE to 1024 before > running the test suite, I get times around 18 (!) seconds for re. This seems to point to re being significantly *faster* than regexp, even in matching, and as such may be something the author would want to look into. Nick writes: > That still fits with the compile/match performance trade-off changes > between re and regex though. The performance trade-off should make regex slower with sufficiently small compiled regex cache, when a lot of time is wasted on compilation. But as the cache gets larger (and, for fairness, of the same size in both implementations), regex should outperform re. Georg, would you care to measure if there is a difference in performance with an even larger cache? From brett at python.org Fri Jul 23 12:39:38 2010 From: brett at python.org (Brett Cannon) Date: Fri, 23 Jul 2010 11:39:38 +0100 Subject: [Python-Dev] PEP 360 has outdated contents In-Reply-To: <20100722154848.75996b4b@pitrou.net> References: <20100722154848.75996b4b@pitrou.net> Message-ID: On Thu, Jul 22, 2010 at 14:48, Antoine Pitrou wrote: > > Hello all, > > PEP 360 - ?Externally Maintained Packages? seems to have outdated > contents. > First of all, I don't think Optik and wsgiref are externally > maintained anymore (both seem unmaintained by their original authors). > Added a note. > Second, the version numbers mentioned there could be out of date too > (especially for ElementTree whose stdlib copy has recently been bumped > by Florent). > I ripped out the sync dates since keeping them up is just a pain. > Third, it seems json should be added to the list, despite the prominent > warning at the beginning that: > > ?No new modules are to be added to this PEP. It has been deemed > dangerous to codify external maintenance of any code checked into > Python's code repository. Code contributers should expect Python's > development methodology to be used for any and all code checked into > Python's code repository.? > > The PEP lists external maintenance that we agreed to. Bob isn't supposed to be maintaining json outside of the stdlib, so I am not going to add it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Jul 23 12:46:57 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 23 Jul 2010 20:46:57 +1000 Subject: [Python-Dev] Python 3 optimizations... In-Reply-To: References: Message-ID: On Fri, Jul 23, 2010 at 1:00 AM, David Cournapeau wrote: > On Thu, Jul 22, 2010 at 10:08 PM, stefan brunthaler > wrote: >>> Is the source code under an open source non-copyleft license? >>> >> I am (unfortunately) not employed or funded by anybody, so I think >> that I can license/release the code as I see fit. > > If you did this work under your PhD program, you may be more > restricted than you think. You may want to check with your adviser > first, This is actually a good point - some universities have (IMO) weird ideas regarding the ownership of thesis material. Best to get that squared away before you publish the code online. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ronaldoussoren at mac.com Fri Jul 23 12:48:11 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Fri, 23 Jul 2010 11:48:11 +0100 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <20100723120252.2e0690cc@snowdog> References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> <63B36042-C4F7-4BCA-8623-5182405D8852@mac.com> <20100723120252.2e0690cc@snowdog> Message-ID: <25347613-8488-40C3-B9EF-9C85D0C47340@mac.com> On 23 Jul, 2010, at 11:02, Barry Warsaw wrote: > On Jul 22, 2010, at 03:58 PM, Ronald Oussoren wrote: > >> I guess this is not an explicit goal of this PEP, but the structure is >> very close to supporting multiple system architectures at the same >> time. I regularly develop code that needs to run on Windows, Linux >> and OSX and it is very convenient to do so in a shared directory tree >> (locally on one machine and accessed using remote mounts on the other >> ones). This works fine for pure python code, but I currently have to >> resort to tricks for extension modules. > > [...] > >> A way to generically solve my problem is to add the platform name as >> well, such as "foo.cpython-32m-darwin.so" or >> "foo.cpython-32mu-linux2.so". > > This could certainly be done in the Windows build, but that wouldn't help > bridge the gap among different POSIX systems. The windows port isn't a problem for this, it uses a different suffix (".pyd") than the unix ports. > I'd be open to adding the > platform name to the tag, but I'd probably define it as part of the > implementation field, e.g. foo.cpython-linux2-32m.so. Or maybe start with the > platform name, e.g. foo.linux2-cpython-32m. This isn't a strong preference > though. I don't have a strong opionion, but placing the platform name at the start is probably better to be consistent with sysconfig.get_platform(). Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From barry at python.org Fri Jul 23 12:54:40 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 23 Jul 2010 12:54:40 +0200 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <25347613-8488-40C3-B9EF-9C85D0C47340@mac.com> References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> <63B36042-C4F7-4BCA-8623-5182405D8852@mac.com> <20100723120252.2e0690cc@snowdog> <25347613-8488-40C3-B9EF-9C85D0C47340@mac.com> Message-ID: <20100723125440.721860bf@snowdog> On Jul 23, 2010, at 11:48 AM, Ronald Oussoren wrote: >> I'd be open to adding the >> platform name to the tag, but I'd probably define it as part of the >> implementation field, e.g. foo.cpython-linux2-32m.so. Or maybe >> start with the platform name, e.g. foo.linux2-cpython-32m. This >> isn't a strong preference though. > >I don't have a strong opionion, but placing the platform name at the >start is probably better to be consistent with >sysconfig.get_platform(). What about the architecture (i386, amd64)? With every increase in length I start to get more concerned. We could encode the platform and architecture, but that gets into cryptic territory. OTOH, would you really co-install i386 and amd64 shared libraries on the same machine? (hello NFS ;). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From ncoghlan at gmail.com Fri Jul 23 12:56:05 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 23 Jul 2010 20:56:05 +1000 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <20100722164036.7a80d27c@snowdog> References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> Message-ID: On Fri, Jul 23, 2010 at 12:40 AM, Barry Warsaw wrote: > Python implementations *MAY* include additional flags in the file name > tag as appropriate. ?For example, on POSIX systems these flags will > also contribute to the file name: > > ?* ``--with-pydebug`` (flag: ``d``) > ?* ``--with-pymalloc`` (flag: ``m``) > ?* ``--with-wide-unicode`` (flag: ``u``) > > By default in Python 3.2, ``configure`` enables ``--with-pymalloc`` so > shared library file names would appear as ``foo.cpython-32m.so``. > When the other two flags are also enabled, the file names would be > ``foo.cpython-32dmu.so``. > > (This PEP only addresses build issues on POSIX systems that use the > ``configure`` script. ?While Windows or other platform support is not > explicitly disallowed under this PEP, platform expertise is needed in > order to evaluate, describe, and implement support on such platforms.) This leads me to a question: how do these configure options affect the PEP 384 stable ABI? That PEP is currently silent on the issue, while PEP 3149 appears to implicitly assume that "abi3" completely specifies the ABI. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From brett at python.org Fri Jul 23 12:57:30 2010 From: brett at python.org (Brett Cannon) Date: Fri, 23 Jul 2010 11:57:30 +0100 Subject: [Python-Dev] PEP 382 progress: import hooks In-Reply-To: <20100722181939.1FA273A40D9@sparrow.telecommunity.com> References: <4C483EDF.7050002@v.loewis.de> <20100722181939.1FA273A40D9@sparrow.telecommunity.com> Message-ID: On Thu, Jul 22, 2010 at 19:19, P.J. Eby wrote: > At 01:51 PM 7/22/2010 +0100, Martin v. L?wis wrote: > >> At EuroPython, I sat down with Brett and we propose an approach >> how namespace packages get along with import hooks. I reshuffled >> the order in which things get done a little bit, and added a >> section that elaborates on the hooks. >> >> Basically, a finder will need to support a find_path method, >> return all .pth files, and a loader will need to support a >> load_module_with_path method, to initialize __path__. >> >> Please comment if you think that this needs further changes; >> > > I'm not certain I understand it precisely. There seem to be some > ambiguities in the spec, e.g.: > > "If fullname is not found, is not a package, or does not have any *.pth > files, None must be returned." > > What does "is not a package" actually mean in that context? The module is a module but not a package. > What happens if an empty list is returned - does that mean the importer is > saying, "this is a package, whether it has an __init__.py or not?" > > As for the "list of strings" returned, is each string the entire contents > of the .pth file? Is it to be \n-separated, or is any > universal-newlines-compatible string accepted? Is there a particular order > in which .pth file contents are to be returned? > > Regarding load_module_with_path(), how does its specification differ from > simply creating a module in sys.modules, setting its __path__, and then > invoking the standard load_module()? (i.e., is this method actually needed, > since a correct PEP 302 loader *must* reuse an existing module object in > sys.modules) It must reuse the module itself but a proper reload would reset __path__ as leaving it unchanged is not a proper resetting of the module object. So this module is needed in order to force the loader > > > > I'll hope to start implementing it soon. >> > > Am I correct in understanding that, as written, one would have to redefine > __import__ to implement this in a library for older Python versions? Or is > it implementable as a meta_path importer? > Redefine __import__ (unless Martin and I are missing something, but I tried to think of how to implement this use sys.meta_path and couldn't come up with a solution). -Brett > > > Regards, >> Martin >> > > Thanks for your work on this, I was just thinking about pinging to see how > it was going. ;-) > > (I want setuptools 0.7 to be able to supply an add-on module for supporting > this PEP in older Pythons, so that its current .pth hacks for implementing > namespace packages can be dropped.) > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ronaldoussoren at mac.com Fri Jul 23 13:01:22 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Fri, 23 Jul 2010 12:01:22 +0100 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <20100723125440.721860bf@snowdog> References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> <63B36042-C4F7-4BCA-8623-5182405D8852@mac.com> <20100723120252.2e0690cc@snowdog> <25347613-8488-40C3-B9EF-9C85D0C47340@mac.com> <20100723125440.721860bf@snowdog> Message-ID: <764F1595-5C8B-44D8-86C5-937B326B2FCD@mac.com> On 23 Jul, 2010, at 11:54, Barry Warsaw wrote: > On Jul 23, 2010, at 11:48 AM, Ronald Oussoren wrote: > >>> I'd be open to adding the >>> platform name to the tag, but I'd probably define it as part of the >>> implementation field, e.g. foo.cpython-linux2-32m.so. Or maybe >>> start with the platform name, e.g. foo.linux2-cpython-32m. This >>> isn't a strong preference though. >> >> I don't have a strong opionion, but placing the platform name at the >> start is probably better to be consistent with >> sysconfig.get_platform(). > > What about the architecture (i386, amd64)? With every increase in length I > start to get more concerned. We could encode the platform and architecture, > but that gets into cryptic territory. OTOH, would you really co-install i386 > and amd64 shared libraries on the same machine? (hello NFS ;). I don't need this, but then again I primarily use a platform where the vendor has a proper solution for having binaries for multiple architectures ;-) Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From ncoghlan at gmail.com Fri Jul 23 13:20:28 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 23 Jul 2010 21:20:28 +1000 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: <4C496BE4.1000608@avl.com> References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> <20100713163544.5f3a60d2@pitrou.net> <4C496BE4.1000608@avl.com> Message-ID: On Fri, Jul 23, 2010 at 8:16 PM, Hrvoje Niksic wrote: > The performance trade-off should make regex slower with sufficiently small > compiled regex cache, when a lot of time is wasted on compilation. ?But as > the cache gets larger (and, for fairness, of the same size in both > implementations), regex should outperform re. ?Georg, would you care to > measure if there is a difference in performance with an even larger cache? That's not quite accurate. The figure that matters is "matches per compilation", and that is ultimately governed by the nature of the application once the cache is sufficiently large (e.g. an application that compiles 10 different regular expressions, but also only matches each one against a single string is going to be slowed down significantly by a switch to regex no matter how big the compilation cache may be). The total runtime for a particular re matching exercise is "(average compilation time)*(number of times compiled) + (average match time)*(number of matches performed)" We know that the new regex module speeds up matching at the cost of slowing down compilation. Whether that is a net increase for the application as a whole depends on 4 things: X: the average speed change in compilation for the application Y: the average speed change in matching for the application A: the proportion of time the application currently spends compiling regular expressions B: the proportion of time the application currently spends matching regular expressions The overall proportional speed change for re operations in a given application is then just X*A + Y*B. For regex to be an improvement for that application, the resulting figure needs to be less than 1. For example, suppose X=2, Y=0.5 (i.e. matching is twice as fast on average, but compilation also takes twice as long), then the overall speed change would be 2*A + 0.5*B. For that to be a wash, the original application needs to spend 1/3 of its expression matching time compiling the expressions and 2/3 actually matching them (the application with the new engine would then spend 2/3 of its time compiling and only 1/3 matching). That's going to be a tough assessment to make in practice, but I think Georg's pygments test suite example shows that there is real world code out there that already spends a lot of time on compilation. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From steve at holdenweb.com Fri Jul 23 13:25:39 2010 From: steve at holdenweb.com (Steve Holden) Date: Fri, 23 Jul 2010 12:25:39 +0100 Subject: [Python-Dev] PEP 382 progress: import hooks In-Reply-To: References: <4C483EDF.7050002@v.loewis.de> <20100722181939.1FA273A40D9@sparrow.telecommunity.com> Message-ID: On 7/23/2010 11:57 AM, Brett Cannon wrote: > > > On Thu, Jul 22, 2010 at 19:19, P.J. Eby > wrote: > > At 01:51 PM 7/22/2010 +0100, Martin v. L?wis wrote: > > At EuroPython, I sat down with Brett and we propose an approach > how namespace packages get along with import hooks. I reshuffled > the order in which things get done a little bit, and added a > section that elaborates on the hooks. > > Basically, a finder will need to support a find_path method, > return all .pth files, and a loader will need to support a > load_module_with_path method, to initialize __path__. > > Please comment if you think that this needs further changes; > > > I'm not certain I understand it precisely. There seem to be some > ambiguities in the spec, e.g.: > > "If fullname is not found, is not a package, or does not have any > *.pth files, None must be returned." > > What does "is not a package" actually mean in that context? > > > The module is a module but not a package. > so s/is not a package/is a module rather than a package/ perhaps? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From schmir at gmail.com Fri Jul 23 13:46:03 2010 From: schmir at gmail.com (schmir at gmail.com) Date: Fri, 23 Jul 2010 13:46:03 +0200 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <764F1595-5C8B-44D8-86C5-937B326B2FCD@mac.com> (Ronald Oussoren's message of "Fri, 23 Jul 2010 12:01:22 +0100") References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> <63B36042-C4F7-4BCA-8623-5182405D8852@mac.com> <20100723120252.2e0690cc@snowdog> <25347613-8488-40C3-B9EF-9C85D0C47340@mac.com> <20100723125440.721860bf@snowdog> <764F1595-5C8B-44D8-86C5-937B326B2FCD@mac.com> Message-ID: <87mxti8lqc.fsf@muni.brainbot.com> Ronald Oussoren writes: > On 23 Jul, 2010, at 11:54, Barry Warsaw wrote: > >> >> What about the architecture (i386, amd64)? With every increase in length I >> start to get more concerned. We could encode the platform and architecture, >> but that gets into cryptic territory. OTOH, would you really co-install i386 >> and amd64 shared libraries on the same machine? (hello NFS ;). > > I don't need this, but then again I primarily use a platform where the vendor has > a proper solution for having binaries for multiple architectures ;-) Well, Apple doesn't prevent people from building 32/64 bit-only python installations. Doesn't that give you 3 choices i386, amd64, fat?? And you can have framework or non-framework builds. Doesn't anybody else think this is lost work for very little gain? My /usr/lib/python2.6/site-packages directory consumes 200MB on disk. I couldn't care less if my /usr/lib/python2.5/site-packages consumed the same amount of disk space... - Ralf From g.brandl at gmx.net Fri Jul 23 15:44:10 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 23 Jul 2010 14:44:10 +0100 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> <20100713163544.5f3a60d2@pitrou.net> <4C496BE4.1000608@avl.com> Message-ID: Am 23.07.2010 12:20, schrieb Nick Coghlan: > On Fri, Jul 23, 2010 at 8:16 PM, Hrvoje Niksic wrote: >> The performance trade-off should make regex slower with sufficiently small >> compiled regex cache, when a lot of time is wasted on compilation. But as >> the cache gets larger (and, for fairness, of the same size in both >> implementations), regex should outperform re. Georg, would you care to >> measure if there is a difference in performance with an even larger cache? > > That's not quite accurate. The figure that matters is "matches per > compilation", and that is ultimately governed by the nature of the > application once the cache is sufficiently large (e.g. an application > that compiles 10 different regular expressions, but also only matches > each one against a single string is going to be slowed down > significantly by a switch to regex no matter how big the compilation > cache may be). > > The total runtime for a particular re matching exercise is "(average > compilation time)*(number of times compiled) + (average match > time)*(number of matches performed)" > > We know that the new regex module speeds up matching at the cost of > slowing down compilation. Whether that is a net increase for the > application as a whole depends on 4 things: > > X: the average speed change in compilation for the application > Y: the average speed change in matching for the application > A: the proportion of time the application currently spends compiling > regular expressions > B: the proportion of time the application currently spends matching > regular expressions > > The overall proportional speed change for re operations in a given > application is then just X*A + Y*B. For regex to be an improvement for > that application, the resulting figure needs to be less than 1. > > For example, suppose X=2, Y=0.5 (i.e. matching is twice as fast on > average, but compilation also takes twice as long), then the overall > speed change would be 2*A + 0.5*B. For that to be a wash, the original > application needs to spend 1/3 of its expression matching time > compiling the expressions and 2/3 actually matching them (the > application with the new engine would then spend 2/3 of its time > compiling and only 1/3 matching). > > That's going to be a tough assessment to make in practice, but I think > Georg's pygments test suite example shows that there is real world > code out there that already spends a lot of time on compilation. And for Pygments, it's not only the case for the test suite, since many applications consist of highlighting one file/one piece of code calling it as a program. When using Pygments as a library, regexes are only compiled once per lexer class, so e.g. in a web appliation that highlights files (like a repository browser), the additional cost is negligible. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Fri Jul 23 15:45:52 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 23 Jul 2010 14:45:52 +0100 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: <4C496BE4.1000608@avl.com> References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> <20100713163544.5f3a60d2@pitrou.net> <4C496BE4.1000608@avl.com> Message-ID: Am 23.07.2010 11:16, schrieb Hrvoje Niksic: > On 07/22/2010 01:34 PM, Georg Brandl wrote: >> Timings (seconds to run the test suite): >> >> re 26.689 26.015 26.008 >> regex 26.066 25.797 25.865 >> >> So, I thought there wasn't a difference in performance for this use case >> (which is compiling a lot of regexes and matching most of them only a >> few times in comparison). However, I found that looking at the regex >> caching is very important in this case: re._MAXCACHE is by default set to >> 100, and regex._MAXCACHE to 1024. When I set re._MAXCACHE to 1024 before >> running the test suite, I get times around 18 (!) seconds for re. > > This seems to point to re being significantly *faster* than regexp, even > in matching, and as such may be something the author would want to look > into. > > Nick writes: > > > That still fits with the compile/match performance trade-off changes > > between re and regex though. > > The performance trade-off should make regex slower with sufficiently > small compiled regex cache, when a lot of time is wasted on compilation. > But as the cache gets larger (and, for fairness, of the same size in > both implementations), regex should outperform re. Georg, would you > care to measure if there is a difference in performance with an even > larger cache? I did measure that, and there are no significant differences in timing. I also did the check the other way around, and restricting regex._MAXCACHE to 100 I got from 26 seconds to 42 seconds. (Nick, is that enough data to calculate A and B now? ;) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From status at bugs.python.org Fri Jul 23 18:07:37 2010 From: status at bugs.python.org (Python tracker) Date: Fri, 23 Jul 2010 18:07:37 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20100723160737.1AB2D7822A@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2010-07-16 - 2010-07-23) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2826 open (+62) / 18349 closed (+21) / 21175 total (+83) Open issues with patches: 1151 Average duration of open issues: 692 days. Median duration of open issues: 480 days. Open Issues Breakdown open 2780 (+60) languishing 16 ( +0) pending 29 ( +2) Issues Created Or Reopened (89) _______________________________ No float formatting in PyString_FromFormat 2010-07-19 http://bugs.python.org/issue2813 reopened mark.dickinson patch os.curdir as the default argument for os.listdir 2010-07-23 CLOSED http://bugs.python.org/issue6095 reopened tarek patch weak dict iterators are fragile because of unpredictable GC ru 2010-07-23 http://bugs.python.org/issue7105 reopened mark.dickinson patch subprocess.Popen documentation should contain a good warning a 2010-07-22 http://bugs.python.org/issue7950 reopened r.david.murray patch code.InteractiveInterpreter fails to change locals when invoke 2010-07-16 http://bugs.python.org/issue9274 created ericp python 2.7 OS X installer no longer installs /usr/local/bin sy 2010-07-16 CLOSED http://bugs.python.org/issue9275 created ned.deily pickle should support methods 2010-07-16 http://bugs.python.org/issue9276 created exarkun test_struct failure on ARM 2010-07-16 http://bugs.python.org/issue9277 created mark.dickinson patch, buildbot rename 2to3 & pydoc to 2to3.py & pydoc.py 2010-07-16 CLOSED http://bugs.python.org/issue9278 created srid get rid of pdb.doc 2010-07-16 CLOSED http://bugs.python.org/issue9279 created georg.brandl easy sharedinstall target doesn't depend on sharedmods target 2010-07-17 CLOSED http://bugs.python.org/issue9280 created Arfrever patch Race condition in distutils.dir_util.mkpath() 2010-07-17 http://bugs.python.org/issue9281 created Arfrever patch Bug in --listfuncs option of trace.py 2010-07-17 CLOSED http://bugs.python.org/issue9282 created eli.bendersky patch buggy repr for os.environ 2010-07-17 http://bugs.python.org/issue9283 created pitrou inspect.findsource() cannot find source for doctest code 2010-07-17 http://bugs.python.org/issue9284 created djc A decorator for cProfile and profile modules 2010-07-17 http://bugs.python.org/issue9285 created giampaolo.rodola patch, patch, needs review email.utils.parseaddr returns garbage for invalid input 2010-07-17 http://bugs.python.org/issue9286 created merwok Cosmetic fix in OtherFileTests.testOpenDir 2010-07-17 http://bugs.python.org/issue9287 created ocean-city patch, easy Disambiguate :option: and :cmdoption: 2010-07-17 CLOSED http://bugs.python.org/issue9288 created merwok patch, needs review Skip test_long_key(test_winreg) on win2k + py2.x 2010-07-18 http://bugs.python.org/issue9289 created ocean-city patch IDLE and Command line present different behavior for sys.stdin 2010-07-18 http://bugs.python.org/issue9290 created Geraldo.Xexeo mimetypes initialization fails on Windows because of non-Latin 2010-07-18 http://bugs.python.org/issue9291 created Dmitry.Jemerov patch, easy Dead code in Modules/pyexpat.c 2010-07-18 http://bugs.python.org/issue9292 created pitrou Unsupported IO operations should raise UnsupportedOperation 2010-07-18 http://bugs.python.org/issue9293 created pitrou easy Dead code in Objects/object.c 2010-07-18 http://bugs.python.org/issue9294 created pitrou easy test_close_open_print_buffered(test_file) sometimes crashes 2010-07-18 http://bugs.python.org/issue9295 created ocean-city patch json module skipkeys handling changed exception types in 2.7 2010-07-18 CLOSED http://bugs.python.org/issue9296 created doughellmann patch SMTP with Sqlite3 file attached problem 2010-07-20 http://bugs.python.org/issue9297 reopened murilobr binary email attachment issue with base64 encoding 2010-07-19 http://bugs.python.org/issue9298 created vunruh os.mkdir() and os.makedirs() add a keyword argument to suppres 2010-07-19 http://bugs.python.org/issue9299 created ysj.ray patch c/profile Profile class is not documented 2010-07-19 CLOSED http://bugs.python.org/issue9300 created giampaolo.rodola urllib.quote(None) returns None in 2.7 (raised TypeError befor 2010-07-19 CLOSED http://bugs.python.org/issue9301 created djc Distutils document problem? 2010-07-19 http://bugs.python.org/issue9302 created ysj.ray patch Migrate sqlite3 module to _v2 API to enhance performance 2010-07-19 http://bugs.python.org/issue9303 created feuermurmel unreproducible example in the memoryview documentation 2010-07-19 CLOSED http://bugs.python.org/issue9304 created chaica_ Don't use east/west of UTC in date/time documentation 2010-07-19 http://bugs.python.org/issue9305 created belopolsky easy distutils: raise informative error message when cmd_class is N 2010-07-19 http://bugs.python.org/issue9306 created srid Py_TPFLAGS_LONG_SUBCLASS is not documented 2010-07-19 http://bugs.python.org/issue9307 created belopolsky Remove redundant coding cookies from 3.x stdlib 2010-07-19 http://bugs.python.org/issue9308 created belopolsky patch Add environment variable complementing command line option -no 2010-07-19 http://bugs.python.org/issue9309 created AlexanderDreyer patch Intermittent failures in test_logging 2010-07-19 CLOSED http://bugs.python.org/issue9310 created skrah patch, buildbot os.access can return bogus values when run as superuser 2010-07-20 http://bugs.python.org/issue9311 created yaneurabeya Fix usage of :option: markup in stdlib ReST docs 2010-07-20 http://bugs.python.org/issue9312 created eli.bendersky patch distutils error on MSVC older than 8 2010-07-20 http://bugs.python.org/issue9313 created ocean-city patch inconsistent result when concatenating list with iterators 2010-07-20 CLOSED http://bugs.python.org/issue9314 created djc The trace module lacks unit tests 2010-07-20 http://bugs.python.org/issue9315 created eli.bendersky typos in zipimport docstrings 2010-07-21 CLOSED http://bugs.python.org/issue9316 created rfk patch Incorrect coverage file from trace test_pickle.py 2010-07-21 http://bugs.python.org/issue9317 created belopolsky patch Py3k compilation on old MSVC 2010-07-21 http://bugs.python.org/issue9318 created ocean-city patch segfault when searching modules with help() 2010-07-21 http://bugs.python.org/issue9319 created ron_adam os.environ is global for threads 2010-07-22 CLOSED http://bugs.python.org/issue9320 reopened techtonik CGIHTTPServer cleanup htbin 2010-07-21 http://bugs.python.org/issue9321 created techtonik bdist_rpm ext_modules absolute source path fail 2010-07-21 http://bugs.python.org/issue9322 created indiedan trace.py bug with the main file being traced 2010-07-21 CLOSED http://bugs.python.org/issue9323 created eli.bendersky patch signal.signal(bogus_signal, handler) segfaults on Windows 2010-07-21 http://bugs.python.org/issue9324 created midnightdf patch Add an option to pdb/trace/profile to run library module as a 2010-07-21 http://bugs.python.org/issue9325 created belopolsky Error message for incorrect number of (function) args is incor 2010-07-21 http://bugs.python.org/issue9326 created midnightdf doctest DocFileCase setUp/tearDown asymmetry 2010-07-21 http://bugs.python.org/issue9327 created boogenhagn str.__format__.__doc__ backport glitch 2010-07-21 http://bugs.python.org/issue9328 created tjreedy easy freeze tool cannot handle JSON module properly 2010-07-22 http://bugs.python.org/issue9329 created sam.sp assertIn should check for membership support before testing 2010-07-22 CLOSED http://bugs.python.org/issue9330 created brian.curtin patch, patch, needs review sys.setprofile is not described as CPython implementation deta 2010-07-22 http://bugs.python.org/issue9331 created belopolsky Document requirements for os.symlink usage on Windows 2010-07-22 http://bugs.python.org/issue9332 created brian.curtin Expose a way to enable os.symlink on Windows 2010-07-22 http://bugs.python.org/issue9333 created brian.curtin patch argparse does not accept options taking arguments beginning wi 2010-07-22 http://bugs.python.org/issue9334 created andersk LC_CTYPE system setting not respected by setlocale() 2010-07-23 CLOSED http://bugs.python.org/issue9335 created antlong string.letters should display locale based equivalent of a-Z 2010-07-23 CLOSED http://bugs.python.org/issue9336 created antlong Make float.__str__ behave identically to float.__repr__ 2010-07-23 http://bugs.python.org/issue9337 created mark.dickinson argparse optionals with nargs='+' can't be followed by positio 2010-07-23 http://bugs.python.org/issue9338 reopened bethard threading is_alive documnetation is unclear 2010-07-23 CLOSED http://bugs.python.org/issue9339 created bbrazil patch argparse parse_known_args does not work with subparsers 2010-07-23 http://bugs.python.org/issue9340 created bethard allow argparse subcommands to be grouped 2010-07-23 http://bugs.python.org/issue9341 created bethard Tests for monthrange in calendar.py module 2010-07-23 CLOSED http://bugs.python.org/issue9342 created John.Chandler patch Document that argparse "parents" must be fully declared before 2010-07-23 http://bugs.python.org/issue9343 created bethard please add posix.getgrouplist() 2010-07-23 http://bugs.python.org/issue9344 created ronaldoussoren easy argparse wrap tests are sensitive to terminal size 2010-07-23 http://bugs.python.org/issue9345 created bethard Improve threading tests 2010-07-23 CLOSED http://bugs.python.org/issue9346 created bbrazil patch Calling argparse add_argument with a sequence as 'type' causes 2010-07-23 http://bugs.python.org/issue9347 created bethard Calling argparse's add_argument with the wrong number of metav 2010-07-23 http://bugs.python.org/issue9348 created bethard document argparse's help=SUPPRESS 2010-07-23 http://bugs.python.org/issue9349 created bethard add remove_argument_group to argparse 2010-07-23 http://bugs.python.org/issue9350 created bethard argparse set_defaults on subcommands should override top level 2010-07-23 http://bugs.python.org/issue9351 created bethard argparse eats characters when parsing multiple merged short op 2010-07-23 http://bugs.python.org/issue9352 created bethard patch argparse __all__ is incomplete 2010-07-23 http://bugs.python.org/issue9353 created bethard file_wrapper fails to provide getsockopt() 2010-07-23 http://bugs.python.org/issue9354 created lukasz.langa patch argparse add_mutually_exclusive_group more than once has incor 2010-07-23 http://bugs.python.org/issue9355 created bethard Improved tests and style for fnmatch 2010-07-23 http://bugs.python.org/issue9356 created bbrazil patch hotshot.stats.load fails with AssertionError 2010-07-19 http://bugs.python.org/issue900092 reopened brian.curtin patch urllib.quote throws exception on Unicode URL 2010-07-22 http://bugs.python.org/issue1712522 reopened orsenthil patch, easy Issues Now Closed (74) ______________________ Allow array.array to be parsed by the t# format unit. 1027 days http://bugs.python.org/issue1200 BreamoreBoy IDLE - patch Delegator to support callables 1015 days http://bugs.python.org/issue1252 taleinat patch Race condition in os.makedirs 943 days http://bugs.python.org/issue1675 tjreedy easy Remove cmp parameter to list.sort() and builtin.sorted() 924 days http://bugs.python.org/issue1771 metal Test 908 days http://bugs.python.org/issue1902 loewis optimize out local variables at end of function 876 days http://bugs.python.org/issue2181 BreamoreBoy patch, patch 2to3 converts long(itude) argument to int 813 days http://bugs.python.org/issue2734 maubp Remove mimetools usage from the stdlib 588 days http://bugs.python.org/issue2848 merwok patch Expect methods for testing. 699 days http://bugs.python.org/issue3615 michael.foord patch test_run_abort triggers CrashReporter on MacOS X 656 days http://bugs.python.org/issue4047 ronaldoussoren Faster opcode dispatch on gcc 570 days http://bugs.python.org/issue4753 pitrou patch Python on Windows disables all C runtime library assertions 564 days http://bugs.python.org/issue4804 BreamoreBoy patch, needs review 3.1 cannot unpickle 2.7-created pickle 525 days http://bugs.python.org/issue5180 belopolsky patch array.fromfile not checking I/O errors 508 days http://bugs.python.org/issue5395 pitrou patch, easy _winreg.OpenKey() is documented with keyword arguments, but do 462 days http://bugs.python.org/issue5774 stutzbach patch, needs review Right-click behavior from Windows Explorer 454 days http://bugs.python.org/issue5817 Mkopinsky os.curdir as the default argument for os.listdir 0 days http://bugs.python.org/issue6095 tarek patch subprocess doesn't pass arguments correctly on Linux when shel 344 days http://bugs.python.org/issue6689 berwyn patch char buffer in function posix_getcwdu should not be fix length 322 days http://bugs.python.org/issue6817 skrah patch getcwd hangs and leaks mem on Solaris <= 9 in very long file n 301 days http://bugs.python.org/issue6976 skrah curses crash on FreeBSD 236 days http://bugs.python.org/issue7384 skrah patch, buildbot dis module documentation gives no indication of the dangers of 16 days http://bugs.python.org/issue7829 brett.cannon Mac 10.6 mac_ver() crashes with USING_FORK_WITHOUT_EXEC_IS _NO 164 days http://bugs.python.org/issue7895 ronaldoussoren easy PYTHON_API_VERSION needs to be bumped? 134 days http://bugs.python.org/issue8118 ronaldoussoren Multiple Python Interpreter Memory Leak 114 days http://bugs.python.org/issue8258 amaury.forgeotdarc test_float fails on ARM Linux EABI with soft floating point 111 days http://bugs.python.org/issue8265 mark.dickinson buildbot unit tests rather light on testing __import__(..., level) 101 days http://bugs.python.org/issue8392 brett.cannon patch, needs review Segmentation error happens in Embedding Python. 43 days http://bugs.python.org/issue8917 tanaga CGIHTTPServer supports only Python CGI scripts 35 days http://bugs.python.org/issue9007 techtonik Simplify Py_CHARMASK 30 days http://bugs.python.org/issue9036 skrah patch, needs review Tkinter module missing from install on OS X 10.6.4 29 days http://bugs.python.org/issue9073 ronaldoussoren contextlib.ContextDecorator 3 days http://bugs.python.org/issue9110 ncoghlan patch os.getcwd causes infinite loop on solaris 12 days http://bugs.python.org/issue9185 skrah patch, needs review Improve CFLAGS handling 12 days http://bugs.python.org/issue9189 jyasskin document that the 'test' package is for core developers only 10 days http://bugs.python.org/issue9255 merwok easy Incorrect name passed as arg[0] when shell=True and executable 5 days http://bugs.python.org/issue9265 skrah Add annotation option to pickletools.dis 2 days http://bugs.python.org/issue9268 belopolsky patch, easy python 2.7 OS X installer no longer installs /usr/local/bin sy 7 days http://bugs.python.org/issue9275 ronaldoussoren rename 2to3 & pydoc to 2to3.py & pydoc.py 4 days http://bugs.python.org/issue9278 srid get rid of pdb.doc 2 days http://bugs.python.org/issue9279 georg.brandl easy sharedinstall target doesn't depend on sharedmods target 1 days http://bugs.python.org/issue9280 benjamin.peterson patch Bug in --listfuncs option of trace.py 3 days http://bugs.python.org/issue9282 belopolsky patch Disambiguate :option: and :cmdoption: 1 days http://bugs.python.org/issue9288 georg.brandl patch, needs review json module skipkeys handling changed exception types in 2.7 3 days http://bugs.python.org/issue9296 doughellmann patch c/profile Profile class is not documented 0 days http://bugs.python.org/issue9300 giampaolo.rodola urllib.quote(None) returns None in 2.7 (raised TypeError befor 0 days http://bugs.python.org/issue9301 orsenthil unreproducible example in the memoryview documentation 0 days http://bugs.python.org/issue9304 pitrou Intermittent failures in test_logging 1 days http://bugs.python.org/issue9310 vinay.sajip patch, buildbot inconsistent result when concatenating list with iterators 1 days http://bugs.python.org/issue9314 belopolsky typos in zipimport docstrings 0 days http://bugs.python.org/issue9316 brian.curtin patch os.environ is global for threads 0 days http://bugs.python.org/issue9320 georg.brandl trace.py bug with the main file being traced 0 days http://bugs.python.org/issue9323 belopolsky patch assertIn should check for membership support before testing 0 days http://bugs.python.org/issue9330 brian.curtin patch, patch, needs review LC_CTYPE system setting not respected by setlocale() 1 days http://bugs.python.org/issue9335 belopolsky string.letters should display locale based equivalent of a-Z 0 days http://bugs.python.org/issue9336 r.david.murray threading is_alive documnetation is unclear 0 days http://bugs.python.org/issue9339 brett.cannon patch Tests for monthrange in calendar.py module 0 days http://bugs.python.org/issue9342 brett.cannon patch Improve threading tests 0 days http://bugs.python.org/issue9346 brett.cannon patch test_timeout updates 2637 days http://bugs.python.org/issue728815 belopolsky patch iterators broken for weak dicts 2447 days http://bugs.python.org/issue839159 mark.dickinson patch hotshot.runctx: builtins missing 1972 days http://bugs.python.org/issue1149798 BreamoreBoy patch Support non-file source/dest in marshal 1871 days http://bugs.python.org/issue1214879 BreamoreBoy patch subprocess: wait for a period of time 1660 days http://bugs.python.org/issue1396825 skrah email.Utils.parseaddr() gives arcane result 1641 days http://bugs.python.org/issue1409460 merwok Use PyOS_snprintf for static buffers 1621 days http://bugs.python.org/issue1428655 BreamoreBoy patch Multiple simultaneous sendtos on AF_UNIX socket broken. 1600 days http://bugs.python.org/issue1441984 BreamoreBoy syntax errors on continuation lines 1504 days http://bugs.python.org/issue1501979 BreamoreBoy patch Add "compose" function to the functools 1499 days http://bugs.python.org/issue1506122 amaury.forgeotdarc sgmllib should recover from unmatched quotes 1480 days http://bugs.python.org/issue1515142 BreamoreBoy patch, easy Tweak to make string.Templates more customizable 1456 days http://bugs.python.org/issue1528167 BreamoreBoy patch email parser incorrectly breaks headers with a CRLF at 8192 1406 days http://bugs.python.org/issue1555570 r.david.murray patch Add os.symlink() and os.path.islink() support for Windows 1375 days http://bugs.python.org/issue1578269 brian.curtin patch, needs review Minor AST tweaks 1250 days http://bugs.python.org/issue1659410 benjamin.peterson patch splitext of dotfiles, incl backwards compat and migration 1219 days http://bugs.python.org/issue1681842 michael.foord patch Top Issues Most Discussed (10) ______________________________ 27 Make gettimeofday available in time module 29 days open http://bugs.python.org/issue9079 21 doctest _load_testfile function -- newline handling seems incor 923 days open http://bugs.python.org/issue1812 21 urllib.quote throws exception on Unicode URL 2 days open http://bugs.python.org/issue1712522 20 LC_CTYPE system setting not respected by setlocale() 1 days closed http://bugs.python.org/issue9335 18 Incorrect coverage file from trace test_pickle.py 3 days open http://bugs.python.org/issue9317 15 os.mkdir() and os.makedirs() add a keyword argument to suppress 4 days open http://bugs.python.org/issue9299 14 inconsistent result when concatenating list with iterators 1 days closed http://bugs.python.org/issue9314 14 Remove redundant coding cookies from 3.x stdlib 4 days open http://bugs.python.org/issue9308 12 test_close_open_print_buffered(test_file) sometimes crashes 5 days open http://bugs.python.org/issue9295 11 json module skipkeys handling changed exception types in 2.7 3 days closed http://bugs.python.org/issue9296 From brett at python.org Fri Jul 23 18:49:10 2010 From: brett at python.org (Brett Cannon) Date: Fri, 23 Jul 2010 17:49:10 +0100 Subject: [Python-Dev] PEP 3148 ready for pronouncement [ACCEPTED] In-Reply-To: References: Message-ID: On Mon, Jul 12, 2010 at 02:13, Jesse Noller wrote: > On Sat, May 22, 2010 at 11:38 AM, Guido van Rossum > wrote: > [snip] > > Great points Jesse! Since I really don't have the time or expertise to > > make a judgment on this PEP, I hereby appoint you chair of the > > approval process for this PEP. That basically means that when you > > think it's ready to be approved, you say so, and it's a done deal. The > > remaining feedback cycle is up to you now -- it sounds like you're > > ready for closure, which sounds good to me (again, without having read > > the PEP or tried to write something using the proposed code). You can > > do it however you like: you can declare it approved now, or read it > > over once more yourself and suggest some final changes, or set a > > period (e.g. 48 hours) during which final comments have to be > > received, which you then will judge by merit or by your whim, or you > > can flip a coin or say a prayer... (I've tried most of those myself in > > the past and haven't done too badly if I say so myself. :-) You're the > > boss now. I know you will do the right thing for this PEP. > > > > -- > > --Guido van Rossum (python.org/~guido) > > > > So, after some cool down - and the last rounds of discussion which > triggered some jiggery-pokery on Brian's part, I'm accepting PEP 3148 > "futures - execute computations asynchronously". I feel that it's a > good addition, and a good start for something bigger down the road. > > Brian - you'll need to provide someone such as Martin or Georg your > public key for ssh access into SVN, and you'll need developer access > to the bug tracker. > Brian, did you ever get your keys to Martin or Georg? If not please do (or send it to me) and let us know what your bugs.python.org username is to get Developer privileges. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pje at telecommunity.com Fri Jul 23 18:54:33 2010 From: pje at telecommunity.com (P.J. Eby) Date: Fri, 23 Jul 2010 12:54:33 -0400 Subject: [Python-Dev] PEP 382 progress: import hooks In-Reply-To: References: <4C483EDF.7050002@v.loewis.de> <20100722181939.1FA273A40D9@sparrow.telecommunity.com> Message-ID: <20100723165435.BB08D3A409B@sparrow.telecommunity.com> At 11:57 AM 7/23/2010 +0100, Brett Cannon wrote: >On Thu, Jul 22, 2010 at 19:19, P.J. Eby ><pje at telecommunity.com> wrote: > >What does "is not a package" actually mean in that context? > > >The module is a module but not a package. Um... that's not any clearer. Are you saying that a module of the same name takes precedence over a package? Is that the current precedence as well? >Regarding load_module_with_path(), how does its specification differ >from simply creating a module in sys.modules, setting its __path__, >and then invoking the standard load_module()? ? (i.e., is this >method actually needed, since a correct PEP 302 loader *must* reuse >an existing module object in sys.modules) > > >It must reuse the module itself but a proper reload would reset >__path__ as leaving it unchanged is not a proper resetting of the >module object. So this module is needed in order to force the loader? Um, no. Reloading doesn't reset the module contents, not even __path__. Never has, from Python 2.2 through 2.7 -- even in 3.1. At least, not for normal filesystem .py/.pyc files. (I tested with 'os', adding an extra 'foo' attribute, and also setting a __path__; both were unaffected by reload(), in all 7 Python versions. Perhaps you're saying this happens with zipfiles, or packages that already have a __path__, or...? >? > >Am I correct in understanding that, as written, one would have to >redefine __import__ to implement this in a library for older Python >versions? ? Or is it implementable as a meta_path importer? > > >Redefine __import__ (unless Martin and I are missing something, but >I tried to think of how to implement this use sys.meta_path and >couldn't come up with a solution). I'm thinking it *could* be done with a meta_path hook, but only by doubling the search length in the event that the search failed. That seems a bit icky, but replacing the entire import process seems ickier (more code surface to maintain, more bug potential) in the case of supporting older Pythons. From reid.kleckner at gmail.com Fri Jul 23 19:19:23 2010 From: reid.kleckner at gmail.com (Reid Kleckner) Date: Fri, 23 Jul 2010 10:19:23 -0700 Subject: [Python-Dev] Python 3 optimizations... In-Reply-To: References: Message-ID: On Fri, Jul 23, 2010 at 1:58 AM, stefan brunthaler wrote: >> Do I understand correctly that you modify the byte code of modules/functions >> at runtime? >> > Yes. Quickening is runtime only optimization technique that rewrites > instructions from a generic instruction to an optimized derivative > (orignally for the Java virtual machine). It is completely hidden from > the compiler and has no other dependencies than the interpreter > dispatch routine itself. How do you generate the specialized opcode implementations? Presumably that is done ahead of time, or you'd have to use a JIT, which is what you're avoiding. I'm guessing from your comments below about cross-module inlining that you generate a separate .c file with the specialized opcode bodies and then call through to them via a table of function pointers indexed by opcode, but I could be totally wrong. :) > Another benefit of using my technique is that a compiler could decide > to inline all of the functions of the optimized derivatives (e.g., the > float_add function call inside my FLOAT_ADD interpreter instruction). > Unfortunately, however, gcc currently does not allow for cross-module > inlining (AFAIR). (Preliminary tests with manually changing the > default inlining size for ceval.c resulted in speedups of up to 1.3 on > my machine, so I think inlinling of function bodies for the optimized > derivatives would boost performance noticeably.) There are a variety of solutions to getting cross-module inlining these days. Clang+LLVM support link-time optimization (LTO) via a plugin for gold. GCC has LTO and LIPO as well. >> Such an approach would also be very useful for Cython. Think of a profiler >> that runs a program in CPython and tells you exactly what static type >> annotations to put where in your Python code to make it compile to a fast >> binary with Cython. Or, even better, it could just spit out a .pxd file that >> you drop next to your .py file and that provides the static type information >> for you. This would be interesting. We have (obviously) have similar instrumentation in unladen swallow to gather type feedback. We talked with Craig Citro about finding a way to feed that back to Cython for exactly this reason, but we haven't really pursued it. Reid From alexander.belopolsky at gmail.com Fri Jul 23 19:42:59 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Fri, 23 Jul 2010 13:42:59 -0400 Subject: [Python-Dev] Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: Thanks, everyone who followed up here and on the tracker. I am readying the patch for check in, but as I look back through the messages, I don't really see anyone's answer to the question in the subject: * Include datetime.py in stdlib or not? I hope this means an implied "yes, include." Since committing the patch will case a bit of file movement, I would like to pre-announce the commit in case my timing is not right. Here is the link to the issue: http://bugs.python.org/issue7989 . From benjamin at python.org Fri Jul 23 19:44:07 2010 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 23 Jul 2010 12:44:07 -0500 Subject: [Python-Dev] Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: 2010/7/23 Alexander Belopolsky : > Thanks, everyone who followed up here and on the tracker. ?I am > readying the patch for check in, but as I look back through the > messages, I don't really see anyone's answer to the question in the > subject: > > * ?Include datetime.py in stdlib or not? > > I hope this means an implied "yes, include." ?Since committing the > patch will case a bit of file movement, I would like to pre-announce > the commit in case my timing is not right. It's a bit hard to get a strong affirmative because it's not really anyone's decision. I would say, though, given that there haven't been any strong -1s, you can go ahead. -- Regards, Benjamin From cesare.di.mauro at gmail.com Fri Jul 23 20:09:11 2010 From: cesare.di.mauro at gmail.com (Cesare Di Mauro) Date: Fri, 23 Jul 2010 20:09:11 +0200 Subject: [Python-Dev] Python 3 optimizations... In-Reply-To: References: Message-ID: 2010/7/23 stefan brunthaler > > This sounds like wpython (a CPython derivative with a wider set of byte > code > > commands) could benefit from it. > > > I am aware of the wpython project of Cesare di Mauro. wpython has reached 1.1 final version. If you are interested, you can find it here: http://code.google.com/p/wpython2/ and you can download the new slides that cover the improvements over 1.0 alpha. > I change the > instruction format from bytecode to wordcode, too (because it allows > for more efficient instruction decoding). Did you used wpython wordcode format, or a new one? > Contrary to his approach, > however, I do not change the instruction encoding to pack in > additional optimizations. (I hope to have put that correctly; I have > seen his slides about a year ago.) > Yes, you're right. wpython approach is to encode as much information as it can to save space, decoding time, "specialize" some opcodes, etc.. Cesare -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan at brunthaler.net Fri Jul 23 20:26:27 2010 From: stefan at brunthaler.net (stefan brunthaler) Date: Fri, 23 Jul 2010 20:26:27 +0200 Subject: [Python-Dev] Python 3 optimizations... In-Reply-To: References: Message-ID: > How do you generate the specialized opcode implementations? > I have a small code generator written in Python that uses Mako templates to generate C files that can be included in the main interpreter. It is a data driven approach that uses type information gathered by gdb and check whether given types implement for instance a nb_add method. > Presumably that is done ahead of time, or you'd have to use a JIT, > which is what you're avoiding. > Yes, and yes: I execute the code generator before compiling the Python interpreter, and I am interested in purely interpretative optimization techniques. > I'm guessing from your comments below about cross-module inlining that > you generate a separate .c file with the specialized opcode bodies and > then call through to them via a table of function pointers indexed by > opcode, but I could be totally wrong. ?:) > No, dead on ;) Probably a small example from the top of my head illustrates what is going on: TARGET(FLOAT_ADD): w= POP(); v= TOP(); x= PyFloat_Type.tp_as_number->nb_add(v, w); SET_TOP(x); if (x != NULL) FAST_DISPATCH(); break; And I extend the standard indirect threaded code dispatch table to support the FLOAT_ADD operation. > There are a variety of solutions to getting cross-module inlining > these days. ?Clang+LLVM support link-time optimization (LTO) via a > plugin for gold. ?GCC has LTO and LIPO as well. > A PhD colleague from our institute pointed the gold stuff out to me yesterday, I am going to check out if any of these solutions would work. A deeper problem here is that the heuristics of the compilers are ill-suited to the needs of compiling an interpreter dispatch routine -- I will investigate this further in future research. > This would be interesting. ?We have (obviously) have similar > instrumentation in unladen swallow to gather type feedback. ?We talked > with Craig Citro about finding a way to feed that back to Cython for > exactly this reason, but we haven't really pursued it. > Ok; I think it would actually be fairly easy to use the type information gathered at runtime by the quickening approach. Several auxiliary functions for dealing with these types could be generated by my code generator as well. It is probably worth looking into this, though my current top-priority is my PhD research, so I cannot promise to being able to allocate vast amounts of time for such endeavours. Best, --stefan From stefan at brunthaler.net Fri Jul 23 20:34:16 2010 From: stefan at brunthaler.net (stefan brunthaler) Date: Fri, 23 Jul 2010 20:34:16 +0200 Subject: [Python-Dev] Python 3 optimizations... In-Reply-To: References: Message-ID: > wpython has reached 1.1 final version. If you are interested, you can find > it here: http://code.google.com/p/wpython2/ and you can download the new > slides that cover the improvements over 1.0 alpha. > Thanks for the hint, I will definitely check your new slides. > Did you used wpython wordcode format, or a new one? > No, actually I was well into working on my stuff when you announced wpython last year. My latest instruction format uses a native machine word (64bit) that contains two 32bit halves with the opcode in the lower half and the operand in the upper half. While the opcode certainly does not exceed 10bits or so, I need more than a byte to support more operations (my latest interpreter has 395 instructions). Our instruction decoding is almost identical, though. > Yes, you're right. wpython approach is to encode as much information as it > can to save space, decoding time, "specialize" some opcodes, etc.. > Yes, I see that wpython eliminates common instruction sequences. From my understanding, it corresponds to using static superinstructions in combination with a changed instruction format. Aside of the optimizations in the operation implementation wpython allows to eliminate some instruction dispatches, which are notoriously inefficient. I think it is a very nice approach and some form of inline caching with quickening might well boost performance even more. Cheers, --stefan From reid.kleckner at gmail.com Fri Jul 23 20:57:55 2010 From: reid.kleckner at gmail.com (Reid Kleckner) Date: Fri, 23 Jul 2010 11:57:55 -0700 Subject: [Python-Dev] Python 3 optimizations... In-Reply-To: References: Message-ID: On Fri, Jul 23, 2010 at 11:26 AM, stefan brunthaler wrote: >> I'm guessing from your comments below about cross-module inlining that >> you generate a separate .c file with the specialized opcode bodies and >> then call through to them via a table of function pointers indexed by >> opcode, but I could be totally wrong. ?:) >> > No, dead on ;) > Probably a small example from the top of my head illustrates what is going on: > > TARGET(FLOAT_ADD): > ?w= POP(); > ?v= TOP(); > ?x= PyFloat_Type.tp_as_number->nb_add(v, w); > ?SET_TOP(x); > ?if (x != NULL) FAST_DISPATCH(); > ?break; > > And I extend the standard indirect threaded code dispatch table to > support the FLOAT_ADD operation. I think I was wrong, but now I understand. The inlining you want is to get the nb_add body, not the opcode body. The example you've given brings up a correctness issue. It seems you'd want to add checks that verify that the operands w and v are both floats, and jump to BINARY_ADD if the guard fails. It would require reshuffling the stack operations to defer the pop until after the check, but it shouldn't be a problem. >> This would be interesting. ?We have (obviously) have similar >> instrumentation in unladen swallow to gather type feedback. ?We talked >> with Craig Citro about finding a way to feed that back to Cython for >> exactly this reason, but we haven't really pursued it. >> > Ok; I think it would actually be fairly easy to use the type > information gathered at runtime by the quickening approach. Several > auxiliary functions for dealing with these types could be generated by > my code generator as well. It is probably worth looking into this, > though my current top-priority is my PhD research, so I cannot promise > to being able to allocate vast amounts of time for such endeavours. I think you also record (via gdb) exactly the information that we record. I now see three consumers of type feedback from the CPython interpreter: you or any quickening approach, Cython, and Unladen Swallow. It might be worth converging on a standard way to record this information and serialize it so that it can be analyzed offline. Our feedback recording mechanism currently uses LLVM data structures, but the point of quickening is to avoid that kind of dependency, so we'd need to rewrite it before it would really be useful to you. Reid From stefan at brunthaler.net Fri Jul 23 21:19:15 2010 From: stefan at brunthaler.net (stefan brunthaler) Date: Fri, 23 Jul 2010 21:19:15 +0200 Subject: [Python-Dev] Python 3 optimizations... In-Reply-To: References: Message-ID: > I think I was wrong, but now I understand. ?The inlining you want is > to get the nb_add body, not the opcode body. > Exactly. This would increase performace by quite a bit -- I will start experimentation with that stuff a.s.a.p. > The example you've given brings up a correctness issue. ?It seems > you'd want to add checks that verify that the operands w and v are > both floats, and jump to BINARY_ADD if the guard fails. ?It would > require reshuffling the stack operations to defer the pop until after > the check, but it shouldn't be a problem. > That's usually the problem when you're doing something from the top of your head, especially when its already 9pm :) You're right, a simple way around this is either: TARGET(FLOAT_ADD): if (!(TOP()->ob_type == SECOND()->ob_type && TOP()->ob_type == &PyFloat_Type)) goto TARGET_BINARY_ADD_SKIP_DECODE; ...remains the same... Note: the skip_decode is necessary because otherwise it would advance the instruction pointer. Another, simple approach is to add another set of labels to denote inline cache misses, e.g.: TARGET(BINARY_ADD): w= POP(); v= TOP(); BINARY_ADD_CACHE_MISS: x= PyNumber_Add(v, w); ... TARGET(FLOAT_ADD): w= POP(); v= TOP(); if (v->ob_type != w->ob_type || v->ob_type != &PyFloat_Type) goto BINARY_ADD_CACHE_MISS; ... You could also call the PyNumber_Add function yourself instead of the goto, but I did not implement it this way... > I think you also record (via gdb) exactly the information that we > record. ?I now see three consumers of type feedback from the CPython > interpreter: you or any quickening approach, Cython, and Unladen > Swallow. ?It might be worth converging on a standard way to record > this information and serialize it so that it can be analyzed offline. > Indeed. Probably a bigger set of types of frequently used C extension modules would be useful as well. I am doing the simplest possible thing here: since the gdb pretty print already is pretty close to a Python data type definition, I just copied this and did a few search and replace commands to convert it to a valid Python data type. > Our feedback recording mechanism currently uses LLVM data structures, > but the point of quickening is to avoid that kind of dependency, so > we'd need to rewrite it before it would really be useful to you. > Ok. --stefan From barry at python.org Sat Jul 24 00:19:41 2010 From: barry at python.org (Barry Warsaw) Date: Sat, 24 Jul 2010 00:19:41 +0200 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <20100723125440.721860bf@snowdog> References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> <63B36042-C4F7-4BCA-8623-5182405D8852@mac.com> <20100723120252.2e0690cc@snowdog> <25347613-8488-40C3-B9EF-9C85D0C47340@mac.com> <20100723125440.721860bf@snowdog> Message-ID: <20100724001941.6adad036@snowdog> On Jul 23, 2010, at 12:54 PM, Barry Warsaw wrote: >On Jul 23, 2010, at 11:48 AM, Ronald Oussoren wrote: > >>> I'd be open to adding the >>> platform name to the tag, but I'd probably define it as part of the >>> implementation field, e.g. foo.cpython-linux2-32m.so. Or maybe >>> start with the platform name, e.g. foo.linux2-cpython-32m. This >>> isn't a strong preference though. >> >>I don't have a strong opionion, but placing the platform name at the >>start is probably better to be consistent with >>sysconfig.get_platform(). > >What about the architecture (i386, amd64)? With every increase in >length I start to get more concerned. We could encode the platform >and architecture, but that gets into cryptic territory. OTOH, would >you really co-install i386 and amd64 shared libraries on the same >machine? (hello NFS ;). Thinking about this some more, I'd rather *not* include the platform or architecture in the tag by default. They aren't really necessary to support the instigating use case and will probably be fairly uncommon. I'd be okay including a configure option to allow you to add whatever you want after the implementation, version, and flags. E.g. something like: ./configure --with-abi-tag-extension=linux2 would lead to foo.cpython-32m-linux2.so, so not the nicer names we'd prefer but probably good enough for your purposes. Would that work for you? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Sat Jul 24 00:22:52 2010 From: barry at python.org (Barry Warsaw) Date: Sat, 24 Jul 2010 00:22:52 +0200 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <87mxti8lqc.fsf@muni.brainbot.com> References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> <63B36042-C4F7-4BCA-8623-5182405D8852@mac.com> <20100723120252.2e0690cc@snowdog> <25347613-8488-40C3-B9EF-9C85D0C47340@mac.com> <20100723125440.721860bf@snowdog> <764F1595-5C8B-44D8-86C5-937B326B2FCD@mac.com> <87mxti8lqc.fsf@muni.brainbot.com> Message-ID: <20100724002252.19d91267@snowdog> On Jul 23, 2010, at 01:46 PM, schmir at gmail.com wrote: >Doesn't anybody else think this is lost work for very little gain? My >/usr/lib/python2.6/site-packages directory consumes 200MB on disk. I >couldn't care less if my /usr/lib/python2.5/site-packages consumed the >same amount of disk space... Right, you probably don't care now that your extension modules live in foo.so so it probably won't make much difference if they were named foo-blahblah.so, as long as they import. :) If you use Debian or Ubuntu though, it'll be a win for you by allow us to make Python support much more robust. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Sat Jul 24 00:25:09 2010 From: barry at python.org (Barry Warsaw) Date: Sat, 24 Jul 2010 00:25:09 +0200 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> Message-ID: <20100724002509.0ad8d359@snowdog> On Jul 23, 2010, at 08:56 PM, Nick Coghlan wrote: >On Fri, Jul 23, 2010 at 12:40 AM, Barry Warsaw >wrote: >> Python implementations *MAY* include additional flags in the file >> name tag as appropriate. ?For example, on POSIX systems these flags >> will also contribute to the file name: >> >> ?* ``--with-pydebug`` (flag: ``d``) >> ?* ``--with-pymalloc`` (flag: ``m``) >> ?* ``--with-wide-unicode`` (flag: ``u``) >> >> By default in Python 3.2, ``configure`` enables ``--with-pymalloc`` >> so shared library file names would appear as ``foo.cpython-32m.so``. >> When the other two flags are also enabled, the file names would be >> ``foo.cpython-32dmu.so``. >> >> (This PEP only addresses build issues on POSIX systems that use the >> ``configure`` script. ?While Windows or other platform support is not >> explicitly disallowed under this PEP, platform expertise is needed in >> order to evaluate, describe, and implement support on such >> platforms.) > >This leads me to a question: how do these configure options affect the >PEP 384 stable ABI? That PEP is currently silent on the issue, while >PEP 3149 appears to implicitly assume that "abi3" completely specifies >the ABI. It's a great question - perhaps Martin can chime in? It may be that 'abiX' isn't enough to fully specify compatible extension modules even when that module is written entirely and solely against PEP 384. In that case, we may need to include the configure flags in the tag, e.g. foo.abi3-dmu.so. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From breamoreboy at yahoo.co.uk Sat Jul 24 00:26:08 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Fri, 23 Jul 2010 23:26:08 +0100 Subject: [Python-Dev] http://bugs.python.org/issue231540 Message-ID: Is there any money to pay for the forthcoming 10th birthday party for this issue? Is the OP still alive? Kindest regards. Mark Lawrence. From p.f.moore at gmail.com Sat Jul 24 01:09:52 2010 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 24 Jul 2010 00:09:52 +0100 Subject: [Python-Dev] http://bugs.python.org/issue231540 In-Reply-To: References: Message-ID: On 23 July 2010 23:26, Mark Lawrence wrote: > Is there any money to pay for the forthcoming 10th birthday party for this > issue? ?Is the OP still alive? I'm not sure the sarcasm helps much. What do you suggest should be done with the request? Nobody has provided a patch, so there's nothing to commit. Closing it as "won't fix" seems unreasonable, as I imagine that should a suitable patch be supplied, it would be accepted. There's no magical means by which such a patch would appear, though. The OP clearly[1] is either not interested enough or doesn't have the skills to provide a patch, and no-one else has stepped up to do so. Note that it's been classified as a feature request, not a bug. So there's nothing wrong, as such, with it remaining unresolved. Paul. [1] I say "clearly" - it may be that he could provide a patch if asked. Maybe it would be worth you contacting him to ask if the issue is still a problem for him, and whether he can assist in resolving it. From alexander.belopolsky at gmail.com Sat Jul 24 01:20:46 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Fri, 23 Jul 2010 19:20:46 -0400 Subject: [Python-Dev] Python profiler and other tools Message-ID: I am changing the subject from "http://bugs.python.org/issue231540," because if there was a prize for a non-descriptive subject, OP would win it. There must be a good reason why traditional software development tools such as debugger, profiler and coverage are mostly neglected in python. Terry and I have recently discovered that the trace (coverage) module does not have any unit tests [1] and pretty much does not work in 3.x. [2] One of the recently fixed trace module problems [3] is likely affecting profile module as well. I tried to figure out who works on profile to add them to the nosy list and could not find anyone. I seem to remember someone suggesting that python's dynamic nature makes traditional development tools unnecessary. A print statement is the only debugging tool that you need. There may be some truth to it, but in this case, python should not distribute these tools. I am personally in the opposite camp: I love pdb and I am a big believer in profiling and coverage testing. I have entered myself as a maintainer for the trace module which is the simplest of the three, but I would like to learn from developers of pdb and profile on a range of overlapping issues. [1] http://bugs.python.org/issue9315 [2] http://bugs.python.org/issue9317 [3] http://bugs.python.org/issue9323 On Fri, Jul 23, 2010 at 6:26 PM, Mark Lawrence wrote: > Is there any money to pay for the forthcoming 10th birthday party for this > issue? ?Is the OP still alive? > > Kindest regards. > > Mark Lawrence. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/alexander.belopolsky%40gmail.com > From breamoreboy at yahoo.co.uk Sat Jul 24 01:39:49 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sat, 24 Jul 2010 00:39:49 +0100 Subject: [Python-Dev] http://bugs.python.org/issue231540 In-Reply-To: References: Message-ID: On 24/07/2010 00:09, Paul Moore wrote: > On 23 July 2010 23:26, Mark Lawrence wrote: >> Is there any money to pay for the forthcoming 10th birthday party for this >> issue? Is the OP still alive? > > I'm not sure the sarcasm helps much. What do you suggest should be > done with the request? Nobody has provided a patch, so there's nothing > to commit. Closing it as "won't fix" seems unreasonable, as I imagine > that should a suitable patch be supplied, it would be accepted. > > There's no magical means by which such a patch would appear, though. > The OP clearly[1] is either not interested enough or doesn't have the > skills to provide a patch, and no-one else has stepped up to do so. > > Note that it's been classified as a feature request, not a bug. So > there's nothing wrong, as such, with it remaining unresolved. > > Paul. > > [1] I say "clearly" - it may be that he could provide a patch if > asked. Maybe it would be worth you contacting him to ask if the issue > is still a problem for him, and whether he can assist in resolving it. Paul, I'm on the verge of giving up my time because the whole system is a complete and utter waste of my time. I feel quite happy that in my brief tenure I've closed 46 issues, but there's so many more that could have been closed, but yet again you don't even get the courtesy of a response when there's more in the pipeline that could be closed. I'd quote the issue numbers here and now, but I'm just too flaming tired to do so, though a quick count indicates I've got 23 ongoing that I'm attempting to sort. As it happens, I have been having discussions offline in an attempt to shift the culture of Python development but I don't believe that anything will come out of it. Let's face it, development is much more interesting than bug fixes. And once again, if some stupid idiot volunteer bothers to put in a patch to the code and/or the unit test, and it sits and rots for five years, is that person likely to come back to Python? Strangely, some do. Sorry, I'm off to bed. Yours feeling most disillusioned with python-dev. Mark Lawrence. From tjreedy at udel.edu Sat Jul 24 01:40:34 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 23 Jul 2010 19:40:34 -0400 Subject: [Python-Dev] http://bugs.python.org/issue231540 In-Reply-To: References: Message-ID: On 7/23/2010 6:26 PM, Mark Lawrence wrote: > Is there any money to pay for the forthcoming 10th birthday party for > this issue? Is the OP still alive? This reminds me of some low priority items on my todo list ;-) Hmmm. Right now there are 756 open feature requests out of 2779 open issues. There have been bug days. Maybe after 3.2 is out, we should have a feature day to see if some feature requests cannot be closed as obsolete, unwise, unwanted, or too difficult. It is possible that PEP3099, Rejected ideas, could be expanded to help triagers close already rejected ideas. Something I/someone might do is make some manageable lists of related ideas (say, additions to builtin functions) and post them on python-ideas for discussion and voting. -- Terry Jan Reedy From rrr at ronadam.com Sat Jul 24 02:05:03 2010 From: rrr at ronadam.com (Ron Adam) Date: Fri, 23 Jul 2010 19:05:03 -0500 Subject: [Python-Dev] [isssue 2001] Pydoc enhancement patch questions Message-ID: This regards the following feature request. http://bugs.python.org/issue2001 Summery: The pydoc "gui" enhancement patch adds a navigation bar on each page with features that correspond to abilities that are currently available to help. ie.. a get field, to get help on an item, a find field, to search for modules by keyword, plus modules, topics, and keyword index links. The file links read the python file in as text and inserts it into a web page instead of relying on the browser to do the right thing. (often enough it doesn't) To achieve this I reworked the existing pydoc server into one that can respond to a navigation bar at the top of the served pages. The new "local_text_server" will exist in the http package where the other server related modules are. The callback function passed to the server does all the pydoc specific work so the server itself is a simple general purpose text server useful for making browser interface front ends. This also removed the need for tkinter in pydoc and allows pydoc -g to work on computers where tk isn't installed. As per discussion on the tracker: > Alexander Belopolsky added the comment: > > +:program:`pydoc` :option:`-g` will start the server and additionally open a web > +browser to a module index page. Each served page has a navagation bar at the > +top where you can 'get' help on a individual item, 'find' all modules with a > +keyword in thier synopsis line, and goto indexes for 'modules', 'topics' and > +'keywords'. > > I am not sure I like the fact that the browser is started automatically. > Please bring this up on python-dev. This may be an opportunity to > rethink pydoc command line switches. For example, -p and -g are > currently exclusive, but it would make sense for -g to start server on > the port specified by -p. So are any thoughts on starting the web browser automatically, and on how the -g and -p command line switches work? I'm also not sure about the name for the server. I've use local_text_server, but it may also be useful in non-local cases as well. The newest patch is... http://bugs.python.org/file18165/pydoc_server3.diff Any feedback will be welcome. Ron From simon at ikanobori.jp Sat Jul 24 02:18:17 2010 From: simon at ikanobori.jp (Simon de Vlieger) Date: Sat, 24 Jul 2010 02:18:17 +0200 Subject: [Python-Dev] http://bugs.python.org/issue231540 In-Reply-To: References: Message-ID: On 24 July 2010 01:39, Mark Lawrence wrote: > On 24/07/2010 00:09, Paul Moore wrote: >> >> On 23 July 2010 23:26, Mark Lawrence ?wrote: >>> >>> Is there any money to pay for the forthcoming 10th birthday party for >>> this >>> issue? ?Is the OP still alive? >> >> I'm not sure the sarcasm helps much. What do you suggest should be >> done with the request? Nobody has provided a patch, so there's nothing >> to commit. Closing it as "won't fix" seems unreasonable, as I imagine >> that should a suitable patch be supplied, it would be accepted. >> >> There's no magical means by which such a patch would appear, though. >> The OP clearly[1] is either not interested enough or doesn't have the >> skills to provide a patch, and no-one else has stepped up to do so. >> >> Note that it's been classified as a feature request, not a bug. So >> there's nothing wrong, as such, with it remaining unresolved. >> >> Paul. >> >> [1] I say "clearly" - it may be that he could provide a patch if >> asked. Maybe it would be worth you contacting him to ask if the issue >> is still a problem for him, and whether he can assist in resolving it. > > Paul, > > I'm on the verge of giving up my time because the whole system is a complete > and utter waste of my time. ?I feel quite happy that in my brief tenure I've > closed 46 issues, but there's so many more that could have been closed, but > yet again you don't even get the courtesy of a response when there's more in > the pipeline that could be closed. ?I'd quote the issue numbers here and > now, but I'm just too flaming tired to do so, though a quick count indicates > I've got 23 ongoing that I'm attempting to sort. > > As it happens, I have been having discussions offline in an attempt to shift > the culture of Python development but I don't believe that anything will > come out of it. ?Let's face it, development is much more interesting than > bug fixes. And once again, if some stupid idiot volunteer bothers to put in > a patch to the code and/or the unit test, and it sits and rots for five > years, is that person likely to come back to Python? ?Strangely, some do. > > Sorry, I'm off to bed. > > Yours feeling most disillusioned with python-dev. > > Mark Lawrence. Mark, when I read your emails it seems to me as if you have the greatest concern with improving Python, the language and improving the state of the bug tracker. This is a great thing, people like you are much needed. However, I do seem to notice you try to take a business-like approach here on this mailinglist. Most people on python-dev are volunteers who (like you) spend their free time helping and working on Python. People who work in their free time are less likely to feel obliged to respond immediately to an issue. They are also less likely to keep paying attention to the bugs they were assigned. I think a person like you is needed, someone who weeds through the rotting bug reports (not the feature requests) and tries to follow up on them. Is the issue persistent for the user, has it been fixed as collateral on another fix, etcetera. However, I think you would get more done if you switched from a business philosophy to accepting that most people here are volunteers, don't try to pressure volunteers. Try to do the best *you* can within the community and let that help the project further. Oh, and with business philosophy I mean: mails like the one you start this thread with are interpreted by me as being very pushy, overly sarcastic and if my project manager at the office sends me an email like that I know I have to do it right now. I would dislike to be spoken to like this in an voluntary environment. Do note that I do understand where your feelings come from. Regards, Simon de Vlieger P.S. a feature day sounds like a great idea! From debatem1 at gmail.com Sat Jul 24 03:06:32 2010 From: debatem1 at gmail.com (geremy condra) Date: Fri, 23 Jul 2010 18:06:32 -0700 Subject: [Python-Dev] http://bugs.python.org/issue231540 In-Reply-To: References: Message-ID: On Fri, Jul 23, 2010 at 5:18 PM, Simon de Vlieger wrote: > On 24 July 2010 01:39, Mark Lawrence wrote: >> On 24/07/2010 00:09, Paul Moore wrote: >>> >>> On 23 July 2010 23:26, Mark Lawrence ?wrote: >>>> >>>> Is there any money to pay for the forthcoming 10th birthday party for >>>> this >>>> issue? ?Is the OP still alive? >>> >>> I'm not sure the sarcasm helps much. What do you suggest should be >>> done with the request? Nobody has provided a patch, so there's nothing >>> to commit. Closing it as "won't fix" seems unreasonable, as I imagine >>> that should a suitable patch be supplied, it would be accepted. >>> >>> There's no magical means by which such a patch would appear, though. >>> The OP clearly[1] is either not interested enough or doesn't have the >>> skills to provide a patch, and no-one else has stepped up to do so. >>> >>> Note that it's been classified as a feature request, not a bug. So >>> there's nothing wrong, as such, with it remaining unresolved. >>> >>> Paul. >>> >>> [1] I say "clearly" - it may be that he could provide a patch if >>> asked. Maybe it would be worth you contacting him to ask if the issue >>> is still a problem for him, and whether he can assist in resolving it. >> >> Paul, >> >> I'm on the verge of giving up my time because the whole system is a complete >> and utter waste of my time. ?I feel quite happy that in my brief tenure I've >> closed 46 issues, but there's so many more that could have been closed, but >> yet again you don't even get the courtesy of a response when there's more in >> the pipeline that could be closed. ?I'd quote the issue numbers here and >> now, but I'm just too flaming tired to do so, though a quick count indicates >> I've got 23 ongoing that I'm attempting to sort. >> >> As it happens, I have been having discussions offline in an attempt to shift >> the culture of Python development but I don't believe that anything will >> come out of it. ?Let's face it, development is much more interesting than >> bug fixes. And once again, if some stupid idiot volunteer bothers to put in >> a patch to the code and/or the unit test, and it sits and rots for five >> years, is that person likely to come back to Python? ?Strangely, some do. >> >> Sorry, I'm off to bed. >> >> Yours feeling most disillusioned with python-dev. >> >> Mark Lawrence. > > Mark, > > when I read your emails it seems to me as if you have the greatest > concern with improving Python, the language and improving the state of > the bug tracker. > > This is a great thing, people like you are much needed. However, I do > seem to notice you try to take a business-like approach here on this > mailinglist. Most people on python-dev are volunteers who (like you) > spend their free time helping and working on Python. > > People who work in their free time are less likely to feel obliged to > respond immediately to an issue. They are also less likely to keep > paying attention to the bugs they were assigned. > > I think a person like you is needed, someone who weeds through the > rotting bug reports (not the feature requests) and tries to follow up > on them. Is the issue persistent for the user, has it been fixed as > collateral on another fix, etcetera. > > However, I think you would get more done if you switched from a > business philosophy to accepting that most people here are volunteers, > don't try to pressure volunteers. Try to do the best *you* can within > the community and let that help the project further. > > Oh, and with business philosophy I mean: mails like the one you start > this thread with are interpreted by me as being very pushy, overly > sarcastic and if my project manager at the office sends me an email > like that I know I have to do it right now. I would dislike to be > spoken to like this in an voluntary environment. Do note that I do > understand where your feelings come from. > > Regards, > > Simon de Vlieger > > P.S. a feature day sounds like a great idea! I don't care if he yells and rants and raves, he's a volunteer too, he does a ton of work that nobody else has stepped up to do, and IMO we're very lucky to have him doing it. Geremy Condra From greg at krypto.org Sat Jul 24 03:31:55 2010 From: greg at krypto.org (Gregory P. Smith) Date: Fri, 23 Jul 2010 18:31:55 -0700 Subject: [Python-Dev] Set the namespace free! In-Reply-To: References: <4C484FD0.2080803@zlotniki.pl> Message-ID: On Thu, Jul 22, 2010 at 9:24 AM, wrote: > I agree with the idea, but a far less radical change is needed to get the > desired result. > The basic idea is this: it should be possible to use any name as an > identifier in the syntax, including names > like 'while' and 'import'. But there is no need to mess up the entire > language to allow this > (either by quoting all the identifiers, perl-style, or by marking the > keywords). > Yuck. Anyone who feels they need a variable named the same a reserved word simply feels wrong and needs reeducation. New words are introduced very rarely and we do care about the ramifications when we do it. What next? An optional way to support case insensitive names using a unicode character prefix? -gps > > All that is needed is something like this: > > foo = 7 > :foo = 7 # exactly like foo=7 > :while= 3 # assigns 3 to variable 'while' > globals()['while']=3 # current way to do this > > print element.:for # from example below > # > # keyword parameters to a function call: > # > BuildUrlQuery( lang='en', item='monsoon', :class='normal') # -> > "?lang=en&query=monsoon&class=normal" > > > The generic keyword function call is a really nice language feature, but > it's rather impaired by the need to avoid > those names which happen to be keywords. > > The lack of this is most painful when you are auto-generating python code > which forms a bridge to another language with its own > namespace (as in XML example). It's a pain when some of the names you might > generate could conflict with python keywords. > So, you end up using dicts and getattrs for everything and the code gets > much less readable. With a simple escape like :name available, > it's worthwhile to do everything with identifiers and generate the escape > only as needed for these. > > > One of the great strengths of python is the ability to form cleans and > comprehensive bridges to other languages and environments (thus, > in many cases, avoiding the need to write programs in those other > environments :-) ). This feature would fill a gap there. > > The python tcl/tk interface is a bit messed up since tcl/tk uses some names > for options which conflict with python keywords, > and so you need to add '_' to those names. > > There is a feature like this in VHDL: \name\ and \while\ are identifiers, > the backslashes are not part of the name, but just > quote it. In VHDL you can write identifiers like \22\, and > \!This?is=Strange\ as well; since VHDL creates modules that > have named ports, and those modules can interface to things generated by > other environments, they needed a > way to assign any name to a port. > > For python, I'm not sure it makes sense to allow identifiers that doesn't > follow the basic rule "[A-Za-z_][A-Za-z_0-9]*" -- that could > break some debugging tools which expect variable names to be well-formed -- > but it would be useful > to be able to use any well-formed name as an identifier, including those > which happen to be keywords. > > I've suggested :name, which doesn't break old code, and doesn't require > using any new punctuation. Syntax would not change, > just the lexical definition of 'identifier'. If the intent is to allow > arbitrary names (not just well-formed ones), then n'name' would > work better (and is consistent with existing stuff). > > > > > Date: Thu, 22 Jul 2010 10:41:39 -0400 > > From: jnoller at gmail.com > > To: bartosz-tarnowski at zlotniki.pl > > CC: python-dev at python.org > > Subject: Re: [Python-Dev] Set the namespace free! > > > > > On Thu, Jul 22, 2010 at 10:04 AM, Bartosz Tarnowski > > wrote: > > > > > > Hello, guys. > > > > > > Python has more and more reserved words over time. It becomes quite > > > annoying, since you can not use variables and attributes of such names. > > > Suppose I want to make an XML parser that reads a document and returns > an > > > object with attributes corresponding to XML element attributes: > > > > > >> elem = parse_xml("") > > >> print elem.param > > > boo > > > > > > What should I do then, when the attribute is a reserver word? I could > use > > > trailing underscore, but this is quite ugly and introduces ambiguity. > > > > > >> elem = parse_xml("") > > >> print elem.for_ #????? > > >> elem = parse_xml("") > > >> print elem.for__ #????? > > > > > > My proposal: let's make a syntax change. > > > > > > Let all reserved words be preceded with some symbol, i.e. "!" > (exclamation > > > mark). This goes also for standard library global identifiers. > > > > > > !for boo in foo: > > > !if boo is !None: > > > !print(hoo) > > > !else: > > > !return !sorted(woo) > > > > > > > > > This would allow the user to declare any identifier with any name: > > > > > > for = with(return) + try > > > > > > What do you think of it? It is a major change, but I think Python needs > it. > > > > > > -- > > > haael > > > > > > > I'm not a fan of this - I'd much prefer[1] that we use the exclamation > > point to determine scope: > > > > foobar - local > > !foobar - one up > > !!foobar - higher than the last one > > !!!foobar - even higher in scope > > > > We could do the inverse as well; if you append ! you can push variable > > down in scope. > > > > Jesse > > > > > > [1] I am not serious. > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/gsmith%40alumni.uwaterloo.ca > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/greg%40krypto.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.curtin at gmail.com Sat Jul 24 04:42:12 2010 From: brian.curtin at gmail.com (Brian Curtin) Date: Fri, 23 Jul 2010 21:42:12 -0500 Subject: [Python-Dev] http://bugs.python.org/issue231540 In-Reply-To: References: Message-ID: On Fri, Jul 23, 2010 at 18:39, Mark Lawrence wrote: > On 24/07/2010 00:09, Paul Moore wrote: > >> On 23 July 2010 23:26, Mark Lawrence wrote: >> >>> Is there any money to pay for the forthcoming 10th birthday party for >>> this >>> issue? Is the OP still alive? >>> >> >> I'm not sure the sarcasm helps much. What do you suggest should be >> done with the request? Nobody has provided a patch, so there's nothing >> to commit. Closing it as "won't fix" seems unreasonable, as I imagine >> that should a suitable patch be supplied, it would be accepted. >> >> There's no magical means by which such a patch would appear, though. >> The OP clearly[1] is either not interested enough or doesn't have the >> skills to provide a patch, and no-one else has stepped up to do so. >> >> Note that it's been classified as a feature request, not a bug. So >> there's nothing wrong, as such, with it remaining unresolved. >> >> Paul. >> >> [1] I say "clearly" - it may be that he could provide a patch if >> asked. Maybe it would be worth you contacting him to ask if the issue >> is still a problem for him, and whether he can assist in resolving it. >> > > Paul, > > I'm on the verge of giving up my time because the whole system is a > complete and utter waste of my time. I feel quite happy that in my brief > tenure I've closed 46 issues, but there's so many more that could have been > closed, but yet again you don't even get the courtesy of a response when > there's more in the pipeline that could be closed. I'd quote the issue > numbers here and now, but I'm just too flaming tired to do so, though a > quick count indicates I've got 23 ongoing that I'm attempting to sort. > > As it happens, I have been having discussions offline in an attempt to > shift the culture of Python development but I don't believe that anything > will come out of it. Let's face it, development is much more interesting > than bug fixes. And once again, if some stupid idiot volunteer bothers to > put in a patch to the code and/or the unit test, and it sits and rots for > five years, is that person likely to come back to Python? Strangely, some > do. > > Sorry, I'm off to bed. > > Yours feeling most disillusioned with python-dev. Mark, First off, thanks for your time. One thing I don't feel is very beneficial is to focus so much on the number of issues you are able to close, because that number by itself is too high-level. Closing 10 issues might not be better than closing 5 issues. If you follow baseball, it's like a pitcher's win total. A higher number doesn't really mean the pitcher is better, and a lower number doesn't mean the pitcher is worse. You find the better pitcher by drilling down into more specific statistics. Closing an obviously invalid issue takes little effort, but it's a -1 for the issue total. Closing an issue as fixed might take a little more effort in tracking down a revision number, but it's a -1 for the total and some varying value for performance, coverage, etc. Closing an issue early because no one commented or no one proposed a patch in some certain time span is, to me, +1 to the total, possibly more. Not closing an issue is a +/- 0. An issue is an issue once confirmed, regardless of a lack of comment or a lack of attention from core development. I agree that it's unfortunate having issues drag on, but time is one of the biggest bottlenecks in the development of open source software. We have plenty of qualified, quality, smart people working on Python, but there are only so many hours in the day that we can spend on it. With that said, I hope you'll continue your efforts. What helped me when I started out in your position was to pick issues I knew I could work on with relative ease and come out without having to duck too many times. I knew I wasn't going to remove the GIL, but I could start with some of the low hanging fruit like making zipfile.ZipFile work as a context manager (first thing I fixed, I think). Like you, I wanted to make an impact, and in order to make a longer term impact I wanted to have some early success to get further into the game. No one wants to keep going if it's not fun, and this should be fun. My suggestion is to throw more effort into less issues. Going back to baseball, if you can advance the runners without getting on base yourself (e.g., a sacrifice fly out), that's still a good thing for the team. Providing a code review to a review-less issue is more valuable than requesting that someone else provide comments -- a review moves the issue along. Updating a patch to a current revision is worth more than seeing if anyone still runs AIX -- updating moves the issue along. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sat Jul 24 05:20:10 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 24 Jul 2010 13:20:10 +1000 Subject: [Python-Dev] http://bugs.python.org/issue231540 In-Reply-To: References: Message-ID: <201007241320.10574.steve@pearwood.info> On Sat, 24 Jul 2010 12:42:12 pm Brian Curtin wrote: > First off, thanks for your time. One thing I don't feel is very > beneficial is to focus so much on the number of issues you are able > to close, because that number by itself is too high-level. Closing 10 > issues might not be better than closing 5 issues. If you follow > baseball, it's like a pitcher's win total. [...] > My suggestion is to throw more effort into less issues. Going back to > baseball Mark is from the UK. He has probably heard of baseball, but otherwise baseball analogies will probably go right over his head (as they went over mine). -- Steven D'Aprano From steve at pearwood.info Sat Jul 24 05:45:06 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 24 Jul 2010 13:45:06 +1000 Subject: [Python-Dev] http://bugs.python.org/issue231540 In-Reply-To: References: Message-ID: <201007241345.06866.steve@pearwood.info> On Sat, 24 Jul 2010 09:39:49 am Mark Lawrence wrote: > I'm on the verge of giving up my time because the whole system is a > complete and utter waste of my time. [...] Mark, you have entirely avoided the issues Paul raised. It's one thing to fire off a snarky email implying that the Python developers are lazy slackers for allowing an issue to approach ten years old, but Paul's response is valid. What do you suggest should be done with it? Close it as "Won't fix"? That doesn't help anyone. If you have the appropriate patch and unit tests, then great, you should say so. But I believe trying to guilt others into writing the needed tests is neither fair nor effective in the long term. Snarking publicly might make you feel better, but it makes for an unpleasant community and is likely to alienate more people than it will inspire. > And once again, if some stupid idiot > volunteer bothers to put in a patch to the code and/or the unit test, > and it sits and rots for five years, is that person likely to come > back to Python? Strangely, some do. I'm not sure that describing people who provide patches as "stupid idiot volunteers" is a good way to motivate more people to provide patches. But either way, I think you answer your own question: apparently some people *are* understanding of the reasons that issues sometimes get missed or neglected. -- Steven D'Aprano From ronaldoussoren at mac.com Sat Jul 24 10:54:25 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sat, 24 Jul 2010 09:54:25 +0100 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <20100724001941.6adad036@snowdog> References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> <63B36042-C4F7-4BCA-8623-5182405D8852@mac.com> <20100723120252.2e0690cc@snowdog> <25347613-8488-40C3-B9EF-9C85D0C47340@mac.com> <20100723125440.721860bf@snowdog> <20100724001941.6adad036@snowdog> Message-ID: On 23 Jul, 2010, at 23:19, Barry Warsaw wrote: > On Jul 23, 2010, at 12:54 PM, Barry Warsaw wrote: > >> On Jul 23, 2010, at 11:48 AM, Ronald Oussoren wrote: >> >>>> I'd be open to adding the >>>> platform name to the tag, but I'd probably define it as part of the >>>> implementation field, e.g. foo.cpython-linux2-32m.so. Or maybe >>>> start with the platform name, e.g. foo.linux2-cpython-32m. This >>>> isn't a strong preference though. >>> >>> I don't have a strong opionion, but placing the platform name at the >>> start is probably better to be consistent with >>> sysconfig.get_platform(). >> >> What about the architecture (i386, amd64)? With every increase in >> length I start to get more concerned. We could encode the platform >> and architecture, but that gets into cryptic territory. OTOH, would >> you really co-install i386 and amd64 shared libraries on the same >> machine? (hello NFS ;). > > Thinking about this some more, I'd rather *not* include the platform or > architecture in the tag by default. They aren't really necessary to support > the instigating use case and will probably be fairly uncommon. > > I'd be okay including a configure option to allow you to add whatever you want > after the implementation, version, and flags. E.g. something like: > > ./configure --with-abi-tag-extension=linux2 > > would lead to foo.cpython-32m-linux2.so, so not the nicer names we'd prefer > but probably good enough for your purposes. > > Would that work for you? That would certainly work. That said, I'm also fine with not adding the platform information or configure argument at all. My usecase is fairly exotic and I do have a feasible workaround. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From ncoghlan at gmail.com Sat Jul 24 12:19:57 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 24 Jul 2010 20:19:57 +1000 Subject: [Python-Dev] Include datetime.py in stdlib or not? In-Reply-To: References: Message-ID: On Sat, Jul 24, 2010 at 3:44 AM, Benjamin Peterson wrote: > 2010/7/23 Alexander Belopolsky : >> Thanks, everyone who followed up here and on the tracker. ?I am >> readying the patch for check in, but as I look back through the >> messages, I don't really see anyone's answer to the question in the >> subject: >> >> * ?Include datetime.py in stdlib or not? >> >> I hope this means an implied "yes, include." ?Since committing the >> patch will case a bit of file movement, I would like to pre-announce >> the commit in case my timing is not right. > > It's a bit hard to get a strong affirmative because it's not really > anyone's decision. I would say, though, given that there haven't been > any strong -1s, you can go ahead. Yep, that sounds right to me too. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sat Jul 24 12:37:23 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 24 Jul 2010 20:37:23 +1000 Subject: [Python-Dev] [isssue 2001] Pydoc enhancement patch questions In-Reply-To: References: Message-ID: On Sat, Jul 24, 2010 at 10:05 AM, Ron Adam wrote: >> I am not sure I like the fact that the browser is started automatically. >> Please bring this up on python-dev. ?This may be an opportunity to >> rethink pydoc command line switches. ?For example, -p and -g are >> currently exclusive, but it would make sense for -g to start server on >> the port specified by -p. > > So are any thoughts on starting the web browser automatically, and on how > the -g and -p command line switches work? My suggestion: - leave the "-g" option alone (including the tk gui), but make sure other options still work when tk is unavailable - add a "-b" option to start the server and open the webbrowser automatically - allow "-p" to be combined with either "-b" or "-g" to specify where the server should run (or is running) For the "-b" option, if the server is already running (and hence the port is in use), catch the exception, print a message and start the webbrowser anyway. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From guido at python.org Sat Jul 24 16:08:31 2010 From: guido at python.org (Guido van Rossum) Date: Sat, 24 Jul 2010 07:08:31 -0700 Subject: [Python-Dev] Thoughts fresh after EuroPython Message-ID: While the EuroPython sprints are still going on, I am back home, and after a somewhat restful night of sleep, I have some thoughts I'd like to share before I get distracted. Note, I am jumping wildly between topics. - Commit privileges: Maybe we've been too careful with only giving commit privileges to to experienced and trusted new developers. I spoke to Ezio Melotti and from his experience with getting commit privileges, it seems to be a case of "the lion is much more afraid of you than you are afraid of the lion". I.e. having got privileges he was very concerned about doing something wrong, worried about the complexity of SVN, and so on. Since we've got lots of people watching the commit stream, I think that there really shouldn't need to be a worry at all about a new committer doing something malicious, and there shouldn't be much worry about honest beginners' mistakes either -- the main worry remains that new committers don't use their privileges enough. So, my recommendation (which surely is a turn-around of my *own* attitude in the past) is to give out more commit privileges sooner. - Concurrency and parallelism: Russel Winder and Sarah Mount pushed the idea of CSP (http://en.wikipedia.org/wiki/Communicating_sequential_processes) in several talks at the conference. They (at least Russell) emphasized the difference between concurrency (interleaved event streams) and parallelism (using many processors to speed things up). Their prediction is that as machines with many processing cores become more prevalent, the relevant architecture will change from cores sharing a single coherent memory (the model on which threads are based) to one where each core has a limited amount of private memory, and communication is done via message passing between the cores. This gives them (and me :-) hope that the GIL won't be a problem as long as we adopt a parallel processing model. Two competing models are the Actor model, which is based on asynchronous communication, and CSP, which is synchronous (when a writer writes to a channel, it blocks until a reader reads that value -- a rendezvous). At least Sarah suggested that both models are important. She also mentioned that a merger is under consideration between the two major CSP-for-Python packages, Py-CSP and Python-CSP. I also believe that the merger will be based on the stdlib multiprocessing package, but I'm not sure. I do expect that we may get some suggestions from that corner to make some minor changes to details of multiprocessing (and perhaps threading), and I think we should be open to those (I expect these will be good suggestions for small tweaks, not major overhauls). - After seeing Raymond's talk about monocle (search for it on PyPI) I am getting excited again about PEP 380 (yield from, return values from generators). Having read the PEP on the plane back home I didn't see anything wrong with it, so it could just be accepted in its current form. Implementation will still have to wait for Python 3.3 because of the moratorium. (Although I wouldn't mind making an exception to get it into 3.2.) - This made me think of how the PEP process should evolve so as to not require my personal approval for every PEP. I think the model for future PEPs should be the one we used for PEP 3148 (futures, which was just approved by Jesse): the discussion is led and moderated by one designated "PEP handler" (a different one for each PEP) and the PEP handler, after reviewing the discussion, decides when the PEP is approved. A PEP handler should be selected for each PEP as soon as possible; without a PEP handler, discussing a PEP is not all that useful. The PEP handler should be someone respected by the community with an interest in the subject of the PEP but at an arms' length (at least) from the PEP author. The PEP handler will have to moderate feedback, separating useful comments from (too much) bikeshedding, repetitious lines of questioning, and other forms of obstruction. The PEP handler should also set and try to maintain a schedule for the discussion. Note that a schedule should not be used to break a tie -- it should be used to stop bikeshedding and repeat discussions, while giving all interested parties a chance to comment. (I should say that this is probably similar to the role of an IETF working group director with respect to RFCs.) - Specifically, if Raymond is interested, I wouldn't mind seeing him as the PEP handler for PEP 380. For some of Martin von L?wis's PEPs (382, 384) I think a PEP handler is sorely lacking -- from the language summit it appeared as if nobody besides Martin understands these PEPs. - A lot of things seem to be happening to make PyPI better. Is this being summarized somewhere? Based on some questions I received during my keynote Q&A (http://bit.ly/bdflqa) I think not enough people are aware of what we are already doing in this area. Frankly, I'm not sure I do, either: I think I've heard of a GSOC student and of plans to take over pypi.appspot.com (with the original developer's permission) to become a full and up-to-date mirror. Mirroring apparently also requires some client changes. Oh, and there's a proposed solution for the "register user" problem where apparently the clients had been broken by a unilateral change to the server to require a certain "yes I agree" checkbox. For a hopefully eventually exhaustive overview of what was accomplished at EuroPython, go to http://wiki.europython.eu/After -- and if you know some blog about EuroPython not yet listed, please add it there. -- --Guido van Rossum (python.org/~guido) From alexander.belopolsky at gmail.com Sat Jul 24 17:16:56 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sat, 24 Jul 2010 11:16:56 -0400 Subject: [Python-Dev] [isssue 2001] Pydoc enhancement patch questions In-Reply-To: References: Message-ID: On Sat, Jul 24, 2010 at 6:37 AM, Nick Coghlan wrote: .. > For the "-b" option, if the server is already running (and hence the > port is in use), catch the exception, print a message and start the > webbrowser anyway. I was going to make a similar suggestion, but then realized that there it may not be easy or desirable for pydoc to figure out whether the service running on the used port is in fact pydoc. Any query that pydoc would send may be disruptive depending on what program is listening on the port. It may also get easily confused by a pydoc service from a different version of python. It may be better to search for an unused port in the error case and pass it to the browser. From alexander.belopolsky at gmail.com Sat Jul 24 18:20:03 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sat, 24 Jul 2010 12:20:03 -0400 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: On Thu, Jul 22, 2010 at 6:33 PM, Nick Coghlan wrote: > On Fri, Jul 23, 2010 at 3:30 AM, Alexander Belopolsky > wrote: >> I see three solutions: >> >> 1. ?Minimal: ?do not rename test_trace in 2.7 and add trace module >> tests to the existing file. ?Whether to revert test_trace to >> test_line_tracing renaming in 3.2 can be decided after we have 2.7 >> tests written. >> 2. Rename test_trace in 2.7/3.1 and test_line_tracing in 3.2 to >> test_sys_settrace. >> 3. Rename test_trace in 2.7/3.1 to test_line_tracing. >> >> My preference in #2, but I can live with #1 ?(assuming Eli is the one >> who will write the tests:-). > > I'd say just fix it and go with #2 (just update the comment in > test_sys as well!) I have looked further into setprofile tests for inspiration and discovered that it is covered in test_profilehooks. Note that sys.settrace and sys.setprofile are very similar. They take a function with the same signature and report events in the same format. The only difference is in the kind of events that get reported. I made an experiment replacing [sg]etprofile with [sg]ettrace in test-profilehooks and all tests worked reporting failures in comparing outputs. I would like to use this opportunity to merge settrace and setprofile tests. Here is the proposal: Step 1: Rename test_trace to test_sys_settrace and test_profilehooks to test_sys_setprofile. Step 2: Create tracetester helper file with abstract test cases made from the union of test_sys_settrace and test_sys_setprofile test cases and replace the concrete test cases in test_sys_set* with subclasses that define setmethod, getmethod and expected_output. An alternative proposal is to put all tests for both trace and profile into a single test_sys_tracing file. What do you think? (Another option is to leave test_profilehooks and use rename test_trace to test_tracehooks in the step 1 above.) From guido at python.org Sat Jul 24 19:30:52 2010 From: guido at python.org (Guido van Rossum) Date: Sat, 24 Jul 2010 10:30:52 -0700 Subject: [Python-Dev] Python profiler and other tools In-Reply-To: References: Message-ID: On Fri, Jul 23, 2010 at 4:20 PM, Alexander Belopolsky wrote: > There must be a good reason why traditional software development tools > such as debugger, profiler and coverage are mostly neglected in > python. Most such tools are probably better developed outside the standard library. There are many reasons for that, but I would guess that it mostly has to do with very different development cycles for tools. Tool development usually goes in quick bursts (driven by urgent needs and perhaps improving tool developer skills) unrelated to the stdlib release cycle (and releasing the stdlib separately won't make much of a difference). Also tools often have rough edges, and the stdlib (except for the stuff added before 1995 or so :-) has a high perfectionism standard. -- --Guido van Rossum (python.org/~guido) From rrr at ronadam.com Sat Jul 24 21:33:09 2010 From: rrr at ronadam.com (Ron Adam) Date: Sat, 24 Jul 2010 14:33:09 -0500 Subject: [Python-Dev] [isssue 2001] Pydoc enhancement patch questions In-Reply-To: References: Message-ID: On 07/24/2010 10:16 AM, Alexander Belopolsky wrote: > On Sat, Jul 24, 2010 at 6:37 AM, Nick Coghlan wrote: > .. >> For the "-b" option, if the server is already running (and hence the >> port is in use), catch the exception, print a message and start the >> webbrowser anyway. > > I was going to make a similar suggestion, but then realized that there > it may not be easy or desirable for pydoc to figure out whether the > service running on the used port is in fact pydoc. Any query that > pydoc would send may be disruptive depending on what program is > listening on the port. It may also get easily confused by a pydoc > service from a different version of python. It may be better to > search for an unused port in the error case and pass it to the > browser. I'll try to look into improving how pydoc handles these types of errors. In the mean time if others have experience with browser apps and these kind of situations I'd like to hear about it. Does this have to be in this particular patch? I don't see any problem adding better error recovery later. This isn't something new, both the "-p" and "-g" modes have this issue. Ron From rrr at ronadam.com Sat Jul 24 21:34:38 2010 From: rrr at ronadam.com (Ron Adam) Date: Sat, 24 Jul 2010 14:34:38 -0500 Subject: [Python-Dev] [isssue 2001] Pydoc enhancement patch questions In-Reply-To: References: Message-ID: <4C4B404E.2090007@ronadam.com> On 07/24/2010 05:37 AM, Nick Coghlan wrote: > On Sat, Jul 24, 2010 at 10:05 AM, Ron Adam wrote: >>> I am not sure I like the fact that the browser is started automatically. >>> Please bring this up on python-dev. This may be an opportunity to >>> rethink pydoc command line switches. For example, -p and -g are >>> currently exclusive, but it would make sense for -g to start server on >>> the port specified by -p. >> >> So are any thoughts on starting the web browser automatically, and on how >> the -g and -p command line switches work? > > My suggestion: > > - leave the "-g" option alone (including the tk gui), but make sure > other options still work when tk is unavailable I was hoping it would be ok to drop the tk gui in pydoc. Keeping it requires rewriting the tk gui interface to use the new server because the server code and the gui code are not cleanly separate. I can do this if it's really wanted. (Nothing against tKinter, I use it for my own gui apps.) Or are you suggesting having pydoc work either with the tk gui behavior without any of the new features, or with the new features without the tk gui, depending on how it's started? I'd prefer not to do this because it would duplicate the server code and possibly other functions to produce some of the web page outputs. That would make pydoc.py both larger and harder to maintain. It may also make enhancing pydoc further more difficult as well. The current patch without the tk gui definitely makes things easier to maintain IMHO. Are there any compelling reasons for keeping the tk gui? BTW, the synopsis search feature is currently broken in python 3.2. See issue: http://bugs.python.org/issue9319 Once that is fixed, you can then play around with the search features with and without this patch and see how they compare. > - add a "-b" option to start the server and open the webbrowser automatically > - allow "-p" to be combined with either "-b" or "-g" to specify where > the server should run (or is running) I also agree the "-p" option should work with the "-b" and/or "-g". Using "-b" instead of reusing "-g" for browser only, makes sense to me. Depending on weather or not the tk gui is kept, the "-g" option can either open the tk gui or give a message to use the "-b" option instead. Ron From tjreedy at udel.edu Sat Jul 24 23:05:58 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 24 Jul 2010 17:05:58 -0400 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On 7/24/2010 10:08 AM, Guido van Rossum wrote: > - Commit privileges: Maybe we've been too careful with only giving > commit privileges to to experienced and trusted new developers. I > spoke to Ezio Melotti and from his experience with getting commit > privileges, it seems to be a case of "the lion is much more afraid of > you than you are afraid of the lion". I.e. having got privileges he > was very concerned about doing something wrong, worried about the > complexity of SVN, and so on. Since we've got lots of people watching > the commit stream, I think that there really shouldn't need to be a > worry at all about a new committer doing something malicious, and > there shouldn't be much worry about honest beginners' mistakes either > -- the main worry remains that new committers don't use their > privileges enough. My initial inclination is to start with 1 or 2 line patches that I am 99.99% certain are correct. But it has occurred to me that it might be better for Python if I were willing to take a greater than 1/10000 chance of making a mistake. But how much greater? What error rate do *you* consider acceptable? > - Concurrency and parallelism: Russel Winder and Sarah Mount pushed > the idea of CSP > (http://en.wikipedia.org/wiki/Communicating_sequential_processes) in > several talks at the conference. They (at least Russell) emphasized > the difference between concurrency (interleaved event streams) This speeds perceived and maybe actual responsiveness to user input. > and parallelism (using many processors to speed things up). This reduces total time. > Their > prediction is that as machines with many processing cores become more > prevalent, the relevant architecture will change from cores sharing a > single coherent memory (the model on which threads are based) to one > where each core has a limited amount of private memory, and > communication is done via message passing between the cores. I believe this is a prediction that current prototypes, if not current products, will be both technically and commercially successful. My impression is enough better than 50/50 to be worth taking into account. It does not seem like much of a leap from private caches that write through to common memory to private memory that is not written through, especially on 64 bit machines with memory space to spare. > - After seeing Raymond's talk about monocle (search for it on PyPI) I > am getting excited again about PEP 380 (yield from, return values from > generators). Having read the PEP on the plane back home I didn't see > anything wrong with it, so it could just be accepted in its current > form. Implementation will still have to wait for Python 3.3 because of > the moratorium. (Although I wouldn't mind making an exception to get > it into 3.2.) While initially -0, I now think the moratorium was a good idea. It seems to be successful at letting and even encouraging people to target 3.2 by working with 3.1. A big exception like this would probably annoy lots of people who had *their* 'equally good' ideas put off and might annoy alternative implementors counting on core 3.2 being as it. So the only exception I would make would one that had a really good technical reason, like making Python work better on multi-core processors > - This made me think of how the PEP process should evolve so as to not > require my personal approval for every PEP. I think the model for > future PEPs should be the one we used for PEP 3148 (futures, which was > just approved by Jesse): +1 -- Terry Jan Reedy From alexander.belopolsky at gmail.com Sat Jul 24 23:29:22 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sat, 24 Jul 2010 17:29:22 -0400 Subject: [Python-Dev] [isssue 2001] Pydoc enhancement patch questions In-Reply-To: <4C4B404E.2090007@ronadam.com> References: <4C4B404E.2090007@ronadam.com> Message-ID: On Sat, Jul 24, 2010 at 3:34 PM, Ron Adam wrote: > > > On 07/24/2010 05:37 AM, Nick Coghlan wrote: .. >> - leave the "-g" option alone (including the tk gui), but make sure >> other options still work when tk is unavailable > > I was hoping it would be ok to drop the tk gui in pydoc. ?Keeping it > requires rewriting the tk gui interface to use the new server because the > server code and the gui code are not cleanly separate. ?I can do this if > it's really wanted. (Nothing against tKinter, I use it for my own gui apps.) > FWIW, I am +1 on dropping tkinter interface. Tkinter window looks foreign next to browser and server-side GUI that opens a new client window with each search topic does not strike me as most usable design. Furthermore, I just tried to use it on my OSX laptop and it crashed after I searched for pydoc and clicked on the first entry. (Another issue is that search window pops under the terminal window.) I think Tkinter interface to pydoc may make sense in IDLE, but not in the main pydoc GUI. If the equivalent functionality is available in the browser (preferably in the style familiar to docs.python.org users, I don't see why we need to keep old GUI and hide new behind a new option. From schmir at gmail.com Sat Jul 24 23:59:14 2010 From: schmir at gmail.com (schmir at gmail.com) Date: Sat, 24 Jul 2010 23:59:14 +0200 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <20100724002252.19d91267@snowdog> (Barry Warsaw's message of "Sat, 24 Jul 2010 00:22:52 +0200") References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> <63B36042-C4F7-4BCA-8623-5182405D8852@mac.com> <20100723120252.2e0690cc@snowdog> <25347613-8488-40C3-B9EF-9C85D0C47340@mac.com> <20100723125440.721860bf@snowdog> <764F1595-5C8B-44D8-86C5-937B326B2FCD@mac.com> <87mxti8lqc.fsf@muni.brainbot.com> <20100724002252.19d91267@snowdog> Message-ID: <87aapgbky5.fsf@brainbot.com> Barry Warsaw writes: > On Jul 23, 2010, at 01:46 PM, schmir at gmail.com wrote: > >>Doesn't anybody else think this is lost work for very little gain? My >>/usr/lib/python2.6/site-packages directory consumes 200MB on disk. I >>couldn't care less if my /usr/lib/python2.5/site-packages consumed the >>same amount of disk space... > > Right, you probably don't care now that your extension modules live in foo.so > so it probably won't make much difference if they were named foo-blahblah.so, > as long as they import. :) Most of the time it won't make much difference, right. But I can assure you, that it will bite some people and there is some code to be adapted. > > If you use Debian or Ubuntu though, it'll be a win for you by allow us to make > Python support much more robust. I'd much prefer to have cleanly separated environments by having separate directories for my python modules. Sharing the source code and complicating things will not lead to increased robustness. - Ralf From rrr at ronadam.com Sun Jul 25 00:19:36 2010 From: rrr at ronadam.com (Ron Adam) Date: Sat, 24 Jul 2010 17:19:36 -0500 Subject: [Python-Dev] [isssue 2001] Pydoc enhancement patch questions In-Reply-To: References: <4C4B404E.2090007@ronadam.com> Message-ID: On 07/24/2010 04:29 PM, Alexander Belopolsky wrote: > On Sat, Jul 24, 2010 at 3:34 PM, Ron Adam wrote: >> >> >> On 07/24/2010 05:37 AM, Nick Coghlan wrote: > .. >>> - leave the "-g" option alone (including the tk gui), but make sure >>> other options still work when tk is unavailable >> >> I was hoping it would be ok to drop the tk gui in pydoc. Keeping it >> requires rewriting the tk gui interface to use the new server because the >> server code and the gui code are not cleanly separate. I can do this if >> it's really wanted. (Nothing against tKinter, I use it for my own gui apps.) >> > FWIW, I am +1 on dropping tkinter interface. Tkinter window looks > foreign next to browser and server-side GUI that opens a new client > window with each search topic does not strike me as most usable > design. Furthermore, I just tried to use it on my OSX laptop and it > crashed after I searched for pydoc and clicked on the first entry. > (Another issue is that search window pops under the terminal window.) > I think Tkinter interface to pydoc may make sense in IDLE, but not in > the main pydoc GUI. If the equivalent functionality is available in > the browser (preferably in the style familiar to docs.python.org > users, I don't see why we need to keep old GUI and hide new behind a > new option. The information returned by the new find field in the browser navigation bar is the same as that returned to the tk gui window. Each item is a link followed by the synopsis. The style is similar to the other pydoc pages with the navigation bar at the top that makes it easy to do other searches or return to an index page. There should be a link on each pydoc module page to the online docs. I have to look at how pydoc decides to include that or not. The patched version does not include it. I don't think I did anything to remove that, but will check and add it back if I did. Both python 2.6 and 3.1 versions of pydoc currently adds a "Module Docs" link to module pages to the 2.7 online docs. (A bug?, I'm using Ubuntu) Having the search results in the browser instead of the tk gui allows you to print the results and you can also right click on the links and choose open in a new tab or window. (Firefox Browser) Ron From guido at python.org Sun Jul 25 01:05:19 2010 From: guido at python.org (Guido van Rossum) Date: Sat, 24 Jul 2010 16:05:19 -0700 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On Sat, Jul 24, 2010 at 2:05 PM, Terry Reedy wrote: > On 7/24/2010 10:08 AM, Guido van Rossum wrote: > >> - Commit privileges: Maybe we've been too careful with only giving >> commit privileges to to experienced and trusted new developers. I >> spoke to Ezio Melotti and from his experience with getting commit >> privileges, it seems to be a case of "the lion is much more afraid of >> you than you are afraid of the lion". I.e. having got privileges he >> was very concerned about doing something wrong, worried about the >> complexity of SVN, and so on. Since we've got lots of people watching >> the commit stream, I think that there really shouldn't need to be a >> worry at all about a new committer doing something malicious, and >> there shouldn't be much worry about honest beginners' mistakes either >> -- the main worry remains that new committers don't use their >> privileges enough. > > My initial inclination is to start with 1 or 2 line patches that I am 99.99% > certain are correct. But it has occurred to me that it might be better for > Python if I were willing to take a greater than 1/10000 chance of making a > mistake. But how much greater? What error rate do *you* consider acceptable? Mistakes get made all the time, mostly by experienced committers. When caught quickly they are easy to roll back (that's arguably much of the point of source control :-). New committers can also start with things like docs where there are fewer risks, and more little things that can easily get fixed but aren't for lack of attention. This will help them figure out the source control tools and workflow, which will build up their (and our) confidence, making future success even more likely. -- --Guido van Rossum (python.org/~guido) From breamoreboy at yahoo.co.uk Sun Jul 25 01:23:58 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Sun, 25 Jul 2010 00:23:58 +0100 Subject: [Python-Dev] http://bugs.python.org/issue231540 In-Reply-To: References: Message-ID: On 23/07/2010 23:26, Mark Lawrence wrote: > Is there any money to pay for the forthcoming 10th birthday party for > this issue? Is the OP still alive? > > Kindest regards. > > Mark Lawrence. > Hi all, I admit that when I sent this off last night, I felt rather better after for having let off a bit of steam. Whatever, but in future, could you all please be so kind as to take this with a supertanker's worth of salt, I don't really mean it. To repeat myself for those who are not aware, I have long term physical and mental health problems. Thinking about it, by definition there must be others using Python who are in the same league as myself. I do sympathise. I have to admit that this morning I was rather afraid of reading my email, I thought I was going to get a right slagging, and deservedly so. I have been absolutely staggered by the responses that I've received, some of them offline, but I don't think that matters at all. They're all along the lines of:- "great job Mark, keep your chin up, keep plugging" "Thanks, I'd forgotten that one, I'll get it fixed" And so on, and so forth. Nobody has been at all really negative. I think that this is a tremendous way of saying what a wonderful community Python is, long may it be that way. So at least when I rant, everyone to my knowledge sees it as a positive rant, if you can understand that. Perhaps those who frequent c.l.py will understand the negative rants, which fortunately come from a very small minority of people. How can I sum up what I think about Python, that's fairly easy. www.python.org is bookmarked as Home Sweet Home, can I say any more? Thank you all for your kindness, and please note that I've again been plugging away today and rather enjoying myself. Kindest regards. Mark Lawrence. p.s. I don't know much about baseball, but did have a rather wonderful chat at my nephew's wedding last September in Conneticut. Uncle George (who's claim to fame is allegedly providing the optics for a little bit of kit called the Hubble Space Telescope) spent 20 minutes with myself discussing baseball compared to cricket. He was baffled by the fact that 11 players can have so many fielding positions in cricket, but we got there in the end. More importantly, the aerodynamics of the two balls were of particular interest. How sad can you get? From steve at holdenweb.com Sun Jul 25 02:04:57 2010 From: steve at holdenweb.com (Steve Holden) Date: Sun, 25 Jul 2010 01:04:57 +0100 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: <4C4B7FA9.2010001@holdenweb.com> On 7/24/2010 3:08 PM, Guido van Rossum wrote: > While the EuroPython sprints are still going on, I am back home, and > after a somewhat restful night of sleep, I have some thoughts I'd like > to share before I get distracted. Note, I am jumping wildly between > topics. > > - Commit privileges: Maybe we've been too careful with only giving > commit privileges to to experienced and trusted new developers. I > spoke to Ezio Melotti and from his experience with getting commit > privileges, it seems to be a case of "the lion is much more afraid of > you than you are afraid of the lion". I.e. having got privileges he > was very concerned about doing something wrong, worried about the > complexity of SVN, and so on. Since we've got lots of people watching > the commit stream, I think that there really shouldn't need to be a > worry at all about a new committer doing something malicious, and > there shouldn't be much worry about honest beginners' mistakes either > -- the main worry remains that new committers don't use their > privileges enough. So, my recommendation (which surely is a > turn-around of my *own* attitude in the past) is to give out more > commit privileges sooner. > +1. I think this would have a very positive effect on the way that the Python development community id perceived from the outside. In reality it's probably mostly an acceptance of the fact that new protocols are appropriate with a rather larger development community. > - Concurrency and parallelism: Russel Winder and Sarah Mount pushed > the idea of CSP > (http://en.wikipedia.org/wiki/Communicating_sequential_processes) in > several talks at the conference. They (at least Russell) emphasized > the difference between concurrency (interleaved event streams) and > parallelism (using many processors to speed things up). Their > prediction is that as machines with many processing cores become more > prevalent, the relevant architecture will change from cores sharing a > single coherent memory (the model on which threads are based) to one > where each core has a limited amount of private memory, and > communication is done via message passing between the cores. This > gives them (and me :-) hope that the GIL won't be a problem as long as > we adopt a parallel processing model. Two competing models are the > Actor model, which is based on asynchronous communication, and CSP, > which is synchronous (when a writer writes to a channel, it blocks > until a reader reads that value -- a rendezvous). At least Sarah > suggested that both models are important. She also mentioned that a > merger is under consideration between the two major CSP-for-Python > packages, Py-CSP and Python-CSP. I also believe that the merger will > be based on the stdlib multiprocessing package, but I'm not sure. I do > expect that we may get some suggestions from that corner to make some > minor changes to details of multiprocessing (and perhaps threading), > and I think we should be open to those (I expect these will be good > suggestions for small tweaks, not major overhauls). > > - After seeing Raymond's talk about monocle (search for it on PyPI) I > am getting excited again about PEP 380 (yield from, return values from > generators). Having read the PEP on the plane back home I didn't see > anything wrong with it, so it could just be accepted in its current > form. Implementation will still have to wait for Python 3.3 because of > the moratorium. (Although I wouldn't mind making an exception to get > it into 3.2.) > I can understand the temptation, but hope you can manage to resist it. The downside of allowing such exceptions is that people won't take these pronouncements seriously if they see that a sufficiently desirable goal is a reason for ignoring them. Everyone should be subject to the same rules. > - This made me think of how the PEP process should evolve so as to not > require my personal approval for every PEP. I think the model for > future PEPs should be the one we used for PEP 3148 (futures, which was > just approved by Jesse): the discussion is led and moderated by one > designated "PEP handler" (a different one for each PEP) and the PEP > handler, after reviewing the discussion, decides when the PEP is > approved. A PEP handler should be selected for each PEP as soon as > possible; without a PEP handler, discussing a PEP is not all that > useful. The PEP handler should be someone respected by the community > with an interest in the subject of the PEP but at an arms' length (at > least) from the PEP author. The PEP handler will have to moderate > feedback, separating useful comments from (too much) bikeshedding, > repetitious lines of questioning, and other forms of obstruction. The > PEP handler should also set and try to maintain a schedule for the > discussion. Note that a schedule should not be used to break a tie -- > it should be used to stop bikeshedding and repeat discussions, while > giving all interested parties a chance to comment. (I should say that > this is probably similar to the role of an IETF working group director > with respect to RFCs.) > I think the process where Jesse steered PEP 3148 worked well. > - Specifically, if Raymond is interested, I wouldn't mind seeing him > as the PEP handler for PEP 380. For some of Martin von L?wis's PEPs > (382, 384) I think a PEP handler is sorely lacking -- from the > language summit it appeared as if nobody besides Martin understands > these PEPs. > > - A lot of things seem to be happening to make PyPI better. Is this > being summarized somewhere? Based on some questions I received during > my keynote Q&A (http://bit.ly/bdflqa) I think not enough people are > aware of what we are already doing in this area. Frankly, I'm not sure > I do, either: I think I've heard of a GSOC student and of plans to > take over pypi.appspot.com (with the original developer's permission) > to become a full and up-to-date mirror. Mirroring apparently also > requires some client changes. Oh, and there's a proposed solution for > the "register user" problem where apparently the clients had been > broken by a unilateral change to the server to require a certain "yes > I agree" checkbox. > There is indeed. PyPi has been troublesome in the past mostly because while many people have strong opinions on how it should work, only Martin has put in much time to try and improve it (and has had precious little thanks for that). The site will definitely benefit from wider community involvement, and I think that your suggestions about opening up the developer community will also help relay the message that people are welcome to help on other aspects of Python such as the cheese shop. > For a hopefully eventually exhaustive overview of what was > accomplished at EuroPython, go to http://wiki.europython.eu/After -- > and if you know some blog about EuroPython not yet listed, please add > it there. > Cool idea. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From ben+python at benfinney.id.au Sun Jul 25 02:45:21 2010 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 25 Jul 2010 10:45:21 +1000 Subject: [Python-Dev] http://bugs.python.org/issue231540 References: Message-ID: <871vas5qzi.fsf@benfinney.id.au> Mark Lawrence writes: > I admit that when I sent this off last night, I felt rather better > after for having let off a bit of steam. [?] > Nobody has been at all really negative. I think that this is a > tremendous way of saying what a wonderful community Python is, long > may it be that way. So at least when I rant, everyone to my knowledge > sees it as a positive rant, if you can understand that. [?] I'm glad that this was a positive experience in the end for you, but I do wish it could have been done without the rant actually appearing here. In my experience, there will likely be many people who were affected negatively by the message containing that rant and simply didn't say so, but they will enjoy this community less as a result. What I find very helpful in these cases is to put all my vitriol into the rant, bask in it, and then *delete it unsent*. That way, I'm not spreading the bad emotion any further than my desk, and it truly is cathartic so I can sleep soundly. A win all around. > How can I sum up what I think about Python, that's fairly easy. > www.python.org is bookmarked as Home Sweet Home, can I say any more? > Thank you all for your kindness, and please note that I've again been > plugging away today and rather enjoying myself. Thanks again for your achievements in improving Python, and best wishes in dealing with whatever life deals you. -- \ ?Here is a test to see if your mission on earth is finished. If | `\ you are alive, it isn't.? ?Francis Bacon | _o__) | Ben Finney From pje at telecommunity.com Sun Jul 25 03:51:37 2010 From: pje at telecommunity.com (P.J. Eby) Date: Sat, 24 Jul 2010 21:51:37 -0400 Subject: [Python-Dev] PEP 380 - return value question and prototype implementation (was Thoughts fresh after EuroPython) In-Reply-To: References: Message-ID: <20100725015135.4AAB83A409B@sparrow.telecommunity.com> At 07:08 AM 7/24/2010 -0700, Guido van Rossum wrote: >- After seeing Raymond's talk about monocle (search for it on PyPI) I >am getting excited again about PEP 380 (yield from, return values from >generators). Having read the PEP on the plane back home I didn't see >anything wrong with it, so it could just be accepted in its current >form. I would like to reiterate (no pun intended) the suggestion of a special syntactic form for the return, such as "yield return x", or "return with x" or something similar, to distinguish it from a normal generator return. I think that when people are getting used to the idea of generators, it's important for them to get the idea that the function's "return value" isn't really a value, it's an iterator object. Allowing a return value, but then having that value silently disappear, seems like it would delay that learning, so, a special form might help to make it clear that the generator in question is intended for use with a corresponding "yield from", and help avoid confusion on this. (I could of course be wrong, and would defer to anyone who sees a better way to explain/teach around this issue. In any event, I'm +1 on the PEP otherwise.) By the way, the PEP's "optimized" implementation could probably be done just by making generator functions containing yield-from statements return an object of a different type than the standard geniter. Here's a Python implementation sketch, using a helper class and a decorator -- translation to a C version is likely straightforward, as it'll basically be this plus a light sprinkling of syntactic sugar. So, in the pure-Python prototype (without syntax sugaring), usage would look like this: @From.container def some_generator(...): ... yield From(other_generator(...)) # equivalent to 'yield from' ... def other_generator(...): ... raise StopIteration(value) # equivalent to 'return value' We mark some_generator() with @From.container to indicate that it uses 'yield from' internally (which would happen automatically in the C/syntax sugar version). We don't mark other_generator(), though, because it doesn't contain a 'yield from'. Now, the implementation code (a slightly altered/watered-down version of a trampoline I've used before in 2.x, hopefully altered correctly for Python 3.x syntax/semantics): class From: @classmethod def container(cls, func): def decorated(*args, **kw): return cls(func(*args, **kw)) # wrap generator in a From() instance return decorated def __new__(cls, geniter): if isinstance(geniter, cls): # It's already a 'From' instance, just return it return geniter self = object.__new__(cls, geniter) self.stack = [geniter] return self def __iter__(self): return self def __next__(self): return self._step() def send(self, value): return self._step(value) def throw(self, *exc_info): return self._step(None, exc_info) def _step(self, value=None, exc_info=()): if not self.stack: raise RuntimeError("Can't resume completed generator") try: while self.stack: try: it = self.stack[-1] if exc_info: try: rv = it.throw(*exc_info) finally: exc_info = () elif value is not None: rv = it.send(value) else: rv = it.next() except: value = None exc_info = sys.exc_info() if exc_info[0] is StopIteration: # pass return value up the stack value, = exc_info[1].args or (None,) exc_info = () # but not the error self.stack.pop() else: if isinstance(rv, From): stack.extend(rv.stack) # Call subgenerator value, exc_info, rv = None, (), None else: return rv # it's a value to yield/return else: # Stack's empty, so exit w/current return value or error if exc_info: raise exc_info[1] else: return value finally: exc_info = () # don't let this create garbage def close(self): if self.stack: try: # There's probably a cleaner way to do this in Py 3, I just # don't know what it is off the top of my head... raise GeneratorExit except GeneratorExit as e: try: self.throw(*sys.exc_info()) except (StopIteration, GeneratorExit): pass else: raise RuntimeError("Generator(s) failed to close()") # XXX probably needs some more code here to clean up stack def __del__(self): try: self.close() except: pass As you can (hopefully) see, the main code path simply ends up delegating next/send/close etc. directly to the iterator on the top of the stack, so there isn't any multi-layer passthru going on, as in the "non-optimized" version of the spec in the PEP. From ncoghlan at gmail.com Sun Jul 25 05:18:24 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 25 Jul 2010 13:18:24 +1000 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: On Sun, Jul 25, 2010 at 2:20 AM, Alexander Belopolsky wrote: > I would like to use this opportunity to merge settrace and setprofile > tests. ?Here is the proposal: > > Step 1: Rename test_trace to test_sys_settrace and test_profilehooks > to test_sys_setprofile. > Step 2: Create tracetester helper file with abstract test cases made > from the union of test_sys_settrace and ?test_sys_setprofile test > cases and replace the concrete test cases in test_sys_set* with > subclasses that define setmethod, getmethod and expected_output. > > An alternative proposal is to put all tests for both trace and profile > into a single test_sys_tracing file. > > What do you think? > > (Another option is to leave test_profilehooks and use rename > test_trace to test_tracehooks in the step 1 above.) A tracetester helper module + the two test module renames sounds like a fine idea to me (your proposed new names should be clearer than the existing *hooks naming, given the existence of "profile" and "trace" modules). You may want to do a grep for references to the existing test files in comments in the rest of the test suite though. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From guido at python.org Sun Jul 25 05:21:03 2010 From: guido at python.org (Guido van Rossum) Date: Sat, 24 Jul 2010 20:21:03 -0700 Subject: [Python-Dev] PEP 380 - return value question and prototype implementation (was Thoughts fresh after EuroPython) In-Reply-To: <20100725015135.4AAB83A409B@sparrow.telecommunity.com> References: <20100725015135.4AAB83A409B@sparrow.telecommunity.com> Message-ID: On Sat, Jul 24, 2010 at 6:51 PM, P.J. Eby wrote: > At 07:08 AM 7/24/2010 -0700, Guido van Rossum wrote: >> >> - After seeing Raymond's talk about monocle (search for it on PyPI) I >> am getting excited again about PEP 380 (yield from, return values from >> generators). Having read the PEP on the plane back home I didn't see >> anything wrong with it, so it could just be accepted in its current >> form. > > I would like to reiterate (no pun intended) the suggestion of a special > syntactic ?form for the return, such as "yield return x", or "return with x" > or something similar, to distinguish it from a normal generator return. > > I think that when people are getting used to the idea of generators, it's > important for them to get the idea that the function's "return value" isn't > really a value, it's an iterator object. ?Allowing a return value, but then > having that value silently disappear, seems like it would delay that > learning, so, a special form might help to make it clear that the generator > in question is intended for use with a corresponding "yield from", and help > avoid confusion on this. > > (I could of course be wrong, and would defer to anyone who sees a better way > to explain/teach around this issue. ?In any event, I'm +1 on the PEP > otherwise.) Hm... I somehow really like the idea that the return statement always translates into raising StopIteration -- it seems so "right" when writing the trampoline code. I wonder if we could enforce this at runtime, by raising (e.g.) TypeError if a StopIteration is caught with a value other than None in a place where the value will be ignored. Such a check feels somewhat similar to the check that .send() with a value other than None is not used on a generator in its initial state. Also I think that confusing "yield X" and "return X", while possible, is relatively easy to debug, because things go so horribly wrong the first time. FWIW, the thing that was harder to debug when I tried to write some code involving generators and a trampoline recently, was thinking of a function as a generator without actually putting a yield in it (because a particular version of a coroutine pattern didn't need to block at all). Monocle uses a decorator to flag all coroutines which fixes this up in the right way, which I think is clever, but I'm torn about the need to flag every coroutine with a decorator -- Monocle makes the decorator really short (@_o) because, as Raymond (not Monocle's author but its advocate at EuroPython) said, "you'll be using this hundreds of times". Which I find disturbing in itself. > By the way, the PEP's "optimized" implementation could probably be done just > by making generator functions containing yield-from statements return an > object of a different type than the standard geniter. [...] Not a bad idea. -- --Guido van Rossum (python.org/~guido) From ncoghlan at gmail.com Sun Jul 25 05:29:17 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 25 Jul 2010 13:29:17 +1000 Subject: [Python-Dev] Python profiler and other tools In-Reply-To: References: Message-ID: On Sun, Jul 25, 2010 at 3:30 AM, Guido van Rossum wrote: > On Fri, Jul 23, 2010 at 4:20 PM, Alexander Belopolsky > wrote: >> There must be a good reason why traditional software development tools >> such as debugger, profiler and coverage are mostly neglected in >> python. > > Most such tools are probably better developed outside the standard > library. There are many reasons for that, but I would guess that it > mostly has to do with very different development cycles for tools. > Tool development usually goes in quick bursts (driven by urgent needs > and perhaps improving tool developer skills) unrelated to the stdlib > release cycle (and releasing the stdlib separately won't make much of > a difference). Also tools often have rough edges, and the stdlib > (except for the stuff added before 1995 or so :-) has a high > perfectionism standard. Part of me agrees with you regarding the generally different tool lifecycle, but another part says we need these tools in the standard library or we risk inadvertently breaking the hooks they would still rely on, even as third party projects. I suspect a major factor here relates to the degree of unit test coverage for these components - for areas that aren't of direct interest to me, I'll still deal with it if something I do breaks that code's unit tests, but if the tests don't start failing I'll never even think to check for an incompatibility. (e.g. I seem to recall line tracing for with statements and/or generator expressions being broken for a while before we fixed it) PEP 306 (the one about changing the grammar) and PEP 339 (which includes a short section on changing the bytecode definition) may also be missing some steps to make sure that any syntax and opcode changes are correctly covered by the disassembly, debugging, profiling and tracing tests. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sun Jul 25 05:38:59 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 25 Jul 2010 13:38:59 +1000 Subject: [Python-Dev] [isssue 2001] Pydoc enhancement patch questions In-Reply-To: <4C4B404E.2090007@ronadam.com> References: <4C4B404E.2090007@ronadam.com> Message-ID: On Sun, Jul 25, 2010 at 5:34 AM, Ron Adam wrote: > > > On 07/24/2010 05:37 AM, Nick Coghlan wrote: >> >> On Sat, Jul 24, 2010 at 10:05 AM, Ron Adam ?wrote: >>>> >>>> I am not sure I like the fact that the browser is started automatically. >>>> Please bring this up on python-dev. ?This may be an opportunity to >>>> rethink pydoc command line switches. ?For example, -p and -g are >>>> currently exclusive, but it would make sense for -g to start server on >>>> the port specified by -p. >>> >>> So are any thoughts on starting the web browser automatically, and on how >>> the -g and -p command line switches work? >> >> My suggestion: >> >> - leave the "-g" option alone (including the tk gui), but make sure >> other options still work when tk is unavailable > > I was hoping it would be ok to drop the tk gui in pydoc. ?Keeping it > requires rewriting the tk gui interface to use the new server because the > server code and the gui code are not cleanly separate. ?I can do this if > it's really wanted. (Nothing against tKinter, I use it for my own gui apps.) I'd be completely fine with dropping the "Search For" box from the GUI interface, but the persistent window listing the served port and providing "Open Browser" and "Quit Serving" buttons still seems quite useful even without the search box (When running "python -m pydoc -p 8080 &", it took me a moment to figure out how to kill the server I had started). You could even tidy it up a bit and include things like the Python version in the GUI window. Specifying both -b and -g should open both the (limited) GUI and the web browser. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sun Jul 25 05:44:15 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 25 Jul 2010 13:44:15 +1000 Subject: [Python-Dev] [isssue 2001] Pydoc enhancement patch questions In-Reply-To: References: Message-ID: On Sun, Jul 25, 2010 at 1:16 AM, Alexander Belopolsky wrote: > On Sat, Jul 24, 2010 at 6:37 AM, Nick Coghlan wrote: > .. >> For the "-b" option, if the server is already running (and hence the >> port is in use), catch the exception, print a message and start the >> webbrowser anyway. > > I was going to make a similar suggestion, but then realized that there > it may not be easy or desirable for pydoc to figure out whether the > service running on the used port is in fact pydoc. ?Any query that > pydoc would send may be disruptive depending on what program is > listening on the port. ?It may also get easily confused by a pydoc > service from a different version of python. ? It may be better to > search for an unused port in the error case and pass it to the > browser. I don't think there's any need to check if it is an actual pydoc server running on the port - if it isn't pydoc, the user will be able to tell that by what shows up in the webbrowser window. To request automatic assignment of a local port number, "-p 0" could be made to work correctly (i.e. print out the actual port the OS assigned rather than the 0 that was passed in on the command line as it does currently). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From pje at telecommunity.com Sun Jul 25 05:46:16 2010 From: pje at telecommunity.com (P.J. Eby) Date: Sat, 24 Jul 2010 23:46:16 -0400 Subject: [Python-Dev] PEP 380 - return value question and prototype implementation (was Thoughts fresh after EuroPython) In-Reply-To: References: <20100725015135.4AAB83A409B@sparrow.telecommunity.com> Message-ID: <20100725034610.D97563A409B@sparrow.telecommunity.com> At 08:21 PM 7/24/2010 -0700, Guido van Rossum wrote: >FWIW, the thing that was harder to debug when I tried to write some >code involving generators and a trampoline recently, was thinking of a >function as a generator without actually putting a yield in it >(because a particular version of a coroutine pattern didn't need to >block at all). Monocle uses a decorator to flag all coroutines which >fixes this up in the right way, which I think is clever, but I'm torn >about the need to flag every coroutine with a decorator -- Monocle >makes the decorator really short (@_o) because, as Raymond (not >Monocle's author but its advocate at EuroPython) said, "you'll be >using this hundreds of times". Which I find disturbing in itself. I haven't used Monocle, but in all the libraries I've written myself for this sort of thing (Trellis and peak.events), a decorator is only required for a generator that is a "root" task; everything else is just a normal generator. For example, in Trellis you use @Task.factory to mark a function as spawning an independent task each time it's called, but subgenerator functions called within the task don't need to be marked, and in fact the "yield from" is just a "yield" - the trampoline expects all yields of generators to be subgenerator calls. (PEP 380 can't do this of course, since it also doubles as a sort of 'yield *' - i.e., you may care about the yielded values) Note, though, that even in the sketch I just gave, you don't *really* need to decorate every function, just the ones that need to be called from *non*-decorated functions... i.e. "root" coroutines. Even then, you could *still* skip the decorator and replace: an_iter = decorated_root_function() with: an_iter = From(undecorated_root_function()) and not need to decorate *anything*. From ncoghlan at gmail.com Sun Jul 25 05:51:56 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 25 Jul 2010 13:51:56 +1000 Subject: [Python-Dev] http://bugs.python.org/issue231540 In-Reply-To: <871vas5qzi.fsf@benfinney.id.au> References: <871vas5qzi.fsf@benfinney.id.au> Message-ID: On Sun, Jul 25, 2010 at 10:45 AM, Ben Finney wrote: > What I find very helpful in these cases is to put all my vitriol into > the rant, bask in it, and then *delete it unsent*. That way, I'm not > spreading the bad emotion any further than my desk, and it truly is > cathartic so I can sleep soundly. A win all around. I can vouch for the effectiveness of this trick. I can also point to cases where I didn't use it and dealing with the consequences proved to be a lot more hassle than whatever had been irritating me in the first place :) >> How can I sum up what I think about Python, that's fairly easy. >> www.python.org is bookmarked as Home Sweet Home, can I say any more? >> Thank you all for your kindness, and please note that I've again been >> plugging away today and rather enjoying myself. > > Thanks again for your achievements in improving Python, and best wishes > in dealing with whatever life deals you. Indeed. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From steve at pearwood.info Sun Jul 25 06:16:36 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 25 Jul 2010 14:16:36 +1000 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: <4C4B7FA9.2010001@holdenweb.com> References: <4C4B7FA9.2010001@holdenweb.com> Message-ID: <201007251416.37181.steve@pearwood.info> On Sun, 25 Jul 2010 10:04:57 am Steve Holden wrote: > > - After seeing Raymond's talk about monocle (search for it on PyPI) > > I am getting excited again about PEP 380 (yield from, return values > > from generators). Having read the PEP on the plane back home I > > didn't see anything wrong with it, so it could just be accepted in > > its current form. Implementation will still have to wait for Python > > 3.3 because of the moratorium. (Although I wouldn't mind making an > > exception to get it into 3.2.) > > I can understand the temptation, but hope you can manage to resist > it. > > The downside of allowing such exceptions is that people won't take > these pronouncements seriously if they see that a sufficiently > desirable goal is a reason for ignoring them. Everyone should be > subject to the same rules. I have no opinion on PEP 380 specifically, but surely a *sufficiently* desirable goal *should* be a reason for breaking the rules? Obedience to some abstract rule just because it is the rule is not a virtue. The moratorium is there to advance Python as a whole, and if (a big "if") it becomes a hindrance instead, then Guido should make an exception. I promise that I won't cease taking his pronouncements seriously if he does :) -- Steven D'Aprano From debatem1 at gmail.com Sun Jul 25 06:34:45 2010 From: debatem1 at gmail.com (geremy condra) Date: Sat, 24 Jul 2010 21:34:45 -0700 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: <201007251416.37181.steve@pearwood.info> References: <4C4B7FA9.2010001@holdenweb.com> <201007251416.37181.steve@pearwood.info> Message-ID: On Sat, Jul 24, 2010 at 9:16 PM, Steven D'Aprano wrote: > On Sun, 25 Jul 2010 10:04:57 am Steve Holden wrote: >> > - After seeing Raymond's talk about monocle (search for it on PyPI) >> > I am getting excited again about PEP 380 (yield from, return values >> > from generators). Having read the PEP on the plane back home I >> > didn't see anything wrong with it, so it could just be accepted in >> > its current form. Implementation will still have to wait for Python >> > 3.3 because of the moratorium. (Although I wouldn't mind making an >> > exception to get it into 3.2.) >> >> I can understand the temptation, but hope you can manage to resist >> it. >> >> The downside of allowing such exceptions is that people won't take >> these pronouncements seriously if they see that a sufficiently >> desirable goal is a reason for ignoring them. Everyone should be >> subject to the same rules. > > > I have no opinion on PEP 380 specifically, but surely a *sufficiently* > desirable goal *should* be a reason for breaking the rules? Obedience > to some abstract rule just because it is the rule is not a virtue. The > moratorium is there to advance Python as a whole, and if (a big "if") > it becomes a hindrance instead, then Guido should make an exception. > > I promise that I won't cease taking his pronouncements seriously if he > does :) I wasn't for the moratorium in the first place, so take this with a grain of salt, but ISTM that if you feel this doesn't impact the moratorium's goals then there's nothing logically inconsistent about allowing it through. Of course, if you feel like it does and you decide to let it through anyway, I think it would be worth explaining really well why exactly that happened. Geremy Condra From nagle at animats.com Sun Jul 25 07:08:07 2010 From: nagle at animats.com (John Nagle) Date: Sat, 24 Jul 2010 22:08:07 -0700 Subject: [Python-Dev] Python-Dev Digest, Vol 84, Issue 112 In-Reply-To: References: Message-ID: <4C4BC6B7.7080203@animats.com> > Date: Sat, 24 Jul 2010 23:59:14 +0200 > From:schmir at gmail.com > To: Barry Warsaw > Cc: Ronald Oussoren,python-dev at python.org > Subject: Re: [Python-Dev] versioned .so files for Python 3.2 > Message-ID:<87aapgbky5.fsf at brainbot.com> > Content-Type: text/plain; charset=us-ascii > > Barry Warsaw writes: > >> > On Jul 23, 2010, at 01:46 PM,schmir at gmail.com wrote: >> > >>> >>Doesn't anybody else think this is lost work for very little gain? My >>> >>/usr/lib/python2.6/site-packages directory consumes 200MB on disk. I >>> >>couldn't care less if my /usr/lib/python2.5/site-packages consumed the >>> >>same amount of disk space... >> > >> > Right, you probably don't care now that your extension modules live in foo.so >> > so it probably won't make much difference if they were named foo-blahblah.so, >> > as long as they import.:) > Most of the time it won't make much difference, right. But I can assure > you, that it will bite some people and there is some code to be adapted. > >> > >> > If you use Debian or Ubuntu though, it'll be a win for you by allow us to make >> > Python support much more robust. > I'd much prefer to have cleanly separated environments by having > separate directories for my python modules. Sharing the source code and > complicating things will not lead to increased robustness. > > - Ralf Debian's policy on Python packaging calls for maximum separation between versions. See "http://www.debian.org/doc/packaging-manuals/python-policy/" This keeps the system updaters from becoming confused, and reduces the risk that an update to one version of Python will break another version. John Nagle From ncoghlan at gmail.com Sun Jul 25 08:29:32 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 25 Jul 2010 16:29:32 +1000 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: <201007251416.37181.steve@pearwood.info> References: <4C4B7FA9.2010001@holdenweb.com> <201007251416.37181.steve@pearwood.info> Message-ID: On Sun, Jul 25, 2010 at 2:16 PM, Steven D'Aprano wrote: > On Sun, 25 Jul 2010 10:04:57 am Steve Holden wrote: >> > - After seeing Raymond's talk about monocle (search for it on PyPI) >> > I am getting excited again about PEP 380 (yield from, return values >> > from generators). Having read the PEP on the plane back home I >> > didn't see anything wrong with it, so it could just be accepted in >> > its current form. Implementation will still have to wait for Python >> > 3.3 because of the moratorium. (Although I wouldn't mind making an >> > exception to get it into 3.2.) >> >> I can understand the temptation, but hope you can manage to resist >> it. >> >> The downside of allowing such exceptions is that people won't take >> these pronouncements seriously if they see that a sufficiently >> desirable goal is a reason for ignoring them. Everyone should be >> subject to the same rules. > > > I have no opinion on PEP 380 specifically, but surely a *sufficiently* > desirable goal *should* be a reason for breaking the rules? Obedience > to some abstract rule just because it is the rule is not a virtue. The > moratorium is there to advance Python as a whole, and if (a big "if") > it becomes a hindrance instead, then Guido should make an exception. > > I promise that I won't cease taking his pronouncements seriously if he > does :) We knew PEP 380 would be hurt by the moratorium when the moratorium PEP went through. The goals of the moratorium itself, in making it possible to have a 3.2 release that is fully supported by all of the major Python implementations, still apply, and I believe making an exception for PEP 380 *would* make those goals much harder to achieve. So, while I can understand Guido's temptation (PEP 380 *is* pretty cool), I'm among those that hope he resists that temptation. Letting these various ideas bake a little longer without syntactic support likely won't hurt either. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From stefan_ml at behnel.de Sun Jul 25 08:54:02 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 25 Jul 2010 08:54:02 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: <4C4B7FA9.2010001@holdenweb.com> <201007251416.37181.steve@pearwood.info> Message-ID: Nick Coghlan, 25.07.2010 08:29: > We knew PEP 380 would be hurt by the moratorium when the moratorium > PEP went through. > > The goals of the moratorium itself, in making it possible to have a > 3.2 release that is fully supported by all of the major Python > implementations, still apply, and I believe making an exception for > PEP 380 *would* make those goals much harder to achieve. IMO, it would be worth asking the other implementations if that is the case. It may well be that they are interested in implementing it anyway, so getting it into CPython and the other implementations at the same time may actually be possible. It wouldn't meet the moratorium as such, but it would absolutely comply with its goals. Stefan From g.brandl at gmx.net Sun Jul 25 12:24:29 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 25 Jul 2010 12:24:29 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: <4C4B7FA9.2010001@holdenweb.com> <201007251416.37181.steve@pearwood.info> Message-ID: Am 25.07.2010 08:54, schrieb Stefan Behnel: > Nick Coghlan, 25.07.2010 08:29: >> We knew PEP 380 would be hurt by the moratorium when the moratorium >> PEP went through. >> >> The goals of the moratorium itself, in making it possible to have a >> 3.2 release that is fully supported by all of the major Python >> implementations, still apply, and I believe making an exception for >> PEP 380 *would* make those goals much harder to achieve. > > IMO, it would be worth asking the other implementations if that is the > case. It may well be that they are interested in implementing it anyway, so > getting it into CPython and the other implementations at the same time may > actually be possible. It wouldn't meet the moratorium as such, but it would > absolutely comply with its goals. +1. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From guido at python.org Sun Jul 25 14:43:48 2010 From: guido at python.org (Guido van Rossum) Date: Sun, 25 Jul 2010 05:43:48 -0700 Subject: [Python-Dev] Python profiler and other tools In-Reply-To: References: Message-ID: On Sat, Jul 24, 2010 at 8:29 PM, Nick Coghlan wrote: > On Sun, Jul 25, 2010 at 3:30 AM, Guido van Rossum wrote: >> On Fri, Jul 23, 2010 at 4:20 PM, Alexander Belopolsky >> wrote: >>> There must be a good reason why traditional software development tools >>> such as debugger, profiler and coverage are mostly neglected in >>> python. >> >> Most such tools are probably better developed outside the standard >> library. There are many reasons for that, but I would guess that it >> mostly has to do with very different development cycles for tools. >> Tool development usually goes in quick bursts (driven by urgent needs >> and perhaps improving tool developer skills) unrelated to the stdlib >> release cycle (and releasing the stdlib separately won't make much of >> a difference). Also tools often have rough edges, and the stdlib >> (except for the stuff added before 1995 or so :-) has a high >> perfectionism standard. > > Part of me agrees with you regarding the generally different tool > lifecycle, but another part says we need these tools in the standard > library or we risk inadvertently breaking the hooks they would still > rely on, even as third party projects. The hooks need to be exercised by unittests, for sure. Hooks are APIs with a contract and a promise of stability (though sometimes it's complicated). > I suspect a major factor here relates to the degree of unit test > coverage for these components - for areas that aren't of direct > interest to me, I'll still deal with it if something I do breaks that > code's unit tests, but if the tests don't start failing I'll never > even think to check for an incompatibility. (e.g. I seem to recall > line tracing for with statements and/or generator expressions being > broken for a while before we fixed it) Sure, but I don't see how the presence of a tracing tool in the stdlib would have helped -- in fact, there's pdb which does use the tracing hooks, but that was a new use case and nobody thought of checking it. > PEP 306 (the one about changing the grammar) and PEP 339 (which > includes a short section on changing the bytecode definition) may also > be missing some steps to make sure that any syntax and opcode changes > are correctly covered by the disassembly, debugging, profiling and > tracing tests. That's a good idea regardless. -- --Guido van Rossum (python.org/~guido) From alexander.belopolsky at gmail.com Sun Jul 25 17:50:04 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 25 Jul 2010 11:50:04 -0400 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: On Sat, Jul 24, 2010 at 11:18 PM, Nick Coghlan wrote: > On Sun, Jul 25, 2010 at 2:20 AM, Alexander Belopolsky > wrote: .. >> Step 1: Rename test_trace to test_sys_settrace and test_profilehooks >> to test_sys_setprofile. .. > A tracetester helper module + the two test module renames sounds like > a fine idea to me (your proposed new names should be clearer than the > existing *hooks naming, given the existence of "profile" and "trace" > modules). I have implemented the step 1 above in 3.2 and 2.7 branches, but svnmerge failed in 3.1 and I suspect that this was because r79102 did not make it to release31-maint. It is my understanding that test improvements are normally backported to maintenance branches and r79102 is a test-only revision. Unfortunately, I was not able to figure out how to convince svnmerge to merge r79102 in 3.1. Can someone help? From pje at telecommunity.com Sun Jul 25 18:20:13 2010 From: pje at telecommunity.com (P.J. Eby) Date: Sun, 25 Jul 2010 12:20:13 -0400 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: <4C4B7FA9.2010001@holdenweb.com> <201007251416.37181.steve@pearwood.info> Message-ID: <20100725162008.0B3C03A4093@sparrow.telecommunity.com> At 04:29 PM 7/25/2010 +1000, Nick Coghlan wrote: >So, while I can understand Guido's temptation (PEP 380 *is* pretty >cool), I'm among those that hope he resists that temptation. Letting >these various ideas bake a little longer without syntactic support >likely won't hurt either. Well, if somebody wants to clean up my syntax-sugar-free version a little (maybe adding a "From.return_(value)" staticmethod that raises StopIteration(value)) and throw it in the stdlib, then people can certainly experiment with the feature in 3.2, and get an opportunity to iron out any implementation issues before going to the C-and-sugared version later. From rrr at ronadam.com Sun Jul 25 18:32:16 2010 From: rrr at ronadam.com (Ron Adam) Date: Sun, 25 Jul 2010 11:32:16 -0500 Subject: [Python-Dev] [isssue 2001] Pydoc enhancement patch questions In-Reply-To: References: <4C4B404E.2090007@ronadam.com> Message-ID: On 07/24/2010 10:38 PM, Nick Coghlan wrote: > On Sun, Jul 25, 2010 at 5:34 AM, Ron Adam wrote: >> >> >> On 07/24/2010 05:37 AM, Nick Coghlan wrote: >>> >>> On Sat, Jul 24, 2010 at 10:05 AM, Ron Adam wrote: >>>>> >>>>> I am not sure I like the fact that the browser is started automatically. >>>>> Please bring this up on python-dev. This may be an opportunity to >>>>> rethink pydoc command line switches. For example, -p and -g are >>>>> currently exclusive, but it would make sense for -g to start server on >>>>> the port specified by -p. >>>> >>>> So are any thoughts on starting the web browser automatically, and on how >>>> the -g and -p command line switches work? >>> >>> My suggestion: >>> >>> - leave the "-g" option alone (including the tk gui), but make sure >>> other options still work when tk is unavailable >> >> I was hoping it would be ok to drop the tk gui in pydoc. Keeping it >> requires rewriting the tk gui interface to use the new server because the >> server code and the gui code are not cleanly separate. I can do this if >> it's really wanted. (Nothing against tKinter, I use it for my own gui apps.) > > I'd be completely fine with dropping the "Search For" box from the GUI > interface, but the persistent window listing the served port and > providing "Open Browser" and "Quit Serving" buttons still seems quite > useful even without the search box (When running "python -m pydoc -p > 8080&", it took me a moment to figure out how to kill the server I > had started). You could even tidy it up a bit and include things like > the Python version in the GUI window. Good ideas. > Specifying both -b and -g should open both the (limited) GUI and the > web browser. I see what you mean. The point is to always enable a way to communicate to the server directly. I've fairly often have had to kill the server with the task manager. or on ubuntu, use the system monitor to stop the server. That's definitely not novice friendly. Currently the port is printed on the console when started from the console, but there isn't a clean way to stop the server and no message to say how. The gui is needed for when pydoc is started without a console ie.. by an icon in windows. Another way to communicate to the server would be to add a link in the browser to open a server status page. For example my router has a configure page where I can check it's status and do other things. That might be something worth exploring at some later date. Here are some thoughts... The gui window can have a panal to display the server output. Currently it goes to the console window which may not be visible. This would replace the search results panel. If the gui window is not opened, ie... tk isn't installed or the user prefers the console, have the console window accept commands that correspond to the tk window. Both can have a way to turn on and off verbosity which will display the server activity. Cheers, Ron I'll be away from my computer most of today, I will start making some of the changes we have been discussing later tonight. From alexander.belopolsky at gmail.com Sun Jul 25 19:01:45 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 25 Jul 2010 13:01:45 -0400 Subject: [Python-Dev] [isssue 2001] Pydoc enhancement patch questions In-Reply-To: References: <4C4B404E.2090007@ronadam.com> Message-ID: On Sun, Jul 25, 2010 at 12:32 PM, Ron Adam wrote: .. >> I'd be completely fine with dropping the "Search For" box from the GUI >> interface, but the persistent window listing the served port and >> providing "Open Browser" and "Quit Serving" buttons still seems quite >> useful even without the search box (When running "python -m pydoc -p >> 8080&", it took me a moment to figure out how to kill the server I >> had started). Why not simply have "Quit Serving" next to the search button in the served pages? The server can even serve a friendly page explaining how it can be restarted before quitting. .. > Another way to communicate to the server would be to add a link in the > browser to open a server status page. ?For example my router has a configure > page where I can check it's status and do other things. ?That might be > something worth exploring at some later date. This would work as well, but for starters, I think "Search" and "Quit" buttons next to each other will be most familiar to the users of the current Tk-based control window. From eric at trueblade.com Sun Jul 25 19:13:14 2010 From: eric at trueblade.com (Eric Smith) Date: Sun, 25 Jul 2010 13:13:14 -0400 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: References: <4C470E2B.5060406@timgolden.me.uk> <4C4884CA.9080603@trueblade.com> Message-ID: <4C4C70AA.1000002@trueblade.com> On 7/23/10 2:44 AM, Guido van Rossum wrote: > Indeed, we meant b'...{}...{}...'.format(x, y). The problem is that it > can't invoke x.__format__() or y.__format__() since those will return > text strings instead of bytes. A proposed solution was to try > x.__bformat__() etc. Another proposed solution was to limit x and y to > a small set of common types and hard-code their meanings (e.g. > numbers, bytes, and dates). If there were an __bformat__ method, what would object.__bformat__() return? -- Eric. From martin at v.loewis.de Sun Jul 25 19:22:38 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 25 Jul 2010 19:22:38 +0200 Subject: [Python-Dev] Python profiler and other tools In-Reply-To: References: Message-ID: <4C4C72DE.9080701@v.loewis.de> > Part of me agrees with you regarding the generally different tool > lifecycle, but another part says we need these tools in the standard > library or we risk inadvertently breaking the hooks they would still > rely on, even as third party projects. > > I suspect a major factor here relates to the degree of unit test > coverage for these components - for areas that aren't of direct > interest to me, I'll still deal with it if something I do breaks that > code's unit tests, but if the tests don't start failing I'll never > even think to check for an incompatibility. (e.g. I seem to recall > line tracing for with statements and/or generator expressions being > broken for a while before we fixed it) It is at best entertaining to ponder about reasons here; I doubt anything productive can come out of such a discussion. It may be fun posting a message to a ten-year-old issue saying "ACT NOW (you sluggards)", but I question that this will, on average, improve things. Likewise, speculating that tool support is deliberately ignored probably won't improve things (specifically *not* responding to Nick here). On the other hand, posting actual patches that fix actual bugs can make a lot of a difference. Also, having a maintainer who is willing to look into these patches and accept the good ones will make a lot of a difference. Regards, Martin From martin at v.loewis.de Sun Jul 25 19:29:09 2010 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 25 Jul 2010 19:29:09 +0200 Subject: [Python-Dev] http://bugs.python.org/issue231540 In-Reply-To: References: Message-ID: <4C4C7465.4080700@v.loewis.de> > Oh, and with business philosophy I mean: mails like the one you start > this thread with are interpreted by me as being very pushy, overly > sarcastic and if my project manager at the office sends me an email > like that I know I have to do it right now. I would dislike to be > spoken to like this in an voluntary environment. Do note that I do > understand where your feelings come from. +1. Regards, Martin From alexander.belopolsky at gmail.com Sun Jul 25 19:32:58 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 25 Jul 2010 13:32:58 -0400 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: References: <4C470E2B.5060406@timgolden.me.uk> <4C4884CA.9080603@trueblade.com> Message-ID: On Fri, Jul 23, 2010 at 2:44 AM, Guido van Rossum wrote: .. > Indeed, we meant b'...{}...{}...'.format(x, y). The problem is that it > can't invoke x.__format__() or y.__format__() since those will return > text strings instead of bytes. A proposed solution was to try > x.__bformat__() etc. Another proposed solution was to limit x and y to > a small set of common types and hard-code their meanings (e.g. > numbers, bytes, and dates). Reaching an agreement on what to hard-code for dates may be harder than for the other common types. [1] Implementation of date.__bformat__ will also have it's unique challenges because at the moment date.__format__ is date.strftime and there is no locale independent strftime variant. A simple solution would be to have date.__bformat__ or hard-coded meaning to be just b'YYYY-MM-DD', but when it gets to datetime, I am sure users will start complaining about inability to specify RFC 3339 or other internet-standard formats. [1] http://bugs.python.org/issue8913 From alexander.belopolsky at gmail.com Sun Jul 25 19:41:54 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 25 Jul 2010 13:41:54 -0400 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: <4C4C70AA.1000002@trueblade.com> References: <4C470E2B.5060406@timgolden.me.uk> <4C4884CA.9080603@trueblade.com> <4C4C70AA.1000002@trueblade.com> Message-ID: On Sun, Jul 25, 2010 at 1:13 PM, Eric Smith wrote: > On 7/23/10 2:44 AM, Guido van Rossum wrote: >> >> Indeed, we meant b'...{}...{}...'.format(x, y). The problem is that it >> can't invoke x.__format__() or y.__format__() since those will return >> text strings instead of bytes. A proposed solution was to try >> x.__bformat__() etc. Another proposed solution was to limit x and y to >> a small set of common types and hard-code their meanings (e.g. >> numbers, bytes, and dates). > > If there were an __bformat__ method, what would object.__bformat__() return? Maybe self.__format__(..).encode('ascii')? ...encode('utf-8') is a tempting alternative as well. From bartosz-tarnowski at zlotniki.pl Sun Jul 25 20:55:37 2010 From: bartosz-tarnowski at zlotniki.pl (Bartosz Tarnowski) Date: Sun, 25 Jul 2010 20:55:37 +0200 Subject: [Python-Dev] Set the namespace free! In-Reply-To: <4C490813.30903@canterbury.ac.nz> References: <4C484FD0.2080803@zlotniki.pl> <4C490813.30903@canterbury.ac.nz> Message-ID: <4C4C88A9.8010502@zlotniki.pl> So OK, thank you for response. No, I wasn't joking. I'm sorry, I didn't know that you Python guys get offended from being compared to PHP or Perl. Perhaps that shouldn't surprise me, though. I have posted all of this here, because I was hoping this feature would be implemented secretly, without anyone knowing. Life isn't that simple it seems, I would have to go through all this PEP publication and stuff. Blame yourselves. Reconsidering various ideas you have posted, I think the following solution would be best: All alphanumeric sequences preceded by a dot (".") should not be treated as reserved words. This would allow now reserved names as attributes without any change. Highest level identifiers would need to be simply preceded by a dot. > boo.if = 3 > boo.with = hoo.as > .while = 1 > .for = .def > fun(.class="a") This doesn't require any major change, does not break anything and as a side effect makes some very old code _valid_, at least syntactically. I am going to repost this now on a valid list. Wish me luck, you opportunists. -- Regards, haael From jnoller at gmail.com Sun Jul 25 20:26:02 2010 From: jnoller at gmail.com (Jesse Noller) Date: Sun, 25 Jul 2010 14:26:02 -0400 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On Sat, Jul 24, 2010 at 10:08 AM, Guido van Rossum wrote: > While the EuroPython sprints are still going on, I am back home, and > after a somewhat restful night of sleep, I have some thoughts I'd like > to share before I get distracted. Note, I am jumping wildly between > topics. > > - Commit privileges: Maybe we've been too careful with only giving > commit privileges to to experienced and trusted new developers. I > spoke to Ezio Melotti and from his experience with getting commit > privileges, it seems to be a case of "the lion is much more afraid of > you than you are afraid of the lion". I.e. having got privileges he > was very concerned about doing something wrong, worried about the > complexity of SVN, and so on. Since we've got lots of people watching > the commit stream, I think that there really shouldn't need to be a > worry at all about a new committer doing something malicious, and > there shouldn't be much worry about honest beginners' mistakes either > -- the main worry remains that new committers don't use their > privileges enough. So, my recommendation (which surely is a > turn-around of my *own* attitude in the past) is to give out more > commit privileges sooner. I'd agree with this as well; speaking as someone who was terrified when I first got privileges, I can really back this up. However, I wonder if voluntary code reviews for bigger ticket items would be a good thing to encourage more. I know some people use rietveld - but maybe we should be using it more, plus emails to python-committers asking for reviews. As time has gone on, I've become more and more a firm believer in doing peer reviews. > - Concurrency and parallelism: Russel Winder and Sarah Mount pushed > the idea of CSP > (http://en.wikipedia.org/wiki/Communicating_sequential_processes) in > several talks at the conference. They (at least Russell) emphasized > the difference between concurrency (interleaved event streams) and > parallelism (using many processors to speed things up). Their > prediction is that as machines with many processing cores become more > prevalent, the relevant architecture will change from cores sharing a > single coherent memory (the model on which threads are based) to one > where each core has a limited amount of private memory, and > communication is done via message passing between the cores. This > gives them (and me :-) hope that the GIL won't be a problem as long as > we adopt a parallel processing model. Two competing models are the > Actor model, which is based on asynchronous communication, and CSP, > which is synchronous (when a writer writes to a channel, it blocks > until a reader reads that value -- a rendezvous). At least Sarah > suggested that both models are important. She also mentioned that a > merger is under consideration between the two major CSP-for-Python > packages, Py-CSP and Python-CSP. I also believe that the merger will > be based on the stdlib multiprocessing package, but I'm not sure. I do > expect that we may get some suggestions from that corner to make some > minor changes to details of multiprocessing (and perhaps threading), > and I think we should be open to those (I expect these will be good > suggestions for small tweaks, not major overhauls). I'm open to changes; but remain skeptical the CSP will suddenly take over the world :) There's other, competing philosophies and toolkits out there as well. Additionally; the patches would have to be relicensed - python-csp is under GPLv2 AFAICT. > - After seeing Raymond's talk about monocle (search for it on PyPI) I > am getting excited again about PEP 380 (yield from, return values from > generators). Having read the PEP on the plane back home I didn't see > anything wrong with it, so it could just be accepted in its current > form. Implementation will still have to wait for Python 3.3 because of > the moratorium. (Although I wouldn't mind making an exception to get > it into 3.2.) I, like others, want PEP 380 to be in and done (it's exciting!). However, we knew going into the moratorium that it would negatively affect PEP 380 - as a co-author, it was one of the few things which made me second-guess the implementation of the moratorium. So; in this case I'd have to vote no, we knew going in it would do this. > - This made me think of how the PEP process should evolve so as to not > require my personal approval for every PEP. I think the model for > future PEPs should be the one we used for PEP 3148 (futures, which was > just approved by Jesse): the discussion is led and moderated by one > designated "PEP handler" (a different one for each PEP) and the PEP > handler, after reviewing the discussion, decides when the PEP is > approved. A PEP handler should be selected for each PEP as soon as > possible; without a PEP handler, discussing a PEP is not all that > useful. The PEP handler should be someone respected by the community > with an interest in the subject of the PEP but at an arms' length (at > least) from the PEP author. The PEP handler will have to moderate > feedback, separating useful comments from (too much) bikeshedding, > repetitious lines of questioning, and other forms of obstruction. The > PEP handler should also set and try to maintain a schedule for the > discussion. Note that a schedule should not be used to break a tie -- > it should be used to stop bikeshedding and repeat discussions, while > giving all interested parties a chance to comment. (I should say that > this is probably similar to the role of an IETF working group director > with respect to RFCs.) > This reminds me of discussions I've had in some venues about Python adopting a lieutenant-based system ala the linux kernel. I'm a fan of what you're suggesting, but would take it one step further - the PEP handler should also be a committer, someone with experience working with the current core group so that the PEP handler can eventually help mentor (if needed) the new code (and possibly committer) through the integration process. From alexander.belopolsky at gmail.com Sun Jul 25 20:34:09 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 25 Jul 2010 14:34:09 -0400 Subject: [Python-Dev] Python profiler and other tools In-Reply-To: <4C4C72DE.9080701@v.loewis.de> References: <4C4C72DE.9080701@v.loewis.de> Message-ID: On Sun, Jul 25, 2010 at 1:22 PM, "Martin v. L?wis" wrote: .. > It is at best entertaining to ponder about reasons here; I doubt > anything productive can come out of such a discussion. > I disagree. Depending on the reasons for the relative lack of attention to these components, several alternative actions can be taken to improve the situation. Let's focus on the trace module because it is the simplest and in my opinion the most neglected of the three. Guido suggested that a tool like this is best developed outside of stdlib and python-dev should focus on providing stable interfaces such as sys.settrace to third party projects. On one hand, I would agree that it is better not to distribute the trace module at all than to distribute a broken implementation. (Quality of coverage tools is particularly important now when many developers try to improve up their test coverage before or during transition to 3.x.) On the other hand, using sys.settrace interface requires knowledge of the frame object structure which AFAICT is an implementation detail and may change from version to version. I think stdlib should expose a more abstract tracing API such as one implemented by the trace module which would hide frame object details from the user and give direct access to the data extracted from the frames such as file names and line numbers. A higher level module within stdlib can take advantage of implementation details and not be burdened by having to support multiple releases as long as its own user facing API is stable. > It may be fun posting a message to a ten-year-old issue saying > "ACT NOW (you sluggards)", but I question that this will, on average, > improve things. Likewise, speculating that tool support is deliberately > ignored probably won't improve things (specifically *not* responding > to Nick here). > I assume this is directed at Mark Lawrence even though I made the top post under this subject. While I expressed my own share of criticism of Mark's ways, I do believe that what he is doing is extremely valuable. I did not expect such an outcome, but Mark had a lot of success closing the issues that simply slipped under the radar, but were resolved or close to resolution in every other respect. He is apparently putting a lot of his volunteer hours into this work and this is reason enough to overlook occasional rants or poor choice of words. Mark's breadth-first approach to the tracker is unique and does not work for a lot of issues, but at the same time it does work maybe 10-20% of the time and as long as Mark himself does not get discouraged by the low success rate and learns how to communicate more effectively, I think his work will continue benefiting python. > On the other hand, posting actual patches that fix actual bugs can > make a lot of a difference. Also, having a maintainer who is willing > to look into these patches and accept the good ones will make a lot > of a difference. With respect to the trace module, I started both of these things already, but I would really like to hear feedback from people interested in the future of pdb and profile modules as well. From fuzzyman at voidspace.org.uk Sun Jul 25 20:38:59 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 25 Jul 2010 19:38:59 +0100 Subject: [Python-Dev] Python profiler and other tools In-Reply-To: References: <4C4C72DE.9080701@v.loewis.de> Message-ID: <4C4C84C3.9060700@voidspace.org.uk> On 25/07/2010 19:34, Alexander Belopolsky wrote: > [snip...] >> On the other hand, posting actual patches that fix actual bugs can >> make a lot of a difference. Also, having a maintainer who is willing >> to look into these patches and accept the good ones will make a lot >> of a difference. >> > With respect to the trace module, I started both of these things > already, but I would really like to hear feedback from people > interested in the future of pdb and profile modules as well. > Antonio Cuni demoed some *great* improvements to the pdb module at EuroPython. A project called pdb++ http://codespeak.net/svn/user/antocuni/hack/pdb.py Among its improvements it includes the ability to debug through generator frames, which pdb can't. It would be great to see some of these fixes folded back into the standard library version. All the best, Michael Foord > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From raymond.hettinger at gmail.com Sun Jul 25 20:53:48 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Sun, 25 Jul 2010 11:53:48 -0700 Subject: [Python-Dev] Python profiler and other tools In-Reply-To: <4C4C84C3.9060700@voidspace.org.uk> References: <4C4C72DE.9080701@v.loewis.de> <4C4C84C3.9060700@voidspace.org.uk> Message-ID: <05BC511D-55C2-48C9-B300-E94775A316C1@gmail.com> On Jul 25, 2010, at 11:38 AM, Michael Foord wrote: > On 25/07/2010 19:34, Alexander Belopolsky wrote: >> [snip...] >>> On the other hand, posting actual patches that fix actual bugs can >>> make a lot of a difference. Also, having a maintainer who is willing >>> to look into these patches and accept the good ones will make a lot >>> of a difference. >>> >> With respect to the trace module, I started both of these things >> already, but I would really like to hear feedback from people >> interested in the future of pdb and profile modules as well. >> > > Antonio Cuni demoed some *great* improvements to the pdb module at EuroPython. A project called pdb++ > > http://codespeak.net/svn/user/antocuni/hack/pdb.py > > Among its improvements it includes the ability to debug through generator frames, which pdb can't. It would be great to see some of these fixes folded back into the standard library version. > +1 Raymond From leonhard.vogt at gmx.ch Sun Jul 25 20:58:55 2010 From: leonhard.vogt at gmx.ch (Leonhard Vogt) Date: Sun, 25 Jul 2010 20:58:55 +0200 Subject: [Python-Dev] Patch submitted for 7447 Message-ID: <4C4C896F.6070202@gmx.ch> Hi I have made a documentation patch for issue 7447. I cannot change the stage to patch-review - is this intentional? Would be great if someone could comment on the patch. Leonhard From greg at krypto.org Sun Jul 25 21:19:31 2010 From: greg at krypto.org (Gregory P. Smith) Date: Sun, 25 Jul 2010 12:19:31 -0700 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On Sat, Jul 24, 2010 at 7:08 AM, Guido van Rossum wrote: > > > - Commit privileges: Maybe we've been too careful with only giving > commit privileges to to experienced and trusted new developers. I > spoke to Ezio Melotti and from his experience with getting commit > privileges, it seems to be a case of "the lion is much more afraid of > you than you are afraid of the lion". I.e. having got privileges he > was very concerned about doing something wrong, worried about the > complexity of SVN, and so on. Since we've got lots of people watching > the commit stream, I think that there really shouldn't need to be a > worry at all about a new committer doing something malicious, and > there shouldn't be much worry about honest beginners' mistakes either > -- the main worry remains that new committers don't use their > privileges enough. So, my recommendation (which surely is a > turn-around of my *own* attitude in the past) is to give out more > commit privileges sooner. > +1 agreed! > > - Concurrency and parallelism: Russel Winder and Sarah Mount pushed > the idea of CSP > (http://en.wikipedia.org/wiki/Communicating_sequential_processes) in > several talks at the conference. They (at least Russell) emphasized > the difference between concurrency (interleaved event streams) and > parallelism (using many processors to speed things up). Their > prediction is that as machines with many processing cores become more > prevalent, the relevant architecture will change from cores sharing a > single coherent memory (the model on which threads are based) to one > where each core has a limited amount of private memory, and > communication is done via message passing between the cores. This > I do not believe this prediction. The dominant systems being deployed today have 4,6,8,12 cores on a single memory bus with coherent memory. Sure, NUMA is clearly the dominant architecture but any subdivisions will still have multiple cores with access to the same coherent memory. We'll just end up with multiples of that in one system. The future architecture is _not_ the Cell processor. gives them (and me :-) hope that the GIL won't be a problem as long as > we adopt a parallel processing model. Two competing models are the Actor model, which is based on asynchronous communication, and CSP, > which is synchronous (when a writer writes to a channel, it blocks > until a reader reads that value -- a rendezvous). At least Sarah > suggested that both models are important. She also mentioned that a > merger is under consideration between the two major CSP-for-Python > packages, Py-CSP and Python-CSP. I also believe that the merger will > be based on the stdlib multiprocessing package, but I'm not sure. I do > expect that we may get some suggestions from that corner to make some > minor changes to details of multiprocessing (and perhaps threading), > and I think we should be open to those (I expect these will be good > suggestions for small tweaks, not major overhauls). > The async communication model is good regardless of individual architecture because it more readily grows out beyond a single computer to a network of computers when you want to scale an application up. So yes, anything we can do to help that is good. Hoping that the GIL won't be a problem has been a strategy for over a decade. It has failed. It is limiting what people can and will do with Python today. It isn't going to magically un-become a problem. -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Sun Jul 25 23:08:42 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 25 Jul 2010 23:08:42 +0200 Subject: [Python-Dev] [Idle-dev] IDLE contributors and committers In-Reply-To: References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> Message-ID: <4C4CA7DA.8070103@v.loewis.de> >> There's no reason why Tal should be obstructed in his goal of making >> IDLE at least acceptable again. It's fairly obvious thaat there aren't >> any committers who have both the inclination /and/ the time to do this, >> so adding Tal (and other interested parties) as a developer makes a lot >> of sense. > > I would certainly accept this as a first step. Although I now use IDLE > much less than I have in previous years, I would be willing to put in > some time towards fixing the major current issues and integrating the > few polished enhancements. If you want commit access, please send me your ssh key. Regards, Martin From martin at v.loewis.de Sun Jul 25 23:25:36 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 25 Jul 2010 23:25:36 +0200 Subject: [Python-Dev] IDLE contributors and committers In-Reply-To: References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> <4C41991A.9040103@voidspace.org.uk> Message-ID: <4C4CABD0.2080202@v.loewis.de> > Interested, yes. But until either a) I can commit patches, or b) there > is someone who will respond to commit review recommendations with "No, > here is why not" or "Yes, committed", I will work on other issues, such > as doc issues, for which I can get responses. If you are then still interested in getting commit access, please send me your SSH key. Regards, Martin From ncoghlan at gmail.com Sun Jul 25 23:57:47 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 26 Jul 2010 07:57:47 +1000 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: On Mon, Jul 26, 2010 at 1:50 AM, Alexander Belopolsky wrote: > On Sat, Jul 24, 2010 at 11:18 PM, Nick Coghlan wrote: >> On Sun, Jul 25, 2010 at 2:20 AM, Alexander Belopolsky >> wrote: > .. >>> Step 1: Rename test_trace to test_sys_settrace and test_profilehooks >>> to test_sys_setprofile. > .. >> A tracetester helper module + the two test module renames sounds like >> a fine idea to me (your proposed new names should be clearer than the >> existing *hooks naming, given the existence of "profile" and "trace" >> modules). > > I have implemented the step 1 above in 3.2 and 2.7 branches, but > svnmerge failed in 3.1 and I suspect that this was because r79102 did > not make it to release31-maint. ?It is my understanding that test > improvements are normally backported to maintenance branches and > r79102 is a test-only revision. ?Unfortunately, I was not able to > figure out how to convince svnmerge to merge r79102 in 3.1. ?Can > someone help? Looking at r79102, it updates a number of tests to use the new assert* unittest methods that aren't available in Python 3.1. So while the patch itself is test-only, it can't be backported because it depends on new features. While the general preference is to backport tests, it is also acknowledged that that can become overly difficult as the test cases diverge. Up to you if you want to manually fix your patch for 3.1, drop the test_profilehooks changes, or else just not backport at all. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From alexander.belopolsky at gmail.com Mon Jul 26 00:20:04 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 25 Jul 2010 18:20:04 -0400 Subject: [Python-Dev] Does trace modules have a unit test? In-Reply-To: References: Message-ID: On Sun, Jul 25, 2010 at 5:57 PM, Nick Coghlan wrote: .. > While the general preference is to backport tests, it is also > acknowledged that that can become overly difficult as the test cases > diverge. Up to you if you want to manually fix your patch for 3.1, > drop the test_profilehooks changes, or else just not backport at all. Thanks for explanation, Nick. Since my change is for the benefit of tests that are planned to go in 2.7, it makes little sense to skip 3.1. The change introduced by r79102 in the relevant files is trivial (s/== None/is None/). I just added it manually. See rev 83148. From tjreedy at udel.edu Mon Jul 26 00:56:19 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 25 Jul 2010 18:56:19 -0400 Subject: [Python-Dev] Patch submitted for 7447 In-Reply-To: <4C4C896F.6070202@gmx.ch> References: <4C4C896F.6070202@gmx.ch> Message-ID: On 7/25/2010 2:58 PM, Leonhard Vogt wrote: > I have made a documentation patch for issue 7447. > I cannot change the stage to patch-review - is this intentional? > Would be great if someone could comment on the patch. Done x 3 -- Terry Jan Reedy From tjreedy at udel.edu Mon Jul 26 01:11:47 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 25 Jul 2010 19:11:47 -0400 Subject: [Python-Dev] PEP 380 - return value question and prototype implementation (was Thoughts fresh after EuroPython) In-Reply-To: References: <20100725015135.4AAB83A409B@sparrow.telecommunity.com> Message-ID: On 7/24/2010 11:21 PM, Guido van Rossum wrote: > On Sat, Jul 24, 2010 at 6:51 PM, P.J. Eby wrote: >> By the way, the PEP's "optimized" implementation could probably be done just >> by making generator functions containing yield-from statements return an >> object of a different type than the standard geniter. + 1 > Not a bad idea. I have not read the PEP thoroughly enough to understand the new behavior and suggest another name, but I would really like to not have to change current references to 'generator instances' in texts to "generators instances created by generator functions that only contain 'yield' and not 'yield from'". -- Terry Jan Reedy From greg.ewing at canterbury.ac.nz Mon Jul 26 01:31:42 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 26 Jul 2010 11:31:42 +1200 Subject: [Python-Dev] PEP 380 - return value question and prototype implementation (was Thoughts fresh after EuroPython) In-Reply-To: <20100725015135.4AAB83A409B@sparrow.telecommunity.com> References: <20100725015135.4AAB83A409B@sparrow.telecommunity.com> Message-ID: <4C4CC95E.7080002@canterbury.ac.nz> P.J. Eby wrote: > I would like to reiterate (no pun intended) the suggestion of a special > syntactic form for the return > > Allowing a return > value, but then having that value silently disappear, seems like it > would delay ... learning If I remember correctly, all these arguments were made at the time, even by Guido himself, and he eventually came around to the view that treating generator returns specially wasn't worth the trouble. -- Greg From jackdied at gmail.com Mon Jul 26 01:53:18 2010 From: jackdied at gmail.com (Jack Diederich) Date: Sun, 25 Jul 2010 19:53:18 -0400 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On Sun, Jul 25, 2010 at 2:26 PM, Jesse Noller wrote: > On Sat, Jul 24, 2010 at 10:08 AM, Guido van Rossum wrote: >> - After seeing Raymond's talk about monocle (search for it on PyPI) I >> am getting excited again about PEP 380 (yield from, return values from >> generators). Having read the PEP on the plane back home I didn't see >> anything wrong with it, so it could just be accepted in its current >> form. Implementation will still have to wait for Python 3.3 because of >> the moratorium. (Although I wouldn't mind making an exception to get >> it into 3.2.) > > I, like others, want PEP 380 to be in and done (it's exciting!). > However, we knew going into the moratorium that it would negatively > affect PEP 380 - as a co-author, it was one of the few things which > made me second-guess the implementation of the moratorium. So; in this > case I'd have to vote no, we knew going in it would do this. I was/am pro PEP 380 and pro Moratorium. We knew going in to the moratorium that PEP 380 wouldn't be included and talked about it extensively. We should honor that now for the same reasons we talked about then: declaring "no syntax changes" allows for a focus on the stdlib. -Jack From tjreedy at udel.edu Mon Jul 26 02:24:02 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 25 Jul 2010 20:24:02 -0400 Subject: [Python-Dev] View tracker patches with ViewVC? Message-ID: To review a patch on the tracker, I have to read and try to make sense of the raw diff file. Sometimes that is easy, sometimes not. *After* a patch is applied, I can click the revxxxx link and then the 'text changed' link and see a nice, colored, side-by-side web-pace view created by ViewVC. Is there any way a 'view' button could be added, along with the current edit and remove buttons, to produce the same web page and make it easier to review patches *before* commitment? -- Terry Jan Reedy From martin at v.loewis.de Mon Jul 26 02:35:21 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 26 Jul 2010 02:35:21 +0200 Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: References: Message-ID: <4C4CD849.9090108@v.loewis.de> Am 26.07.2010 02:24, schrieb Terry Reedy: > To review a patch on the tracker, I have to read and try to make sense > of the raw diff file. Sometimes that is easy, sometimes not. > > *After* a patch is applied, I can click the revxxxx link and then the > 'text changed' link and see a nice, colored, side-by-side web-pace view > created by ViewVC. Is there any way a 'view' button could be added, > along with the current edit and remove buttons, to produce the same web > page and make it easier to review patches *before* commitment? You should be using Rietveld for that. Regards, Martin From guido at python.org Mon Jul 26 03:46:28 2010 From: guido at python.org (Guido van Rossum) Date: Sun, 25 Jul 2010 18:46:28 -0700 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: References: <4C470E2B.5060406@timgolden.me.uk> <4C4884CA.9080603@trueblade.com> <4C4C70AA.1000002@trueblade.com> Message-ID: On Sun, Jul 25, 2010 at 10:41 AM, Alexander Belopolsky wrote: > On Sun, Jul 25, 2010 at 1:13 PM, Eric Smith wrote: >> On 7/23/10 2:44 AM, Guido van Rossum wrote: >>> >>> Indeed, we meant b'...{}...{}...'.format(x, y). The problem is that it >>> can't invoke x.__format__() or y.__format__() since those will return >>> text strings instead of bytes. A proposed solution was to try >>> x.__bformat__() etc. Another proposed solution was to limit x and y to >>> a small set of common types and hard-code their meanings (e.g. >>> numbers, bytes, and dates). >> >> If there were an __bformat__ method, what would object.__bformat__() return? > > Maybe self.__format__(..).encode('ascii')? ?...encode('utf-8') is a > tempting alternative as well. -1 That would bring back the "it fails for some users but passes for the developer" problem. (True, if the developer calls .encode('ascii') it may also break, but then at least it is something the developer chose to do.) How hard would it be to recode the sprintf language but with the locale fixed to "C"? That would always be ASCII. Otherwise, str(x).encode('ascii') might work, that's like the ISO format with the 'T' replaced by a space. (One issue: it's not fixed-width, since it shows the microseconds iff they are nonzero. If I had to choose I'd never show the microseconds.) -- --Guido van Rossum (python.org/~guido) From guido at python.org Mon Jul 26 03:53:04 2010 From: guido at python.org (Guido van Rossum) Date: Sun, 25 Jul 2010 18:53:04 -0700 Subject: [Python-Dev] Python profiler and other tools In-Reply-To: <05BC511D-55C2-48C9-B300-E94775A316C1@gmail.com> References: <4C4C72DE.9080701@v.loewis.de> <4C4C84C3.9060700@voidspace.org.uk> <05BC511D-55C2-48C9-B300-E94775A316C1@gmail.com> Message-ID: On Sun, Jul 25, 2010 at 11:53 AM, Raymond Hettinger wrote: > On Jul 25, 2010, at 11:38 AM, Michael Foord wrote: >> Antonio Cuni demoed some *great* improvements to the pdb module at EuroPython. A project called pdb++ >> >> http://codespeak.net/svn/user/antocuni/hack/pdb.py >> >> Among its improvements it includes the ability to debug through generator frames, which pdb can't. It would be great to see some of these fixes folded back into the standard library version. >> > > +1 +1 on adding tracing through generators. Not so much about the other pdb++ features that were demoed. While I liked the demo at EuroPython (and applauded), in general I am wary of adding this kind of UI cleverness to the stdlib -- things like the (seemingly curses-based) updating of the listing in-line are usually pretty fragile. Also the development cycle of a tool like this is likely 1-2 orders of magnitude over that of the stdlib. (For this reason I also think it's right that ipython is still a 3rd party tool). -- --Guido van Rossum (python.org/~guido) From alexander.belopolsky at gmail.com Mon Jul 26 03:53:43 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 25 Jul 2010 21:53:43 -0400 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: References: <4C470E2B.5060406@timgolden.me.uk> <4C4884CA.9080603@trueblade.com> <4C4C70AA.1000002@trueblade.com> Message-ID: On Sun, Jul 25, 2010 at 9:46 PM, Guido van Rossum wrote: .. > If I had to choose I'd never show the microseconds. Or the timezone offset, right? From alexander.belopolsky at gmail.com Mon Jul 26 04:20:03 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sun, 25 Jul 2010 22:20:03 -0400 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: References: <4C470E2B.5060406@timgolden.me.uk> <4C4884CA.9080603@trueblade.com> <4C4C70AA.1000002@trueblade.com> Message-ID: On Sun, Jul 25, 2010 at 9:46 PM, Guido van Rossum wrote: .. >> Maybe self.__format__(..).encode('ascii')? ?...encode('utf-8') is a >> tempting alternative as well. > > -1 > > That would bring back the "it fails for some users but passes for the > developer" problem. (True, if the developer calls .encode('ascii') it > may also break, but then at least it is something the developer chose > to do.) > > How hard would it be to recode the sprintf language but with the > locale fixed to "C"? That would always be ASCII. This is exactly what I proposed at http://bugs.python.org/issue7584#msg110240 not so long ago. Given that stftime language uses every English letter as one of its codes (both caps and lower case), it would be an effort, but coding it in python should not be too hard. A C implementation would be harder, but there must be implementations around available under a suitable license that can be reused. In short, definitely +1. > Otherwise, > str(x).encode('ascii') might work, that's like the ISO format with the > 'T' replaced by a space. Before proposing format(x, ..).encode('ascii') above, I considered str(x).encode('ascii') , but then realized that for user-defined classes, str(x) is as likely to contain non-ASCII characters as format(x, ..). What about .encode('utf-8')? I thought it was not supposed to break for any unicode. From guido at python.org Mon Jul 26 04:52:52 2010 From: guido at python.org (Guido van Rossum) Date: Sun, 25 Jul 2010 19:52:52 -0700 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: References: <4C470E2B.5060406@timgolden.me.uk> <4C4884CA.9080603@trueblade.com> <4C4C70AA.1000002@trueblade.com> Message-ID: On Sun, Jul 25, 2010 at 7:20 PM, Alexander Belopolsky wrote: > On Sun, Jul 25, 2010 at 9:46 PM, Guido van Rossum wrote: > .. >>> Maybe self.__format__(..).encode('ascii')? ?...encode('utf-8') is a >>> tempting alternative as well. >> >> -1 >> >> That would bring back the "it fails for some users but passes for the >> developer" problem. (True, if the developer calls .encode('ascii') it >> may also break, but then at least it is something the developer chose >> to do.) >> >> How hard would it be to recode the sprintf language but with the >> locale fixed to "C"? That would always be ASCII. > > This is exactly what I proposed at > http://bugs.python.org/issue7584#msg110240 not so long ago. ?Given > that stftime language uses every English letter as one of its codes > (both caps and lower case), it would be an effort, but coding it in > python should not be too hard. ? A C implementation would be harder, > but there must be implementations around available under a suitable > license that can be reused. > > In short, definitely +1. For b"...".format() we could also support a subset (still better than hardcoding something str()-based). >> Otherwise, >> str(x).encode('ascii') might work, that's like the ISO format with the >> 'T' replaced by a space. > > Before proposing format(x, ..).encode('ascii') above, I considered > str(x).encode('ascii') , but then realized that for user-defined > classes, str(x) is as likely to contain non-ASCII characters as > format(x, ..). > > What about .encode('utf-8')? ?I thought it was not supposed to break > for any unicode. But it still makes an assumption about the output encoding. (So does ASCII, technically, but way more encodings are ASCII supersets, and some bytes methods already assume ASCII, e.g. .lower() and friends.) -- --Guido van Rossum (python.org/~guido) From peter.a.portante at gmail.com Mon Jul 26 05:31:41 2010 From: peter.a.portante at gmail.com (Peter Portante) Date: Sun, 25 Jul 2010 23:31:41 -0400 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: Message-ID: On 7/25/10 3:19 PM, "Gregory P. Smith" wrote: > > On Sat, Jul 24, 2010 at 7:08 AM, Guido van Rossum wrote: >> >> - Commit privileges: Maybe we've been too careful with only giving >> commit privileges to to experienced and trusted new developers. I >> spoke to Ezio Melotti and from his experience with getting commit >> privileges, it seems to be a case of "the lion is much more afraid of >> you than you are afraid of the lion". I.e. having got privileges he >> was very concerned about doing something wrong, worried about the >> complexity of SVN, and so on. Since we've got lots of people watching >> the commit stream, I think that there really shouldn't need to be a >> worry at all about a new committer doing something malicious, and >> there shouldn't be much worry about honest beginners' mistakes either >> -- the main worry remains that new committers don't use their >> privileges enough. So, my recommendation (which surely is a >> turn-around of my *own* attitude in the past) is to give out more >> commit privileges sooner. > > +1 agreed! > ? >> >> - Concurrency and parallelism: Russel Winder and Sarah Mount pushed >> the idea of CSP >> (http://en.wikipedia.org/wiki/Communicating_sequential_processes) in >> several talks at the conference. They (at least Russell) emphasized >> the difference between concurrency (interleaved event streams) and >> parallelism (using many processors to speed things up). Their >> prediction is that as machines with many processing cores become more >> prevalent, the relevant architecture will change from cores sharing a >> single coherent memory (the model on which threads are based) to one >> where each core has a limited amount of private memory, and >> communication is done via message passing between the cores. This > ? > I do not believe this prediction. ?The dominant systems being deployed today > have 4,6,8,12 cores on a single memory bus with coherent memory. ?Sure, NUMA > is clearly the dominant architecture but any subdivisions will still have > multiple cores with access to the same coherent memory. ?We'll just end up > with multiples of that in one system. ?The future architecture is _not_ the > Cell processor. +1 And there is so much legacy software out there that depends on single coherency systems that it is unlikely such a memory architecture would become pervasive, given the hurdle of rewriting those programs, python programs included. >> gives them (and me :-) hope that the GIL won't be a problem as long as >> we adopt a parallel processing model. Two competing models are the >> Actor model, which is based on asynchronous communication, and CSP, >> which is synchronous (when a writer writes to a channel, it blocks >> until a reader reads that value -- a rendezvous). At least Sarah >> suggested that both models are important. She also mentioned that a >> merger is under consideration between the two major CSP-for-Python >> packages, Py-CSP and Python-CSP. I also believe that the merger will >> be based on the stdlib multiprocessing package, but I'm not sure. I do >> expect that we may get some suggestions from that corner to make some >> minor changes to details of multiprocessing (and perhaps threading), >> and I think we should be open to those (I expect these will be good >> suggestions for small tweaks, not major overhauls). > > The async communication model is good regardless of individual architecture > because it more readily grows out beyond a single computer to a network of > computers when you want to scale an application up. ?So yes, anything we can > do to help that is good. +1 > Hoping that the GIL won't be a problem has been a strategy for over a decade. > It has failed. It is limiting what people can and will do with Python today. > It isn't going to magically un-become a problem. +1 FWIW: We use Python at Tabblo, straddled across Python 2.5.4 and 2.6.5. They work. And they work well. But we make light use of threads (mostly background I/O handling), and heavy use of multiple processes because we can't take advantage of our multi-core systems otherwise. -peter > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/peter.a.portante%40gmail.com From guido at python.org Mon Jul 26 05:42:14 2010 From: guido at python.org (Guido van Rossum) Date: Sun, 25 Jul 2010 20:42:14 -0700 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On Sun, Jul 25, 2010 at 8:31 PM, Peter Portante wrote: > FWIW: We use Python at Tabblo, straddled across Python 2.5.4 and 2.6.5. They > work. And they work well. But we make light use of threads (mostly > background I/O handling), and heavy use of multiple processes because we > can't take advantage of our multi-core systems otherwise. Isn't this an indication that the GIL is, in fact, not (much of) a problem? I wish those trying to get rid of the GIL well. But it may not be the panacea some folks are hoping for. Multi-threaded programming remains hard (and removing the GIL might actually make it harder). Jython and IronPython don't have a GIL, and I think PyPy may not either. Does anyone have experience with GIL-free programming in one of those? -- --Guido van Rossum (python.org/~guido) From peter.a.portante at gmail.com Mon Jul 26 08:40:04 2010 From: peter.a.portante at gmail.com (Peter Portante) Date: Mon, 26 Jul 2010 02:40:04 -0400 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: Message-ID: On 7/25/10 11:42 PM, "Guido van Rossum" wrote: > On Sun, Jul 25, 2010 at 8:31 PM, Peter Portante > wrote: >> FWIW: We use Python at Tabblo, straddled across Python 2.5.4 and 2.6.5. They >> work. And they work well. But we make light use of threads (mostly >> background I/O handling), and heavy use of multiple processes because we >> can't take advantage of our multi-core systems otherwise. > > Isn't this an indication that the GIL is, in fact, not (much of) a problem? Meaning, we have a working system, so the GIL is not (much of) a problem? Or that we have successfully spent a lot of time and effort rewriting embarrassingly parallel multithreaded algorithms into a bit more complex message passing multi-process algorithms because we can't get the language implementation to make efficient use of multiple CPUs, thus avoiding the GIL from being (much of) a problem? Perhaps we have to ask what does it mean to say the GIL is a problem? If what we mean is that the existence of the GIL does not cause a CPython based program to fail, then yes, it is not a problem at all. In fact, it is a testament to the level of excellence the code has achieved through the hard work folks have put in over the years. If what we mean is that the existence of the GIL prevents a multithreaded CPython application from taking advantage of multiple CPUs, then yes, it is a "problem". So the above statement says that the GIL is not a problem, and that it is THE problem, depending on your definition. :) > I wish those trying to get rid of the GIL well. But it may not be the > panacea some folks are hoping for. You are right, getting rid of the GIL is not a panacea for anything. Removing the GIL means that there will be other changes to the behavioral landscape of the language implementation which folks will have to learn and understand well to write multi-threaded programs that perform well. Anybody wishing to make a whole system run well must engage in that process of learning and discovery. Yet, shouldn't we be able to write a simple embarrassingly parallel multithreaded algorithm in python (no C-extensions) and have its execution use all the cores on a system using CPython? Python is a beautiful language in which to express algorithms. Having to resort to other languages, C extensions, or other implementations of Python, in order to express those algorithms that rely on execution entities sharing a coherent memory space is a limitation imposed by the existence of the GIL in CPython. Is that limitation worth the effort to remove? Perhaps. Perhaps not. Perhaps Jython, or IronPython, or other implementations of Python that don't have a GIL provide a path forward for folks that need that. Those implementations don't currently provide a path forward for what we are doing, so we avoid the use of threads with CPython. > Multi-threaded programming remains > hard (and removing the GIL might actually make it harder). Could we make a statement that the perceived difficulty of multithreaded programming would only increase if a CPython implementation had undocumented behaviors, or undefined behaviors that should be defined? In other words, the difficulty of multithreaded programming is independent of the existence of a/the GIL, but is dependent on the thorough documentation of all language implementation behaviors. > Jython and IronPython don't have a GIL, and I think PyPy may not > either. FWIW: We have considered switching to Jython because it does not have a GIL. Unfortunately, we'd have to find replacements for some of the C-extension modules we use. Sincerely, -peter > Does anyone have experience with GIL-free programming in one > of those? From pydev at rebertia.com Mon Jul 26 08:42:06 2010 From: pydev at rebertia.com (Chris Rebert) Date: Sun, 25 Jul 2010 23:42:06 -0700 Subject: [Python-Dev] marking os.system() as deprecated in the docs Message-ID: Hello, I'd like to propose formally marking os.system() as deprecated in the docs for next release of Python (v3.2 ?). The docs for os.system() /already/ include the following paragraph, which is basically tantamount to calling system() deprecated and very much resembles the deprecation notes for the os.popen*() family. """ The subprocess module provides more powerful facilities for spawning new processes and retrieving their results; using that module is preferable to using this function. Use the subprocess module. Check especially the Replacing Older Functions with the subprocess Module section. """ I'm merely suggesting slapping a formal ".. deprecated::" tag & box around this paragraph to make system()'s status more obvious. I am ***not*** suggesting the function be scheduled for removal or anything like that. I figure since this would be a substantive change (the docs currently don't outright say "deprecated"), it would need dev approval, hence this message. I've already written the absolutely /trivial/ docs patch. So, any objections? Cheers, Chris From anders.sandvig at gmail.com Mon Jul 26 10:59:52 2010 From: anders.sandvig at gmail.com (Anders Sandvig) Date: Mon, 26 Jul 2010 10:59:52 +0200 Subject: [Python-Dev] Set the namespace free! In-Reply-To: References: <4C484FD0.2080803@zlotniki.pl> Message-ID: On Sat, Jul 24, 2010 at 3:31 AM, Gregory P. Smith wrote: > Yuck. ?Anyone who feels they need a variable named the same a reserved word > simply feels wrong and needs reeducation. ?[...] While I agree with you in principle, I have been finding it frustrating trying to calculate yield in my financial applications lately... ;) That being said, yield is pretty much the only (reserved) word I have had problems with finding (descriptive) alternative (variable) names for, so far. Anders From fuzzyman at voidspace.org.uk Mon Jul 26 12:00:05 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 26 Jul 2010 11:00:05 +0100 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: <4C4D5CA5.7020809@voidspace.org.uk> On 26/07/2010 04:42, Guido van Rossum wrote: > On Sun, Jul 25, 2010 at 8:31 PM, Peter Portante > wrote: > >> FWIW: We use Python at Tabblo, straddled across Python 2.5.4 and 2.6.5. They >> work. And they work well. But we make light use of threads (mostly >> background I/O handling), and heavy use of multiple processes because we >> can't take advantage of our multi-core systems otherwise. >> > Isn't this an indication that the GIL is, in fact, not (much of) a problem? > > I wish those trying to get rid of the GIL well. But it may not be the > panacea some folks are hoping for. Multi-threaded programming remains > hard (and removing the GIL might actually make it harder). > > Jython and IronPython don't have a GIL, and I think PyPy may not > either. Does anyone have experience with GIL-free programming in one > of those? > > At Resolver Systems we created a "calculation system" that does large calculations on background threads using IronPython. Doing them on a background thread allows the ui to remain responsive. Several calculations could run simultaneously using multiple cores. As the calculation operates on a large object graph (which the ui then needs access to in order to display it) using multiprocessing would have imposed a very big overhead due to serialization / deserialization (the program runs on windows). Using CPython would have made the program a lot slower due to the GIL. All the best, Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From fuzzyman at voidspace.org.uk Mon Jul 26 12:02:16 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 26 Jul 2010 11:02:16 +0100 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: <4C4D5D28.90603@voidspace.org.uk> On 26/07/2010 04:42, Guido van Rossum wrote: > On Sun, Jul 25, 2010 at 8:31 PM, Peter Portante > wrote: > >> FWIW: We use Python at Tabblo, straddled across Python 2.5.4 and 2.6.5. They >> work. And they work well. But we make light use of threads (mostly >> background I/O handling), and heavy use of multiple processes because we >> can't take advantage of our multi-core systems otherwise. >> > Isn't this an indication that the GIL is, in fact, not (much of) a problem? > > I wish those trying to get rid of the GIL well. But it may not be the > panacea some folks are hoping for. Multi-threaded programming remains > hard (and removing the GIL might actually make it harder). > > Jython and IronPython don't have a GIL, and I think PyPy may not > either. Does anyone have experience with GIL-free programming in one > of those? > > Oh, and PyPy does have a GIL but the developers say it wouldn't be a huge amount of work to remove it. Presumably they would have to add locking in the right places - which would then impact performance. As PyPy doesn't use reference counting adding locking shouldn't impact performance as much as previous attempts with CPython have. Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From phil at freehackers.org Mon Jul 26 12:16:09 2010 From: phil at freehackers.org (Philippe Fremy) Date: Mon, 26 Jul 2010 12:16:09 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: <4C4D6069.3010909@freehackers.org> Guido van Rossum wrote: > While the EuroPython sprints are still going on, I am back home, and > after a somewhat restful night of sleep, I have some thoughts I'd like > to share before I get distracted. Note, I am jumping wildly between > topics. > > - Commit privileges: Maybe we've been too careful with only giving > commit privileges to to experienced and trusted new developers. I > spoke to Ezio Melotti and from his experience with getting commit > privileges, it seems to be a case of "the lion is much more afraid of > you than you are afraid of the lion". I.e. having got privileges he > was very concerned about doing something wrong, worried about the > complexity of SVN, and so on. Since we've got lots of people watching > the commit stream, I think that there really shouldn't need to be a > worry at all about a new committer doing something malicious, and > there shouldn't be much worry about honest beginners' mistakes either > -- the main worry remains that new committers don't use their > privileges enough. So, my recommendation (which surely is a > turn-around of my *own* attitude in the past) is to give out more > commit privileges sooner. > I would like to highlight that other open source projects have used more liberal commit right policies, without the project breaking into pieces, to the contrary. For example, in KDE, you usually get commit rights on your second patch submisssion. The contributors have reported that it really helped to convert from "occasional contributors" to "active contributors" and were encouraged by the trust given by the project community. They felt a new sense of responsibility toward the project with the ability to contribute directly. There was never any malicious commits done to KDE using this liberal policy. The newcomers tend to be extremely careful. If you keep the newcomers under the umbrella of a mentor for a few months and with the additional security of post-commit reviews, I am sure that you are not taking any real risks on the codebase. cheers, Philippe From fijall at gmail.com Mon Jul 26 12:56:25 2010 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 26 Jul 2010 12:56:25 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: <4C4D5D28.90603@voidspace.org.uk> References: <4C4D5D28.90603@voidspace.org.uk> Message-ID: On Mon, Jul 26, 2010 at 12:02 PM, Michael Foord wrote: > On 26/07/2010 04:42, Guido van Rossum wrote: >> >> On Sun, Jul 25, 2010 at 8:31 PM, Peter Portante >> ?wrote: >> >>> >>> FWIW: We use Python at Tabblo, straddled across Python 2.5.4 and 2.6.5. >>> They >>> work. And they work well. But we make light use of threads (mostly >>> background I/O handling), and heavy use of multiple processes because we >>> can't take advantage of our multi-core systems otherwise. >>> >> >> Isn't this an indication that the GIL is, in fact, not (much of) a >> problem? >> >> I wish those trying to get rid of the GIL well. But it may not be the >> panacea some folks are hoping for. Multi-threaded programming remains >> hard (and removing the GIL might actually make it harder). >> >> Jython and IronPython don't have a GIL, and I think PyPy may not >> either. Does anyone have experience with GIL-free programming in one >> of those? >> >> > > Oh, and PyPy does have a GIL but the developers say it wouldn't be a huge > amount of work to remove it. It wouldn't be as huge as on CPython, since we don't have reference counting, but it's still *a lot* of work and someone would have to step and take this task (since none core pypy dev is that interested in that). > > Presumably they would have to add locking in the right places - which would > then impact performance. As PyPy doesn't use reference counting adding > locking shouldn't impact performance as much as previous attempts with > CPython have. That's one thing but the other thing is that JIT can remove a lot of locks (like it does no JVM), but that's yet another batch of work to be done. Cheers, fijal From ziade.tarek at gmail.com Mon Jul 26 13:02:37 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 26 Jul 2010 13:02:37 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On Sat, Jul 24, 2010 at 4:08 PM, Guido van Rossum wrote: [...] > - A lot of things seem to be happening to make PyPI better. Is this > being summarized somewhere? Based on some questions I received during > my keynote Q&A (http://bit.ly/bdflqa) I think not enough people are > aware of what we are already doing in this area. Even people very involved in packaging are not fully aware of what's going on. I am not for instance. I think that we lack to communicate and synchronize on our efforts on the PyPI development. The last example I have in mind is that I have announced here that I was working on a patch for the checkbox problem, then Martin announced today on catalog-sig it was fixed by Georg and updated in production :) I think we need to improve this: it can be a very frustrating experience to contribute to PyPI. Possible improvements: - Have a PyPI component at bugs.python.org so all work on bugs/new features would be known and followed by at the same level than other packaging components we maintain, and that depend on PyPI (distutils, distutils2) -- e.g. drop the sourceforge tracker - Make it easier to contribute by moving the PyPI code base to hg.python.org. Unlike Python, this is a very simple move. > Frankly, I'm not sure > I do, either: I think I've heard of a GSOC student and of plans to > take over pypi.appspot.com (with the original developer's permission) > to become a full and up-to-date mirror. That would be great if the student could promote his work at Catalog-SIG. > Mirroring apparently also > requires some client changes. Mirrors can be used as long as you manually point a mirror when using them. We we are working on making the switch automatic. Regards, Tarek -- Tarek Ziad? | http://ziade.org From debatem1 at gmail.com Mon Jul 26 13:20:36 2010 From: debatem1 at gmail.com (geremy condra) Date: Mon, 26 Jul 2010 04:20:36 -0700 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On Mon, Jul 26, 2010 at 4:02 AM, Tarek Ziad? wrote: > On Sat, Jul 24, 2010 at 4:08 PM, Guido van Rossum wrote: >> Mirroring apparently also >> requires some client changes. > > Mirrors can be used as long as you manually point a mirror when using > them. We we are working on making the > switch automatic. I think we've talked briefly about this before, but let me reiterate that getting this right from a security point of view is quite a bit harder than it at first appears, and IMHO it is worth getting right. Geremy Condra From ziade.tarek at gmail.com Mon Jul 26 13:52:12 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 26 Jul 2010 13:52:12 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On Mon, Jul 26, 2010 at 1:20 PM, geremy condra wrote: > On Mon, Jul 26, 2010 at 4:02 AM, Tarek Ziad? wrote: >> On Sat, Jul 24, 2010 at 4:08 PM, Guido van Rossum wrote: > > > >>> Mirroring apparently also >>> requires some client changes. >> >> Mirrors can be used as long as you manually point a mirror when using >> them. We we are working on making the >> switch automatic. > > I think we've talked briefly about this before, but let me reiterate > that getting this right from a security point of view is quite a bit > harder than it at first appears, and IMHO it is worth getting right. FWIW, Martin has added a section about mirror authenticity in the PEP: http://www.python.org/dev/peps/pep-0381/#mirror-authenticity > > Geremy Condra > -- Tarek Ziad? | http://ziade.org From debatem1 at gmail.com Mon Jul 26 14:10:02 2010 From: debatem1 at gmail.com (geremy condra) Date: Mon, 26 Jul 2010 05:10:02 -0700 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On Mon, Jul 26, 2010 at 4:52 AM, Tarek Ziad? wrote: > On Mon, Jul 26, 2010 at 1:20 PM, geremy condra wrote: >> On Mon, Jul 26, 2010 at 4:02 AM, Tarek Ziad? wrote: >>> On Sat, Jul 24, 2010 at 4:08 PM, Guido van Rossum wrote: >> >> >> >>>> Mirroring apparently also >>>> requires some client changes. >>> >>> Mirrors can be used as long as you manually point a mirror when using >>> them. We we are working on making the >>> switch automatic. >> >> I think we've talked briefly about this before, but let me reiterate >> that getting this right from a security point of view is quite a bit >> harder than it at first appears, and IMHO it is worth getting right. > > FWIW, Martin has added a section about mirror authenticity in the PEP: > > http://www.python.org/dev/peps/pep-0381/#mirror-authenticity This is more-or-less what was discussed earlier, and from what's described here I think the concerns I voiced stand. What's the right way to do disclosure on this sort of issue? Geremy Condra From ncoghlan at gmail.com Mon Jul 26 14:41:50 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 26 Jul 2010 22:41:50 +1000 Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: <4C4CD849.9090108@v.loewis.de> References: <4C4CD849.9090108@v.loewis.de> Message-ID: On Mon, Jul 26, 2010 at 10:35 AM, "Martin v. L?wis" wrote: > Am 26.07.2010 02:24, schrieb Terry Reedy: >> To review a patch on the tracker, I have to read and try to make sense >> of the raw diff file. Sometimes that is easy, sometimes not. >> >> *After* a patch is applied, I can click the revxxxx link and then the >> 'text changed' link and see a nice, colored, side-by-side web-pace view >> created by ViewVC. Is there any way a 'view' button could be added, >> along with the current edit and remove buttons, to produce the same web >> page and make it easier to review patches *before* commitment? > > You should be using Rietveld for that. And I believe there's already an item in the meta-tracker about improving the integration between bugs.python.org and Rietveld - it just doesn't exist yet. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From barry at python.org Mon Jul 26 16:06:50 2010 From: barry at python.org (Barry Warsaw) Date: Mon, 26 Jul 2010 10:06:50 -0400 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: <20100726100650.1700c8b7@heresy> On Jul 24, 2010, at 07:08 AM, Guido van Rossum wrote: >privileges enough. So, my recommendation (which surely is a >turn-around of my *own* attitude in the past) is to give out more >commit privileges sooner. +1, though I'll observe that IME, actual commit privileges become much less of a special badge once a dvcs-based workflow is put in place. In the absence of that, I agree that we have enough checks and balances in place to allow more folks to commit changes. >approved. A PEP handler should be selected for each PEP as soon as >possible; without a PEP handler, discussing a PEP is not all that >useful. The PEP handler should be someone respected by the community >with an interest in the subject of the PEP but at an arms' length (at >least) from the PEP author. The PEP handler will have to moderate This is a good idea, and certainly helps "scale Guido" better. We might also consider designating experts who can collaborate on PEP wrangling for certain topics. For example, if Martin would normally be the Cheeseshop PEP handler, but submits his own PEPs on the topic, the handful of experts can take up the slack when Martin recuses himself on his own PEPs. I'd hope that we could always find at least two people to wrangle any PEP, or we've got a bigger problem to deal with! Sounds like EuroPython was fun! -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From ziade.tarek at gmail.com Mon Jul 26 16:21:28 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 26 Jul 2010 16:21:28 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On Mon, Jul 26, 2010 at 2:10 PM, geremy condra wrote: > On Mon, Jul 26, 2010 at 4:52 AM, Tarek Ziad? wrote: >> On Mon, Jul 26, 2010 at 1:20 PM, geremy condra wrote: >>> On Mon, Jul 26, 2010 at 4:02 AM, Tarek Ziad? wrote: >>>> On Sat, Jul 24, 2010 at 4:08 PM, Guido van Rossum wrote: >>> >>> >>> >>>>> Mirroring apparently also >>>>> requires some client changes. >>>> >>>> Mirrors can be used as long as you manually point a mirror when using >>>> them. We we are working on making the >>>> switch automatic. >>> >>> I think we've talked briefly about this before, but let me reiterate >>> that getting this right from a security point of view is quite a bit >>> harder than it at first appears, and IMHO it is worth getting right. >> >> FWIW, Martin has added a section about mirror authenticity in the PEP: >> >> http://www.python.org/dev/peps/pep-0381/#mirror-authenticity > > This is more-or-less what was discussed earlier, and from what's > described here I think the concerns I voiced stand. What's the right > way to do disclosure on this sort of issue? I would recommend discussing it in Distutils-SIG and proposing a change to that PEP. Notice that this PEP is not accepted yet. I am not sure what would be the best moment to have it accepted. I guess once we have experimented enough on the client side. Tarek -- Tarek Ziad? | http://ziade.org From debatem1 at gmail.com Mon Jul 26 16:29:14 2010 From: debatem1 at gmail.com (geremy condra) Date: Mon, 26 Jul 2010 07:29:14 -0700 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On Mon, Jul 26, 2010 at 7:21 AM, Tarek Ziad? wrote: > On Mon, Jul 26, 2010 at 2:10 PM, geremy condra wrote: >> On Mon, Jul 26, 2010 at 4:52 AM, Tarek Ziad? wrote: >>> On Mon, Jul 26, 2010 at 1:20 PM, geremy condra wrote: >>>> On Mon, Jul 26, 2010 at 4:02 AM, Tarek Ziad? wrote: >>>>> On Sat, Jul 24, 2010 at 4:08 PM, Guido van Rossum wrote: >>>> >>>> >>>> >>>>>> Mirroring apparently also >>>>>> requires some client changes. >>>>> >>>>> Mirrors can be used as long as you manually point a mirror when using >>>>> them. We we are working on making the >>>>> switch automatic. >>>> >>>> I think we've talked briefly about this before, but let me reiterate >>>> that getting this right from a security point of view is quite a bit >>>> harder than it at first appears, and IMHO it is worth getting right. >>> >>> FWIW, Martin has added a section about mirror authenticity in the PEP: >>> >>> http://www.python.org/dev/peps/pep-0381/#mirror-authenticity >> >> This is more-or-less what was discussed earlier, and from what's >> described here I think the concerns I voiced stand. What's the right >> way to do disclosure on this sort of issue? > > I would recommend discussing it in Distutils-SIG and proposing a > change to that PEP. I've noticed that I don't have a lot of success in shifting this kind of debate, so I'm not sure it's a good idea to publicly discuss vulnerabilities in something that may wind up being implemented as-is, but it's up to you guys. Geremy Condra From guido at python.org Mon Jul 26 16:29:50 2010 From: guido at python.org (Guido van Rossum) Date: Mon, 26 Jul 2010 07:29:50 -0700 Subject: [Python-Dev] marking os.system() as deprecated in the docs In-Reply-To: References: Message-ID: Unless, it's as simple as replacing "os.system(x)" with "subprocess.system(x)", I'm against this removal of a handy shorthand. Ditto for popen. On Sun, Jul 25, 2010 at 11:42 PM, Chris Rebert wrote: > Hello, > > I'd like to propose formally marking os.system() as deprecated in the > docs for next release of Python (v3.2 ?). > > The docs for os.system() /already/ include the following paragraph, > which is basically tantamount to calling system() deprecated and very > much resembles the deprecation notes for the os.popen*() family. > """ > The subprocess module provides more powerful facilities for spawning > new processes and retrieving their results; using that module is > preferable to using this function. Use the subprocess module. Check > especially the Replacing Older Functions with the subprocess Module > section. > """ > > I'm merely suggesting slapping a formal ".. deprecated::" tag & box > around this paragraph to make system()'s status more obvious. I am > ***not*** suggesting the function be scheduled for removal or anything > like that. > > I figure since this would be a substantive change (the docs currently > don't outright say "deprecated"), it would need dev approval, hence > this message. I've already written the absolutely /trivial/ docs > patch. > > So, any objections? > > Cheers, > Chris > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) From ronaldoussoren at mac.com Mon Jul 26 16:33:48 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Mon, 26 Jul 2010 07:33:48 -0700 (PDT) Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: <4C4D5CA5.7020809@voidspace.org.uk> Message-ID: <368164bf-6540-66f8-f2d0-b986226d9587@me.com> On 26 Jul, 2010,at 12:00 PM, Michael Foord wrote: On 26/07/2010 04:42, Guido van Rossum wrote: > On Sun, Jul 25, 2010 at 8:31 PM, Peter Portante > wrote: > >> FWIW: We use Python at Tabblo, straddled across Python 2.5.4 and 2.6.5. They >> work. And they work well. But we make light use of threads (mostly >> background I/O handling), and heavy use of multiple processes because we >> can't take advantage of our multi-core systems otherwise. >> > Isn't this an indication that the GIL is, in fact, not (much of) a problem? > > I wish those trying to get rid of the GIL well But it may not be the > panacea some folks are hoping for. Multi-threaded programming remains > hard (and removing the GIL might actually make it harder). > > Jython and IronPython don't have a GIL, and I think PyPy may not > either. Does anyone have experience with GIL-free programming in one > of those? > > At Resolver Systems we created a "calculation system" that does large calculations on background threads using IronPython. Doing them on a background thread allows the ui to remain responsive. Several calculations could run simultaneously using multiple cores. As the calculation operates on a large object graph (which the ui then needs access to in order to display it) using multiprocessing would have imposed a very big overhead due to serialization / deserialization (the program runs on windows). Using CPython would have made the program a lot slower due to the GIL. ? I have a simular usecase, although in my case it's more using large blobs instead of complicated datastructures. I'm not hurt by the GIL because most threads run C code most of the time, which enables us to use multiple CPU cores without getting hurt by the GIL. In my opinion the GIL is a weak point of CPython and it would be nice if it could be fixed. ?That is however easier said than done, a number of people have tried in the past and ran into implementation limitations like our refcounting garbage collector that make hard to remove the GIL without either rewriting lots of code, or running into a brick wall performance-wise. The HotPy presentation at EuroPython shows that it is possible to remove the GIL, although at the cost of replacing the garbage collector and most likely breaking existing C extensions (although the HotPy author seemed to have a possible workaround for that) Ronald -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Jul 26 16:36:13 2010 From: guido at python.org (Guido van Rossum) Date: Mon, 26 Jul 2010 07:36:13 -0700 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: <4C4D5CA5.7020809@voidspace.org.uk> References: <4C4D5CA5.7020809@voidspace.org.uk> Message-ID: On Mon, Jul 26, 2010 at 3:00 AM, Michael Foord wrote: > At Resolver Systems we created a "calculation system" that does large > calculations on background threads using IronPython. Doing them on a > background thread allows the ui to remain responsive. Several calculations > could run simultaneously using multiple cores. > > As the calculation operates on a large object graph (which the ui then needs > access to in order to display it) using multiprocessing would have imposed a > very big overhead due to serialization / deserialization (the program runs > on windows). > > Using CPython would have made the program a lot slower due to the GIL. Sure. Note that using threads with the GIL, it is not a problem to keep the UI responsive even if background calculations are going on (at worst it requires some tweaking of sys.setcheckinterval() or its new-GIL equivalent). However with the GIL multiple calculations would be limited to a single core. According to CSP advicates, this approach will break down when you need more than 8-16 cores since cache coherence breaks down at 16 cores. Then you would have to figure out a message-passing approach (but the messages would have to be very fast). -- --Guido van Rossum (python.org/~guido) From stefan_ml at behnel.de Mon Jul 26 16:36:37 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 26 Jul 2010 16:36:37 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: geremy condra, 26.07.2010 16:29: > I've noticed that I don't have a lot of success in shifting this kind > of debate, so I'm not sure it's a good idea to publicly discuss > vulnerabilities in something that may wind up being implemented as-is, > but it's up to you guys. Hmm, security by obscurity? That's a good idea. Let's do that more often. Stefan From ziade.tarek at gmail.com Mon Jul 26 16:51:44 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 26 Jul 2010 16:51:44 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On Mon, Jul 26, 2010 at 4:29 PM, geremy condra wrote: ... > I've noticed that I don't have a lot of success in shifting this kind > of debate, so I'm not sure it's a good idea to publicly discuss > vulnerabilities in something that may wind up being implemented as-is, > but it's up to you guys. I think its best to have this discussed there publicly. In any case, mirrors are run by trusted people, so the risks are not very high AFAIK I think this discussion didn't have a lot of participant because most of us (that includes me) are not expert all all, if not ignorant, in this topic. A complete patch to the PEP, including a detailed description, is the best thing to do I think, to move this forward. Regards Tarek From guido at python.org Mon Jul 26 16:55:58 2010 From: guido at python.org (Guido van Rossum) Date: Mon, 26 Jul 2010 07:55:58 -0700 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On Mon, Jul 26, 2010 at 7:36 AM, Stefan Behnel wrote: > geremy condra, 26.07.2010 16:29: >> >> I've noticed that I don't have a lot of success in shifting this kind >> of debate, so I'm not sure it's a good idea to publicly discuss >> vulnerabilities in something that may wind up being implemented as-is, >> but it's up to you guys. > > Hmm, security by obscurity? That's a good idea. Let's do that more often. FWIW, security by obscurity has a bad rep in some circles, but it is an essential component of any serious security policy. It just should never be the *only* component. (In fact, any serious security policy should have multiple disparate components.) In this case, it looks like (a) the cat is already out of the bag, and (b) it's easy to figure out from the PEPs where the vulnerabilities lie, so I don't think we'll gain much by shushing it up. -- --Guido van Rossum (python.org/~guido) From debatem1 at gmail.com Mon Jul 26 16:56:21 2010 From: debatem1 at gmail.com (geremy condra) Date: Mon, 26 Jul 2010 07:56:21 -0700 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On Mon, Jul 26, 2010 at 7:36 AM, Stefan Behnel wrote: > geremy condra, 26.07.2010 16:29: >> >> I've noticed that I don't have a lot of success in shifting this kind >> of debate, so I'm not sure it's a good idea to publicly discuss >> vulnerabilities in something that may wind up being implemented as-is, >> but it's up to you guys. > > Hmm, security by obscurity? That's a good idea. Let's do that more often. Usually it's termed responsible disclosure, but I'm a lot more interested in fixing things than playing semantics. Geremy Condra From steve at pearwood.info Mon Jul 26 17:10:23 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 27 Jul 2010 01:10:23 +1000 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: <201007270110.23638.steve@pearwood.info> On Tue, 27 Jul 2010 12:36:37 am Stefan Behnel wrote: > geremy condra, 26.07.2010 16:29: > > I've noticed that I don't have a lot of success in shifting this > > kind of debate, so I'm not sure it's a good idea to publicly > > discuss vulnerabilities in something that may wind up being > > implemented as-is, but it's up to you guys. > > Hmm, security by obscurity? That's a good idea. Let's do that more > often. Shhh! Don't tell anybody! *wink* But seriously, I don't think Geremy is suggesting security by obscurity. It seems to me that he's merely suggesting that we are discreet about discussing vulnerabilities unless we have a plan to fix them. Whether such discretion is useful is an open question. It may be that the cat is already out of the bag and it's too late to be discreet, so we might as well not bother. -- Steven D'Aprano From ebo at 4geeks.de Mon Jul 26 17:04:23 2010 From: ebo at 4geeks.de (=?ISO-8859-1?Q?J=F6rg_Blank?=) Date: Mon, 26 Jul 2010 17:04:23 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: <4C4D5CA5.7020809@voidspace.org.uk> Message-ID: <4C4DA3F7.4010308@4geeks.de> > According to CSP advicates, this approach will break down when you > need more than 8-16 cores since cache coherence breaks down at 16 > cores. Then you would have to figure out a message-passing approach > (but the messages would have to be very fast). It does break down, and probably always will be. Imho this gets worse with NUMA architectures becoming more prevalent. But even with 50 cores you may be happy to have something run away with 4-8 threads shared memory from time to time. Developing good message based schemes is important for the long run, but I think multithreaded parallelization will become more common, before we see a general switch to messages. Regards, Joerg Blank From steve at pearwood.info Mon Jul 26 17:19:18 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 27 Jul 2010 01:19:18 +1000 Subject: [Python-Dev] marking os.system() as deprecated in the docs In-Reply-To: References: Message-ID: <201007270119.19041.steve@pearwood.info> On Mon, 26 Jul 2010 04:42:06 pm Chris Rebert wrote: > Hello, > > I'd like to propose formally marking os.system() as deprecated in the > docs for next release of Python (v3.2 ?). -0.5 on that. I often use os.system() for quick and dirty scripts that just call an external process without caring what the output is. But perhaps os.system() should become an alias for subprocess.system() in 3.2, then deprecated in 3.3 and removed (from os only) in 3.4? -- Steven D'Aprano From rrr at ronadam.com Mon Jul 26 17:25:28 2010 From: rrr at ronadam.com (Ron Adam) Date: Mon, 26 Jul 2010 10:25:28 -0500 Subject: [Python-Dev] [isssue 2001] Pydoc enhancement patch questions In-Reply-To: References: <4C4B404E.2090007@ronadam.com> Message-ID: On 07/25/2010 12:01 PM, Alexander Belopolsky wrote: > On Sun, Jul 25, 2010 at 12:32 PM, Ron Adam wrote: .. >>> I'd be completely fine with dropping the "Search For" box from the >>> GUI interface, but the persistent window listing the served port >>> and providing "Open Browser" and "Quit Serving" buttons still seems >>> quite useful even without the search box (When running "python -m >>> pydoc -p 8080&", it took me a moment to figure out how to kill the >>> server I had started). > > Why not simply have "Quit Serving" next to the search button in the > served pages? The server can even serve a friendly page explaining how > it can be restarted before quitting. .. >> Another way to communicate to the server would be to add a link in >> the browser to open a server status page. For example my router has a >> configure page where I can check it's status and do other things. >> That might be something worth exploring at some later date. > > This would work as well, but for starters, I think "Search" and "Quit" > buttons next to each other will be most familiar to the users of the > current Tk-based control window. Nick, what do you think about the "Quit" link in the browser along with improving the server startup message on the console window? Ron From ianb at colorstudy.com Mon Jul 26 17:50:15 2010 From: ianb at colorstudy.com (Ian Bicking) Date: Mon, 26 Jul 2010 10:50:15 -0500 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: <20100726100650.1700c8b7@heresy> References: <20100726100650.1700c8b7@heresy> Message-ID: On Mon, Jul 26, 2010 at 9:06 AM, Barry Warsaw wrote: > On Jul 24, 2010, at 07:08 AM, Guido van Rossum wrote: > >privileges enough. So, my recommendation (which surely is a > >turn-around of my *own* attitude in the past) is to give out more > >commit privileges sooner. > > +1, though I'll observe that IME, actual commit privileges become much less > of > a special badge once a dvcs-based workflow is put in place. In the absence > of > that, I agree that we have enough checks and balances in place to allow > more > folks to commit changes > Even with DVCS in place, commit privileges allow the person who cares about a change to move it forward, including the more mechanical aspects. E.g. if there are positive reviews of a person's changes in their fork, they can push those changes in. Or more generally, there's a lot of ways of getting approval, but limited commit privileges means all approval must ultimately be funneled through someone with commit. Also different parts of the codebase should have different levels of review and conservativism; e.g., adding clarifications to the docs requires a different level of review than changing stuff in the core. We could try to build that into the tools, but it's a lot easier to make the tools permissive and build these distinctions into social structures. -- Ian Bicking | http://blog.ianbicking.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From amk at amk.ca Mon Jul 26 17:52:40 2010 From: amk at amk.ca (A.M. Kuchling) Date: Mon, 26 Jul 2010 11:52:40 -0400 Subject: [Python-Dev] marking os.system() as deprecated in the docs In-Reply-To: <201007270119.19041.steve@pearwood.info> References: <201007270119.19041.steve@pearwood.info> Message-ID: <20100726155240.GA24947@amk-desktop.matrixgroup.net> The current text is: The subprocess module provides more powerful facilities for spawning new processes and retrieving their results; using that module is preferable to using this function. See the Replacing Older Functions with the subprocess Module section in the subprocess documentation for some helpful recipes. We could soften "using that module is preferable" to say "may be", or just delete that whole clause completely. --amk From alexander.belopolsky at gmail.com Mon Jul 26 17:55:14 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 26 Jul 2010 11:55:14 -0400 Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: References: Message-ID: On Sun, Jul 25, 2010 at 8:24 PM, Terry Reedy wrote: > Is there any way a 'view' button could be added, along with the > current edit and remove buttons, to produce the same web page and make it > easier to review patches *before* commitment? One thing that patch submitters can do already is to make sure that their patches are recognized as text/plain by the tracker. This can be achieved using standard extensions such as .patch, .diff, or .txt and avoiding non-satndard ones such as .patch2. Note that if you accidentally uploaded a text file with a wrong extension, you can go to the edit page and change the file name and/or content type. I think the tracker may become a little more proactive in enforcing plain text attachments by rejecting unrecognized extensions or warning users about them. On a similar note, I wonder if tracker should simply reject binary attachments with a possible exception of well formed UTF-8 text. From ronaldoussoren at mac.com Mon Jul 26 17:39:39 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Mon, 26 Jul 2010 17:39:39 +0200 Subject: [Python-Dev] [isssue 2001] Pydoc enhancement patch questions In-Reply-To: References: <4C4B404E.2090007@ronadam.com> Message-ID: <0280CF24-5D99-45B7-B2E3-6AAE81A2B602@mac.com> On 24 Jul, 2010, at 23:29, Alexander Belopolsky wrote: > Furthermore, I just tried to use it on my OSX laptop and it > crashed after I searched for pydoc and clicked on the first entry. > (Another issue is that search window pops under the terminal window.) Could you please file a bug for this? I have no idea whether or not it is an easy to fix one, but without an issue in the tracker this is something that is unlikely to get fixed. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From alexander.belopolsky at gmail.com Mon Jul 26 18:45:45 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 26 Jul 2010 12:45:45 -0400 Subject: [Python-Dev] [isssue 2001] Pydoc enhancement patch questions In-Reply-To: <0280CF24-5D99-45B7-B2E3-6AAE81A2B602@mac.com> References: <4C4B404E.2090007@ronadam.com> <0280CF24-5D99-45B7-B2E3-6AAE81A2B602@mac.com> Message-ID: On Mon, Jul 26, 2010 at 11:39 AM, Ronald Oussoren wrote: .. > Could you please file a bug for this? I have no idea whether or not it is an easy to fix one, > but without an issue in the tracker this is something that is unlikely to get fixed. http://bugs.python.org/issue9384 From tjreedy at udel.edu Mon Jul 26 19:14:56 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 26 Jul 2010 13:14:56 -0400 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On 7/26/2010 2:40 AM, Peter Portante wrote: > Yet, shouldn't we be able to write a simple embarrassingly parallel > multithreaded algorithm in python (no C-extensions) and have its execution > use all the cores on a system using CPython? Abstractly, yes, and I believe you can do that now with some implementations. The actual questions are along the lines of ... What would be the cost of making that happen with CPython? Who would be disadvanged making that happen with CPython? and for both of those, Is the tradeoff worth it? Another way to put it is Should CPython be optimized for 1, 2, 3, or 4 or more cores? The answer to this is obviously changing. I will soon replace a single core with a 4/6 core machine, so would be right in the middle on that, except that my current work is all single-threaded anyway. But that could change. Should all implementation be optimized the same way? Of course, with several developers focused on these issues, we could have a compile time switch and distribute multiple Windows binaries, but this does not seem like fun, volunteer-type stuff. -- Terry Jan Reedy From techtonik at gmail.com Mon Jul 26 19:48:06 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Mon, 26 Jul 2010 20:48:06 +0300 Subject: [Python-Dev] marking os.system() as deprecated in the docs In-Reply-To: <20100726155240.GA24947@amk-desktop.matrixgroup.net> References: <201007270119.19041.steve@pearwood.info> <20100726155240.GA24947@amk-desktop.matrixgroup.net> Message-ID: Is it possible to use subprocess functionality in os.system()? Just exactly describe what it does instead of "This is implemented by calling the Standard C function system(), and has the same limitations." type sentences. -- anatoly t. On Mon, Jul 26, 2010 at 6:52 PM, A.M. Kuchling wrote: > The current text is: > > ? ?The subprocess module provides more powerful facilities for > ? ?spawning new processes and retrieving their results; using that > ? ?module is preferable to using this function. See the Replacing > ? ?Older Functions with the subprocess Module section in the > ? ?subprocess documentation for some helpful recipes. > > We could soften "using that module is preferable" to say "may be", or > just delete that whole clause completely. > > --amk > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/techtonik%40gmail.com > From barry at python.org Mon Jul 26 20:08:49 2010 From: barry at python.org (Barry Warsaw) Date: Mon, 26 Jul 2010 14:08:49 -0400 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: <20100726100650.1700c8b7@heresy> Message-ID: <20100726140849.44466164@heresy> On Jul 26, 2010, at 10:50 AM, Ian Bicking wrote: >On Mon, Jul 26, 2010 at 9:06 AM, Barry Warsaw wrote: > >> On Jul 24, 2010, at 07:08 AM, Guido van Rossum wrote: >> >privileges enough. So, my recommendation (which surely is a >> >turn-around of my *own* attitude in the past) is to give out more >> >commit privileges sooner. >> >> +1, though I'll observe that IME, actual commit privileges become >> much less of >> a special badge once a dvcs-based workflow is put in place. In the >> absence of >> that, I agree that we have enough checks and balances in place to >> allow more >> folks to commit changes >> > >Even with DVCS in place, commit privileges allow the person who cares >about a change to move it forward, including the more mechanical >aspects. E.g. if there are positive reviews of a person's changes in >their fork, they can push those changes in. Or more generally, >there's a lot of ways of getting approval, but limited commit >privileges means all approval must ultimately be funneled through >someone with commit. Right, but with a dvcs workflow, it's really only the very last step that requires commit privileges. There is much less chance of having those fork branches get stale, much greater ability for those branches to be reviewed, tested, and commented on, etc. You can more easily do everything right up until the final merge to the master branch without commit privileges, so it's much less of a blocker to progress. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From techtonik at gmail.com Mon Jul 26 20:16:31 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Mon, 26 Jul 2010 21:16:31 +0300 Subject: [Python-Dev] OpenID login for Roundup bug trackers In-Reply-To: <4C4595B5.8090409@v.loewis.de> References: <4C437BCF.5000409@v.loewis.de> <87r5iy8n9q.fsf_-_@benfinney.id.au> <4C4595B5.8090409@v.loewis.de> Message-ID: +On Tue, Jul 20, 2010 at 3:25 PM, "Martin v. L?wis" wrote: >> Thanks for drawing my attention to that; if the people who made OpenID >> auth happen are observing this, then thank you all very much! > > You're welcome! > >> Any hope of feeding those changes back upstream so it's available to all >> Roundup users at some point? > > Hope dies last. > > It's main foundation is an openid library that I wrote > (http://pypi.python.org/pypi/openid2rp). As for the roundup integration, I > likely won't have much time to contribute it back. I could do the merge if the license were the same as Roundup or something similar like MIT or ISC. -- anatoly t. From techtonik at gmail.com Mon Jul 26 20:29:31 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Mon, 26 Jul 2010 21:29:31 +0300 Subject: [Python-Dev] Readability of hex strings (Was: Use of coding cookie in 3.x stdlib) Message-ID: I find "\xXX\xXX\xXX\xXX..." notation for binary data totally unreadable. Everybody who uses and analyses binary data is more familiar with plain hex dumps in the form of "XX XX XX XX...". I wonder if it is possible to introduce an effective binary string type that will be represented as h"XX XX XX" in language syntax? It will be much easier to analyze printed binary data and copy/paste such data as-is from hex editors/views. On Mon, Jul 19, 2010 at 9:45 AM, Guido van Rossum wrote: > Sounds like a good idea to try to remove redundant cookies *and* to > remove most occasional use of non-ASCII characters outside comments > (except for unittests specifically trying to test Unicode features). > Personally I would use \xXX escapes instead of spelling out the > characters in shlex.py, for example. > > Both with or without the coding cookies, many ways of displaying text > files garble characters outside the ASCII range, so it's better to > stick to ASCII as much as possible. > > --Guido > > On Mon, Jul 19, 2010 at 1:21 AM, Alexander Belopolsky > wrote: >> I was looking at the inspect module and noticed that it's source >> starts with "# -*- coding: iso-8859-1 -*-". ? I have checked and there >> are no non-ascii characters in the file. ? There are several other >> modules that still use the cookie: >> >> Lib/ast.py:# -*- coding: utf-8 -*- >> Lib/getopt.py:# -*- coding: utf-8 -*- >> Lib/inspect.py:# -*- coding: iso-8859-1 -*- >> Lib/pydoc.py:# -*- coding: latin-1 -*- >> Lib/shlex.py:# -*- coding: iso-8859-1 -*- >> Lib/encodings/punycode.py:# -*- coding: utf-8 -*- >> Lib/msilib/__init__.py:# -*- coding: utf-8 -*- >> Lib/sqlite3/__init__.py:#-*- coding: ISO-8859-1 -*- >> Lib/sqlite3/dbapi2.py:#-*- coding: ISO-8859-1 -*- >> Lib/test/bad_coding.py:# -*- coding: uft-8 -*- >> Lib/test/badsyntax_3131.py:# -*- coding: utf-8 -*- >> >> I understand that coding: utf-8 is strictly redundant in 3.x. ?There >> are cases such as Lib/shlex.py where using encoding other than utf-8 >> is justified. ?(See >> http://svn.python.org/view?view=rev&revision=82560). ?What are the >> guidelines for other cases? ?Should redundant cookies be removed? >> Since not all editors respect the ?-*- cookie, I think the answer >> should be "yes" particularly when the cookie is setting encoding other >> than utf-8. >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org >> > > > > -- > --Guido van Rossum (python.org/~guido) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/techtonik%40gmail.com > From barry at python.org Mon Jul 26 20:53:58 2010 From: barry at python.org (Barry Warsaw) Date: Mon, 26 Jul 2010 14:53:58 -0400 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> <63B36042-C4F7-4BCA-8623-5182405D8852@mac.com> <20100723120252.2e0690cc@snowdog> <25347613-8488-40C3-B9EF-9C85D0C47340@mac.com> <20100723125440.721860bf@snowdog> <20100724001941.6adad036@snowdog> Message-ID: <20100726145358.1eade6ac@heresy> On Jul 24, 2010, at 09:54 AM, Ronald Oussoren wrote: >> I'd be okay including a configure option to allow you to add >> whatever you want after the implementation, version, and flags. >> E.g. something like: >> >> ./configure --with-abi-tag-extension=linux2 >> >> would lead to foo.cpython-32m-linux2.so, so not the nicer names we'd >> prefer but probably good enough for your purposes. >> >> Would that work for you? > >That would certainly work. That said, I'm also fine with not adding >the platform information or configure argument at all. My usecase is >fairly exotic and I do have a feasible workaround. Cool. In that case, I won't add it. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Mon Jul 26 21:07:05 2010 From: barry at python.org (Barry Warsaw) Date: Mon, 26 Jul 2010 15:07:05 -0400 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <87aapgbky5.fsf@brainbot.com> References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> <63B36042-C4F7-4BCA-8623-5182405D8852@mac.com> <20100723120252.2e0690cc@snowdog> <25347613-8488-40C3-B9EF-9C85D0C47340@mac.com> <20100723125440.721860bf@snowdog> <764F1595-5C8B-44D8-86C5-937B326B2FCD@mac.com> <87mxti8lqc.fsf@muni.brainbot.com> <20100724002252.19d91267@snowdog> <87aapgbky5.fsf@brainbot.com> Message-ID: <20100726150705.778ebf65@heresy> On Jul 24, 2010, at 11:59 PM, schmir at gmail.com wrote: >Barry Warsaw writes: > >> On Jul 23, 2010, at 01:46 PM, schmir at gmail.com wrote: >> >>>Doesn't anybody else think this is lost work for very little gain? My >>>/usr/lib/python2.6/site-packages directory consumes 200MB on disk. I >>>couldn't care less if my /usr/lib/python2.5/site-packages consumed >>>the same amount of disk space... >> >> Right, you probably don't care now that your extension modules live >> in foo.so so it probably won't make much difference if they were >> named foo-blahblah.so, as long as they import. :) > >Most of the time it won't make much difference, right. But I can assure >you, that it will bite some people and there is some code to be >adapted. Do you have concrete examples? Without that it's just speculation I can't do much to address. Is the problem big or small? Easy to work around or not? "Change is bad" isn't a constructive argument. ;) >> If you use Debian or Ubuntu though, it'll be a win for you by allow >> us to make Python support much more robust. > >I'd much prefer to have cleanly separated environments by having >separate directories for my python modules. Sharing the source code and >complicating things will not lead to increased robustness. That's fine, but it's not the way Debian/Ubuntu works today. PEP 3149 adoption will definitely remove significant complication for deploying multiple Python versions at the same time on those systems. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From alexandre at peadrop.com Mon Jul 26 20:42:07 2010 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Mon, 26 Jul 2010 11:42:07 -0700 Subject: [Python-Dev] Readability of hex strings (Was: Use of coding cookie in 3.x stdlib) In-Reply-To: References: Message-ID: [+Python-ideas -Python-Dev] import binascii def h(s): return binascii.unhexlify("".join(s.split())) h("DE AD BE EF CA FE BA BE") -- Alexandre On Mon, Jul 26, 2010 at 11:29 AM, anatoly techtonik wrote: > I find "\xXX\xXX\xXX\xXX..." notation for binary data totally > unreadable. Everybody who uses and analyses binary data is more > familiar with plain hex dumps in the form of "XX XX XX XX...". > > I wonder if it is possible to introduce an effective binary string > type that will be represented as h"XX XX XX" in language syntax? It > will be much easier to analyze printed binary data and copy/paste such > data as-is from hex editors/views. > > On Mon, Jul 19, 2010 at 9:45 AM, Guido van Rossum wrote: >> Sounds like a good idea to try to remove redundant cookies *and* to >> remove most occasional use of non-ASCII characters outside comments >> (except for unittests specifically trying to test Unicode features). >> Personally I would use \xXX escapes instead of spelling out the >> characters in shlex.py, for example. >> >> Both with or without the coding cookies, many ways of displaying text >> files garble characters outside the ASCII range, so it's better to >> stick to ASCII as much as possible. >> >> --Guido >> >> On Mon, Jul 19, 2010 at 1:21 AM, Alexander Belopolsky >> wrote: >>> I was looking at the inspect module and noticed that it's source >>> starts with "# -*- coding: iso-8859-1 -*-". ? I have checked and there >>> are no non-ascii characters in the file. ? There are several other >>> modules that still use the cookie: >>> >>> Lib/ast.py:# -*- coding: utf-8 -*- >>> Lib/getopt.py:# -*- coding: utf-8 -*- >>> Lib/inspect.py:# -*- coding: iso-8859-1 -*- >>> Lib/pydoc.py:# -*- coding: latin-1 -*- >>> Lib/shlex.py:# -*- coding: iso-8859-1 -*- >>> Lib/encodings/punycode.py:# -*- coding: utf-8 -*- >>> Lib/msilib/__init__.py:# -*- coding: utf-8 -*- >>> Lib/sqlite3/__init__.py:#-*- coding: ISO-8859-1 -*- >>> Lib/sqlite3/dbapi2.py:#-*- coding: ISO-8859-1 -*- >>> Lib/test/bad_coding.py:# -*- coding: uft-8 -*- >>> Lib/test/badsyntax_3131.py:# -*- coding: utf-8 -*- >>> >>> I understand that coding: utf-8 is strictly redundant in 3.x. ?There >>> are cases such as Lib/shlex.py where using encoding other than utf-8 >>> is justified. ?(See >>> http://svn.python.org/view?view=rev&revision=82560). ?What are the >>> guidelines for other cases? ?Should redundant cookies be removed? >>> Since not all editors respect the ?-*- cookie, I think the answer >>> should be "yes" particularly when the cookie is setting encoding other >>> than utf-8. >>> _______________________________________________ >>> Python-Dev mailing list >>> Python-Dev at python.org >>> http://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org >>> >> >> >> >> -- >> --Guido van Rossum (python.org/~guido) >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/techtonik%40gmail.com >> > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/alexandre%40peadrop.com > From techtonik at gmail.com Mon Jul 26 22:18:05 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Mon, 26 Jul 2010 23:18:05 +0300 Subject: [Python-Dev] Define a place for code review in Python workflow Message-ID: http://bugs.python.org/issue9376 This issue discussed docs on the proper way to create diff on windows (as it is doesn't have the tool) for sending the patch. The current proper way is to use "svn diff" which will be replaced with "hg diff". I proposed using Rietveld like: > python -m easy_install review > python -m review But Brian said using Rietveld at all is not a good idea, and didn't want to answer what should be done for it to become good idea. Probably Brian is too busy, that's why I'd like to ask people here. What do you need or expect to happen to start using code review in Python workflow? == Intro == Small introduction for insiders not familiar with outsider's problem of maintaining patches in tracker. Please forgive the tone I write about things I dislike, but I am not devoting my life to earn a title of polite bastard (this one is obligatory disclaimer I find it still doesn't work for all people, so expect some irrelevant flame in follow-ups). Ok, for an outsider like me "svn diff" 'best practice' suxx greatly, because in non-ideal conditions (and it is about 90% of cases) patches are often needed to be edited, and reuploaded. There are usually too few insiders to review you patch, and they are usually too busy to edit a small typo, and they also deny that they need an online tool to see if patch applies clearly, so if your patch doesn't apply clearly you will be asked to check where the problem is. The "svn diff" upload story continues time after time. The problem of too few insiders is that there are too few of them, and in case your patch is somehow complicated it can enjoy some years of loneliness. During this time not only the code can change, but the codebase itself can move to some other place. Some of the few are devoting great time to review new contributions and everything could be fine, but.. there is still a big time lapse, big enough that your patches start to overlap.. Trying to get out of this maintenance nightmare you will start learning Mercurial, then Mercurial Queues, then you go some weeks practicing or.. you will just give up right away, because time is more valuable than money. If you're an insider, you can propose to review patches in ViewVC, but it is just plain wrong. Just because. Just because there are tools that do the job better. Even if they have awful usability interface. That can be improved though. By stealing templates from Gerrit. == How Rietveld helps to save time == Assuming that Python has "easy_install" packaging solution bundled by default (which it doesn't of course): > python -m easy_install review This is run once to install the Rietveld script that is used from command like like: > python -m review and allows you to: 1. Create issue for patch review on Rietveld site 2. Run "svn diff" 3. Upload the patch 4. Supply comment for the patch everything above in one step. To upload an updated patch, you just do: > python -m review -i XXXXX -m "log of fixes made in comparison with previous patch" Everybody can go to Rietveld site to view either patch or the whole file code _with_ the patch. Everybody can add comments _directly_ near patch lines. Everybody receives notifications about new code review comments. As code comments are hard to keep offtopic, the signal to noise ration appears to be is quite high. The patch upload script is designed in a way that every project can tune it for their needs and place into the root of source code. "review" project at PyPI.is uploaded from source at http://bitbucket.org/techtonik/pypi-rietveld - it can be tuned to address needs specific for Python development. What do you need to start using code review for contributed Python patches? Why you wouldn't recommend this practice to outsiders? -- anatoly t. From techtonik at gmail.com Mon Jul 26 22:28:04 2010 From: techtonik at gmail.com (anatoly techtonik) Date: Mon, 26 Jul 2010 23:28:04 +0300 Subject: [Python-Dev] Fwd: Readability of hex strings (Was: Use of coding cookie in 3.x stdlib) In-Reply-To: References: Message-ID: Don't you think it is a bad idea to require people to subscribe to post to python-ideas of redirected there? -- anatoly t. ---------- Forwarded message ---------- From: Date: Mon, Jul 26, 2010 at 11:24 PM Subject: Re: [Python-Dev] Readability of hex strings (Was: Use of coding cookie in 3.x stdlib) To: techtonik at gmail.com You are not allowed to post to this mailing list, and your message has been automatically rejected. ?If you think that your messages are being rejected in error, contact the mailing list owner at python-ideas-owner at python.org. ---------- Forwarded message ---------- From:?anatoly techtonik To:?Alexandre Vassalotti Date:?Mon, 26 Jul 2010 23:24:15 +0300 Subject:?Re: [Python-Dev] Readability of hex strings (Was: Use of coding cookie in 3.x stdlib) On Mon, Jul 26, 2010 at 9:42 PM, Alexandre Vassalotti wrote: > [+Python-ideas -Python-Dev] > > import binascii > def h(s): > ?return binascii.unhexlify("".join(s.split())) > > h("DE AD BE EF CA FE BA BE") The idea is to make "print f.read(50)" usable. Your current solution is: -Beautiful is better than ugly. -Explicit is better than implicit. -Simple is better than complex. Complex is better than complicated. Flat is better than nested. -Sparse is better than dense. -Readability counts. Special cases aren't special enough to break the rules. -Although practicality beats purity. Errors should never pass silently. ... -- anatoly t. > -- Alexandre > > On Mon, Jul 26, 2010 at 11:29 AM, anatoly techtonik wrote: >> I find "\xXX\xXX\xXX\xXX..." notation for binary data totally >> unreadable. Everybody who uses and analyses binary data is more >> familiar with plain hex dumps in the form of "XX XX XX XX...". >> >> I wonder if it is possible to introduce an effective binary string >> type that will be represented as h"XX XX XX" in language syntax? It >> will be much easier to analyze printed binary data and copy/paste such >> data as-is from hex editors/views. >> >> On Mon, Jul 19, 2010 at 9:45 AM, Guido van Rossum wrote: >>> Sounds like a good idea to try to remove redundant cookies *and* to >>> remove most occasional use of non-ASCII characters outside comments >>> (except for unittests specifically trying to test Unicode features). >>> Personally I would use \xXX escapes instead of spelling out the >>> characters in shlex.py, for example. >>> >>> Both with or without the coding cookies, many ways of displaying text >>> files garble characters outside the ASCII range, so it's better to >>> stick to ASCII as much as possible. >>> >>> --Guido >>> >>> On Mon, Jul 19, 2010 at 1:21 AM, Alexander Belopolsky >>> wrote: >>>> I was looking at the inspect module and noticed that it's source >>>> starts with "# -*- coding: iso-8859-1 -*-". ? I have checked and there >>>> are no non-ascii characters in the file. ? There are several other >>>> modules that still use the cookie: >>>> >>>> Lib/ast.py:# -*- coding: utf-8 -*- >>>> Lib/getopt.py:# -*- coding: utf-8 -*- >>>> Lib/inspect.py:# -*- coding: iso-8859-1 -*- >>>> Lib/pydoc.py:# -*- coding: latin-1 -*- >>>> Lib/shlex.py:# -*- coding: iso-8859-1 -*- >>>> Lib/encodings/punycode.py:# -*- coding: utf-8 -*- >>>> Lib/msilib/__init__.py:# -*- coding: utf-8 -*- >>>> Lib/sqlite3/__init__.py:#-*- coding: ISO-8859-1 -*- >>>> Lib/sqlite3/dbapi2.py:#-*- coding: ISO-8859-1 -*- >>>> Lib/test/bad_coding.py:# -*- coding: uft-8 -*- >>>> Lib/test/badsyntax_3131.py:# -*- coding: utf-8 -*- >>>> >>>> I understand that coding: utf-8 is strictly redundant in 3.x. ?There >>>> are cases such as Lib/shlex.py where using encoding other than utf-8 >>>> is justified. ?(See >>>> http://svn.python.org/view?view=rev&revision=82560). ?What are the >>>> guidelines for other cases? ?Should redundant cookies be removed? >>>> Since not all editors respect the ?-*- cookie, I think the answer >>>> should be "yes" particularly when the cookie is setting encoding other >>>> than utf-8. >>>> _______________________________________________ >>>> Python-Dev mailing list >>>> Python-Dev at python.org >>>> http://mail.python.org/mailman/listinfo/python-dev >>>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org >>>> >>> >>> >>> >>> -- >>> --Guido van Rossum (python.org/~guido) >>> _______________________________________________ >>> Python-Dev mailing list >>> Python-Dev at python.org >>> http://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/techtonik%40gmail.com >>> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/alexandre%40peadrop.com >> > From brian.curtin at gmail.com Mon Jul 26 22:29:14 2010 From: brian.curtin at gmail.com (Brian Curtin) Date: Mon, 26 Jul 2010 15:29:14 -0500 Subject: [Python-Dev] Define a place for code review in Python workflow In-Reply-To: References: Message-ID: On Mon, Jul 26, 2010 at 15:18, anatoly techtonik wrote: > http://bugs.python.org/issue9376 > This issue discussed docs on the proper way to create diff on windows > (as it is doesn't have the tool) for sending the patch. The current > proper way is to use "svn diff" which will be replaced with "hg diff". > > I proposed using Rietveld like: > > python -m easy_install review > > python -m review > > But Brian said using Rietveld at all is not a good idea, and didn't > want to answer what should be done for it to become good idea. > -- > anatoly t. This is not at all what I said. I do not think it is a good idea to make subtle changes to the dev FAQ to replace patch submission with Rietveld, like you had suggested. Comments on the issue tracker are not the place to debate/suggest/discuss changes in development workflow. My second sentence in that comment even says that I think Rietveld will be used in the future (hopefully sooner than later). -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Jul 26 22:34:24 2010 From: guido at python.org (Guido van Rossum) Date: Mon, 26 Jul 2010 13:34:24 -0700 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: <20100726140849.44466164@heresy> References: <20100726100650.1700c8b7@heresy> <20100726140849.44466164@heresy> Message-ID: FWIW, a leading magazine (IEEE Spectrum) this week has an interesting opinion piece about multicore. http://spectrum.ieee.org/computing/software/the-trouble-with-multicore -- --Guido van Rossum (python.org/~guido) From rdmurray at bitdance.com Mon Jul 26 22:38:50 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 26 Jul 2010 16:38:50 -0400 Subject: [Python-Dev] Define a place for code review in Python workflow In-Reply-To: References: Message-ID: <20100726203850.3870D21CC3B@kimball.webabinitio.net> On Mon, 26 Jul 2010 23:18:05 +0300, anatoly techtonik wrote: > > python -m review > > and allows you to: > > 1. Create issue for patch review on Rietveld site > 2. Run "svn diff" > 3. Upload the patch > 4. Supply comment for the patch > > everything above in one step. To upload an updated patch, you just do: > > > python -m review -i XXXXX -m "log of fixes made in comparison with previous patch" > > Everybody can go to Rietveld site to view either patch or the whole > file code _with_ the patch. Everybody can add comments _directly_ near > patch lines. > Everybody receives notifications about new code review comments. We do use rietveld for reviews, though (so far) not usually for small patches. That could change. So: 1) write the tool 2) upload it to pypi 3) see to what extent it gets adopted. There's nothing in your proposal that is outside of your control, as far as I can tell. (Well, except for easy_install not being in the stdlib, but that's no barrier to adoption of the proposed tool.) -- R. David Murray www.bitdance.com From g.brandl at gmx.net Mon Jul 26 22:39:10 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 26 Jul 2010 22:39:10 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: Am 26.07.2010 13:02, schrieb Tarek Ziad?: > On Sat, Jul 24, 2010 at 4:08 PM, Guido van Rossum wrote: > [...] >> - A lot of things seem to be happening to make PyPI better. Is this >> being summarized somewhere? Based on some questions I received during >> my keynote Q&A (http://bit.ly/bdflqa) I think not enough people are >> aware of what we are already doing in this area. > > Even people very involved in packaging are not fully aware of what's > going on. I am not for instance. > I think that we lack to communicate and synchronize on our efforts on > the PyPI development. Basically, I think what you'd like to have is Martin saying "I'm going to work on this feature", in addition to "I implemented this feature now" afterwards. That shouldn't be too hard. On related news, PyPI now has a JSONRPC interface. I'll leave it to you to figure out how to use it, and that it wasn't Martin who added it in secret ;) > The last example I have in mind is that I have announced here that I > was working on a patch for > the checkbox problem, then Martin announced today on catalog-sig it > was fixed by Georg and > updated in production :) I'd like to add to this that Martin didn't know I was working on the patch (I wrote the patch on the day after I came home from EP), and having worked a bit in the PyPI codebase during the sprints, I just decided to fix this issue, which I had perceived to be quite urgent to some people. (Also, the patch really wasn't a huge thing.) > I think we need to improve this: it can be a very frustrating > experience to contribute to PyPI. I did not experience it this way. On the contrary, I tried to run PyPI locally for testing purposes, but didn't want to compile and run Postgres, so we figured how hard it was to use Sqlite instead. Martin put in quite an effort to make it possible to have a local instance run with an sqlite db, and I could sprint productively on PyPI. > Possible improvements: > > - Have a PyPI component at bugs.python.org so all work on bugs/new > features would be known and > followed by at the same level than other packaging components we > maintain, and that depend on PyPI > (distutils, distutils2) -- e.g. drop the sourceforge tracker I wouldn't do that -- PyPI is not distributed with Python. (I'm equally skeptical about Distutils2, but it will at least be part of Python at some point in the future.) I would support a move to a separate bugs.python.org/pypi tracker, however. Not having to deal with SourceForge is still a good thing. > - Make it easier to contribute by moving the PyPI code base to > hg.python.org. Unlike Python, this is > a very simple move. +1 to that. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Mon Jul 26 22:40:13 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 26 Jul 2010 22:40:13 +0200 Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: References: <4C4CD849.9090108@v.loewis.de> Message-ID: Am 26.07.2010 14:41, schrieb Nick Coghlan: > On Mon, Jul 26, 2010 at 10:35 AM, "Martin v. L?wis" wrote: >> Am 26.07.2010 02:24, schrieb Terry Reedy: >>> To review a patch on the tracker, I have to read and try to make sense >>> of the raw diff file. Sometimes that is easy, sometimes not. >>> >>> *After* a patch is applied, I can click the revxxxx link and then the >>> 'text changed' link and see a nice, colored, side-by-side web-pace view >>> created by ViewVC. Is there any way a 'view' button could be added, >>> along with the current edit and remove buttons, to produce the same web >>> page and make it easier to review patches *before* commitment? >> >> You should be using Rietveld for that. > > And I believe there's already an item in the meta-tracker about > improving the integration between bugs.python.org and Rietveld - it > just doesn't exist yet. Sounds like something Ezio could easily do -- adapt Rietveld's upload.py to a Roundup extension that submits attachments as patches, adds people on nosy to Rietveld CC, &c. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Mon Jul 26 22:42:51 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 26 Jul 2010 22:42:51 +0200 Subject: [Python-Dev] Readability of hex strings (Was: Use of coding cookie in 3.x stdlib) In-Reply-To: References: Message-ID: Am 26.07.2010 20:42, schrieb Alexandre Vassalotti: > [+Python-ideas -Python-Dev] > > import binascii > def h(s): > return binascii.unhexlify("".join(s.split())) > > h("DE AD BE EF CA FE BA BE") In Py3k: h = bytes.fromhex Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Mon Jul 26 22:50:09 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 26 Jul 2010 22:50:09 +0200 Subject: [Python-Dev] Fwd: Readability of hex strings (Was: Use of coding cookie in 3.x stdlib) In-Reply-To: References: Message-ID: Am 26.07.2010 22:28, schrieb anatoly techtonik: > Don't you think it is a bad idea to require people to subscribe to > post to python-ideas of redirected there? It doesn't matter *why* you post to a mailing list -- you just have to be subscribed to post. Whoever redirects you there is usually not in the position of automatically subscribing anyone, and most people wouldn't much like being subscribed automatically, don't you think? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ralf at brainbot.com Mon Jul 26 22:53:16 2010 From: ralf at brainbot.com (Ralf Schmitt) Date: Mon, 26 Jul 2010 22:53:16 +0200 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <20100726150705.778ebf65@heresy> (Barry Warsaw's message of "Mon, 26 Jul 2010 15:07:05 -0400") References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> <63B36042-C4F7-4BCA-8623-5182405D8852@mac.com> <20100723120252.2e0690cc@snowdog> <25347613-8488-40C3-B9EF-9C85D0C47340@mac.com> <20100723125440.721860bf@snowdog> <764F1595-5C8B-44D8-86C5-937B326B2FCD@mac.com> <87mxti8lqc.fsf@muni.brainbot.com> <20100724002252.19d91267@snowdog> <87aapgbky5.fsf@brainbot.com> <20100726150705.778ebf65@heresy> Message-ID: <87ocdu3qyr.fsf@brainbot.com> Barry Warsaw writes: > > Do you have concrete examples? Without that it's just speculation I can't do > much to address. Is the problem big or small? Easy to work around or not? Some of the things that need to be adapted are e.g. Makefiles (basically anything that assumes modules have a certain name), all of the freezers (cxFreeze, py2exe, ...). The biggest problem probably will be that an import will load the wrong module or no module at all. I'm just speculating here... > "Change is bad" isn't a constructive argument. ;) Did I make that argument? > > That's fine, but it's not the way Debian/Ubuntu works today. PEP 3149 > adoption will definitely remove significant complication for deploying > multiple Python versions at the same time on those systems. You're just moving that complication into python. - Ralf From g.brandl at gmx.net Mon Jul 26 22:55:05 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 26 Jul 2010 22:55:05 +0200 Subject: [Python-Dev] Define a place for code review in Python workflow In-Reply-To: References: Message-ID: Am 26.07.2010 22:18, schrieb anatoly techtonik: > Small introduction for insiders not familiar with outsider's problem > of maintaining patches in tracker. Please forgive the tone I write > about things I dislike, but I am not devoting my life to earn a title > of polite bastard (this one is obligatory disclaimer I find it still > doesn't work for all people, so expect some irrelevant flame in > follow-ups). Nice! I wish we'd all do this. We'd have so much more time to do actual work. > As code comments are hard to keep offtopic, the signal to noise ration > appears to be is quite high. > The patch upload script is designed in a way that every project can > tune it for their needs and place into the root of source code. > "review" project at PyPI.is uploaded from source at > http://bitbucket.org/techtonik/pypi-rietveld - it can be tuned to > address needs specific for Python development. > > > What do you need to start using code review for contributed Python patches? > Why you wouldn't recommend this practice to outsiders? I'd welcome any patch submitted to Rietveld for review. However, your proposed "review.py" module does not exist as far as I know, and unless someone writes it, it won't. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ziade.tarek at gmail.com Mon Jul 26 23:03:42 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 26 Jul 2010 23:03:42 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On Mon, Jul 26, 2010 at 10:39 PM, Georg Brandl wrote: ... >> I think we need to improve this: it can be a very frustrating >> experience to contribute to PyPI. > > I did not experience it this way. ?On the contrary, I tried to run > PyPI locally for testing purposes, but didn't want to compile and > run Postgres, so we figured how hard it was to use Sqlite instead. > Martin put in quite an effort to make it possible to have a local > instance run with an sqlite db, and I could sprint productively on > PyPI. I wasn't talking about the technical gap. This is improving all the time (I've improved the quick wsgi launching script so we could use it without dependencies instance). I am talking about synchronization and communication e.g. the project management. For instance, You didn't know that I have started this patch, and Martin didn't know that you started on your side. :-) > >> Possible improvements: >> >> - Have a PyPI component at bugs.python.org so all work on bugs/new >> features would be known and >> ? followed by at the same level than other packaging components we >> maintain, and that depend on PyPI >> ? (distutils, distutils2) -- e.g. drop the sourceforge tracker > > I wouldn't do that -- PyPI is not distributed with Python. ?(I'm > equally skeptical about Distutils2, but it will at least be part > of Python at some point in the future.) > > I would support a move to a separate bugs.python.org/pypi tracker, > however. ?Not having to deal with SourceForge is still a good thing. My PoV is that PyPI is part of the Python ecosystem we provide. The other part is currently in Python (distutils) and can be impacted by changes. The checkbox for instance, is one atomic change across Python and PyPI, But another roundup issue would be a progress already. >> - Make it easier to contribute by moving the PyPI code base to >> hg.python.org. Unlike Python, this is >> ? a very simple move. > > +1 to that. From g.brandl at gmx.net Mon Jul 26 23:04:52 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 26 Jul 2010 23:04:52 +0200 Subject: [Python-Dev] Set the namespace free! In-Reply-To: References: <4C484FD0.2080803@zlotniki.pl> Message-ID: Am 26.07.2010 10:59, schrieb Anders Sandvig: > On Sat, Jul 24, 2010 at 3:31 AM, Gregory P. Smith wrote: >> Yuck. Anyone who feels they need a variable named the same a reserved word >> simply feels wrong and needs reeducation. [...] > > While I agree with you in principle, I have been finding it > frustrating trying to calculate yield in my financial applications > lately... ;) In the spirit of optimistic programming, why not assume a large one and call it Yield? ;) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Mon Jul 26 23:06:25 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 26 Jul 2010 23:06:25 +0200 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C36491C.2050300@mrabarnett.plus.com> <4C366EC2.3000203@mrabarnett.plus.com> Message-ID: Am 22.07.2010 12:53, schrieb Guido van Rossum: > On Fri, Jul 16, 2010 at 6:08 PM, Georg Brandl wrote: >> Nevertheless, the authoritative reference for our regex engine is its >> docs, i.e. http://docs.python.org/library/re.html -- and that states >> clearly that inline flags apply to the whole regex. >> >>> I think with a new regex implementation, not all of this "historical" >>> semantics must be copied, unless there are major real usecases, which >>> would be affected by this. >> >> As I already said, I *have* seen this in real code. As MRAB indicated, >> this was the only silent change in semantics as compared to the old >> regex engine. If we replace re by regex, which I think is the only >> way to get the new features in the stdlib, changing this one aspect is >> a) not backwards compatible and b) in a subtle way that forces everyone >> to review his/her regular expressions. That's definitely not >> acceptable. > > I wonder if the solution could be a new flag that you have to specify > if you want the flags to apply locally only. Let's say add 'L' to the > flags. Then for existing code (not using 'L') the flags should apply > globally, while someone wanting flags to apply to only part of a regex > can add 'L' to the flags syntax. > > Is that acceptable? I guess it would be; however it remains to be proven that this scoping is actually needed in addition to the (backwards compatible) grouped scoping. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Mon Jul 26 23:10:09 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 26 Jul 2010 23:10:09 +0200 Subject: [Python-Dev] :program: markup in .rst documentation In-Reply-To: References: Message-ID: Am 20.07.2010 19:32, schrieb Fred Drake: > Switching to "python -m" is generally good where it applies (as in this case). > > The original intent for :option: and :program: were in their use as > references rather than in sample command lines. > > Georg should be the final arbiter, but I'd be in favor of ``...`` for > command lines. Agreed. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Mon Jul 26 23:15:22 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 26 Jul 2010 23:15:22 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: Am 26.07.2010 23:03, schrieb Tarek Ziad?: > On Mon, Jul 26, 2010 at 10:39 PM, Georg Brandl wrote: > .... >>> I think we need to improve this: it can be a very frustrating >>> experience to contribute to PyPI. >> >> I did not experience it this way. On the contrary, I tried to run >> PyPI locally for testing purposes, but didn't want to compile and >> run Postgres, so we figured how hard it was to use Sqlite instead. >> Martin put in quite an effort to make it possible to have a local >> instance run with an sqlite db, and I could sprint productively on >> PyPI. > > I wasn't talking about the technical gap. This is improving all the time > (I've improved the quick wsgi launching script so we could use it > without dependencies instance). > > I am talking about synchronization and communication e.g. the project > management. > > For instance, You didn't know that I have started this patch, and > Martin didn't know > that you started on your side. :-) This isn't specific to PyPI though. I usually don't announce "I'm going to work on this issue" for a specific Python tracker item, except if I expect it to take more than a few hours. The patch in question is really pretty minor. >>> Possible improvements: >>> >>> - Have a PyPI component at bugs.python.org so all work on bugs/new >>> features would be known and >>> followed by at the same level than other packaging components we >>> maintain, and that depend on PyPI >>> (distutils, distutils2) -- e.g. drop the sourceforge tracker >> >> I wouldn't do that -- PyPI is not distributed with Python. (I'm >> equally skeptical about Distutils2, but it will at least be part >> of Python at some point in the future.) >> >> I would support a move to a separate bugs.python.org/pypi tracker, >> however. Not having to deal with SourceForge is still a good thing. > > My PoV is that PyPI is part of the Python ecosystem we provide. > The other part is currently in Python (distutils) and can be impacted > by changes. > The checkbox for instance, is one atomic change across Python and PyPI, Sure PyPI is part of the ecosystem. But so are quite a lot of other tools, and none of them are tracked in bugs.python.org. (This is also the case for the website.) I'd really like bugs.python.org to remain a tracker for what we ship as the CPython distribution, and nothing else. There's enough content in there already. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Mon Jul 26 23:09:12 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 26 Jul 2010 23:09:12 +0200 Subject: [Python-Dev] IDLE contributors and committers In-Reply-To: References: <20100717133325.5f204ee9@pitrou.net> <4C41985A.4030201@holdenweb.com> <4C41991A.9040103@voidspace.org.uk> Message-ID: Am 19.07.2010 04:22, schrieb Terry Reedy: > On 7/18/2010 7:42 PM, Georg Brandl wrote: > >> That phrasing implies that there is purpose behind letting issues rot. >> Believe me that this is not the case. > > This seems like a good place to mention that doc issues have become a > bright spot in the last few years, with a couple of days turnaround not > unusual for an issue ready for commit review. This is one reason I have > focused a bit on doc issues. It is rewarding. Because it's usually easier to review a doc patch -- you don't have to consider backwards compatibility implications :) > As I remember, a reason to switch from latex to sphinx/rst was so that > more people could/would work on doc issues. That seems to have worked. > And I do not think it bad is the docs catch up to the code by getting > extra attention. Yep, especially in Py3k where some changes were not really given enough docs love. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From martin at v.loewis.de Mon Jul 26 23:33:28 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 26 Jul 2010 23:33:28 +0200 Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: References: <4C4CD849.9090108@v.loewis.de> Message-ID: <4C4DFF28.1090205@v.loewis.de> > Sounds like something Ezio could easily do -- adapt Rietveld's upload.py > to a Roundup extension that submits attachments as patches, adds people > on nosy to Rietveld CC, &c. That may not be so easy - you'll have to authenticate to Rietveld from Roundup. The other way 'round actually works: if you put report at bugs.python.org into the reviewer list when uploading to Rietveld, Rietveld changes get automatically posted to the tracker. If you then also chose a sensible title (ie. one that includes [issueXXX]), Roundup will be able to associate it with the right issue. Regards, Martin From ziade.tarek at gmail.com Mon Jul 26 23:41:10 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 26 Jul 2010 23:41:10 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On Mon, Jul 26, 2010 at 11:15 PM, Georg Brandl wrote: .. > Sure PyPI is part of the ecosystem. ?But so are quite a lot of other tools, > and none of them are tracked in bugs.python.org. ?(This is also the case > for the website.) ?I'd really like bugs.python.org to remain a tracker for > what we ship as the CPython distribution, and nothing else. ?There's enough > content in there already. PyPI is the only tool that gets called from Python AFAIK. From martin at v.loewis.de Mon Jul 26 23:57:46 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 26 Jul 2010 23:57:46 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: <4C4E04DA.5000200@v.loewis.de> > Basically, I think what you'd like to have is Martin saying "I'm going to > work on this feature", in addition to "I implemented this feature now" > afterwards. That shouldn't be too hard. I'm not very good at blogging (more specifically, I never blog). People interested in following even the tinyest changes to PyPI should watch the commit list; it is unlikely that I will post each and every change to catalog-sig. People are then free to blog about the changes as they please (and I would appreciate any help I can get in announcing changes). Regards, Martin From pydev at rebertia.com Tue Jul 27 01:16:31 2010 From: pydev at rebertia.com (Chris Rebert) Date: Mon, 26 Jul 2010 16:16:31 -0700 Subject: [Python-Dev] marking os.system() as deprecated in the docs In-Reply-To: References: Message-ID: > On Sun, Jul 25, 2010 at 11:42 PM, Chris Rebert wrote: >> Hello, >> >> I'd like to propose formally marking os.system() as deprecated in the >> docs for next release of Python (v3.2 ?). >> >> The docs for os.system() /already/ include the following paragraph, >> which is basically tantamount to calling system() deprecated and very >> much resembles the deprecation notes for the os.popen*() family. >> """ >> The subprocess module provides more powerful facilities for spawning >> new processes and retrieving their results; using that module is >> preferable to using this function. Use the subprocess module. Check >> especially the Replacing Older Functions with the subprocess Module >> section. >> """ Emphasizing the following paragraph from my original email: >> I'm merely suggesting slapping a formal ".. deprecated::" tag & box >> around this paragraph to make system()'s status more obvious. I am >> ***not*** suggesting the function be scheduled for removal or anything >> like that. >> >> I figure since this would be a substantive change (the docs currently >> don't outright say "deprecated"), it would need dev approval, hence >> this message. I've already written the absolutely /trivial/ docs >> patch. >> >> So, any objections? On Mon, Jul 26, 2010 at 7:29 AM, Guido van Rossum wrote: > Unless, it's as simple as replacing "os.system(x)" with > "subprocess.system(x)", I'm against this removal of a handy shorthand. [Addressing Steven's reply as well] In hindsight, I should have used a word other than "deprecate" (despite no longer matching the Sphinx directive) that's not overloaded with Python baggage about removal; I intended it here in its plain English meaning of merely "to express disapproval of" or "to play down". Allow me to reiterate: I am *NOT* suggesting os.system() be removed or prepared for removal. I'm only suggesting the docs be /slightly/ tweaked to make it /slightly/ more obvious that os.system() is dispreferred over the subprocess module. Removing os.system() would have backward compatibility and cross-language API problems; I agree it would be a bad idea; which is why I'm not proposing that. > Ditto for popen. Funny you should bring up os.popen(). Its docs are currently MIA in 3.x, for unclear reasons: http://bugs.python.org/issue9382 Cheers, Chris > -- > --Guido van Rossum (python.org/~guido) From ziade.tarek at gmail.com Tue Jul 27 01:18:17 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 27 Jul 2010 01:18:17 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: <4C4E04DA.5000200@v.loewis.de> References: <4C4E04DA.5000200@v.loewis.de> Message-ID: On Mon, Jul 26, 2010 at 11:57 PM, "Martin v. L?wis" wrote: >> Basically, I think what you'd like to have is Martin saying "I'm going to >> work on this feature", in addition to "I implemented this feature now" >> afterwards. ?That shouldn't be too hard. > > I'm not very good at blogging (more specifically, I never blog). > > People interested in following even the tinyest changes to PyPI should > watch the commit list; it is unlikely that I will post each and every > change to catalog-sig. > > People are then free to blog about the changes as they please (and I > would appreciate any help I can get in announcing changes). I think what would be good is to have you discussing the "I'm going to work on this feature" at the ML before it's started. And not discovering it randomly at PyPI or the commit list, leading to a heated discussion or a poll. I would classify the changes in three kinds: - minor: a new feature, a UI bugfix etc - important: a new feature that changes a lot the end-user experience (like the rating system) - major: a change to the APIs (HTTP/XML-RPC) I think you should briefly present your plans for important or major changes in catalog-SIG prior to starting them, so we can discuss them. Regards Tarek -- Tarek Ziad? | http://ziade.org From steve at pearwood.info Tue Jul 27 02:12:43 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 27 Jul 2010 10:12:43 +1000 Subject: [Python-Dev] Readability of hex strings (Was: Use of coding cookie in 3.x stdlib) In-Reply-To: References: Message-ID: <201007271012.44460.steve@pearwood.info> On Tue, 27 Jul 2010 04:29:31 am anatoly techtonik wrote: > I find "\xXX\xXX\xXX\xXX..." notation for binary data totally > unreadable. Everybody who uses and analyses binary data is more > familiar with plain hex dumps in the form of "XX XX XX XX...". > > I wonder if it is possible to introduce an effective binary string > type that will be represented as h"XX XX XX" in language syntax? It > will be much easier to analyze printed binary data and copy/paste > such data as-is from hex editors/views. With the moratorium on new language features, this would not even be considered until Python 3.3. If you are serious in pursuing this idea, it is off-topic for this list and should be taken to python-ideas, or even python-list for community feedback, first. Since it only takes a pair of small helper functions to convert hex dumps in the form "XXXX XXXX ..." to and from byte strings, I don't see the need for new syntax and would vote -1 on the idea. However, I'd vote +0 on a matching bytes.tohex() method to partner with the existing bytes.fromhex(). -- Steven D'Aprano From kevinar18 at hotmail.com Tue Jul 27 02:20:35 2010 From: kevinar18 at hotmail.com (Kevin Ar18) Date: Mon, 26 Jul 2010 20:20:35 -0400 Subject: [Python-Dev] Do any of the 4 multiprocessing methods work using shared memory? Message-ID: Brief Summary: Can I share Python objects between multiple processes using shared memory (using the multiprocessing module)? In particular, do Queues or Pipes work using shared memory? Details: * I have several processes each on a separate CPU core (so they run in parallel). * I want to share certain Python objects between the processes using shared memory (I can handle the locking myself -- only one process can access a variable or object at a time, so there is no conflict anyways). * I strongly prefer that it be shared memory because I do not want to incur the costs of having to copy data back and forth between the processes. I am aware of the multiprocessing module.? It offers: Queues Pipes Shared Memory Map Server Process So, let me ask a few questions about those 4 items: Queues & Pipes These handle Python objects and variables....? However, my question is do they use shared memory or do they require copying the data or some other method that is more costly than shared memory? Shared Memory Map Does not support Python objects or variables -- thus no good to me. Server Process Looks like it requires another process that incurs extra processing just to handle the data exchange -- which, if true, is bad for me. Does this use shared memory?? or does it require copying data back and forth between processes? _________________________________________________________________ The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4 From turnbull at sk.tsukuba.ac.jp Tue Jul 27 03:40:30 2010 From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull) Date: Tue, 27 Jul 2010 10:40:30 +0900 Subject: [Python-Dev] Fwd: Readability of hex strings (Was: Use of coding cookie in 3.x stdlib) In-Reply-To: References: Message-ID: <871vapwvld.fsf@uwakimon.sk.tsukuba.ac.jp> anatoly techtonik writes: > Don't you think it is a bad idea to require people to subscribe to > post to python-ideas of redirected there? No, it's an excellent compromise in the current environment. On lists I manage (with a similar audience to python-dev), my MTA-level filters get 95% of the spam, the member test catches essentially all of the rest. On support lists, I moderate, mostly because drive-by posters usually post in HTML which is another 95% accurate signal of spam. On the dev lists, they get rejected on the principle that drive-by posts to a dev list *are* spam. If you want to post a quick RFE, that's what the tracker is for; people will see your brainstorm when they get around to looking at the tracker, but it just isn't urgent at all. This works well, yours is the first complaint about this kind of policy that I've seen in four or five years. From stephen at xemacs.org Tue Jul 27 03:43:22 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 27 Jul 2010 10:43:22 +0900 Subject: [Python-Dev] Set the namespace free! In-Reply-To: References: <4C484FD0.2080803@zlotniki.pl> Message-ID: <87zkxdvgw5.fsf@uwakimon.sk.tsukuba.ac.jp> Georg Brandl writes: > Am 26.07.2010 10:59, schrieb Anders Sandvig: > > On Sat, Jul 24, 2010 at 3:31 AM, Gregory P. Smith wrote: > >> Yuck. Anyone who feels they need a variable named the same a reserved word > >> simply feels wrong and needs reeducation. [...] > > > > While I agree with you in principle, I have been finding it > > frustrating trying to calculate yield in my financial applications > > lately... ;) > > In the spirit of optimistic programming, why not assume a large one > and call it Yield? ;) That's certainly more workable than the obvious near-synonym, "return". From tjreedy at udel.edu Tue Jul 27 04:14:13 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 26 Jul 2010 22:14:13 -0400 Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: <4C4CD849.9090108@v.loewis.de> References: <4C4CD849.9090108@v.loewis.de> Message-ID: On 7/25/2010 8:35 PM, "Martin v. L?wis" wrote: > Am 26.07.2010 02:24, schrieb Terry Reedy: >> To review a patch on the tracker, I have to read and try to make sense >> of the raw diff file. Sometimes that is easy, sometimes not. >> >> *After* a patch is applied, I can click the revxxxx link and then the >> 'text changed' link and see a nice, colored, side-by-side web-pace view >> created by ViewVC. Is there any way a 'view' button could be added, >> along with the current edit and remove buttons, to produce the same web >> page and make it easier to review patches *before* commitment? > > You should be using Rietveld for that. There is no mention of Rietveld in the tracker docs that I could fine. Did I miss something? That fact that one needs to go to http://codereview.appspot.com/ rather than the rietveld page was not immediately obvious from 'use Rietveld'. > That may not be so easy - you'll have to authenticate to Rietveld from > Roundup. This is why I suggested ViewVC -- it already is at python.org and could potentially be directly accessed with a button. > The other way 'round actually works: if you put report at bugs.python.org > into the reviewer list when uploading to Rietveld, Rietveld changes > get automatically posted to the tracker. If you then also chose a > sensible title (ie. one that includes [issueXXX]), Roundup will be able > to associate it with the right issue. These are less obvious. Here is what I have gathered so far. Open patch in browser tab to get its url. Go to http://codereview.appspot.com/ Login in with Google account Go to Create issue form Copy and paste url into url field To get comments mailed back to tracker, put Issuexxxx in title and put report at bugs.python.org in reviewer list Should I open a tracker issue to add something to the tracker doc? -- Terry Jan Reedy From tjreedy at udel.edu Tue Jul 27 04:43:37 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 26 Jul 2010 22:43:37 -0400 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On 7/26/2010 5:15 PM, Georg Brandl wrote: > Sure PyPI is part of the ecosystem. But so are quite a lot of other tools, > and none of them are tracked in bugs.python.org. (This is also the case > for the website.) I'd really like bugs.python.org to remain a tracker for > what we ship as the CPython distribution, and nothing else. There's enough > content in there already. How about one other tracker, say bugs.python/org/tools (or projects, or ???) for everything else: pypi, distribute2 (until part of release), web site, sandbox projects? It would have to be taught how to turn revxxxx + component into a link to the appropriate repository. -- Terry Jan Reedy From tjreedy at udel.edu Tue Jul 27 04:51:36 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 26 Jul 2010 22:51:36 -0400 Subject: [Python-Dev] Do any of the 4 multiprocessing methods work using shared memory? In-Reply-To: References: Message-ID: On 7/26/2010 8:20 PM, Kevin Ar18 wrote: Questions about the operation and use current python belong on python-list, mirrored on gmane.comp.python.general. This list pydev == g.c.p.devel is for the development of future versions. -- Terry Jan Reedy From rrr at ronadam.com Tue Jul 27 06:51:14 2010 From: rrr at ronadam.com (Ron Adam) Date: Mon, 26 Jul 2010 23:51:14 -0500 Subject: [Python-Dev] [isssue 2001] Pydoc enhancement patch questions In-Reply-To: References: Message-ID: On 07/24/2010 10:44 PM, Nick Coghlan wrote: > To request automatic assignment of a local port number, "-p 0" could > be made to work correctly (i.e. print out the actual port the OS > assigned rather than the 0 that was passed in on the command line as > it does currently). I was able to implement this without too much trouble. I also changed it so -b and -p switches can be used together to override the automatic port selection. The default is port 0. I think idle uses automatic port selection now also. Does it have a way to select a specific port? Is the -p option needed with automatic port selection. I added a command prompt like the help> prompt. Here's what starting the server, opening a browser, and then stopping the server looks like. ra at Gutsy:/media/hda2/home/ra/svn/py3k$ ./python -m pydoc -p 0 Server ready at: http://localhost:50279/ Server commands: [b]rowser, [q]uit server>b server>q Server stopped [83680 refs] ra at Gutsy:/media/hda2/home/ra/svn/py3k$ I can add a [h]elp to get a longer help, but I'm not sure it's needed. I want to have a [r]eport command to list the server requests since the last report command was given. Sort of like a log. It may be useful to find problems with what the server is sending and what the browser is requesting. Should the server have a timeout in case the command window is not reachable? If so, how long? cheers, Ron From martin at v.loewis.de Tue Jul 27 07:37:01 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 27 Jul 2010 07:37:01 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: <4C4E04DA.5000200@v.loewis.de> Message-ID: <4C4E707D.9080808@v.loewis.de> > I would classify the changes in three kinds: > > - minor: a new feature, a UI bugfix etc > - important: a new feature that changes a lot the end-user experience > (like the rating system) > - major: a change to the APIs (HTTP/XML-RPC) > > I think you should briefly present your plans for important or major > changes in catalog-SIG > prior to starting them, so we can discuss them. So would you consider the addition of JSON a major feature (as it introduces a new API)? I doubt Richard would have been willing to wait for the end of some discussion before implementing it. Regards, Martin From greg at krypto.org Tue Jul 27 07:40:30 2010 From: greg at krypto.org (Gregory P. Smith) Date: Mon, 26 Jul 2010 22:40:30 -0700 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> <20100713163544.5f3a60d2@pitrou.net> Message-ID: On Thu, Jul 22, 2010 at 3:26 PM, Nick Coghlan wrote: > On Fri, Jul 23, 2010 at 12:42 AM, Georg Brandl wrote: > > Sure -- I don't think this is a showstopper for regex. However if we > don't > > include regex in a future version, we might think about increasing > MAXCACHE > > a bit, and maybe not clear the cache when it reaches its max length, but > > rather remove another element. > > Yikes, I didn't know it did that. That certainly sounds like it should > be an RFE in its own right - some basic form of Least Recently Used > accounting should be beneficial (although the extra bookkeeping might > hurt scripts that aren't hitting the cache limit). > > A max cache size of 100 was too small. I just increased it to 500 in the py3k branch along with implementing a random replacement cache overflow policy. It now randomly drops 20% of the compiled regular expression cache instead of simply dropping the entire cache on overflow. With the regex_v8 benchmark, the better cache replacement policy sped it up ~7% while raising the cache size on top of that (likely meaning the cache was never overflowing) sped it up ~25%. Random replacement without dropping everything at least means apps thrashing the cache degrade much more gracefully. http://svn.python.org/view?view=rev&revision=83173 This change should be incorporated into MRAB's regex module in order to keep comparisons fair. -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Tue Jul 27 07:42:23 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 27 Jul 2010 07:42:23 +0200 Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: References: <4C4CD849.9090108@v.loewis.de> Message-ID: <4C4E71BF.7060109@v.loewis.de> > Should I open a tracker issue to add something to the tracker doc? I recommend that you use it for some time before changing anything. I also suggest that, instead of uploading the patch to Rietveld yourself, you can ask the submitter to do it. Regards, Martin From greg.ewing at canterbury.ac.nz Tue Jul 27 08:35:45 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 27 Jul 2010 18:35:45 +1200 Subject: [Python-Dev] marking os.system() as deprecated in the docs In-Reply-To: References: Message-ID: <4C4E7E41.7070301@canterbury.ac.nz> Guido van Rossum wrote: > Unless, it's as simple as replacing "os.system(x)" with > "subprocess.system(x)", I'm against this removal of a handy shorthand. Also, seeing as it's a well-known C library facility, and Python likes to provide thin wrappers around the C library wherever reasonable, its lack would seem a bit odd. -- Greg From stefan_ml at behnel.de Tue Jul 27 08:27:35 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 27 Jul 2010 08:27:35 +0200 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> <20100713163544.5f3a60d2@pitrou.net> Message-ID: Gregory P. Smith, 27.07.2010 07:40: > A max cache size of 100 was too small. I just increased it to 500 in the > py3k branch along with implementing a random replacement cache overflow > policy. It now randomly drops 20% of the compiled regular expression cache > instead of simply dropping the entire cache on overflow. > > With the regex_v8 benchmark, the better cache replacement policy sped it up > ~7% while raising the cache size on top of that (likely meaning the cache > was never overflowing) sped it up ~25%. > > Random replacement without dropping everything at least means apps thrashing > the cache degrade much more gracefully. The same algorithm should be helpful in ElementTree's ElementPath module. Stefan From greg.ewing at canterbury.ac.nz Tue Jul 27 09:36:01 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 27 Jul 2010 19:36:01 +1200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: <4C4E8C61.1050205@canterbury.ac.nz> Terry Reedy wrote: > Should CPython be optimized for 1, 2, 3, or 4 or more cores? > The answer to this is obviously changing. I will soon replace a single > core with a 4/6 core machine, I don't think you can answer that just by considering the average number of cores in a CPU. Even if my CPU has 4 cores, most of the time the Python code I run on it isn't going to take advantage of more than one of them, simply because it's not written to be multi-threaded. I would not like to be in a position where I *have* to use some number of cores in order to get reasonable performance from my Python code. -- Greg From greg.ewing at canterbury.ac.nz Tue Jul 27 09:45:25 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 27 Jul 2010 19:45:25 +1200 Subject: [Python-Dev] Readability of hex strings (Was: Use of coding cookie in 3.x stdlib) In-Reply-To: References: Message-ID: <4C4E8E95.9080707@canterbury.ac.nz> anatoly techtonik wrote: > I wonder if it is possible to introduce an effective binary string > type that will be represented as h"XX XX XX" in language syntax? Rather than a new type, maybe bytes objects could just have a bit indicating whether they were best thought of as containing characterish stuff or just raw data. It wouldn't affect the behaviour in any way except that the repr would come out in hex instead of text. Then b"..." and h"..." literals could produce bytes objects with different settings for the raw-data bit. -- Greg From g.brandl at gmx.net Tue Jul 27 09:57:22 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 27 Jul 2010 09:57:22 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: Am 27.07.2010 04:43, schrieb Terry Reedy: > On 7/26/2010 5:15 PM, Georg Brandl wrote: > >> Sure PyPI is part of the ecosystem. But so are quite a lot of other tools, >> and none of them are tracked in bugs.python.org. (This is also the case >> for the website.) I'd really like bugs.python.org to remain a tracker for >> what we ship as the CPython distribution, and nothing else. There's enough >> content in there already. > > How about one other tracker, say bugs.python/org/tools (or projects, or > ???) for everything else: pypi, distribute2 (until part of release), web > site, sandbox projects? It would have to be taught how to turn revxxxx + > component into a link to the appropriate repository. I still think that one tracker per project/site is the better way. However, I would be +0 on a common "infrastructure" tracker (also subsuming the meta-tracker). Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ziade.tarek at gmail.com Tue Jul 27 10:22:57 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 27 Jul 2010 10:22:57 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: <4C4E707D.9080808@v.loewis.de> References: <4C4E04DA.5000200@v.loewis.de> <4C4E707D.9080808@v.loewis.de> Message-ID: 2010/7/27 "Martin v. L?wis" : >> I would classify the changes in three kinds: >> >> - minor: a new feature, a UI bugfix etc >> - important: a new feature that changes a lot the end-user experience >> (like the rating system) >> - major: a change to the APIs (HTTP/XML-RPC) >> >> I think you should briefly present your plans for important or major >> changes in catalog-SIG >> prior to starting them, so we can discuss them. > > So would you consider the addition of JSON a major feature (as it > introduces a new API)? I doubt Richard would have been willing to wait > for the end of some discussion before implementing it. That's "minor" since it's a new feature that does not interfere with existing features -- I hope it doesn't :) "major" is a change to existing APIs that could potentially break existing software. PyPI is not different from other software in that respect, e.g. we need to be careful with the changes we make and push in production. But, if Richard sprints again and change the JSON output --let's say it use to return a mapping, but now it returns a list with a timestamp as the first member and the original mapping as the second member--, he should first explain that change to the ML so people that potentially uses the JSON version can be aware of that. In case of doubt, a change to an existing piece should be mentioned. Ideally, as I said in a previous mail, we should document in a single place (a PEP I guess) the PyPI specification, and maybe version it. Regards, Tarek -- Tarek Ziad? | http://ziade.org From david at deadpansincerity.com Tue Jul 27 10:54:26 2010 From: david at deadpansincerity.com (David) Date: Tue, 27 Jul 2010 09:54:26 +0100 Subject: [Python-Dev] Define a place for code review in Python workflow In-Reply-To: References: Message-ID: > > I'd welcome any patch submitted to Rietveld for review. However, your > proposed "review.py" module does not exist as far as I know, and unless > someone writes it, it won't. > Haven't personally tested that it works with Rietveld due to lack of patches sitting around, but cursory investigation suggests that reports of non-existence may have been exaggerated ;) http://pypi.python.org/pypi/review/r537 Love regards etc David Miller -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Tue Jul 27 12:02:10 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 27 Jul 2010 12:02:10 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython References: Message-ID: <20100727120210.35ea7bb2@pitrou.net> On Tue, 27 Jul 2010 09:57:22 +0200 Georg Brandl wrote: > Am 27.07.2010 04:43, schrieb Terry Reedy: > > On 7/26/2010 5:15 PM, Georg Brandl wrote: > > > >> Sure PyPI is part of the ecosystem. But so are quite a lot of other tools, > >> and none of them are tracked in bugs.python.org. (This is also the case > >> for the website.) I'd really like bugs.python.org to remain a tracker for > >> what we ship as the CPython distribution, and nothing else. There's enough > >> content in there already. > > > > How about one other tracker, say bugs.python/org/tools (or projects, or > > ???) for everything else: pypi, distribute2 (until part of release), web > > site, sandbox projects? It would have to be taught how to turn revxxxx + > > component into a link to the appropriate repository. > > I still think that one tracker per project/site is the better way. Only if they have similar look and feel, and don't require you to register the same login N times, though. Regards Antoine. From steve at holdenweb.com Tue Jul 27 12:49:40 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 27 Jul 2010 11:49:40 +0100 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: <20100727120210.35ea7bb2@pitrou.net> References: <20100727120210.35ea7bb2@pitrou.net> Message-ID: <4C4EB9C4.5060104@holdenweb.com> On 7/27/2010 11:02 AM, Antoine Pitrou wrote: > On Tue, 27 Jul 2010 09:57:22 +0200 > Georg Brandl wrote: > >> Am 27.07.2010 04:43, schrieb Terry Reedy: >>> On 7/26/2010 5:15 PM, Georg Brandl wrote: >>> >>>> Sure PyPI is part of the ecosystem. But so are quite a lot of other tools, >>>> and none of them are tracked in bugs.python.org. (This is also the case >>>> for the website.) I'd really like bugs.python.org to remain a tracker for >>>> what we ship as the CPython distribution, and nothing else. There's enough >>>> content in there already. >>> >>> How about one other tracker, say bugs.python/org/tools (or projects, or >>> ???) for everything else: pypi, distribute2 (until part of release), web >>> site, sandbox projects? It would have to be taught how to turn revxxxx + >>> component into a link to the appropriate repository. >> >> I still think that one tracker per project/site is the better way. > > Only if they have similar look and feel, and don't require you to > register the same login N times, though. > Is it really time to give devs a distributed identity good for a range of systems? Sounds like a potentially hairy management task. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Tue Jul 27 12:49:40 2010 From: steve at holdenweb.com (Steve Holden) Date: Tue, 27 Jul 2010 11:49:40 +0100 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: <20100727120210.35ea7bb2@pitrou.net> References: <20100727120210.35ea7bb2@pitrou.net> Message-ID: <4C4EB9C4.5060104@holdenweb.com> On 7/27/2010 11:02 AM, Antoine Pitrou wrote: > On Tue, 27 Jul 2010 09:57:22 +0200 > Georg Brandl wrote: > >> Am 27.07.2010 04:43, schrieb Terry Reedy: >>> On 7/26/2010 5:15 PM, Georg Brandl wrote: >>> >>>> Sure PyPI is part of the ecosystem. But so are quite a lot of other tools, >>>> and none of them are tracked in bugs.python.org. (This is also the case >>>> for the website.) I'd really like bugs.python.org to remain a tracker for >>>> what we ship as the CPython distribution, and nothing else. There's enough >>>> content in there already. >>> >>> How about one other tracker, say bugs.python/org/tools (or projects, or >>> ???) for everything else: pypi, distribute2 (until part of release), web >>> site, sandbox projects? It would have to be taught how to turn revxxxx + >>> component into a link to the appropriate repository. >> >> I still think that one tracker per project/site is the better way. > > Only if they have similar look and feel, and don't require you to > register the same login N times, though. > Is it really time to give devs a distributed identity good for a range of systems? Sounds like a potentially hairy management task. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 DjangoCon US September 7-9, 2010 http://djangocon.us/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ From doko at ubuntu.com Tue Jul 27 12:58:46 2010 From: doko at ubuntu.com (Matthias Klose) Date: Tue, 27 Jul 2010 12:58:46 +0200 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <87ocdu3qyr.fsf@brainbot.com> References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> <63B36042-C4F7-4BCA-8623-5182405D8852@mac.com> <20100723120252.2e0690cc@snowdog> <25347613-8488-40C3-B9EF-9C85D0C47340@mac.com> <20100723125440.721860bf@snowdog> <764F1595-5C8B-44D8-86C5-937B326B2FCD@mac.com> <87mxti8lqc.fsf@muni.brainbot.com> <20100724002252.19d91267@snowdog> <87aapgbky5.fsf@brainbot.com> <20100726150705.778ebf65@heresy> <87ocdu3qyr.fsf@brainbot.com> Message-ID: <4C4EBBE6.6080603@ubuntu.com> On 26.07.2010 22:53, Ralf Schmitt wrote: > Barry Warsaw writes: >> That's fine, but it's not the way Debian/Ubuntu works today. PEP 3149 >> adoption will definitely remove significant complication for deploying >> multiple Python versions at the same time on those systems. > > You're just moving that complication into python. There is nothing which prevents you to still deploy/use python modules in separate directories, and if you see a python package as a directory, nothing will change for you with this PEP besides the naming of the extensions. > I'd much prefer to have cleanly separated environments by having > separate directories for my python modules. That is your preference, but not what standards like the FHS talk about (i.e. having different locations for data, docs, headers). > Sharing the source code and > complicating things will not lead to increased robustness. Not true. Package managers like dpkg/apt-get, rpm/yum and maybe others do this for ages. And yes, the added "complexity" of package managers does lead to increased robustness. Matthias From g.brandl at gmx.net Tue Jul 27 13:10:31 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 27 Jul 2010 13:10:31 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: <4C4EB9C4.5060104@holdenweb.com> References: <20100727120210.35ea7bb2@pitrou.net> <4C4EB9C4.5060104@holdenweb.com> Message-ID: Am 27.07.2010 12:49, schrieb Steve Holden: > On 7/27/2010 11:02 AM, Antoine Pitrou wrote: >> On Tue, 27 Jul 2010 09:57:22 +0200 >> Georg Brandl wrote: >> >>> Am 27.07.2010 04:43, schrieb Terry Reedy: >>>> On 7/26/2010 5:15 PM, Georg Brandl wrote: >>>> >>>>> Sure PyPI is part of the ecosystem. But so are quite a lot of other tools, >>>>> and none of them are tracked in bugs.python.org. (This is also the case >>>>> for the website.) I'd really like bugs.python.org to remain a tracker for >>>>> what we ship as the CPython distribution, and nothing else. There's enough >>>>> content in there already. >>>> >>>> How about one other tracker, say bugs.python/org/tools (or projects, or >>>> ???) for everything else: pypi, distribute2 (until part of release), web >>>> site, sandbox projects? It would have to be taught how to turn revxxxx + >>>> component into a link to the appropriate repository. >>> >>> I still think that one tracker per project/site is the better way. >> >> Only if they have similar look and feel, and don't require you to >> register the same login N times, though. >> > Is it really time to give devs a distributed identity good for a range > of systems? Sounds like a potentially hairy management task. IMO supporting OpenID is good enough. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Tue Jul 27 13:11:48 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 27 Jul 2010 13:11:48 +0200 Subject: [Python-Dev] Define a place for code review in Python workflow In-Reply-To: References: Message-ID: Am 27.07.2010 10:54, schrieb David: > I'd welcome any patch submitted to Rietveld for review. However, your > > proposed "review.py" module does not exist as far as I know, and unless > someone writes it, it won't. > > > Haven't personally tested that it works with Rietveld due to lack of patches > sitting around, but cursory investigation suggests that reports of non-existence > may have been exaggerated ;) > > http://pypi.python.org/pypi/review/r537 Ah! Well, a link to this instead of tirades would have been helpful from the OP; now at least we know what he's talking about. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From stephen at xemacs.org Tue Jul 27 13:34:24 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 27 Jul 2010 20:34:24 +0900 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: <4C4EB9C4.5060104@holdenweb.com> References: <20100727120210.35ea7bb2@pitrou.net> <4C4EB9C4.5060104@holdenweb.com> Message-ID: <87pqy9upj3.fsf@uwakimon.sk.tsukuba.ac.jp> Steve Holden writes: > > Only if they have similar look and feel, and don't require you to > > register the same login N times, though. > > > Is it really time to give devs a distributed identity good for a range > of systems? Sounds like a potentially hairy management task. Sure, but Python can leave management up to Google, Yahoo, and several other well-known providers. This is what "OpenId" is all about. From ralf at brainbot.com Tue Jul 27 13:54:56 2010 From: ralf at brainbot.com (Ralf Schmitt) Date: Tue, 27 Jul 2010 13:54:56 +0200 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <4C4EBBE6.6080603@ubuntu.com> (Matthias Klose's message of "Tue, 27 Jul 2010 12:58:46 +0200") References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> <63B36042-C4F7-4BCA-8623-5182405D8852@mac.com> <20100723120252.2e0690cc@snowdog> <25347613-8488-40C3-B9EF-9C85D0C47340@mac.com> <20100723125440.721860bf@snowdog> <764F1595-5C8B-44D8-86C5-937B326B2FCD@mac.com> <87mxti8lqc.fsf@muni.brainbot.com> <20100724002252.19d91267@snowdog> <87aapgbky5.fsf@brainbot.com> <20100726150705.778ebf65@heresy> <87ocdu3qyr.fsf@brainbot.com> <4C4EBBE6.6080603@ubuntu.com> Message-ID: <877hkhkulr.fsf@muni.brainbot.com> Matthias Klose writes: > Not true. Package managers like dpkg/apt-get, rpm/yum and maybe others > do this for ages. And yes, the added "complexity" of package managers > does lead to increased robustness. but how does sharing things lead to increased robustness (even if it might be managed by your package manager)? From lvh at laurensvh.be Tue Jul 27 14:00:26 2010 From: lvh at laurensvh.be (Laurens Van Houtven) Date: Tue, 27 Jul 2010 14:00:26 +0200 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: <4C4D5CA5.7020809@voidspace.org.uk> References: <4C4D5CA5.7020809@voidspace.org.uk> Message-ID: On Mon, Jul 26, 2010 at 12:00 PM, Michael Foord wrote: > At Resolver Systems we created a "calculation system" that does large > calculations on background threads using IronPython. Doing them on a > background thread allows the ui to remain responsive. Several calculations > could run simultaneously using multiple cores. > > As the calculation operates on a large object graph (which the ui then needs > access to in order to display it) using multiprocessing would have imposed a > very big overhead due to serialization / deserialization (the program runs > on windows). [...] > All the best, > > Michael Hey, (De)serialization being a much bigger cost than cache invalidation for a low amount of threads that do a lot of work each is definitely a common "problem" (in quotes, because as you mentioned: it actually *works*!). There are a number of ways that CSP tries to solve that (generally involving more locking), but they are not currently applicable to CPython because of the state of the GIL. Unfortunately, CSP theory appears to predict this is something that starts breaking down around 16 or so cores. Since x86-64 CPUs (Opterons) are currently available with 12 cores and their 16 core bigger brother coming in 2011, I guess now would be a good time to start worrying about it :-) I'd like to chime in from my experience with E, because they've ran into this problem (processors want many processes to perform, but (de)serialization makes that prohibitive) and tried to solve it (and I think they did well). As always when I talk about E, I'm not suggesting everyone drops everything and does this, but it might be interesting to look at. (Disclaimer: the following explanations makes minor concessions to pedant-proof levels of purity in the interest of giving everyone an idea of what something is that's correct enough to reason about it on an abstract level -- people who are interested, please read the Wikipedia bits, they're surprisingly good :-)) E introduces a concept called "vats". They have an event queue, their own stack and N objects. Vats run on top of real processes, which have 0..N vats. The advantage is that vats don't share namespaces but can (but don't necessarily do) share memoryspaces. So, messaging between vats *can* be cheap (I'm unfamiliar with threading under .NET, but if it's similar to how it happens on the JVM: same ballpark), but the vat is completely oblivious to the fact it's running on the same process as a different vat or a completely different one running on a CPU on the other side of the world. Because inter-vat message passing is explicit, these vats can also run in parallel with no issues. The simplest way to implement this would be a vat-local GIL (I realise the name GIL no longer makes sense there) for each vat, and then the process (most likely written in C(ython)) and the objects inside each vat contesting it. Or, in closing, but less exciting sounding: we've reinvented threads and they're called vats now! (The advantage is that you get the distributed nature, and only pay for it when you actually need it.) Computers are reasonably good at this sort of scheduling (putting the appropriate vats together), but it wouldn't be unthinkable to have the programmer hint at it. You just have to be careful not to take it too far and get into gcc realm, where higher levels of optimization include things like "ignore programmer hints". Caveat emptor: E has always cared much more about capabilities (so the security aspect) than parallel execution. thanks for reading Laurens From rdmurray at bitdance.com Tue Jul 27 14:05:40 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 27 Jul 2010 08:05:40 -0400 Subject: [Python-Dev] Define a place for code review in Python workflow In-Reply-To: References: Message-ID: <20100727120540.8AE0D21CC19@kimball.webabinitio.net> On Tue, 27 Jul 2010 13:11:48 +0200, Georg Brandl wrote: > Am 27.07.2010 10:54, schrieb David: > > I'd welcome any patch submitted to Rietveld for review. However, your > > proposed "review.py" module does not exist as far as I know, and unless > > someone writes it, it won't. > > > > Haven't personally tested that it works with Rietveld due to lack of patches > > sitting around, but cursory investigation suggests that reports of non-existence > > may have been exaggerated ;) > > > > http://pypi.python.org/pypi/review/r537 > > Ah! Well, a link to this instead of tirades would have been helpful from > the OP; now at least we know what he's talking about. Indeed. If Anatoly had said, "Hey, there's this cool tool called 'review.py' on PyPI, and I think it would improve the tracker workflow if it was used something like this....what do other people think?" *That* kind of post would have gotten a completely different, and most likely more constructive, response. Anatoly, you might want to think about the fact that at this point I suspect most people's reactions to anything you post tend to be along the lines of "oh, another Anatoly rant" and either ignore it or look immediately for what's *wrong* with what you post, instead of seeing the good suggestions that are occasionally hidden inside your rants. In other words, your negative attitude toward us results in us taking a negative attitude toward you. If you engaged positively with the community instead of negatively, you'd have a much better chance of your ideas getting a positive reception. As an author I respect a lot once said, politeness is the grease that keeps the gears of society working smoothly. Your not "wasting time" being a "polite bastard" is the equivalent of deliberately throwing sand in the gears. It is, to say the least, counter-productive to your stated goals of improving the Python workflow for yourself and others. -- R. David Murray www.bitdance.com From tjreedy at udel.edu Tue Jul 27 16:56:32 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 27 Jul 2010 10:56:32 -0400 Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: <4C4E71BF.7060109@v.loewis.de> References: <4C4CD849.9090108@v.loewis.de> <4C4E71BF.7060109@v.loewis.de> Message-ID: On 7/27/2010 1:42 AM, "Martin v. L?wis" wrote: >> Should I open a tracker issue to add something to the tracker doc? > > I recommend that you use it for some time before changing anything. How is someone suppose to use it without instructions? > I also suggest that, instead of uploading the patch to Rietveld > yourself, you can ask the submitter to do it. That adds another step. Let me repeat me original question: Would it be feasible to add a [view] button that I could click to get a nice view of a patch, such as provided by ViewVC? -- Terry Jan Reedy From alexander.belopolsky at gmail.com Tue Jul 27 17:47:20 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 27 Jul 2010 11:47:20 -0400 Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: References: <4C4CD849.9090108@v.loewis.de> <4C4E71BF.7060109@v.loewis.de> Message-ID: On Tue, Jul 27, 2010 at 10:56 AM, Terry Reedy wrote: .. > Let me repeat me original question: Would it be feasible to add a [view] > button that I could click to get a nice view of a patch, such as provided by > ViewVC? I would at best +0 on such an addition. As I mentioned before, the largest obstacle for me in reviewing the patches is being unable to open them in the browser due to the use of a non-standard extension. For small patches, as long as they open and are in unified diff format, I don't have a problem reviewing them in plain text view. For larger patches, colored diff offers only a minor improvement over plain text compared to a large improvement I get from Rietveld. I believe this should be treated as any other RFE. AFAICT, tracker sources are available from SVN and metatracker is the place to post a patch. I don't think ViewVC module is available within roundup and there may be licensing issues that need to be looked into before it can be integrated. Other than that, writing viewvc-style diff viewer plugin for roundup does not strike me as a terribly difficult project, but my +0 vote means that I am not going to write it. From reid.kleckner at gmail.com Tue Jul 27 17:52:02 2010 From: reid.kleckner at gmail.com (Reid Kleckner) Date: Tue, 27 Jul 2010 08:52:02 -0700 Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: References: <4C4CD849.9090108@v.loewis.de> <4C4E71BF.7060109@v.loewis.de> Message-ID: On Tue, Jul 27, 2010 at 7:56 AM, Terry Reedy wrote: >> I also suggest that, instead of uploading the patch to Rietveld >> yourself, you can ask the submitter to do it. > > That adds another step. > > Let me repeat me original question: Would it be feasible to add a [view] > button that I could click to get a nice view of a patch, such as provided by > ViewVC? How are you proposing to use ViewVC to view the patch? I'd think that you'd have to commit it first, unless it has some functionality that I'm unaware of. Anyway, one uses Rietveld mostly via upload.py, not the form above. Instead of running 'svn diff' + uploading the patch file in a web browser and having several versions accumulate, you run `upload.py -i ` and it uploads the diff to rietveld. Rietveld's diff view is quite nice. Would the ViewVC functionality you are proposing look like this? http://svn.python.org/view/python/branches/release27-maint/Demo/classes/Vec.py?r1=82503&r2=83175&pathrev=83175 Rietveld's differ is smarter (it does intra-line diffs) and the inline comments there are a lot better than pasting the diff into an email. It's true that the workflow isn't really described anywhere, so I'll try to outline it in detail here. Author's steps to upload a patch and create an issue: - Discuss issue in the tracker - Hack away at solution in svn checkout - When done, run `upload.py` (no args creates a new issue and prints URL) - When prompted, enter Google account credentials - When prompted, enter the issue title you want to give it, probably by pasting in the tracker title plus IssueXXX - I always check the diff on Rietveld to make sure it looks good to me before sending - Go to the URL printed and click 'Start Review' to send mail Reviewer's steps to add review comments: - Receive mail, click URL to open issue - Click the link to the first file, and read through the colored diff, using 'n' to scroll down and 'j' to go to the next file. - To make a comment, double click the line you want to comment on. This is the most unintuitive part to beginners. - Enter the comment in the textbox that appears. - Repeat until done reading the diff, then go back to the issue page and click 'Publish+Mail Comments' Author's steps to respond to comments: - Open the files in the issue - Read through the comments ('N' skips from comment to comment) - Apply fixes, reply to each comment - Run `upload.py -i ` to add a new patch with your fixes. - Reply by clicking 'Publish+Mail Comments' to let your reviewer know that you've addressed the comments Repeat ad nauseum until reviewer is happy, then commit. === Not sure why I spelled that all out when these docs exist: http://code.google.com/p/rietveld/wiki/CodeReviewHelp Hopefully my outline reflects the Python workflow more accurately, though. :) Reid From alexander.belopolsky at gmail.com Tue Jul 27 18:18:15 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 27 Jul 2010 12:18:15 -0400 Subject: [Python-Dev] mkdir -p in python In-Reply-To: <20100720142005.70AFE2007B4@kimball.webabinitio.net> References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> <20100720142005.70AFE2007B4@kimball.webabinitio.net> Message-ID: On Tue, Jul 20, 2010 at 10:20 AM, R. David Murray wrote: > I'd go with putting it in shutil. +1 I would also call it shutil.mktree which will go well with shutil.rmtree next to it. From tjreedy at udel.edu Tue Jul 27 19:19:01 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 27 Jul 2010 13:19:01 -0400 Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: References: <4C4CD849.9090108@v.loewis.de> <4C4E71BF.7060109@v.loewis.de> Message-ID: On 7/27/2010 11:52 AM, Reid Kleckner wrote: >> Let me repeat me original question: Would it be feasible to add a [view] >> button that I could click to get a nice view of a patch, such as provided by >> ViewVC? > > How are you proposing to use ViewVC to view the patch? I'd think that > you'd have to commit it first, unless it has some functionality that > I'm unaware of. The point of the tracker is to review patches *before* they are committed. The information is the same before and after it is committed. The question of 'feasible' asks whether ViewVC is inextricably linked to an svn repository. If it is, either technically or legally, then it is. > Anyway, one uses Rietveld mostly via upload.py, not the form above. I had never heard of upload.py. Like Rietveld, it is not mentioned in the tracker docs. I see it mentioned on he rietveld page linked below. The page says it requires 2.5 (which newcomers will not have laying aroung)and only says how to run it on *nix, so I would not be surprised if it were to not run on Windows due to using *nix only functions. > Would the ViewVC functionality you are proposing look like this? > http://svn.python.org/view/python/branches/release27-maint/Demo/classes/Vec.py?r1=82503&r2=83175&pathrev=83175 Yes, I presume that is what some people use to do post-commit reviews. > Rietveld's differ is smarter (it does intra-line diffs) Ok, that would be even better. > and the inline comments there are a lot better than out of context comments in a tracker message. > > It's true that the workflow isn't really described anywhere, So don't expect ignorant folks like me to follow it ;-). > so I'll try to outline it in detail here. Like code patches, doc patches in here-today, gone-tomorrow email is not too useful. Better to put it the doc or in a tracker issue for further revision. > Author's steps to upload a patch and create an issue: > - Discuss issue in the tracker > - Hack away at solution in svn checkout > - When done, run `upload.py` (no args creates a new issue and prints URL) > - When prompted, enter Google account credentials > - When prompted, enter the issue title you want to give it, probably > by pasting in the tracker title plus IssueXXX > - I always check the diff on Rietveld to make sure it looks good to me > before sending > - Go to the URL printed and click 'Start Review' to send mail > > Reviewer's steps to add review comments: > - Receive mail, click URL to open issue > - Click the link to the first file, and read through the colored diff, > using 'n' to scroll down and 'j' to go to the next file. > - To make a comment, double click the line you want to comment on. > This is the most unintuitive part to beginners. > - Enter the comment in the textbox that appears. > - Repeat until done reading the diff, then go back to the issue page > and click 'Publish+Mail Comments' > > Author's steps to respond to comments: > - Open the files in the issue > - Read through the comments ('N' skips from comment to comment) > - Apply fixes, reply to each comment > - Run `upload.py -i` to add a new patch with your fixes. > - Reply by clicking 'Publish+Mail Comments' to let your reviewer know > that you've addressed the comments > > Repeat ad nauseum until reviewer is happy, then commit. > > === > > Not sure why I spelled that all out when these docs exist: > http://code.google.com/p/rietveld/wiki/CodeReviewHelp Because a) someone reading http://wiki.python.org/moin/TrackerDocs/ would not know it exists and b) because the above is *much* cleared than that page. Let me try to explain better what my original post was about. There have been many posts on pydev about the need for more patch reviews. A couple of people have even made 5 for 1 offers to encourage more reviews and have emphasized that even partial reviews are more helpful than none. In response to these appeals, including the last point, I recently starting doing more patch reviews than in the past. Simple diffs that replace block A with block B I can read fine. Sufficiently complex diffs, say with 10 interlaced changed, I cannot. A couple of days ago, I got an email that a doc issue I opened was now closed with revxxxxx, patch never posted to the tracker. I followed the link, saw the [text] button, and got the page with the colored, side-by-side display. I thought, "Wow, I wish I could see patches like this while reviewing, before they are committed!". And to do so just as easily, with one click. (As it turns out, the patch needed review because it had a minor error, but that is another issue.) Rietveld may be a bit better, but it potentially is a lot of work to get that bit better, especially the firs time. And it may be overkill for small to medium patches. So what I am suggesting it this: to get more and better patch reviews, especially from new reviewers, make it as easy to get an improved view of diffs as it is now to view the raw file. -- Terry Jan Reedy From alexander.belopolsky at gmail.com Tue Jul 27 19:48:52 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 27 Jul 2010 13:48:52 -0400 Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: References: <4C4CD849.9090108@v.loewis.de> <4C4E71BF.7060109@v.loewis.de> Message-ID: On Tue, Jul 27, 2010 at 1:19 PM, Terry Reedy wrote: .. > A couple of days ago, I got an email that a doc issue I opened was now > closed with revxxxxx, patch never posted to the tracker. I followed the > link, saw the [text] button, and got the page with the colored, side-by-side > display. I thought, "Wow, I wish I could see patches like this while > reviewing, before they are committed!". And to do so just as easily, with > one click. (As it turns out, the patch needed review because it had a minor > error, but that is another issue.) Multicolored diffs may look impressive the first time you see them, but they are not really that useful at the patch review stage. With a text link I often do curl | patch -p0 with pasted after "copy link". That would not work with a rev-style link. Copying code from side by side view may or may not work depending on your browser. Even with doc patches, for a serious review you need to apply and compile the patch (make html in case of the docs.) Post-commit rev NNNN link offers much more than a nice diff: it shows comments, allows you to explore the history etc. Colored diffs do help spotting spelling mistakes, but there is not much harm in finding them post-commit. From tjreedy at udel.edu Tue Jul 27 20:31:17 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 27 Jul 2010 14:31:17 -0400 Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: References: <4C4CD849.9090108@v.loewis.de> <4C4E71BF.7060109@v.loewis.de> Message-ID: On 7/27/2010 1:48 PM, Alexander Belopolsky wrote: > Multicolored diffs may look impressive the first time you see them, Side-by-side was the important part > Copying code > from side by side view may or may not work depending on your browser. It is a nuisance with FireFox. For a patch on the tracker, I would cut from the raw diff. > Even with doc patches, for a serious review you need to apply and > compile the patch (make html in case of the docs.) I know that and I would not commit an issue without doing that. I am talking about partial pre-commit reviews by a beginner who does not have the tools to do that. The kind that have been requested here on pydev. -- Terry Jan Reedy From eric at trueblade.com Tue Jul 27 20:48:26 2010 From: eric at trueblade.com (Eric Smith) Date: Tue, 27 Jul 2010 14:48:26 -0400 Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: References: <4C4CD849.9090108@v.loewis.de> <4C4E71BF.7060109@v.loewis.de> Message-ID: <4C4F29FA.3070900@trueblade.com> On 7/27/10 2:31 PM, Terry Reedy wrote: > On 7/27/2010 1:48 PM, Alexander Belopolsky wrote: > >> Multicolored diffs may look impressive the first time you see them, > > Side-by-side was the important part > > > Copying code >> from side by side view may or may not work depending on your browser. > > It is a nuisance with FireFox. For a patch on the tracker, I would cut > from the raw diff. > >> Even with doc patches, for a serious review you need to apply and >> compile the patch (make html in case of the docs.) > > I know that and I would not commit an issue without doing that. I am > talking about partial pre-commit reviews by a beginner who does not have > the tools to do that. The kind that have been requested here on pydev. > I agree with Terry that this would be a useful feature to have integrated with the tracker. I'd use it. But until someone write it, it's an academic point. -- Eric. From alexander.belopolsky at gmail.com Tue Jul 27 21:01:39 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 27 Jul 2010 15:01:39 -0400 Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: <4C4F29FA.3070900@trueblade.com> References: <4C4CD849.9090108@v.loewis.de> <4C4E71BF.7060109@v.loewis.de> <4C4F29FA.3070900@trueblade.com> Message-ID: On Tue, Jul 27, 2010 at 2:48 PM, Eric Smith wrote: .. > I agree with Terry that this would be a useful feature to have integrated > with the tracker. I'd use it. But until someone write it, it's an academic > point. I don't say it is useless. It is just not useful enough to justify the required development effort. A low hanging fruit in this area would be to teach the tracker to recognize diffs regardless of file extension and present them as text/plain. For me that would give 99% of the benefit a full-blown diff-to-html converter may bring. From eric at trueblade.com Tue Jul 27 21:25:33 2010 From: eric at trueblade.com (Eric Smith) Date: Tue, 27 Jul 2010 15:25:33 -0400 (EDT) Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: References: <4C4CD849.9090108@v.loewis.de> <4C4E71BF.7060109@v.loewis.de> <4C4F29FA.3070900@trueblade.com> Message-ID: > On Tue, Jul 27, 2010 at 2:48 PM, Eric Smith wrote: > .. >> I agree with Terry that this would be a useful feature to have >> integrated >> with the tracker. I'd use it. But until someone write it, it's an >> academic >> point. > > I don't say it is useless. And I never said you said that :) > It is just not useful enough to justify > the required development effort. A low hanging fruit in this area > would be to teach the tracker to recognize diffs regardless of file > extension and present them as text/plain. For me that would give 99% > of the benefit a full-blown diff-to-html converter may bring. That would be sort of handy, but detecting a diff is of course problematic. I'd rather have the viewvc sort of view, but it's moot until someone does the work. I don't want it bad enough to write it. From martin at v.loewis.de Tue Jul 27 21:45:41 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 27 Jul 2010 21:45:41 +0200 Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: References: <4C4CD849.9090108@v.loewis.de> <4C4E71BF.7060109@v.loewis.de> Message-ID: <4C4F3765.1060505@v.loewis.de> Am 27.07.2010 16:56, schrieb Terry Reedy: > On 7/27/2010 1:42 AM, "Martin v. L?wis" wrote: >>> Should I open a tracker issue to add something to the tracker doc? >> >> I recommend that you use it for some time before changing anything. > > How is someone suppose to use it without instructions? I'd expect that most committers would be able to learn using it quickly, without explicit instructions. However, I think there might also be a Rietveld manual somewhere. > Let me repeat me original question: Would it be feasible to add a [view] > button that I could click to get a nice view of a patch, such as > provided by ViewVC? I think this would be very difficult to implement, plus it would be a wasted effort, since we are going to switch to Mercurial, and would stop using ViewVC. Regards, Martin From breamoreboy at yahoo.co.uk Tue Jul 27 21:51:39 2010 From: breamoreboy at yahoo.co.uk (Mark Lawrence) Date: Tue, 27 Jul 2010 20:51:39 +0100 Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: References: Message-ID: On 26/07/2010 01:24, Terry Reedy wrote: > To review a patch on the tracker, I have to read and try to make sense > of the raw diff file. Sometimes that is easy, sometimes not. > > *After* a patch is applied, I can click the revxxxx link and then the > 'text changed' link and see a nice, colored, side-by-side web-pace view > created by ViewVC. Is there any way a 'view' button could be added, > along with the current edit and remove buttons, to produce the same web > page and make it easier to review patches *before* commitment? > Apologies if I've missed this from earlier parts of this thread. On my windows box I have maintainance versions for 2.6, 2.7, 3.1 and 3.2 plus tortoisesvn. Download the patch file, right click, select tortoisesvn then apply patch. Go to the version I'm interested in. Double click to select the unit test file to start things off. If I'm lucky get a coloured output in parallel highlighting removals, additions and conflicts. If I'm unlucky, get the message "The patch seems outdated". Is this what you're asking for? Can this be (simply) implemented on the issue tracker? What do *nix or OS X users do? Cheers. Mark Lawrence. From martin at v.loewis.de Tue Jul 27 22:13:03 2010 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 27 Jul 2010 22:13:03 +0200 Subject: [Python-Dev] View tracker patches with ViewVC? In-Reply-To: References: Message-ID: <4C4F3DCF.9030704@v.loewis.de> > On my windows box I have maintainance versions for 2.6, 2.7, 3.1 and 3.2 > plus tortoisesvn. Download the patch file, right click, select > tortoisesvn then apply patch. Go to the version I'm interested in. > Double click to select the unit test file to start things off. If I'm > lucky get a coloured output in parallel highlighting removals, additions > and conflicts. If I'm unlucky, get the message "The patch seems > outdated". Is this what you're asking for? Can this be (simply) > implemented on the issue tracker? What do *nix or OS X users do? Automatically checking whether a patch still applies? That could certainly be implemented, although it might be expensive in terms of CPU time. Anything more frequent than daily runs is likely not feasible. In addition, it might be tricky to determine what branches a patch is meant to apply to. It would be nice if the patch author gets notified when a patch becomes stale, but that would require to be certain what branches a patch is targetted for. In any case: contributions are welcome. Regards, Martin From barry at python.org Tue Jul 27 23:23:15 2010 From: barry at python.org (Barry Warsaw) Date: Tue, 27 Jul 2010 17:23:15 -0400 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <87ocdu3qyr.fsf@brainbot.com> References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> <63B36042-C4F7-4BCA-8623-5182405D8852@mac.com> <20100723120252.2e0690cc@snowdog> <25347613-8488-40C3-B9EF-9C85D0C47340@mac.com> <20100723125440.721860bf@snowdog> <764F1595-5C8B-44D8-86C5-937B326B2FCD@mac.com> <87mxti8lqc.fsf@muni.brainbot.com> <20100724002252.19d91267@snowdog> <87aapgbky5.fsf@brainbot.com> <20100726150705.778ebf65@heresy> <87ocdu3qyr.fsf@brainbot.com> Message-ID: <20100727172315.1e1edeec@heresy> On Jul 26, 2010, at 10:53 PM, Ralf Schmitt wrote: >Some of the things that need to be adapted are e.g. Makefiles >(basically anything that assumes modules have a certain name), all of >the freezers (cxFreeze, py2exe, ...). The biggest problem probably >will be that an import will load the wrong module or no module at all. >I'm just speculating here... I took a look at cx_freeze - it doesn't support Python 3.2 yet afaict (the build fails but it may be shallow). I'm going to look at py2exe as soon as I can get a Windows VM up and running. Since import is (usually) handled by the built-in dynload_shlib.c code it should generally Just Work I think, unless some application installs custom import hooks. In any case, I think this will be a fairly standard and probably simple, porting effort, which goes along with supporting any new major release. >> "Change is bad" isn't a constructive argument. ;) > >Did I make that argument? Apologies. >> That's fine, but it's not the way Debian/Ubuntu works today. PEP >> 3149 adoption will definitely remove significant complication for >> deploying multiple Python versions at the same time on those systems. > >You're just moving that complication into python. It's a much different level and scope of complexity though. For Python, it's pretty simple: look for an additional .so file name pattern which gets baked into Python at compile time. For that, you're able to remove a huge amount of complexity on Debian/Ubuntu by removing the need to manage multiple directory trees and symlinks between files in those trees. The tools that manage them (i.e. handle package installs and removals) also get much simpler. I think it's a worthwhile trade-off. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Tue Jul 27 23:25:19 2010 From: barry at python.org (Barry Warsaw) Date: Tue, 27 Jul 2010 17:25:19 -0400 Subject: [Python-Dev] versioned .so files for Python 3.2 In-Reply-To: <877hkhkulr.fsf@muni.brainbot.com> References: <20100624115048.4fd152e3@heresy> <20100714195955.4815f980@heresy> <4C40372C.9030101@ubuntu.com> <20100722164036.7a80d27c@snowdog> <63B36042-C4F7-4BCA-8623-5182405D8852@mac.com> <20100723120252.2e0690cc@snowdog> <25347613-8488-40C3-B9EF-9C85D0C47340@mac.com> <20100723125440.721860bf@snowdog> <764F1595-5C8B-44D8-86C5-937B326B2FCD@mac.com> <87mxti8lqc.fsf@muni.brainbot.com> <20100724002252.19d91267@snowdog> <87aapgbky5.fsf@brainbot.com> <20100726150705.778ebf65@heresy> <87ocdu3qyr.fsf@brainbot.com> <4C4EBBE6.6080603@ubuntu.com> <877hkhkulr.fsf@muni.brainbot.com> Message-ID: <20100727172519.200d2cec@heresy> On Jul 27, 2010, at 01:54 PM, Ralf Schmitt wrote: >Matthias Klose writes: > >> Not true. Package managers like dpkg/apt-get, rpm/yum and maybe >> others do this for ages. And yes, the added "complexity" of package >> managers does lead to increased robustness. > >but how does sharing things lead to increased robustness (even if it >might be managed by your package manager)? It removes the need to maintain multiple directory trees and the symlinks between them. The tools that manage all this platform complexity get simpler, and thus easier to maintain, leading to increased robustness on the platform. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From rdmurray at bitdance.com Wed Jul 28 03:43:37 2010 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 27 Jul 2010 21:43:37 -0400 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> <20100713163544.5f3a60d2@pitrou.net> Message-ID: <20100728014337.A7AFA1FDED5@kimball.webabinitio.net> On Tue, 27 Jul 2010 08:27:35 +0200, Stefan Behnel wrote: > Gregory P. Smith, 27.07.2010 07:40: > > A max cache size of 100 was too small. I just increased it to 500 in the > > py3k branch along with implementing a random replacement cache overflow > > policy. It now randomly drops 20% of the compiled regular expression cache > > instead of simply dropping the entire cache on overflow. > > > > With the regex_v8 benchmark, the better cache replacement policy sped it up > > ~7% while raising the cache size on top of that (likely meaning the cache > > was never overflowing) sped it up ~25%. > > > > Random replacement without dropping everything at least means apps thrashing > > the cache degrade much more gracefully. > > The same algorithm should be helpful in ElementTree's ElementPath module. We recently added the old re cache-clearing strategy to fnmatch, because previously its cache would grow indefinitely. It sounds like this should be applied there as well. That's three...time to figure out how to share the code? -- R. David Murray www.bitdance.com From stefan_ml at behnel.de Wed Jul 28 07:29:00 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 28 Jul 2010 07:29:00 +0200 Subject: [Python-Dev] caching in the stdlib? (was: New regex module for 3.2?) In-Reply-To: <20100728014337.A7AFA1FDED5@kimball.webabinitio.net> References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> <20100713163544.5f3a60d2@pitrou.net> <20100728014337.A7AFA1FDED5@kimball.webabinitio.net> Message-ID: R. David Murray, 28.07.2010 03:43: > On Tue, 27 Jul 2010 08:27:35 +0200, Stefan Behnel wrote: >> Gregory P. Smith, 27.07.2010 07:40: >>> A max cache size of 100 was too small. I just increased it to 500 in the >>> py3k branch along with implementing a random replacement cache overflow >>> policy. It now randomly drops 20% of the compiled regular expression cache >>> instead of simply dropping the entire cache on overflow. >>> >>> With the regex_v8 benchmark, the better cache replacement policy sped it up >>> ~7% while raising the cache size on top of that (likely meaning the cache >>> was never overflowing) sped it up ~25%. >>> >>> Random replacement without dropping everything at least means apps thrashing >>> the cache degrade much more gracefully. >> >> The same algorithm should be helpful in ElementTree's ElementPath module. > > We recently added the old re cache-clearing strategy to > fnmatch, because previously its cache would grow indefinitely. > It sounds like this should be applied there as well. > > That's three...time to figure out how to share the code? What about actually putting it visibly into the stdlib? Except for files, I didn't see much about caching there, which seems like a missing battery to me. Why not do it as with the collections module and add stuff as it comes in? Stefan From greg at krypto.org Wed Jul 28 08:50:09 2010 From: greg at krypto.org (Gregory P. Smith) Date: Tue, 27 Jul 2010 23:50:09 -0700 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: <20100728014337.A7AFA1FDED5@kimball.webabinitio.net> References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> <20100713163544.5f3a60d2@pitrou.net> <20100728014337.A7AFA1FDED5@kimball.webabinitio.net> Message-ID: On Tue, Jul 27, 2010 at 6:43 PM, R. David Murray wrote: > On Tue, 27 Jul 2010 08:27:35 +0200, Stefan Behnel > wrote: > > Gregory P. Smith, 27.07.2010 07:40: > > > A max cache size of 100 was too small. I just increased it to 500 in > the > > > py3k branch along with implementing a random replacement cache overflow > > > policy. It now randomly drops 20% of the compiled regular expression > cache > > > instead of simply dropping the entire cache on overflow. > > > > > > With the regex_v8 benchmark, the better cache replacement policy sped > it up > > > ~7% while raising the cache size on top of that (likely meaning the > cache > > > was never overflowing) sped it up ~25%. > > > > > > Random replacement without dropping everything at least means apps > thrashing > > > the cache degrade much more gracefully. > > > > The same algorithm should be helpful in ElementTree's ElementPath module. > > We recently added the old re cache-clearing strategy to > fnmatch, because previously its cache would grow indefinitely. > It sounds like this should be applied there as well. > > That's three...time to figure out how to share the code? > No doubt. Its already a standalone _shrink_cache function with unit tests that doesn't care the dictionaries it is shrinking are composed of. Easy enough to move somewhere more useful. Any proposed stdlib locations? I'll be offline on vacation soon so I may not get to it for a couple weeks but feel free to move it without me if anyone is interested. -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: From hrvoje.niksic at avl.com Wed Jul 28 09:29:33 2010 From: hrvoje.niksic at avl.com (Hrvoje Niksic) Date: Wed, 28 Jul 2010 09:29:33 +0200 Subject: [Python-Dev] mkdir -p in python In-Reply-To: References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> <20100720142005.70AFE2007B4@kimball.webabinitio.net> Message-ID: <4C4FDC5D.2040605@avl.com> On 07/27/2010 06:18 PM, Alexander Belopolsky wrote: > On Tue, Jul 20, 2010 at 10:20 AM, R. David Murray wrote: >> I'd go with putting it in shutil. > > +1 > > I would also call it shutil.mktree which will go well with > shutil.rmtree next to it. Note that mktree is not analogous to rmtree - while rmtree removes a directory tree beneath a specified directory, mktree would only create a single "branch", not an entire tree. I'd imagine a mktree function to accept a data structure describing the tree to be created. If you're going for a short name distinctive from mkdir, I propose mksubdirs. From stephen at xemacs.org Wed Jul 28 12:19:35 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 28 Jul 2010 19:19:35 +0900 Subject: [Python-Dev] mkdir -p in python In-Reply-To: <4C4FDC5D.2040605@avl.com> References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> <20100720142005.70AFE2007B4@kimball.webabinitio.net> <4C4FDC5D.2040605@avl.com> Message-ID: <87aapbvrgo.fsf@uwakimon.sk.tsukuba.ac.jp> Hrvoje Niksic writes: > single "branch", not an entire tree. I'd imagine a mktree function to > accept a data structure describing the tree to be created. -1 on mktree for that reason. > If you're going for a short name distinctive from mkdir, I propose > mksubdirs. A little more accurate might be mkpath, but I'd be happy with either. And of course in Unix jargon "path" is ambiguous. From ncoghlan at gmail.com Wed Jul 28 12:34:48 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 28 Jul 2010 20:34:48 +1000 Subject: [Python-Dev] [isssue 2001] Pydoc enhancement patch questions In-Reply-To: References: <4C4B404E.2090007@ronadam.com> Message-ID: On Tue, Jul 27, 2010 at 1:25 AM, Ron Adam wrote: >>> Another way to communicate to the server would be to add a link in >>> the browser to open a server status page. ?For example my router has a >>> configure page where I can check it's status and do other things. >>> That might be something worth exploring at some later date. >> >> This would work as well, but for starters, I think "Search" and "Quit" >> buttons next to each other will be most familiar to the users of the >> current Tk-based control window. > > Nick, what do you think about the "Quit" link in the browser along with > improving the server startup message on the console window? My main concern was having a clear way to shut down the server - I think your command line changes address that quite nicely. Probably best to keep the quit command separate from the help browser though - it would be annoying to accidentally shutdown your help server when you were just trying to browse it. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From greg.ewing at canterbury.ac.nz Wed Jul 28 12:53:56 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 28 Jul 2010 22:53:56 +1200 Subject: [Python-Dev] mkdir -p in python In-Reply-To: <4C4FDC5D.2040605@avl.com> References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> <20100720142005.70AFE2007B4@kimball.webabinitio.net> <4C4FDC5D.2040605@avl.com> Message-ID: <4C500C44.3050305@canterbury.ac.nz> Hrvoje Niksic wrote: > mktree would only create a > single "branch", not an entire tree. Maybe mkbranch, then? -- Greg From fuzzyman at voidspace.org.uk Wed Jul 28 12:48:56 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 28 Jul 2010 11:48:56 +0100 Subject: [Python-Dev] mkdir -p in python In-Reply-To: <4C500C44.3050305@canterbury.ac.nz> References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> <20100720142005.70AFE2007B4@kimball.webabinitio.net> <4C4FDC5D.2040605@avl.com> <4C500C44.3050305@canterbury.ac.nz> Message-ID: <4C500B18.6060309@voidspace.org.uk> On 28/07/2010 11:53, Greg Ewing wrote: > Hrvoje Niksic wrote: >> mktree would only create a single "branch", not an entire tree. > > Maybe mkbranch, then? > Seeing as we already have a decision to add this functionality to os.makedirs as a switch and not to create a new function, this bikeshedding seems particularly pointless. http://mail.python.org/pipermail/python-dev/2010-July/102132.html All the best, Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From ncoghlan at gmail.com Wed Jul 28 12:50:14 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 28 Jul 2010 20:50:14 +1000 Subject: [Python-Dev] GIL musings (was Re: Thoughts fresh after EuroPython) Message-ID: On Tue, Jul 27, 2010 at 12:33 AM, Ronald Oussoren wrote: > In my opinion the GIL is a weak point of CPython and it would be nice if it > could be fixed. ?That is however easier said than done, a number of people > have tried in the past and ran into implementation limitations like our > refcounting garbage collector that make hard to remove the GIL without > either rewriting lots of code, or running into a brick wall > performance-wise. > > The HotPy presentation at EuroPython shows that it is possible to remove the > GIL, although at the cost of replacing the garbage collector and most likely > breaking existing C extensions (although the HotPy author seemed to have a > possible workaround for that). This is the kind of approach that seems to hold the most promise of removing the GIL without incurring the single-threaded performance hit that has been the achilles heel of previous attempts at creating a free-threaded CPython implementation. With first IronClad and now PyPy blazing the trail in interfacing a garbage collected Python implementation with deterministic refcounting for C extension modules, it seems plausible that this kind of approach may eventually prove acceptable. Furthermore, the with statement now provides a superior alternative to application level tricks that previously relied on deterministic refcounting. While multi-threading does break down beyond a certain number of cores, it *is* possible to do safely (particularly using queues to pass references around) and can avoid plenty of serialisation overhead when dealing with sizable data structures. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From fuzzyman at voidspace.org.uk Wed Jul 28 12:56:16 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 28 Jul 2010 11:56:16 +0100 Subject: [Python-Dev] GIL musings (was Re: Thoughts fresh after EuroPython) In-Reply-To: References: Message-ID: <4C500CD0.3040309@voidspace.org.uk> On 28/07/2010 11:50, Nick Coghlan wrote: > On Tue, Jul 27, 2010 at 12:33 AM, Ronald Oussoren > wrote: > >> In my opinion the GIL is a weak point of CPython and it would be nice if it >> could be fixed. That is however easier said than done, a number of people >> have tried in the past and ran into implementation limitations like our >> refcounting garbage collector that make hard to remove the GIL without >> either rewriting lots of code, or running into a brick wall >> performance-wise. >> >> The HotPy presentation at EuroPython shows that it is possible to remove the >> GIL, although at the cost of replacing the garbage collector and most likely >> breaking existing C extensions (although the HotPy author seemed to have a >> possible workaround for that). >> > This is the kind of approach that seems to hold the most promise of > removing the GIL without incurring the single-threaded performance hit > that has been the achilles heel of previous attempts at creating a > free-threaded CPython implementation. With first IronClad and now PyPy > blazing the trail in interfacing a garbage collected Python > implementation with deterministic refcounting for C extension modules, > it seems plausible that this kind of approach may eventually prove > acceptable. > > Furthermore, the with statement now provides a superior alternative to > application level tricks that previously relied on deterministic > refcounting. > > While multi-threading does break down beyond a certain number of > cores, it *is* possible to do safely (particularly using queues to > pass references around) and can avoid plenty of serialisation overhead > when dealing with sizable data structures. > Breaking binary compatibility with C extensions would be "difficult" once PEP 384 (stable binary ABI) has gone into effect. As you intimate, Ironclad demonstrates that C extensions *can* be interfaced with a different garbage collection system whilst maintaining binary compatibility. It does impose constraints however (which is why the PyPy c-ext implementors chose source compatibility rather than binary compatibility). Michael > Cheers, > Nick. > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From solipsis at pitrou.net Wed Jul 28 13:12:30 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 28 Jul 2010 13:12:30 +0200 Subject: [Python-Dev] GIL musings (was Re: Thoughts fresh after EuroPython) References: <4C500CD0.3040309@voidspace.org.uk> Message-ID: <20100728131230.7dc49719@pitrou.net> On Wed, 28 Jul 2010 11:56:16 +0100 Michael Foord wrote: > > This is the kind of approach that seems to hold the most promise of > > removing the GIL without incurring the single-threaded performance hit > > that has been the achilles heel of previous attempts at creating a > > free-threaded CPython implementation. With first IronClad and now PyPy > > blazing the trail in interfacing a garbage collected Python > > implementation with deterministic refcounting for C extension modules, > > it seems plausible that this kind of approach may eventually prove > > acceptable. > > > > Furthermore, the with statement now provides a superior alternative to > > application level tricks that previously relied on deterministic > > refcounting. > > > > While multi-threading does break down beyond a certain number of > > cores, it *is* possible to do safely (particularly using queues to > > pass references around) and can avoid plenty of serialisation overhead > > when dealing with sizable data structures. > > > > Breaking binary compatibility with C extensions would be "difficult" > once PEP 384 (stable binary ABI) has gone into effect. "Stable" doesn't mean eternal. At worse, we could call the result Python 4.0. It should be noted, though, that a full GC can be detrimental to real-time applications. Kristj?n has already explained how some of his software disabled the cyclic GC, and took care of breaking cycles manually instead. Regards Antoine. From vikas.mahajan12 at gmail.com Wed Jul 28 13:21:47 2010 From: vikas.mahajan12 at gmail.com (Vikas Mahajan) Date: Wed, 28 Jul 2010 16:51:47 +0530 Subject: [Python-Dev] Compiling Free Cad on Rhel5 Message-ID: Hello to all......Today I was trying to install FreeCAD 0.10 from its source code. I am using RHEL5. I have firstly installed python2.6.5 from source and also tested it. Python is working fine. But when I try to compile FreeCAD 0.10, I got following configure error : #./configure --with-python-include=/usr/local/include/python2.6 --with-python-lib=/usr/local/lib/python2.6 --------------------------------------------------------------------------------------------------------------------------------------------------------- checking for a Python interpreter with version >= 2.5... python checking for python... /usr/local/bin/python checking for python version... 2.6 checking for python platform... linux2 checking for python script directory... ${prefix}/lib/python2.6/site-packages checking for python extension module directory... ${exec_prefix}/lib/python2.6/site-packages checking Python.h usability... yes checking Python.h presence... yes checking for Python.h... yes checking for libpython2.6... no configure: error: **** Cannot find Python2.6 devel files. **** ---------------------------------------------------------------------------------------------------------------------------------------------------------- I also searched internet for python-devel source code but was unable to find any such pacage. Please help me to resolve this python error. Thanks Vikas Mahajan -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Wed Jul 28 13:26:51 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 28 Jul 2010 12:26:51 +0100 Subject: [Python-Dev] Compiling Free Cad on Rhel5 In-Reply-To: References: Message-ID: <4C5013FB.5020902@voidspace.org.uk> On 28/07/2010 12:21, Vikas Mahajan wrote: > Hello to all......Today I was trying to install FreeCAD 0.10 from its > source code. I am using RHEL5. I have firstly installed python2.6.5 > from source and also tested it. Python is working fine. But when I try > to compile FreeCAD 0.10, I got following configure error : > Hello Vikas, This list is for the discussion of the development *of* Python, not developing *with* Python. A more appropriate list to use would be the python-list (also known as comp.lang.python). Other Python lists are referenced here: http://www.python.org/community/lists/ Having said that, it is common for Linux distributions to package the bits needed to compile Python separately from their distribution of Python itself (a horrible practise). You probably need to install the python-devel package using your package manager. All the best, Michael > #./configure --with-python-include=/usr/local/include/python2.6 > --with-python-lib=/usr/local/lib/python2.6 > > --------------------------------------------------------------------------------------------------------------------------------------------------------- > checking for a Python interpreter with version >= 2.5... python > checking for python... /usr/local/bin/python > checking for python version... 2.6 > checking for python platform... linux2 > checking for python script directory... > ${prefix}/lib/python2.6/site-packages > checking for python extension module directory... > ${exec_prefix}/lib/python2.6/site-packages > checking Python.h usability... yes > checking Python.h presence... yes > checking for Python.h... yes > checking for libpython2.6... no > configure: error: > **** Cannot find Python2.6 devel files. **** > > ---------------------------------------------------------------------------------------------------------------------------------------------------------- > > I also searched internet for python-devel source code but was unable > to find any such pacage. Please help me to resolve this python error. > > Thanks > > Vikas Mahajan > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. -------------- next part -------------- An HTML attachment was scrubbed... URL: From phd at phd.pp.ru Wed Jul 28 13:27:33 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Wed, 28 Jul 2010 15:27:33 +0400 Subject: [Python-Dev] Compiling Free Cad on Rhel5 In-Reply-To: References: Message-ID: <20100728112733.GC8061@phd.pp.ru> Hello. We are sorry but we cannot help you. This mailing list is to work on developing Python (adding new features to Python itself and fixing bugs); if you're having problems learning, understanding or using Python, please find another forum. Probably python-list/comp.lang.python mailing list/news group is the best place; there are Python developers who participate in it; you may get a faster, and probably more complete, answer there. See http://www.python.org/community/ for other lists/news groups/fora. Thank you for understanding. python-devel should be a package in your distribution - check the list of packages. On Wed, Jul 28, 2010 at 04:51:47PM +0530, Vikas Mahajan wrote: > Hello to all......Today I was trying to install FreeCAD 0.10 from its source > code. I am using RHEL5. I have firstly installed python2.6.5 from source and > also tested it. Python is working fine. But when I try to compile FreeCAD > 0.10, I got following configure error : > > #./configure --with-python-include=/usr/local/include/python2.6 > --with-python-lib=/usr/local/lib/python2.6 > > --------------------------------------------------------------------------------------------------------------------------------------------------------- > checking for a Python interpreter with version >= 2.5... python > checking for python... /usr/local/bin/python > checking for python version... 2.6 > checking for python platform... linux2 > checking for python script directory... > ${prefix}/lib/python2.6/site-packages > checking for python extension module directory... > ${exec_prefix}/lib/python2.6/site-packages > checking Python.h usability... yes > checking Python.h presence... yes > checking for Python.h... yes > checking for libpython2.6... no > configure: error: > **** Cannot find Python2.6 devel files. **** > > ---------------------------------------------------------------------------------------------------------------------------------------------------------- > > I also searched internet for python-devel source code but was unable to find > any such pacage. Please help me to resolve this python error. > > Thanks > > Vikas Mahajan Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From ncoghlan at gmail.com Wed Jul 28 13:29:02 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 28 Jul 2010 21:29:02 +1000 Subject: [Python-Dev] New regex module for 3.2? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> <20100713163544.5f3a60d2@pitrou.net> <20100728014337.A7AFA1FDED5@kimball.webabinitio.net> Message-ID: On Wed, Jul 28, 2010 at 4:50 PM, Gregory P. Smith wrote: > On Tue, Jul 27, 2010 at 6:43 PM, R. David Murray > wrote: >> On Tue, 27 Jul 2010 08:27:35 +0200, Stefan Behnel >> wrote: >> > Gregory P. Smith, 27.07.2010 07:40: >> > > Random replacement without dropping everything at least means apps >> > > thrashing >> > > the cache degrade much more gracefully. >> > >> > The same algorithm should be helpful in ElementTree's ElementPath >> > module. >> >> We recently added the old re cache-clearing strategy to >> fnmatch, because previously its cache would grow indefinitely. >> It sounds like this should be applied there as well. >> >> That's three...time to figure out how to share the code? > > No doubt. Anyone remember off the top of their head what linecache does? ... ah, it's just unbounded unless you call clearcache(). Probably OK for that particular application. > Its already a standalone _shrink_cache function with unit tests that doesn't > care the dictionaries it is shrinking are composed of. ?Easy enough to move > somewhere more useful. ?Any proposed stdlib locations? ?I'll be offline on > vacation soon so I may not get to it for a couple weeks but feel free to > move it without me if anyone is interested. I created a tracker issue for the idea so we don't forget about it: http://bugs.python.org/issue9396 collections seems like a fairly obvious possibility, but the flavour doesn't quite seem right for that. Nowhere else springs to mind though. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ronaldoussoren at mac.com Wed Jul 28 13:43:37 2010 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 28 Jul 2010 04:43:37 -0700 (PDT) Subject: [Python-Dev] GIL musings (was Re: Thoughts fresh after EuroPython) In-Reply-To: <4C500CD0.3040309@voidspace.org.uk> Message-ID: On 28 Jul, 2010,at 12:56 PM, Michael Foord wrote: On 28/07/2010 11:50, Nick Coghlan wrote: > On Tue, Jul 27, 2010 at 12:33 AM, Ronald Oussoren > wrote: > >> In my opinion the GIL is a weak point of CPython and it would be nice if it >> could be fixed. That is however easier said than done, a number of people >> have tried in the past and ran into implementation limitations like our >> refcounting garbage collector that make hard to remove the GIL without >> either rewriting lots of code, or running into a brick wall >> performance-wise. >> >> The HotPy presentation at EuroPython shows that it is possible to remove the >> GIL, although at the cost of replacing the garbage collector and most likely >> breaking existing C extensions (although the HotPy author seemed to have a >> possible workaround for that). >> > This is the kind of approach that seems to hold the most promise of > removing the GIL without incurring the single-threaded performance hit > that has been the achilles heel of previous attempts at creating a > free-threaded CPython implementation. With first IronClad and now PyPy > blazing the trail in interfacing a garbage collected Python > implementation with deterministic refcounting for C extension modules, > it seems plausible that this kind of approach may eventually prove > acceptable. > > Furthermore, the with statement now provides a superior alternative to > application level tricks that previously relied on deterministic > refcounting. > > While multi-threading does break down beyond a certain number of > cores, it *is* possible to do safely (particularly using queues to > pass references around) and can avoid plenty of serialisation overhead > when dealing with sizable data structures. > Breaking binary compatibility with C extensions would be "difficult" once PEP 384 (stable binary ABI) has gone into effect. As you intimate, Ironclad demonstrates that C extensions *can* be interfaced with a different garbage collection system whilst maintaining binary compatibility. It does impose constraints however (which is why the PyPy c-ext implementors chose source compatibility rather than binary compatibility). ? The HotPy author mentioned that he has a scheme where refcounts could be used by C extensions while the system natively uses a copying collector, but I got the impression that this was not fully fleshed out yet. Apple's Objective-C garbage collector has a simular feature: you can use CFRetain/CFRelease to manage refcounts and the GC will only collect objects where the CF reference count is 0. ?This is a non-copying collector in a C environment though, which makes this scheme easier to implement than with a full generational copying collector. It should therefore be possible to have an interpreter where the VM uses a real GC and while extensions using the stable ABI could work as is, but that probably requires that Py_INCREF and Py_DECREF expand into function calls in the stable ABI. Implementing this would still be a significant amount of work. Ronald Michael > Cheers, > Nick. > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Wed Jul 28 13:52:37 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 28 Jul 2010 12:52:37 +0100 Subject: [Python-Dev] GIL musings (was Re: Thoughts fresh after EuroPython) In-Reply-To: References: Message-ID: <4C501A05.7030707@voidspace.org.uk> On 28/07/2010 12:43, Ronald Oussoren wrote: > > > On 28 Jul, 2010,at 12:56 PM, Michael Foord > wrote: > >> On 28/07/2010 11:50, Nick Coghlan wrote: >> > On Tue, Jul 27, 2010 at 12:33 AM, Ronald Oussoren >> > wrote: >> > >> >> In my opinion the GIL is a weak point of CPython and it would be >> nice if it >> >> could be fixed. That is however easier said than done, a number of >> people >> >> have tried in the past and ran into implementation limitations >> like our >> >> refcounting garbage collector that make hard to remove the GIL without >> >> either rewriting lots of code, or running into a brick wall >> >> performance-wise. >> >> >> >> The HotPy presentation at EuroPython shows that it is possible to >> remove the >> >> GIL, although at the cost of replacing the garbage collector and >> most likely >> >> breaking existing C extensions (although the HotPy author seemed >> to have a >> >> possible workaround for that). >> >> >> > This is the kind of approach that seems to hold the most promise of >> > removing the GIL without incurring the single-threaded performance hit >> > that has been the achilles heel of previous attempts at creating a >> > free-threaded CPython implementation. With first IronClad and now PyPy >> > blazing the trail in interfacing a garbage collected Python >> > implementation with deterministic refcounting for C extension modules, >> > it seems plausible that this kind of approach may eventually prove >> > acceptable. >> > >> > Furthermore, the with statement now provides a superior alternative to >> > application level tricks that previously relied on deterministic >> > refcounting. >> > >> > While multi-threading does break down beyond a certain number of >> > cores, it *is* possible to do safely (particularly using queues to >> > pass references around) and can avoid plenty of serialisation overhead >> > when dealing with sizable data structures >> > >> >> Breaking binary compatibility with C extensions would be "difficult" >> once PEP 384 (stable binary ABI) has gone into effect. As you intimate, >> Ironclad demonstrates that C extensions *can* be interfaced with a >> different garbage collection system whilst maintaining binary >> compatibility. It does impose constraints however (which is why the PyPy >> c-ext implementors chose source compatibility rather than binary >> compatibility). > The HotPy author mentioned that he has a scheme where refcounts could > be used by C extensions while the system natively uses a copying > collector, but I got the impression that this was not fully fleshed > out yet. > > Apple's Objective-C garbage collector has a simular feature: you can > use CFRetain/CFRelease to manage refcounts and the GC will only > collect objects where the CF reference count is 0. This is a > non-copying collector in a C environment though, which makes this > scheme easier to implement than with a full generational copying > collector. > > It should therefore be possible to have an interpreter where the VM > uses a real GC and while extensions using the stable ABI could work as > is, but that probably requires that Py_INCREF and Py_DECREF expand > into function calls in the stable ABI. Ironclad artificially inflates the refcount by one when objects are created. If an object is eligible for garbage collection *and* the refcount is 1 (so the C extension doesn't hold any references to it) then Ironclad decrements the refcount to zero and nature takes its course. That's a simplification of course (particularly around what happens when an object is eligible for garbage collection but the refcount is above 1). This allows Py_INCREF and Py_DECREF to remain as macros - switching to functions would have a performance cost I guess. Michael > > Implementing this would still be a significant amount of work. > > Ronald > >> >> >> Michael >> >> >> >> > Cheers, >> > Nick. >> > >> > >> >> >> -- >> http://www.ironpythoninaction.com/ >> http://www.voidspace.org.uk/blog >> >> READ CAREFULLY. By accepting and reading this email you agree, on >> behalf of your employer, to release me from all obligations and >> waivers arising from any and all NON-NEGOTIATED agreements, licenses, >> terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, >> non-disclosure, non-compete and acceptable use policies (?BOGUS >> AGREEMENTS?) that I have entered into with your employer, its >> partners, licensors, agents and assigns, in perpetuity, without >> prejudice to my ongoing rights and privileges. You further represent >> that you have the authority to release me from any BOGUS AGREEMENTS >> on behalf of your employer. >> >> -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From lukasz at langa.pl Wed Jul 28 13:25:42 2010 From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=) Date: Wed, 28 Jul 2010 13:25:42 +0200 Subject: [Python-Dev] Compiling Free Cad on Rhel5 In-Reply-To: References: Message-ID: Wiadomo?? napisana przez Vikas Mahajan w dniu 2010-07-28, o godz. 13:21: > Hello to all......Today I was trying to install FreeCAD 0.10 from its source code. I am using RHEL5. I have firstly installed python2.6.5 from source and also tested it. Python is working fine. But when I try to compile FreeCAD 0.10, I got following configure error : 1. Please post Python usage related problems to the python-users mailing list: http://mail.python.org/mailman/listinfo/python-list 2. That being said, your specific problem can be resolved by compiling Python to a dynamic library. You can do this by specifying `--enable-shared` during the `./configure` phase. -- Best regards, ?ukasz Langa tel. +48 791 080 144 WWW http://lukasz.langa.pl/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From vikas.mahajan12 at gmail.com Wed Jul 28 13:57:40 2010 From: vikas.mahajan12 at gmail.com (Vikas Mahajan) Date: Wed, 28 Jul 2010 17:27:40 +0530 Subject: [Python-Dev] Compiling Free Cad on Rhel5 In-Reply-To: References: Message-ID: > 1. Please post Python usage related problems to the python-users mailing list: > http://mail.python.org/mailman/listinfo/python-list Sorry for posting here. I am new baby to python so I don't know where to post. I will remember this and will not post irrelevant topics here. > 2. That being said, your specific problem can be resolved by compiling Python to a dynamic library. You can do this by specifying `--enable-shared` during the `./configure` phase. Thanks a lot for your help. Your solution helped me a lot. > -- > Best regards, > ?ukasz Langa > tel. +48 791 080 144 > WWW http://lukasz.langa.pl/ -- Regards Vikas Mahajan From ysj.ray at gmail.com Wed Jul 28 10:42:49 2010 From: ysj.ray at gmail.com (Ray Allen) Date: Wed, 28 Jul 2010 16:42:49 +0800 Subject: [Python-Dev] mkdir -p in python In-Reply-To: <4C4FDC5D.2040605@avl.com> References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> <20100720142005.70AFE2007B4@kimball.webabinitio.net> <4C4FDC5D.2040605@avl.com> Message-ID: I believe, in design purpose, the os.mkdir() is to match the system call "mkdir()" exactly, the os.makedirs() is a "Super-mkdir", it provides extra convenience for using when we want to create directories. This is the case makedirs() should deal with. A new function maybe confused with makedirs(). I think os.makedirs() should go to shutil, but we have missed the right time. On Wed, Jul 28, 2010 at 3:29 PM, Hrvoje Niksic wrote: > On 07/27/2010 06:18 PM, Alexander Belopolsky wrote: > >> On Tue, Jul 20, 2010 at 10:20 AM, R. David Murray >> wrote: >> >>> I'd go with putting it in shutil. >>> >> >> +1 >> >> I would also call it shutil.mktree which will go well with >> shutil.rmtree next to it. >> > > Note that mktree is not analogous to rmtree - while rmtree removes a > directory tree beneath a specified directory, mktree would only create a > single "branch", not an entire tree. I'd imagine a mktree function to > accept a data structure describing the tree to be created. > > If you're going for a short name distinctive from mkdir, I propose > mksubdirs. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ysj.ray%2Bpython-dev%40gmail.com > -- Ray Allen Best wishes! -------------- next part -------------- An HTML attachment was scrubbed... URL: From pez at apocalyptech.com Wed Jul 28 16:01:15 2010 From: pez at apocalyptech.com (CJ Kucera) Date: Wed, 28 Jul 2010 09:01:15 -0500 Subject: [Python-Dev] Fast Implementation for ZIP decryption In-Reply-To: <20100718212818.GL20775@apocalyptech.com> References: <20100718212818.GL20775@apocalyptech.com> Message-ID: <20100728140115.GB23384@apocalyptech.com> CJ Kucera wrote: > Hello list, resurrecting a rather old thread from here: > > http://mail.python.org/pipermail/python-dev/2009-August/091450.html ... and one final update from me, mostly just so Google and the like will pick it up. I did actually end up packaging up something I called "czipfile," which is just the stock Python Lib/zipfile.py optimized in Cython, which provides very fast zipfile decryption. It's available here: http://pypi.python.org/pypi/czipfile So, anyone with a need for fast zipfile decryption can now head over that way. FWIW, while playing around with things, I found that you can get some pretty nontrivial performance improvements in pure Python by unrolling the decryption loop. For instance, line 608 in Lib/zipfile.py used to be: newdata = ''.join(map(self.decrypter, newdata)) Substituting the following results in some significant speed improvements (though still quite a lot slower than the C-based extension) - key0 = self.decrypter.key0 key1 = self.decrypter.key1 key2 = self.decrypter.key2 crctable = self.decrypter.crctable datalist = [] for c in newdata: k = key2 | 2 cord = ord(c) ^ (((k * (k^1)) >> 8) & 255) datalist.append(chr(cord)) key0 = ((key0 >> 8) & 0xffffff) ^ crctable[(key0 ^ cord) & 0xff] key1 = (key1 + (key0 & 255)) & 4294967295 key1 = (key1 * 134775813 + 1) & 4294967295 key2 = ((key2 >> 8) & 0xffffff) ^ crctable[(key2 ^ ((key1 >> 24) & 255)) & 0xff] self.decrypter.key0 = key0 self.decrypter.key1 = key1 self.decrypter.key2 = key2 newdata = ''.join(datalist) Anyone looking to speed up decryption who didn't want to absolutely depend on the user having czipfile installed might want to consider bundling their own modified version of Lib/zipfile.py with the above changes. Okay, that's all. Enjoy! -CJ -- WOW: Flemmy | "The ships hung in the sky in much the same pez at apocalyptech.com | way that bricks don't." - Douglas Adams, 24.24.2.3171 | _The Hitchhiker's Guide To The Galaxy_ From guido at python.org Wed Jul 28 16:31:10 2010 From: guido at python.org (Guido van Rossum) Date: Wed, 28 Jul 2010 07:31:10 -0700 Subject: [Python-Dev] caching in the stdlib? (was: New regex module for 3.2?) In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> <20100713163544.5f3a60d2@pitrou.net> <20100728014337.A7AFA1FDED5@kimball.webabinitio.net> Message-ID: On Tue, Jul 27, 2010 at 10:29 PM, Stefan Behnel wrote: > What about actually putting it visibly into the stdlib? Except for files, I > didn't see much about caching there, which seems like a missing battery to > me. Why not do it as with the collections module and add stuff as it comes > in? Caching is a rather large topic to focus on in a single stdlib module or package. It means radically different things to different people. The pattern we're talking about there is probably more something like memoization, so 'memo' or 'memoization' could b a reasonable module name. Maybe it could provide a decorator that you configure with things like how many entries max, the expiration policy (or different policies could have different decorators), and how to compute the memo key from the arguments to the function (the default could be the repr() of the argument tuple). -- --Guido van Rossum (python.org/~guido) From raymond.hettinger at gmail.com Wed Jul 28 16:44:06 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Wed, 28 Jul 2010 07:44:06 -0700 Subject: [Python-Dev] caching in the stdlib? (was: New regex module for 3.2?) In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> <20100713163544.5f3a60d2@pitrou.net> <20100728014337.A7AFA1FDED5@kimball.webabinitio.net> Message-ID: On Jul 28, 2010, at 7:31 AM, Guido van Rossum wrote: > On Tue, Jul 27, 2010 at 10:29 PM, Stefan Behnel wrote: >> What about actually putting it visibly into the stdlib? Except for files, I >> didn't see much about caching there, which seems like a missing battery to >> me. Why not do it as with the collections module and add stuff as it comes >> in? > > Caching is a rather large topic to focus on in a single stdlib module > or package. It means radically different things to different people. > > The pattern we're talking about there is probably more something like > memoization, so 'memo' or 'memoization' could b a reasonable module > name. Maybe it could provide a decorator that you configure with > things like how many entries max, the expiration policy (or different > policies could have different decorators), and how to compute the memo > key from the arguments to the function (the default could be the > repr() of the argument tuple). FWIW, this has been a popular caching/memoization recipe: http://code.activestate.com/recipes/498245-lru-cache-decorator Raymond From stefan_ml at behnel.de Wed Jul 28 16:47:48 2010 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 28 Jul 2010 16:47:48 +0200 Subject: [Python-Dev] caching in the stdlib? In-Reply-To: References: <4C362C8C.80707@mrabarnett.plus.com> <4C39ACF2.6060108@trueblade.com> <201007120942.40778.steve@pearwood.info> <4C3C7627.80203@voidspace.org.uk> <20100713163544.5f3a60d2@pitrou.net> <20100728014337.A7AFA1FDED5@kimball.webabinitio.net> Message-ID: I think this is better suited for python-ideas, so moving it there. Guido van Rossum, 28.07.2010 16:31: > On Tue, Jul 27, 2010 at 10:29 PM, Stefan Behnel wrote: >> What about actually putting it visibly into the stdlib? Except for files, I >> didn't see much about caching there, which seems like a missing battery to >> me. Why not do it as with the collections module and add stuff as it comes >> in? > > Caching is a rather large topic to focus on in a single stdlib module > or package. It means radically different things to different people. Sure. All I would like to see in the stdlib is a dict based cache with a bounded size and a set of different policies that keep it at that size. Maybe a BoundedDict would match that need. > The pattern we're talking about there is probably more something like > memoization, so 'memo' or 'memoization' could b a reasonable module > name. That would be a second interface to it, I'd say. The normal interface would just be the regular mapping interface, maybe even just a .get() method instead of a KeyError raising lookup. > Maybe it could provide a decorator that you configure with > things like how many entries max, the expiration policy (or different > policies could have different decorators), and how to compute the memo > key from the arguments to the function (the default could be the > repr() of the argument tuple). Absolutely. Stefan From tjreedy at udel.edu Wed Jul 28 22:47:42 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 28 Jul 2010 16:47:42 -0400 Subject: [Python-Dev] mkdir -p in python In-Reply-To: References: <4C44DB97.8030108@canterbury.ac.nz> <201007202309.59704.steve@pearwood.info> <20100720142005.70AFE2007B4@kimball.webabinitio.net> <4C4FDC5D.2040605@avl.com> Message-ID: On 7/28/2010 4:42 AM, Ray Allen wrote: > I believe, in design purpose, the os.mkdir() is to match the system call > "mkdir()" exactly, the os.makedirs() is a "Super-mkdir", it provides > extra convenience for using when we want to create directories. This is > the case makedirs() should deal with. After discussion with Guido, the patch on the tracker leaves makedir alone and catches the exception, or not, in makedirs. -- Terry Jan Reedy From benjamin at python.org Wed Jul 28 23:20:52 2010 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 28 Jul 2010 15:20:52 -0600 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: <4C4B7FA9.2010001@holdenweb.com> <201007251416.37181.steve@pearwood.info> Message-ID: 2010/7/25 Stefan Behnel : > Nick Coghlan, 25.07.2010 08:29: >> >> We knew PEP 380 would be hurt by the moratorium when the moratorium >> PEP went through. >> >> The goals of the moratorium itself, in making it possible to have a >> 3.2 release that is fully supported by all of the major Python >> implementations, still apply, and I believe making an exception for >> PEP 380 *would* make those goals much harder to achieve. > > IMO, it would be worth asking the other implementations if that is the case. > It may well be that they are interested in implementing it anyway, so > getting it into CPython and the other implementations at the same time may > actually be possible. It wouldn't meet the moratorium as such, but it would > absolutely comply with its goals. Speaking from the PyPy perspective, syntax is not really a problem. It, for example, took me ~1 week to more PyPy from 2.5 to 2.7 syntax. A more interesting moratorium for us would be one on tests that are not implementation portable. :) -- Regards, Benjamin From fuzzyman at voidspace.org.uk Wed Jul 28 23:23:20 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 28 Jul 2010 22:23:20 +0100 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: <4C4B7FA9.2010001@holdenweb.com> <201007251416.37181.steve@pearwood.info> Message-ID: <4C509FC8.6080900@voidspace.org.uk> On 28/07/2010 22:20, Benjamin Peterson wrote: > 2010/7/25 Stefan Behnel: > >> Nick Coghlan, 25.07.2010 08:29: >> >>> We knew PEP 380 would be hurt by the moratorium when the moratorium >>> PEP went through. >>> >>> The goals of the moratorium itself, in making it possible to have a >>> 3.2 release that is fully supported by all of the major Python >>> implementations, still apply, and I believe making an exception for >>> PEP 380 *would* make those goals much harder to achieve. >>> >> IMO, it would be worth asking the other implementations if that is the case. >> It may well be that they are interested in implementing it anyway, so >> getting it into CPython and the other implementations at the same time may >> actually be possible. It wouldn't meet the moratorium as such, but it would >> absolutely comply with its goals. >> > Speaking from the PyPy perspective, syntax is not really a problem. > It, for example, took me ~1 week to more PyPy from 2.5 to 2.7 syntax. > A more interesting moratorium for us would be one on tests that are > not implementation portable. :) > At the PyCon language summit the IronPython guys said that syntax wasn't an issue for them either but changes on builtins *could* be an issue. All the best, Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From ideasman42 at gmail.com Thu Jul 29 00:11:39 2010 From: ideasman42 at gmail.com (Campbell Barton) Date: Thu, 29 Jul 2010 08:11:39 +1000 Subject: [Python-Dev] Best practice for new namespace (from C/API) Message-ID: Hi, I'm writing because I'm working on a project where the user can run scripts that don't reference a file but use internal application text data. Otherwise we are not doing anything tricky, only that the scripts should each run independently (no cruft left from the previous scripts namespace, sharing sys.modules etc is fine). Something which is unclear to me even after looking over pythonrun.c is the correct way to setup a namespace. For years we have been doing this and it seemed to work fine... PyObject *d = PyDict_New(); // new namespace PyDict_SetItemString(d, "__builtins__", PyEval_GetBuiltins()); PyDict_SetItemString(d, "__file__", PyString_FromString(filename)); // fake, avoids sys.argv[0] being used for warnings. /* --- snip ---*/ PyEval_EvalCode(compiled_text, d, d); Recently a developer reported a bug where pickle wasn't working, it turns out that in a few places python expects the __main__ modules namespace to match that if the running script: _pickle.c's save_global() in this case eg: >>> spam = 10 >>> print(__import__("__main__").__dict__["spam"]) ... 10 Once I found this was the problem it was simple to use __main__'s namespace however there are still things that are not clear about exactly how this should be done. Simplified code... PyObject *item, *dict= PyModule_GetDict(PyImport_AddModule("__main__")); PyDict_Clear(dict); PyDict_SetItemString(dict, "__builtins__", PyImport_AddModule("builtins")); item = PyUnicode_FromString( "__main__" ); PyDict_SetItemString( dict, "__name__", item ); Py_DECREF(item); PyDict_SetItemString(d, "__file__", PyString_FromString(filename)); // fake, avoids sys.argv[0] being used for warnings. /* --- snip ---*/ PyEval_EvalCode(compiled_text, dict, dict); Still this leaves me with the following questions... - Whats the best way to manage the namespace for running multiple scripts one after another? clear and initialize __main__'s dict each time?, keep a copy and restore it after each run? - should the original __main__ namespace be restored after running a script? - pickle expects: __import__("__main__").__dict__ == ***the current namespace***, is this apart of the python spec or just something specific to pickle? - PyDict_SetItemString(d, "__builtins__", PyEval_GetBuiltins()) acts differently to PyDict_SetItemString(dict, "__builtins__", PyImport_AddModule("builtins")), using the PyEval_GetBuiltins() dict adds every member of __builtins__ when running: print(__import__("__main__").__dict__.keys()), rather then just showing __builtins__. -- - Campbell From jnoller at gmail.com Thu Jul 29 00:57:29 2010 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 28 Jul 2010 18:57:29 -0400 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: <4C4B7FA9.2010001@holdenweb.com> <201007251416.37181.steve@pearwood.info> Message-ID: On Wed, Jul 28, 2010 at 5:20 PM, Benjamin Peterson wrote: > 2010/7/25 Stefan Behnel : >> Nick Coghlan, 25.07.2010 08:29: >>> >>> We knew PEP 380 would be hurt by the moratorium when the moratorium >>> PEP went through. >>> >>> The goals of the moratorium itself, in making it possible to have a >>> 3.2 release that is fully supported by all of the major Python >>> implementations, still apply, and I believe making an exception for >>> PEP 380 *would* make those goals much harder to achieve. >> >> IMO, it would be worth asking the other implementations if that is the case. >> It may well be that they are interested in implementing it anyway, so >> getting it into CPython and the other implementations at the same time may >> actually be possible. It wouldn't meet the moratorium as such, but it would >> absolutely comply with its goals. > > Speaking from the PyPy perspective, syntax is not really a problem. > It, for example, took me ~1 week to more PyPy from 2.5 to 2.7 syntax. > A more interesting moratorium for us would be one on tests that are > not implementation portable. :) > > > I thought at the last two pycons, we've all discussed that we should have a system in place for marking tests *and* modules within the stdlib as "will only work on FooPython". I suspect that it's waiting on the shared-stdlib effort, which is waiting on mercurial (and time). jesse From fuzzyman at voidspace.org.uk Thu Jul 29 01:00:32 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 29 Jul 2010 00:00:32 +0100 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: <4C4B7FA9.2010001@holdenweb.com> <201007251416.37181.steve@pearwood.info> Message-ID: <4C50B690.1020101@voidspace.org.uk> On 28/07/2010 23:57, Jesse Noller wrote: > On Wed, Jul 28, 2010 at 5:20 PM, Benjamin Peterson wrote: > >> 2010/7/25 Stefan Behnel: >> >>> Nick Coghlan, 25.07.2010 08:29: >>> >>>> We knew PEP 380 would be hurt by the moratorium when the moratorium >>>> PEP went through. >>>> >>>> The goals of the moratorium itself, in making it possible to have a >>>> 3.2 release that is fully supported by all of the major Python >>>> implementations, still apply, and I believe making an exception for >>>> PEP 380 *would* make those goals much harder to achieve. >>>> >>> IMO, it would be worth asking the other implementations if that is the case. >>> It may well be that they are interested in implementing it anyway, so >>> getting it into CPython and the other implementations at the same time may >>> actually be possible. It wouldn't meet the moratorium as such, but it would >>> absolutely comply with its goals. >>> >> Speaking from the PyPy perspective, syntax is not really a problem. >> It, for example, took me ~1 week to more PyPy from 2.5 to 2.7 syntax. >> A more interesting moratorium for us would be one on tests that are >> not implementation portable. :) >> >> >> >> > I thought at the last two pycons, we've all discussed that we should > have a system in place for marking tests *and* modules within the > stdlib as "will only work on FooPython". I suspect that it's waiting > on the shared-stdlib effort, which is waiting on mercurial (and time). > It is also one of the things Brett intends to work on if his proposal is accepted by the PSF board. Michael > jesse > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From catherine.devlin at gmail.com Thu Jul 29 03:46:50 2010 From: catherine.devlin at gmail.com (Catherine Devlin) Date: Wed, 28 Jul 2010 21:46:50 -0400 Subject: [Python-Dev] audience-instructors for Teach Me Python Bugfixing needed In-Reply-To: References: <20100507020717.7EB991FA9AB@kimball.webabinitio.net> <20100510200423.836CF207EAD@kimball.webabinitio.net> Message-ID: The PyOhio contribu-palooza starts this Saturday! http://www.pyohio.org/Contribute With two talks and a two-day-four-night sprint, I'm very hopeful that it will recruit and train some new core workers. I'm preparing my portion, the teach-the-newbie (me) -to-fix-a-core-bug session, and I want to make sure that I'm prepared in two ways: 1. Any bulky download/compilation steps are complete I pulled the Py3 trunk with svn co http://svn.python.org/projects/python/branches/py3k python, did the compilation steps, and verified that I can fire up the latest build. I also note that http://www.python.org/dev/ doesn't say anything about hg yet. Is there someplace else I should look for hg-centered docs? Should we just teach it using svn if that's better documented? Then again, if hg is the way of the future... I also built the docs (``cd Doc; make html``) Are there other things that I need to do to configure my machine beforehand? Things that are too long/boring for the audience to sit through while I do it live? 2. Have a good set of questions to ask. Here's what I'm planning so far: - While running ``make test`` on the Python trunk, I got an error on "test_os". Is that a problem with my machine's configuration, or with the build? Do I need to report it? Can I ignore it? Now we'll find a bug. - Do I need an account on bugs.python.org? What do I need to do to get one? - How do I find a bug suitable for me to work on? - entry-level - in Python not C - corresponding to my strong points / interests Now we'll "find" a fake bug that David has planted for us. (David, have you planted it yet?) - Can/should I make my edits directly in the trunk that I just pulled down? - Now we'll make the fix... maybe this should involve using a debugging IDE or pdb? - How do I verify that my fix worked? That it didn't break anything else? That it's written with proper style? That it doesn't generally suck? - How do I send my fix back up to the trunk? - How do I record my work in the bug tracker? (If time permits) now let's try writing a test for a gap in test coverage (not necessarily on the code we just worked on - this doesn't have to be fake) DON'T ANSWER THESE! I need to carefully guard my sincere ignorance through Saturday! (Actually, I already have a pretty good idea about some of them, but I don't want my ignorance to become any less sincere than it already is.) But, if you're David or Dan or anybody else who's going to be there, you may want to ponder how you'll guide me through it. But what I want to know from all of you is: what other questions should be on my list? I was going to address this only to David, my primary audience/instructor volunteer, but I thought it wouldn't hurt to get input from the rest of you. Thank you all! -- - Catherine http://catherinedevlin.blogspot.com/ *** PyOhio 2010 * July 31 - Aug 1 * Columbus, OH * pyohio.org *** -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.belopolsky at gmail.com Thu Jul 29 04:33:05 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 28 Jul 2010 22:33:05 -0400 Subject: [Python-Dev] Python Language Summit EuroPython 2010 In-Reply-To: References: <4C470E2B.5060406@timgolden.me.uk> <4C4884CA.9080603@trueblade.com> <4C4C70AA.1000002@trueblade.com> Message-ID: On Sun, Jul 25, 2010 at 10:52 PM, Guido van Rossum wrote: .. >>> How hard would it be to recode the sprintf language but with the >>> locale fixed to "C"? That would always be ASCII. >> >> This is exactly what I proposed at >> http://bugs.python.org/issue7584#msg110240 not so long ago. ?Given >> that stftime language uses every English letter as one of its codes >> (both caps and lower case), it would be an effort, but coding it in >> python should not be too hard. ? A C implementation would be harder, >> but there must be implementations around available under a suitable >> license that can be reused. >> >> In short, definitely +1. > > For b"...".format() we could also support a subset (still better than > hardcoding something str()-based). It appears that the treasure trove that we call the bugs' tracker already has a patch for that: http://bugs.python.org/issue3173 http://bugs.python.org/file10704/strftime.diff From greg.ewing at canterbury.ac.nz Thu Jul 29 06:53:59 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 29 Jul 2010 16:53:59 +1200 Subject: [Python-Dev] GIL musings (was Re: Thoughts fresh after EuroPython) In-Reply-To: <20100728131230.7dc49719@pitrou.net> References: <4C500CD0.3040309@voidspace.org.uk> <20100728131230.7dc49719@pitrou.net> Message-ID: <4C510967.7050707@canterbury.ac.nz> On 28/07/10 23:12, Antoine Pitrou wrote: > It should be noted, though, that a full GC can be detrimental to > real-time applications. Kristj?n has already explained how some of his > software disabled the cyclic GC, and took care of breaking cycles > manually instead. This worries me, too. I'd be upset if I could no longer write games in Python that achieve smooth animation because of unpredictable GC pauses. -- Greg From fijall at gmail.com Thu Jul 29 11:30:07 2010 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 29 Jul 2010 11:30:07 +0200 Subject: [Python-Dev] GIL musings (was Re: Thoughts fresh after EuroPython) In-Reply-To: <4C510967.7050707@canterbury.ac.nz> References: <4C500CD0.3040309@voidspace.org.uk> <20100728131230.7dc49719@pitrou.net> <4C510967.7050707@canterbury.ac.nz> Message-ID: On Thu, Jul 29, 2010 at 6:53 AM, Greg Ewing wrote: > On 28/07/10 23:12, Antoine Pitrou wrote: > >> It should be noted, though, that a full GC can be detrimental to >> real-time applications. Kristj?n has already explained how some of his >> software disabled the cyclic GC, and took care of breaking cycles >> manually instead. > > This worries me, too. I'd be upset if I could no longer write > games in Python that achieve smooth animation because of > unpredictable GC pauses. > There are always concurrent or incremental GCs which do not stop the world. From phd at phd.pp.ru Thu Jul 29 12:23:19 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Thu, 29 Jul 2010 14:23:19 +0400 Subject: [Python-Dev] Best practice for new namespace (from C/API) In-Reply-To: References: Message-ID: <20100729102319.GC1630@phd.pp.ru> Hello. We are sorry but we cannot help you. This mailing list is to work on developing Python (adding new features to Python itself and fixing bugs); if you're having problems learning, understanding or using Python, please find another forum. Probably python-list/comp.lang.python mailing list/news group is the best place; there are Python developers who participate in it; you may get a faster, and probably more complete, answer there. See http://www.python.org/community/ for other lists/news groups/fora. Thank you for understanding. Though I must admit the question is deeper than usual. On Thu, Jul 29, 2010 at 08:11:39AM +1000, Campbell Barton wrote: > - Whats the best way to manage the namespace for running multiple > scripts one after another? clear and initialize __main__'s dict each > time?, keep a copy and restore it after each run? > - should the original __main__ namespace be restored after running a script? > - pickle expects: __import__("__main__").__dict__ == ***the current > namespace***, is this apart of the python spec or just something > specific to pickle? > - PyDict_SetItemString(d, "__builtins__", PyEval_GetBuiltins()) acts > differently to PyDict_SetItemString(dict, "__builtins__", > PyImport_AddModule("builtins")), using the PyEval_GetBuiltins() dict > adds every member of __builtins__ when running: > print(__import__("__main__").__dict__.keys()), rather then just > showing __builtins__. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From ncoghlan at gmail.com Thu Jul 29 14:10:40 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 29 Jul 2010 22:10:40 +1000 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: <4C4B7FA9.2010001@holdenweb.com> <201007251416.37181.steve@pearwood.info> Message-ID: On Thu, Jul 29, 2010 at 8:57 AM, Jesse Noller wrote: > I thought at the last two pycons, we've all discussed that we should > have a system in place for marking tests *and* modules within the > stdlib as "will only work on FooPython". I suspect that it's waiting > on the shared-stdlib effort, which is waiting on mercurial (and time). @skipIf, @cpython_only and @test_impl_detail have been getting sprinkled fairly liberally throughout the test suite, so that part of the effort is already in progress. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Thu Jul 29 14:12:53 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 29 Jul 2010 22:12:53 +1000 Subject: [Python-Dev] Best practice for new namespace (from C/API) In-Reply-To: <20100729102319.GC1630@phd.pp.ru> References: <20100729102319.GC1630@phd.pp.ru> Message-ID: On Thu, Jul 29, 2010 at 8:23 PM, Oleg Broytman wrote: > Hello. > > ? We are sorry but we cannot help you. This mailing list is to work on > developing Python (adding new features to Python itself and fixing bugs); > if you're having problems learning, understanding or using Python, please > find another forum. Probably python-list/comp.lang.python mailing list/news > group is the best place; there are Python developers who participate in it; > you may get a faster, and probably more complete, answer there. See > http://www.python.org/community/ for other lists/news groups/fora. Thank > you for understanding. > ? Though I must admit the question is deeper than usual. capi-sig may also be able to provide some assistance if c.l.p isn't sufficiently helpful. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From jnoller at gmail.com Thu Jul 29 14:15:23 2010 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 29 Jul 2010 08:15:23 -0400 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: <4C4B7FA9.2010001@holdenweb.com> <201007251416.37181.steve@pearwood.info> Message-ID: On Thu, Jul 29, 2010 at 8:10 AM, Nick Coghlan wrote: > On Thu, Jul 29, 2010 at 8:57 AM, Jesse Noller wrote: >> I thought at the last two pycons, we've all discussed that we should >> have a system in place for marking tests *and* modules within the >> stdlib as "will only work on FooPython". I suspect that it's waiting >> on the shared-stdlib effort, which is waiting on mercurial (and time). > > @skipIf, @cpython_only and @test_impl_detail have been getting > sprinkled fairly liberally throughout the test suite, so that part of > the effort is already in progress. > Excellent I have some sprinkling of my own to do then, I guess. From da.waterworth at gmail.com Thu Jul 29 08:32:28 2010 From: da.waterworth at gmail.com (Daniel Waterworth) Date: Thu, 29 Jul 2010 07:32:28 +0100 Subject: [Python-Dev] unexpected import behaviour Message-ID: Hi, I'm not sure if this is a bug or not, I certainly didn't expect it. If you create a file called test.py with the following contents, class Test: pass def test_1(): import test print Test == test.Test if __name__ == '__main__': test_1() and then run it ($ python test.py), it'll print False. Now try: $python import test test.test_1() and it'll print True. Is this behaviour expected? What was the rationale for it if is? Thanks, Daniel -- active-thought.com From phd at phd.pp.ru Thu Jul 29 14:53:26 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Thu, 29 Jul 2010 16:53:26 +0400 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: References: Message-ID: <20100729125326.GA13475@phd.pp.ru> Hello. We are sorry but we cannot help you. This mailing list is to work on developing Python (adding new features to Python itself and fixing bugs); if you're having problems learning, understanding or using Python, please find another forum. Probably python-list/comp.lang.python mailing list/news group is the best place; there are Python developers who participate in it; you may get a faster, and probably more complete, answer there. See http://www.python.org/community/ for other lists/news groups/fora. Thank you for understanding. On Thu, Jul 29, 2010 at 07:32:28AM +0100, Daniel Waterworth wrote: > class Test: > pass > > def test_1(): > import test > print Test == test.Test > > if __name__ == '__main__': > test_1() > > and then run it ($ python test.py), it'll print False. The problem is that when you run the code as a script it is imported as module __main__; when you import it as 'test' you get the second copy of the module. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From ncoghlan at gmail.com Thu Jul 29 14:56:21 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 29 Jul 2010 22:56:21 +1000 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: References: Message-ID: On Thu, Jul 29, 2010 at 4:32 PM, Daniel Waterworth wrote: > Hi, > > I'm not sure if this is a bug or not, I certainly didn't expect it. If > you create a file called test.py with the following contents, > > class Test: > ? ?pass > > def test_1(): > ? ?import test > ? ?print Test == test.Test > > if __name__ == '__main__': > ? ?test_1() > > and then run it ($ python test.py), it'll print False. Now try: > > $python > import test > test.test_1() > > and it'll print True. Is this behaviour expected? What was the > rationale for it if is? The behaviour is expected, but there's no particularly deep rationale for it - the interpreter just doesn't go out of its way to try and figure out what __main__ *would* have been called if it had been imported rather than executed (your script will still print False even if you run it via "python -m test"). We certainly *could* put __main__ into sys.modules under both names (i.e. "__main__" and "test" in your example), but the backwards compatibility implications of doing so aren't particularly clear. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From fuzzyman at voidspace.org.uk Thu Jul 29 15:05:37 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 29 Jul 2010 14:05:37 +0100 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: References: Message-ID: <4C517CA1.8080709@voidspace.org.uk> On 29/07/2010 07:32, Daniel Waterworth wrote: > Hi, > > I'm not sure if this is a bug or not, I certainly didn't expect it. If > you create a file called test.py with the following contents, > The issue is that when your code is executed as a script it is run as the __main__ module and not as the test module. When you import test you then import the same module with a different name - and your Test class is recreated (so __main__.Test is then different from test.Test). When you import your code as test and it then reimports itself it is only created once. This *is* expected behaviour (not a bug), but it frequently confuses even relatively experienced programmers (it can happen by accident and cause hard to track down bugs) and I personally think that Python would be improved by issuing a warning if a __main__ script reimports itself. All the best, Michael Foord > class Test: > pass > > def test_1(): > import test > print Test == test.Test > > if __name__ == '__main__': > test_1() > > and then run it ($ python test.py), it'll print False. Now try: > > $python > import test > test.test_1() > > and it'll print True. Is this behaviour expected? What was the > rationale for it if is? > > Thanks, > > Daniel > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From mount.sarah at gmail.com Thu Jul 29 18:19:40 2010 From: mount.sarah at gmail.com (Sarah Mount) Date: Thu, 29 Jul 2010 17:19:40 +0100 Subject: [Python-Dev] [Concurrency / parallelism] Re: Thoughts fresh after EuroPython In-Reply-To: References: Message-ID: On 25 July 2010 19:26, Jesse Noller wrote: > On Sat, Jul 24, 2010 at 10:08 AM, Guido van Rossum wrote: >> - Concurrency and parallelism: Russel Winder and Sarah Mount pushed >> the idea of CSP >> (http://en.wikipedia.org/wiki/Communicating_sequential_processes) in >> several talks at the conference. They (at least Russell) emphasized >> the difference between concurrency (interleaved event streams) and >> parallelism (using many processors to speed things up). Their >> prediction is that as machines with many processing cores become more >> prevalent, the relevant architecture will change from cores sharing a >> single coherent memory (the model on which threads are based) to one >> where each core has a limited amount of private memory, and >> communication is done via message passing between the cores. This >> gives them (and me :-) hope that the GIL won't be a problem as long as >> we adopt a parallel processing model. Two competing models are the >> Actor model, which is based on asynchronous communication, and CSP, >> which is synchronous (when a writer writes to a channel, it blocks >> until a reader reads that value -- a rendezvous). At least Sarah >> suggested that both models are important. She also mentioned that a >> merger is under consideration between the two major CSP-for-Python >> packages, Py-CSP and Python-CSP. I also believe that the merger will >> be based on the stdlib multiprocessing package, but I'm not sure. I do >> expect that we may get some suggestions from that corner to make some >> minor changes to details of multiprocessing (and perhaps threading), >> and I think we should be open to those (I expect these will be good >> suggestions for small tweaks, not major overhauls). > > I'm open to changes; but remain skeptical the CSP will suddenly take > over the world :) > > There's other, competing philosophies and toolkits out there as well. > Additionally; the patches would have to be relicensed - python-csp is > under GPLv2 AFAICT. Thanks for this write-up. Just a few things to follow-up from the comments here... * There is a discussion taking place, as we speak on the python-csp list, about merging PyCSP and python-csp. Like I said in the talk, it's really a matter of details more than anything else and will likely go ahead, hopefully soon. The issue of licensing is one thing we are talking about. * We do currently use threading and multiprocessing. I think we could achieve a big performance boost by loosing mp and moving to a C implementation of much of our own work, but only because so much of mp is pure Python and implementing a message-passing library requires some rather idiosyncratic programming. That's really an implementation detail for us though, so please don't read it as a comment about mp or threading in general. * In terms of the Python standard library, python-csp was very much written with our own purposes in mind, and in order to provide a platform for the CSP "paradigm" of programming to be used in a Pythonic style. Multiprocessing as it currently stands is great, and provides good support for the sort of design patterns that are currently very common in concurrent and parallel programming, like process-safe queues and so on. That's brilliant, but I would like to see the Python stdlib support a wider range of programming styles, including but not exclusive to message-passing concurrency. java.util.concurrent does this very nicely and has a wide range of constructs such as barriers, futures and so on. Python has are a wide range of packages outside the standard library (trellis, pycells, python-csp, STM implementations, and so on) but not so much that comes inside the stdlib. I think that's a shame. I won't bang on about this too much here, we'll write up some code over the next few months and when there's something more concrete and fully-baked to discuss I think it would a really useful discussion to have. Thanks, Sarah -- Sarah Mount, Senior Lecturer, University of Wolverhampton website:? http://www.snim2.org/ twitter: @snim2 From dickinsm at gmail.com Thu Jul 29 20:47:37 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Thu, 29 Jul 2010 19:47:37 +0100 Subject: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2 Message-ID: Now that we've got the short float repr in Python, there's less value in having float.__str__ truncate to 12 significant digits (as it currently does). For Python 3.2, I propose making float.__str__ use the same algorithm as float.__repr__ for its output (and similarly for complex). Apart from simplifying the internals a little bit, one nice feature of this change is that it removes the differences in formatting between printing a float and printing a container of floats: >>> l = [1/3, 1/5, 1/7] >>> print(l) [0.3333333333333333, 0.2, 0.14285714285714285] >>> print(l[0], l[1], l[2]) 0.333333333333 0.2 0.142857142857 Any thoughts or comments on this? There's a working patch at http://bugs.python.org/issue9337 Mark From fuzzyman at voidspace.org.uk Thu Jul 29 20:52:47 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 29 Jul 2010 19:52:47 +0100 Subject: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2 In-Reply-To: References: Message-ID: <4C51CDFF.30201@voidspace.org.uk> On 29/07/2010 19:47, Mark Dickinson wrote: > Now that we've got the short float repr in Python, there's less value > in having float.__str__ truncate to 12 significant digits (as it > currently does). For Python 3.2, I propose making float.__str__ use > the same algorithm as float.__repr__ for its output (and similarly for > complex). > > Apart from simplifying the internals a little bit, one nice feature of > this change is that it removes the differences in formatting between > printing a float and printing a container of floats: > > >>>> l = [1/3, 1/5, 1/7] >>>> print(l) >>>> > [0.3333333333333333, 0.2, 0.14285714285714285] > >>>> print(l[0], l[1], l[2]) >>>> > 0.333333333333 0.2 0.142857142857 > > Any thoughts or comments on this? > > There's a working patch at http://bugs.python.org/issue9337 > +1 Michael > Mark > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From lukasz at langa.pl Thu Jul 29 20:54:32 2010 From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=) Date: Thu, 29 Jul 2010 20:54:32 +0200 Subject: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2 In-Reply-To: References: Message-ID: <424F8BE7-031B-45A6-AC69-0162A3D9C21B@langa.pl> Wiadomo?? napisana przez Mark Dickinson w dniu 2010-07-29, o godz. 20:47: > Now that we've got the short float repr in Python, there's less value > in having float.__str__ truncate to 12 significant digits (as it > currently does). For Python 3.2, I propose making float.__str__ use > the same algorithm as float.__repr__ for its output (and similarly for > complex). > > Any thoughts or comments on this? +1 -- Best regards, ?ukasz Langa tel. +48 791 080 144 WWW http://lukasz.langa.pl/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.hettinger at gmail.com Thu Jul 29 21:16:35 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Thu, 29 Jul 2010 12:16:35 -0700 Subject: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2 In-Reply-To: References: Message-ID: <55ACAB6D-3D92-4643-B7A5-5599037BA952@gmail.com> On Jul 29, 2010, at 11:47 AM, Mark Dickinson wrote: > Now that we've got the short float repr in Python, there's less value > in having float.__str__ truncate to 12 significant digits (as it > currently does). For Python 3.2, I propose making float.__str__ use > the same algorithm as float.__repr__ for its output (and similarly for > complex). When you proposed the idea at EuroPython, it seemed reasonable but we didn't go into the pros and cons. The downsides include breaking tests, changing the output of report generating scripts that aren't using string formatting, and it introduces another inter-version incompatibility. The only obvious advantage is that it makes float.__repr__ and float.__str__ the same, making one less thing to explain. Can you elaborate on other advantages? Is there something wrong with the current way? IIRC, some other tools like matlab have a relatively compact default display size for floats, perhaps because formatting matrices becomes awkward when there are too many digits shown and because many of those digits are insignificant. Also, I think those tools have a way to globally change the default number of digits. Am curious about your thoughts on the problem we're trying to solve and the implications of changing the default. Raymond From dickinsm at gmail.com Thu Jul 29 22:30:37 2010 From: dickinsm at gmail.com (Mark Dickinson) Date: Thu, 29 Jul 2010 21:30:37 +0100 Subject: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2 In-Reply-To: <55ACAB6D-3D92-4643-B7A5-5599037BA952@gmail.com> References: <55ACAB6D-3D92-4643-B7A5-5599037BA952@gmail.com> Message-ID: On Thu, Jul 29, 2010 at 8:16 PM, Raymond Hettinger wrote: > > On Jul 29, 2010, at 11:47 AM, Mark Dickinson wrote: > >> Now that we've got the short float repr in Python, there's less value >> in having float.__str__ truncate to 12 significant digits (as it >> currently does). ?For Python 3.2, I propose making float.__str__ use >> the same algorithm as float.__repr__ for its output (and similarly for >> complex). > > When you proposed the idea at EuroPython, it seemed reasonable > but we didn't go into the pros and cons. ?The downsides include > breaking tests, changing the output of report generating scripts > that aren't using string formatting, and it introduces another > inter-version incompatibility. Yes, I agree that the change has potential for breakage; it's a change that probably would have been unacceptable for Python 2.7; for Python 3.2 I think there's a little more scope, since 3.x has fewer users. And those users it does have at the moment are the early adopters, who with any luck may be more tolerant of this level of breakage. (By the time we get to 3.2 -> 3.3 that's probably not going to be true any more.) Really, this change should have gone into 3.1. FWIW, the change broke very few of the standard library tests (as Eric Smith verified): there was a (somewhat buggy) doctest in test_tokenize that needed fixing, and test_unicodedata computes a checksum that depends on the str() of various numeric values. Apart from those, only test_float and test_complex needed fixing to reflect the __str__ method changes. > The only obvious advantage is > that it makes float.__repr__ and float.__str__ the same, making > one less thing to explain. ?Can you elaborate on other advantages? > Is there something wrong with the current way? That's one advantage; as mentioned earlier the difference between str and repr causes confusion for floats in containers, where users don't realize that two different operations are being used. This is a genuine problem: I've answered questions about this a couple of times on the #python IRC channel. Another advantage is that is makes 'str' faithful: that is, if x and y are distinct floats then str(x) and str(y) are guaranteed distinct. I know I should know better, but I've been bitten by the lack of faithfulness a couple of times when debugging floating-point problems: I insert a "print(x, y)" line into the code for debugging purposes and still wonder why my 'assertEqual(x, y)' test is failing even though x and y look the same; only then do I remember that I need to use repr instead. As you say, it's just one less surprise, and one less thing to explain: a small shrinkage of the mental footprint of the language. Mark From guido at python.org Thu Jul 29 22:53:33 2010 From: guido at python.org (Guido van Rossum) Date: Thu, 29 Jul 2010 13:53:33 -0700 Subject: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2 In-Reply-To: References: <55ACAB6D-3D92-4643-B7A5-5599037BA952@gmail.com> Message-ID: On Thu, Jul 29, 2010 at 1:30 PM, Mark Dickinson wrote: > On Thu, Jul 29, 2010 at 8:16 PM, Raymond Hettinger > wrote: >> >> On Jul 29, 2010, at 11:47 AM, Mark Dickinson wrote: >> >>> Now that we've got the short float repr in Python, there's less value >>> in having float.__str__ truncate to 12 significant digits (as it >>> currently does). ?For Python 3.2, I propose making float.__str__ use >>> the same algorithm as float.__repr__ for its output (and similarly for >>> complex). >> >> When you proposed the idea at EuroPython, it seemed reasonable >> but we didn't go into the pros and cons. ?The downsides include >> breaking tests, changing the output of report generating scripts >> that aren't using string formatting, and it introduces another >> inter-version incompatibility. > > Yes, I agree that the change has potential for breakage; ?it's a > change that probably would have been unacceptable for Python 2.7; ?for > Python 3.2 I think there's a little more scope, since 3.x has fewer > users. ?And those users it does have at the moment are the early > adopters, who with any luck may be more tolerant of this level of > breakage. ?(By the time we get to 3.2 -> 3.3 that's probably not going > to be true any more.) ?Really, this change should have gone into 3.1. > > FWIW, the change broke very few of the standard library tests (as Eric > Smith verified): ?there was a (somewhat buggy) doctest in > test_tokenize that needed fixing, and test_unicodedata computes a > checksum that depends on the str() of various numeric values. ?Apart > from those, only test_float and test_complex needed fixing to reflect > the __str__ method changes. > >> The only obvious advantage is >> that it makes float.__repr__ and float.__str__ the same, making >> one less thing to explain. ?Can you elaborate on other advantages? >> Is there something wrong with the current way? > > That's one advantage; ?as mentioned earlier the difference between str > and repr causes confusion for floats in containers, where users don't > realize that two different operations are being used. ?This is a > genuine problem: I've answered questions about this a couple of times > on the #python IRC channel. > > Another advantage is that is makes 'str' faithful: ?that is, if x and > y are distinct floats then str(x) and str(y) are guaranteed distinct. > I know I should know better, but I've been bitten by the lack of > faithfulness a couple of times when debugging floating-point problems: > ?I insert a "print(x, y)" line into the code for debugging purposes > and still wonder why my 'assertEqual(x, y)' test is failing even > though x and y look the same; ?only then do I remember that I need to > use repr instead. > > As you say, it's just one less surprise, and one less thing to > explain: a small shrinkage of the mental footprint of the language. +1 from me for all the reasons Mark mentioned. -- --Guido van Rossum (python.org/~guido) From rrr at ronadam.com Thu Jul 29 23:21:40 2010 From: rrr at ronadam.com (Ron Adam) Date: Thu, 29 Jul 2010 16:21:40 -0500 Subject: [Python-Dev] [isssue 2001] Pydoc enhancement patch questions In-Reply-To: References: <4C4B404E.2090007@ronadam.com> Message-ID: > FWIW, I am +1 on dropping tkinter interface. Tkinter window looks > foreign next to browser and server-side GUI that opens a new client > window with each search topic does not strike me as most usable > design. Furthermore, I just tried to use it on my OSX laptop and it > crashed after I searched for pydoc and clicked on the first entry. > (Another issue is that search window pops under the terminal window.) > I think Tkinter interface to pydoc may make sense in IDLE, but not in > the main pydoc GUI. If the equivalent functionality is available in > the browser (preferably in the style familiar to docs.python.org > users, I don't see why we need to keep old GUI and hide new behind a > new option. I agree. What do you think of having a "-i" command line option to enter an interactive help session directly from the command line. This is easy to do. The instruction when entering and leaving need to change a bit, but that isn't hard to do. Cheers, Ron From tjreedy at udel.edu Thu Jul 29 23:28:46 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 29 Jul 2010 17:28:46 -0400 Subject: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2 In-Reply-To: References: <55ACAB6D-3D92-4643-B7A5-5599037BA952@gmail.com> Message-ID: On 7/29/2010 4:30 PM, Mark Dickinson wrote: > > As you say, it's just one less surprise, and one less thing to > explain: a small shrinkage of the mental footprint of the language. With this change, I believe the only difference between str(ob) and repr(ob) will be the addition of quotes. If so, perhaps this could be noted in the repr entry. Terry Jan Reedy From fumanchu at aminus.org Thu Jul 29 23:37:34 2010 From: fumanchu at aminus.org (Robert Brewer) Date: Thu, 29 Jul 2010 14:37:34 -0700 Subject: [Python-Dev] Proposal: make float.__str__ identical tofloat__repr__ in Python 3.2 In-Reply-To: References: Message-ID: Mark Dickinson wrote: > Now that we've got the short float repr in Python, there's less value > in having float.__str__ truncate to 12 significant digits (as it > currently does). For Python 3.2, I propose making float.__str__ use > the same algorithm as float.__repr__ for its output (and similarly for > complex). > > Apart from simplifying the internals a little bit, one nice feature of > this change is that it removes the differences in formatting between > printing a float and printing a container of floats: > > >>> l = [1/3, 1/5, 1/7] > >>> print(l) > [0.3333333333333333, 0.2, 0.14285714285714285] > >>> print(l[0], l[1], l[2]) > 0.333333333333 0.2 0.142857142857 > > Any thoughts or comments on this? > > There's a working patch at http://bugs.python.org/issue9337 Python 2.5.4 (r254:67916, Jan 20 2010, 21:44:03) >>> float("0.142857142857") * 7 0.99999999999899991 >>> float("0.14285714285714285") * 7 1.0 I've made a number of tools in the past that needed to round-trip a float through a string and back. I was under the impression that floats needed 17 decimal digits to avoid losing precision. How does one do that efficiently if neither str nor repr return 17 digits? Robert Brewer fumanchu at aminus.org From fuzzyman at voidspace.org.uk Thu Jul 29 23:53:01 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 29 Jul 2010 22:53:01 +0100 Subject: [Python-Dev] Proposal: make float.__str__ identical tofloat__repr__ in Python 3.2 In-Reply-To: References: Message-ID: <4C51F83D.3090809@voidspace.org.uk> On 29/07/2010 22:37, Robert Brewer wrote: > Mark Dickinson wrote: > >> Now that we've got the short float repr in Python, there's less value >> in having float.__str__ truncate to 12 significant digits (as it >> currently does). For Python 3.2, I propose making float.__str__ use >> the same algorithm as float.__repr__ for its output (and similarly for >> complex). >> >> Apart from simplifying the internals a little bit, one nice feature of >> this change is that it removes the differences in formatting between >> printing a float and printing a container of floats: >> >> >>>>> l = [1/3, 1/5, 1/7] >>>>> print(l) >>>>> >> [0.3333333333333333, 0.2, 0.14285714285714285] >> >>>>> print(l[0], l[1], l[2]) >>>>> >> 0.333333333333 0.2 0.142857142857 >> >> Any thoughts or comments on this? >> >> There's a working patch at http://bugs.python.org/issue9337 >> > Python 2.5.4 (r254:67916, Jan 20 2010, 21:44:03) > >>>> float("0.142857142857") * 7 >>>> > 0.99999999999899991 > >>>> float("0.14285714285714285") * 7 >>>> > 1.0 > > I've made a number of tools in the past that needed to round-trip a > float through a string and back. I was under the impression that floats > needed 17 decimal digits to avoid losing precision. How does one do that > efficiently if neither str nor repr return 17 digits? > Because every floating point number represents a range of values - and the new algorithm uses the shortest possible representation within that range of values that will return the same floating point number. Mark did an excellent presentation on this at EuroPython and the slides are online: http://www.slideshare.net/dickinsm/magical-float-repr Michael Foord > > Robert Brewer > fumanchu at aminus.org > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From raymond.hettinger at gmail.com Fri Jul 30 00:30:45 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Thu, 29 Jul 2010 15:30:45 -0700 Subject: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2 In-Reply-To: References: <55ACAB6D-3D92-4643-B7A5-5599037BA952@gmail.com> Message-ID: <7DE1C1BA-B9FC-4EC1-896B-D9902FD0DFB7@gmail.com> >> When you proposed the idea at EuroPython, it seemed reasonable >> but we didn't go into the pros and cons. The downsides include >> breaking tests, changing the output of report generating scripts >> that aren't using string formatting, and it introduces another >> inter-version incompatibility. > > Yes, I agree that the change has potential for breakage; it's a > change that probably would have been unacceptable for Python 2.7; for > Python 3.2 I think there's a little more scope, since 3.x has fewer > users. +1 for making the change to 3.2 +0 for 2.7 >> The only obvious advantage is >> that it makes float.__repr__ and float.__str__ the same, making >> one less thing to explain. Can you elaborate on other advantages? >> Is there something wrong with the current way? > > That's one advantage; as mentioned earlier the difference between str > and repr causes confusion for floats in containers, where users don't > realize that two different operations are being used. This is a > genuine problem: I've answered questions about this a couple of times > on the #python IRC channel. > > Another advantage is that is makes 'str' faithful: that is, if x and > y are distinct floats then str(x) and str(y) are guaranteed distinct. > I know I should know better, but I've been bitten by the lack of > faithfulness a couple of times when debugging floating-point problems: > I insert a "print(x, y)" line into the code for debugging purposes > and still wonder why my 'assertEqual(x, y)' test is failing even > though x and y look the same; only then do I remember that I need to > use repr instead. > > As you say, it's just one less surprise, and one less thing to > explain: a small shrinkage of the mental footprint of the language. Thanks for listing the advantages. Sounds like it is worth the cost. It also really calls into question whether there are good reasons for other types to have a __str__ that is different than their __repr__. Raymond From fuzzyman at voidspace.org.uk Fri Jul 30 00:55:50 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 29 Jul 2010 23:55:50 +0100 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) Message-ID: <4C5206F6.7090808@voidspace.org.uk> Hello all, My apologies in advance if email mangles whitespace in the code examples. I can reformulate as a PEP if that is deemed useful and this document can be found online at: http://hg.python.org/unittest2/file/tip/description.txt (Please excuse errors and omissions - but do feel free to point them out.) This is a description, and request for feedback, of the unittest plugin system that I am currently prototyping in the plugins branch of unittest2_. My goal is to merge the plugin system back into unittest itself in Python 3.2. .. _unittest2: http://hg.python.org/unittest2 As part of the prototype I have been implementing some example plugins (in unittest2/plugins/) so I can develop the mechanism against real rather than imagined use cases. Jason Pellerin, creator of nose, has been providing me with feedback and has been trying it out by porting some of the nose plugins to unittest [#]_. He comments on the system "it looks very flexible and clean". ;-) Example plugins available and included: * a pep8 and pyflakes checker * a debugger plugin that drops you into pdb on test fail / error * a doctest loader (looks for doctests in all text files in the project) * use a regex for matching files in test discovery instead of a glob * growl notifications on test run start and stop * filter individual *test methods* using a regex * load test functions from modules as well as TestCases * integration with the coverage module for coverage reporting In addition I intend to create a plugin that outputs junit compatible xml from a test run (for integration with tools like the hudson continuous integration server) and a test runner that runs tests in parallel using multiprocessing. Not all of these will be included in the merge to unittest. Which ones will is still an open question. I'd like feedback on the proposal, and hopefully approval to port it into unittest after discussion / amendment / completion. In particular I'd like feedback on the basic system, plus which events should be available and what information should be available in them. Note that the system is *not* complete in the prototype. Enough is implemented to get "the general idea" and to formalise the full system. It still needs extensive tests and the extra work in TestProgram makes it abundantly clear that refactoring there is well overdue... In the details below open questions and todos are noted. I *really* value feedback (but will ignore bikeshedding ;-) .. note:: Throughout this document I refer to the prototype implementation using names like ``unittest2.events.hooks``. Should this proposal be accepted then the names will live in the unittest package instead of unittest2. The core classes for the event system live in the current implementation in the ``unittest2.events`` namespace. Abstract ======== unittest lacks a standard way of extending it to provide commonly requested functionality, other than subclassing and overriding (and reimplementing) parts of its behaviour. This document describes a plugin system already partially prototyped in unittest2. Aspects of the plugin system include: * an events mechanism where handlers can be registered and called during a test run * a Plugin class built over the top of this for easy creation of plugins * a configuration system for specifying which plugins should be loaded and for configuring individual plugins * command line integration * the specific set of events and the information / actions available to them As the plugin system essentially just adds event calls to key places it has few backwards compatibility issues. Unfortunately existing extensions that override the parts of unittest that call these events will not be compatible with plugins that use them. Framework authors who re-implement parts of unittest, for example custom test runners, may want to add calling these events in appropriate places. Rationale ========= Why a plugin system for unittest? unittest is the standard library test framework for Python but in recent years has been eclipsed in functionality by frameworks like nose and py.test. Among the reasons for this is that these frameworks are easier to extend with plugins than unittest. unittest makes itself particularly difficult to extend by using subclassing as its basic extension mechanism. You subclass and override behaviour in its core classes like the loader, runner and result classes. This means that where you have more than one "extension" working in the same area it is very hard for them to work together. Whilst various extensions to unittest do exist (e.g. testtools, zope.testrunner etc) they don't tend to work well together. In contrast the plugin system makes creating extensions to unittest much simpler and less likely that extensions will clash with each other. nose itself exists as a large system built over the top of unittest. Extending unittest in this way was very painful for the creators of nose, and every release of Python breaks nose in some way due to changes in unittest. One of the goals of the extension mechanism is to allow nose2 to be a much thinner set of plugins over unittest(2) that is much simpler to maintain [#]_. The early indications are that the proposed system is a good fit for this goal. Low Level Mechanism ==================== The basic mechanism is having events fired at various points during a test run. Plugins can register event handler functions that will be called with an event object. Multiple functions may be registered to handle an event and event handlers can also be removed. Over the top of this is a ``Plugin`` class that simplifies building plugins on top of this mechanism. This is described in a separate section. The events live on the ``unittest2.events.hooks`` class. Handlers are added using ``+=`` and removed using ``-=``, a syntax borrowed from the .NET system. For example adding a handler for the ``startTestRun`` event:: from unittest2.events import hooks def startTestRun(event): print 'test run started at %s' % event.startTime hooks.startTestRun += startTestRun Handlers are called with an Event object specific to the event. Each event provides different information on its event objects as attributes. For example the attributes available on ``StartTestRunEvent`` objects are: * ``suite`` - the test suite for the full test run * ``runner`` - the test runner * ``result`` - the test result * ``startTime`` The name of events, whether any should be added or removed, and what information is available on the event objects are all valid topics for discussion. Specific events and the information available to them is covered in a section below. An example plugin using events directly is the ``doctestloader`` plugin. Framework authors who re-implement parts of unittest, for example custom test runners, may want to add calling these events in appropriate places. This is very simple. For example the ``pluginsLoaded`` event is fired with a ``PluginsLoadedEvent`` object that is instantiated without parameters:: from unittest2.events import hooks, PluginsLoadedEvent hooks.pluginsLoaded(PluginsLoadedEvent()) Why use event objects and not function parameters? -------------------------------------------------- There are several reasons to use event objects instead of function parameters. The *disadvantage* of this is that the information available to an event is not obvious from the signature of a handler. There are several compelling advantages however: * the signature of all handler functions is identical and therefore easy to remember * backwards compatibility - new attributes can be added to event objects (and parameters deprecated) without breaking existing plugins. Changing the way a function is called (unless all handlers have a ``**kw`` signature) is much harder. * several of the events have a lot of information available. This would make the signature of handlers huge. With an event object handlers only need to be aware of attributes they are interested in and ignore information they aren't interested in ("only pay for what you eat"). * some of the attributes are mutable - the event object is shared between all handlers, this would be less obvious if function parameters were used * calling multiple handlers and still returning a value (see the handled pattern below) The handled pattern -------------------- Several events can be used to *override* the default behaviour. For example the 'matchregexp' plugin uses the ``matchPath`` event to replace the default way of matching files for loading as tests during test discovery. The handler signals that it is handling this event, and the default implementation should not be run, by setting ``event.handled = True``:: def matchRegexp(event): pattern = event.pattern name = event.name event.handled = True path = event.path if matchFullPath: return re.match(pattern, path) return re.match(pattern, name) Where the default implementation returns a value, for example creating a test suite, or in the case of ``matchPath`` deciding if a path matches a file that should be loaded as a test, the handler can return a result. If an event sets handled on an event then no more handlers will be called for that event. Which events can be handled, and which not, is discussed in the events section. The Plugin Class ================ A sometimes-more-convenient way of creating plugins is to subclass the ``unittest2.events.Plugin`` class. By default subclassing ``Plugin`` will auto-instantiate the plugin and store the instance in a list of loaded plugins. Each plugin has a ``register()`` method that auto-hooks up all methods whose names correspond to events. Plugin classes may also provide ``configSection`` and ``commandLineSwitch`` class attributes which simplifies enabling the plugin through the command line and making available a section from the configuration file(s). A simple plugin using this is the 'debugger' plugin that starts ``pdb`` when the ``onTestFail`` event fires:: from unittest2.events import Plugin import pdb import sys class Debugger(Plugin): configSection = 'debugger' commandLineSwitch = ('D', 'debugger', 'Enter pdb on test fail or error') def __init__(self): self.errorsOnly = self.config.as_bool('errors-only', default=False) def onTestFail(self, event): value, tb = event.exc_info[1:] test = event.test if self.errorsOnly and isinstance(value, test.failureException): return original = sys.stdout sys.stdout = sys.__stdout__ try: pdb.post_mortem(tb) finally: sys.stdout = original A plugin that doesn't want to be auto-instantiated (for example a base class used for several plugins) can set ``autoCreate = False`` as a class attribute. (This attribute is only looked for on the class directly and so isn't inherited by subclasses.) If a plugin is auto-instantiated then the instance will be set as the ``instance`` attribute on the plugin class. ``configSection`` and ``commandLineSwitch`` are described in the `configuration system`_ and `command line integration`_ sections. Plugin instances also have an ``unregister`` method that unhooks all events. It doesn't exactly correspond to the ``register`` method (it undoes some of the work done when a plugin is instantiated) and so can only be called once. Plugins to be loaded are specified in configuration files. For frameworks not using the unittest test runner and configuration system APIs for loading plugins are available in the form of the ``loadPlugins`` function (which uses the configuration system to load plugins) and ``loadPlugin`` which loads an individual plugin by module name. Loading plugins just means importing the module containing the plugin. Configuration system ==================== By default the unittest2 test runner (triggered by the unit2 script or for unittest ``python -m unittest``) loads two configuration files to determine which plugins to load. A user configuration file, ~/unittest.cfg (alternative name and location would be possible), can specify plugins that will always be loaded. A per-project configuration file, unittest.cfg which should be located in the current directory when unit2 is launched, can specify plugins for individual projects. To support this system several command line options have been added to the test runner:: --config=CONFIGLOCATIONS Specify local config file location --no-user-config Don't use user config file --no-plugins Disable all plugins Several config files can be specified using ``--config``. If the user config is being loaded then it will be loaded first (if it exists), followed by the project config (if it exists) *or* any config files specified by ``--config``. ``--config`` can point to specific files, or to a directory containing a ``unittest.cfg``. Config files loaded later are merged into already loaded ones. Where a *key* is in both the later key overrides the earlier one. Where a section is in both but with different keys they are merged. (The exception to keys overriding is the 'plugins' key in the unittest section - these are combined to create a full list of plugins. Perhaps multiline values in config files could also be merged?) plugins to be loaded are specified in the ``plugins`` key of the ``unittest`` section:: [unittest] plugins = unittest2.plugins.checker unittest2.plugins.doctestloader unittest2.plugins.matchregexp unittest2.plugins.moduleloading unittest2.plugins.debugger unittest2.plugins.testcoverage unittest2.plugins.growl unittest2.plugins.filtertests The plugins are simply module names. They either hook themselves up manually on import or are created by virtue of subclassing ``Plugin``. A list of all loaded plugins is available as ``unittest2.events.loadedPlugins`` (a list of strings). For accessing config values there is a ``getConfig(sectionName=None)`` function. By default it returns the whole config data-structure but it an also return individual sections by name. If the section doesn't exist an empty section will be returned. The config data-structure is not read-only but there is no mechanism for persisting changes. The config is a dictionary of ``Section`` objects, where a section is a dictionary subclass with some convenience methods for accessing values:: section = getConfig(sectionName) integer = section.as_int('foo', default=3) number = section.as_float('bar', default=0.0) # as_list returns a list with empty lines and comment lines removed items = section.as_list('items', default=[]) # as_bool allows 'true', '1', 'on', 'yes' for True (matched case-insensitively) and # 'false', 'off', '0', 'no', '' for False value = section.as_bool('value', default=True) If a plugin specifies a ``configSection`` as a class attribute then that section will be fetched and set as the ``config`` attribute on instances. By convention plugins should use the 'always-on' key in their config section to specify that the plugin should be switched on by default. If 'always-on' exists and is set to 'True' then the ``register()`` method will be called on the plugin to hook up all events. If you don't want a plugin to be auto-registered you should fetch the config section yourself rather than using ``configSection``. If the plugin is configured to be 'always-on', and is auto-registered, then it doesn't need a command line switch to turn it on (although it may add other command line switches or options) and ``commandLineSwitch`` will be ignored. Command Line Interface ====================== Plugins may add command line options, either switches with a callback function or options that take values and will be added to a list. There are two functions that do this: ``unittest2.events.addOption`` and ``unittest2.events.addDiscoveryOption``. Some of the events are only applicable to test discovery (``matchPath`` is the only one currently I think), options that use these events should use ``addDiscoveryOption`` which will only be used if test discovery is invoked. Both functions have the same signature:: addDiscoveryOption(callback, opt=None, longOpt=None, help=None) addOption(plugin.method, 'X', '--extreme', 'Run tests in extreme mode') * ``callback`` is a callback function (taking no arguments) to be invoked if the option is on *or* a list indicating that this is an option that takes arguments, values passed in at the command line will be added to the list * ``opt`` is a short option for the command (or None) not including the leading '-' * ``longopt`` a long option for the command (or None) not including the leading '--' * ``help`` is optional help text for the option, to be displayed by ``unit2 -h`` Lowercase short options are reserved for use by unittest2 internally. Plugins may only add uppercase short options. If a plugin needs a simple command line switch (on/off) then it can set the ``commandLineSwitch`` class attribute to a tuple of ``(opt, longOpt, help)``. The ``register()`` method will be used as the callback function, automatically hooking the plugin up to events if it is switched on. The Events ========== This section details the events implemented so far, the order they are called in, what attributes are available on the event objects, whether the event is 'handleable' (and what that means for the event), plus the intended use case for the event. Events in rough order are: * ``pluginsLoaded`` * ``handleFile`` * ``matchPath`` * ``loadTestsFromNames`` * ``loadTestsFromName`` * ``loadTestsFromModule`` * ``loadTestsFromTestCase`` * ``getTestCaseNames`` * ``runnerCreated`` * ``startTestRun`` * ``startTest`` * ``onTestFail`` * ``stopTest`` * ``stopTestRun`` pluginsLoaded ------------- This event is useful for plugin initialisation. It is fired after all plugins have been loaded, the config file has been read and command line options processed. The ``PluginsLoadedEvent`` has one attribute: ``loadedPlugins`` which is a list of strings referring to all plugin modules that have been loaded. handleFile ---------- This event is fired when a file is looked at in test discovery or a *filename* is passed at the command line. It can be used for loading tests from non-Python files, like doctests from text files, or adding tests for a file like pep8 and pyflakes checks. A ``HandleFileEvent`` object has the following attributes: * ``extraTests`` - a list, extend this with tests to *add* tests that will be loaded from this file without preventing the default test loading * ``name`` - the name of the file * ``path`` - the full path of the file being looked at * ``loader`` - the ``TestLoader`` in use * ``pattern`` - the pattern being used to match files, or None if not called during test discovery * ``top_level_directory`` - the top level directory of the project tests are being loaded from, or the current working directory if not called during test discovery This event *can* be handled. If it is handled then the handler should return a test suite or None. Returning None means no tests will be loaded from this file. If any plugin has created any ``extraTests`` then these will be used even if a handler handles the event and returns None. If this event is not handled then it will be matched against the pattern (test discovery only) and either be rejected or go through for standard test loading. matchPath --------- ``matchPath`` is called to determine if a file should be loaded as a test module. This event only fires during test discovery. ``matchPath`` is only fired if the filename can be converted to a valid python module name, this is because tests are loaded by importing. If you want to load tests from files whose paths don't translate to valid python identifiers then you should use ``handleFile`` instead. A ``MatchPathEvent`` has the following attributes: * ``path`` - full path to the file * ``name`` - filename only * ``pattern`` - pattern being used for discovery This event *can* be handled. If it is handled then the handler should return True or False to indicate whether or not test loading should be attempted from this file. If this event is not handled then the pattern supplied to test discovery will be used as a glob pattern to match the filename. loadTestsFromNames ------------------ This event is fired when ``TestLoader.loadTestsFromNames`` is called. Attributes on the ``LoadFromNamesEvent`` object are: * ``loader`` - the test loader * ``names`` - a list of the names being loaded * ``module`` - the module passed to ``loader.loadTestsFromNames(...)`` * ``extraTests`` - a list of extra tests to be added to the suites loaded from the names This event can be handled. If it is handled then the handler should return a list of suites or None. Returning None means no tests will be loaded from these names. If any plugin has created any ``extraTests`` then these will be used even if a handler handles the event and returns None. If this event is not handled then ``loader.loadTestFromName`` will be called for each name to build up the list of suites. loadTestsFromName ----------------- This event is fired when ``TestLoader.loadTestsFromName`` is called. Attributes on the ``LoadFromNameEvent`` object are: * ``loader`` - the test loader * ``name`` - the name being loaded * ``module`` - the module passed to ``loader.loadTestsFromName(...)`` * ``extraTests`` - a suite of extra tests to be added to the suite loaded from the name This event can be handled. If it is handled then the handler should return a TestSuite or None. Returning None means no tests will be loaded from this name. If any plugin has created any ``extraTests`` then these will be used even if a handler handles the event and returns None. If the event is not handled then each name will be resolved and tests loaded from it, which may mean calling ``loader.loadTestsFromModule`` or ``loader.loadTestsFromTestCase``. loadTestsFromModule ------------------- This event is fired when ``TestLoader.loadTestsFromModule`` is called. It can be used to customise the loading of tests from a module, for example loading tests from functions as well as from TestCase classes. Attributes on the ``LoadFromModuleEvent`` object are: * ``loader`` - the test loader * ``module`` - the module object tests * ``extraTests`` - a suite of extra tests to be added to the suite loaded from the module This event can be handled. If it is handled then the handler should return a TestSuite or None. Returning None means no tests will be loaded from this module. If any plugin has created any ``extraTests`` then these will be used even if a handler handles the event and returns None. If the event is not handled then ``loader.loadTestsFromTestCase`` will be called for every TestCase in the module. Event if the event is handled, if the module defines a ``load_tests`` function then it *will* be called for the module. This removes the responsibility for implementing the ``load_tests`` protocol from plugin authors. loadTestsFromTestCase --------------------- This event is fired when ``TestLoader.loadTestsFromTestCase`` is called. It could be used to customise the loading of tests from a TestCase, for example loading tests with an alternative prefix or created generative / parameterized tests. Attributes on the ``LoadFromTestCaseEvent`` object are: * ``loader`` - the test loader * ``testCase`` - the test case class being loaded * ``extraTests`` - a suite of extra tests to be added to the suite loaded from the TestCase This event can be handled. If it is handled then the handler should return a TestSuite or None. Returning None means no tests will be loaded from this module. If any plugin has created any ``extraTests`` then these will be used even if a handler handles the event and returns None If the event is not handled then ``loader.getTestCaseNames`` will be called to get method names from the test case and a suite will be created by instantiating the TestCase class with each name it returns. getTestCaseNames ---------------- This event is fired when ``TestLoader.getTestCaseNames`` is called. It could be used to customise the method names used to load tests from a TestCase, for example loading tests with an alternative prefix from the default or filtering for specific names. Attributes on the ``GetTestCaseNamesEvent`` object are: * ``loader`` - the test loader * ``testCase`` - the test case class that tests are being loaded from * ``testMethodPrefix`` - set to None, modify this attribute to *change* the prefix being used for this class * ``extraNames`` - a list of extra names to use for this test case as well as the default ones * ``excludedNames`` - a list of names to exclude from loading from this class This event can be handled. If it is handled it should return a list of strings. Note that if this event returns an empty list (or None which will be replaced with an empty list then ``loadTestsFromTestCase`` will still check to see if the TestCase has a ``runTest`` method. Even if the event is handled ``extraNames`` will still be added to the list, however *excludedNames`` won't be removed as they are filtered out by the default implementation which looks for all attributes that are methods (or callable) whose name begins with ``loader.testMethodPrefix`` (or ``event.testMethodPrefix`` if that is set) and aren't in the list of excluded names (converted to a set first for efficient lookup). The list of names will also be sorted using ``loader.sortTestMethodsUsing``. runnerCreated ------------- This event is fired when the ``TextTestRunner`` is instantiated. It can be used to customize the test runner, for example replace the stream and result class, without needing to write a custom test harness. This should allow the default test runner script (``unit2`` or ``python -m untitest``) to be suitable for a greater range of projects. Projects that want to use custom test reporting should be able to do it through a plugin rather than having to rebuild the runner and result machinery, which also requires writing custom test collection too. The ``RunnerCreatedEvent`` object only has one attribute; ``runner`` which is the runner instance. startTestRun ------------ This event is fired when the test run is started. This is used, for example, by the growl notifier that displays a growl notification when a test run begins. It can also be used for filtering tests after they have all been loaded or for taking over the test run machinery altogether, for distributed testing for example. The ``StartTestRunEvent`` object has the following attributes: * ``test`` - the full suite of all tests to be run (may be modified in place) * ``result`` - the result object * ``startTime`` - the time the test run started Currently this event can be handled. This prevents the normal test run from executing, allowing an alternative implementation, but the return value is unused. Handling this event (as with handling any event) prevents other plugins from executing. This means that the it wouldn't be possible to safely combine a distributed test runner with a plugin that filters the suite. Fixing this issue is one of the open issues with the plugin system. startTest --------- This event is fired immediately before a test is executed (inside ``TestCase.run(...)``). The ``StartTestEvent`` object has the following attributes: * ``test`` - the test to be run * ``result`` - the result object * ``startTime`` - the time the test starts execution This event cannot be handled. onTestFail ---------- This event is fired when a test setUp, a test, a tearDown or a cleanUp fails or errors. It is currently used by the debugger plugin. It is *not* currently called for 'internal' unittest exceptions like ``SkipTest`` or expected failures and unexpected successes. Attributes on the ``TestFailEvent`` are: * ``test`` - the test * ``result`` - the result * ``exc_info`` - the result of ``sys.exc_info()`` after the error / fail * ``when`` - one of 'setUp', 'call', 'tearDown', or 'cleanUp' This event cannot be handled. Should this event be able to suppress raised exceptions? It should also be able to modify the traceback so that bare asserts could be used but still provide useful diagnostic information. Should this event be fired for test skips? stopTest -------- This event is fired when a test execution is completed. It includes a great deal of information about the test and could be used to completely replace test reporting, making the test result potentially obsolete. It will be used by the junit-xml plugin to generate the xml reports describing the test run. If there are errors during a tearDown or clean up functions then this event may be fired several times for a test. For each call the ``stage`` will be different, although there could be several errors during clean up functions. Attributes on the ``StopTestEvent`` are: * ``test`` - the test * ``result`` - the result * ``exc_info`` - the result of ``sys.exc_info()`` after an error / fail or None for success * ``stopTime``- time the test stopped, including tear down and clean up functions * ``timeTaken`` - total time for test execution from setUp to clean up functions * ``stage`` - one of setUp, call, tearDown, cleanUp, or None for success * ``outcome`` - one of passed, failed, error, skipped, unexpectedSuccess, expectedFailure The outcomes all correspond to an attribute that will be set to True or False depending on outcome: * ``passed`` * ``failed`` * ``error`` * ``skipped`` * ``unexpectedSuccess`` * ``expectedFailure`` In addition there is a ``skipReason`` that will be None unless the test was skipped, in which case it will be a string containing the reason. This event cannot be handled. stopTestRun ----------- This event is fired when the test run completes. It is useful for reporting tools. The ``StopTestRunEvent`` event objects have the following attributes: * ``runner`` - the test runner * ``result`` - the test result * ``stopTime`` - the time the test run completes * ``timeTaken`` - total time taken by the test run From fuzzyman at voidspace.org.uk Fri Jul 30 01:03:11 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 30 Jul 2010 00:03:11 +0100 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <4C5206F6.7090808@voidspace.org.uk> References: <4C5206F6.7090808@voidspace.org.uk> Message-ID: <4C5208AF.6080808@voidspace.org.uk> Damn, the email was truncated. Probably my fault. The part missed off is: Not Yet Implemented =================== Except where noted, everything in this document is already working in the prototype. There are a few open issues and things still to be implemented. Certain event attributes should be read only (like extraTests, extraNames etc) because they should only be extended or modified in place instead of being replaced. Add an epilogue to the optparse help messages. A messaging API for plugins that respects verbosity. The ``startTestRun`` event needs an alternative way of replacing the default test run machinery without stopping other plugins from executing. It should only be possible for *one* plugin to use this. Should the ``onTestFailEvent`` be able to supress raised exceptions? It should also be able to modify the traceback so that bare asserts could be used but still provide useful diagnostic information. Should this event be fired for test skips? Command line options to unittest, like verbosity, should be put into the config data structure so that they are accessible to plugins. TestFailEvent needs to be fired on failures in setUpClass and setUpModule etc Plugin.register and Plugin.unregister aren't precisely complementary. This means that unregister can only be called once. This may not be an issue. Plugin.unregister will fail if any events have already been unhooked. This can easily be fixed. If the merge to unittest in Python 3.2 happens we need to decide which of the example plugins will be included in unittest. (They will probably all remain in unittest2.) Custom test outcomes are *not* supported; making interoperating tools or plugins in the presence of custom outcomes can be very hard. Should unittest2 have a different config file from unittest, so they can be configured side-by-side? (``unittest2.cfg`` perhaps.) Alternatively the same config file could be used with a '[unittest2]' section instead of '[unittest]'. Should ``StopTestEvent.timeTaken`` (plus startTime / stopTime etc) include the time for setUp, tearDown and cleanUps? Should this information be available separately perhaps? The discovery command line interface has changed a lot and the tests need reviewing to ensure they still provide full coverage. Should there be an "excluded-plugins" section in the config files, so projects can prevent incompatible plugins from being loaded? Should multiline values in config files be merged instead of overriding each other. (This is effectively what happens for the plugins list.) Should ``handleFile`` be fired when a test name is specified at the command line? (This would be 'tricky' as ``handleFile`` will have to be called for the containing module and then the correct name pulled out of the module.) Additional Changes ================== Alongside, and in support of, the plugin system a few changes have been made to unittest2. These either have no, or very minor, backwards compatibility issues. Changes so far are: TestLoader has a new attribute ``DEFAULT_PATTERN``. This is so that the regex matching plugin can change the default pattern used for test discovery when no pattern is explicitly provided. Command line parsing is all done by optparse, removing the use of getopt. This makes the help messages more consistent but makes the usage messages less useful in some situations. This can be fixed with the use of the optparse epilogue. unit2 (the default test runner) runs test discovery if invoked without any arguments. unit2 can execute tests in filenames as well as module names - so long as the module pointed to by the filename is importable from the current directory. FunctionTestCase.id() returns 'module.funcname' instead of just funcname. Added util.formatTraceback, the default way of formatting tracebacks. TestCase has a new formatTraceback method (delegating to util.formatTraceback). TestCase instances can implement formatTraceback to control how the traceback for errors and failures are represented. Useful for test items that don't represent Python tests, for example the pep8 / pyflakes checker and theoretical javascript test runners such as exist for py.test and nose. If you specify test names (modules, classes etc) at the command line they will be loaded individually using ``loader.loadTestsFromName`` instead of collectively with ``loader.loadTestsFromNames``. This enables individual names to be checked to see if they refer to filenames. References ========== .. [#] See http://bitbucket.org/jpellerin/unittest2/src/tip/unittest2/plugins/attrib.py and http://bitbucket.org/jpellerin/unittest2/src/tip/unittest2/plugins/errormaster.py .. [#] http://lists.idyll.org/pipermail/testing-in-python/2010-March/002799.html On 29/07/2010 23:55, Michael Foord wrote: > Hello all, > > My apologies in advance if email mangles whitespace in the code > examples. I can reformulate as a PEP if that is deemed useful and this > document can be found online at: > > http://hg.python.org/unittest2/file/tip/description.txt > > (Please excuse errors and omissions - but do feel free to point them > out.) > > This is a description, and request for feedback, of the unittest > plugin system that I am currently prototyping in the plugins branch of > unittest2_. My goal is to merge the plugin system back into unittest > itself in Python 3.2. > > .. _unittest2: http://hg.python.org/unittest2 > > As part of the prototype I have been implementing some example plugins > (in unittest2/plugins/) so I can develop the mechanism against real > rather than imagined use cases. Jason Pellerin, creator of nose, has > been providing me with feedback and has been trying it out by porting > some of the nose plugins to unittest [#]_. He comments on the system > "it looks very flexible and clean". ;-) > > Example plugins available and included: > > * a pep8 and pyflakes checker > * a debugger plugin that drops you into pdb on test fail / error > * a doctest loader (looks for doctests in all text files in the project) > * use a regex for matching files in test discovery instead of a glob > * growl notifications on test run start and stop > * filter individual *test methods* using a regex > * load test functions from modules as well as TestCases > * integration with the coverage module for coverage reporting > > In addition I intend to create a plugin that outputs junit compatible > xml from a test run (for integration with tools like the hudson > continuous integration server) and a test runner that runs tests in > parallel using multiprocessing. > > Not all of these will be included in the merge to unittest. Which ones > will is still an open question. > > I'd like feedback on the proposal, and hopefully approval to port it > into unittest after discussion / amendment / completion. In particular > I'd like feedback on the basic system, plus which events should be > available and what information should be available in them. Note that > the system is *not* complete in the prototype. Enough is implemented > to get "the general idea" and to formalise the full system. It still > needs extensive tests and the extra work in TestProgram makes it > abundantly clear that refactoring there is well overdue... > > In the details below open questions and todos are noted. I *really* > value feedback (but will ignore bikeshedding ;-) > > .. note:: > > Throughout this document I refer to the prototype implementation using > names like ``unittest2.events.hooks``. Should this proposal be > accepted then the names will live in the unittest package instead of > unittest2. > > The core classes for the event system live in the current > implementation in the ``unittest2.events`` namespace. > > > Abstract > ======== > > unittest lacks a standard way of extending it to provide commonly > requested functionality, other than subclassing and overriding (and > reimplementing) parts of its behaviour. This document describes a > plugin system already partially prototyped in unittest2. > > Aspects of the plugin system include: > > * an events mechanism where handlers can be registered and called > during a test run > * a Plugin class built over the top of this for easy creation of plugins > * a configuration system for specifying which plugins should be loaded > and for configuring individual plugins > * command line integration > * the specific set of events and the information / actions available > to them > > As the plugin system essentially just adds event calls to key places > it has few backwards compatibility issues. Unfortunately existing > extensions that override the parts of unittest that call these events > will not be compatible with plugins that use them. Framework authors > who re-implement parts of unittest, for example custom test runners, > may want to add calling these events in appropriate places. > > > Rationale > ========= > > Why a plugin system for unittest? > > unittest is the standard library test framework for Python but in > recent years has been eclipsed in functionality by frameworks like > nose and py.test. Among the reasons for this is that these frameworks > are easier to extend with plugins than unittest. unittest makes itself > particularly difficult to extend by using subclassing as its basic > extension mechanism. You subclass and override behaviour in its core > classes like the loader, runner and result classes. > > This means that where you have more than one "extension" working in > the same area it is very hard for them to work together. Whilst > various extensions to unittest do exist (e.g. testtools, > zope.testrunner etc) they don't tend to work well together. In > contrast the plugin system makes creating extensions to unittest much > simpler and less likely that extensions will clash with each other. > > nose itself exists as a large system built over the top of unittest. > Extending unittest in this way was very painful for the creators of > nose, and every release of Python breaks nose in some way due to > changes in unittest. One of the goals of the extension mechanism is to > allow nose2 to be a much thinner set of plugins over unittest(2) that > is much simpler to maintain [#]_. The early indications are that the > proposed system is a good fit for this goal. > > > Low Level Mechanism > ==================== > > The basic mechanism is having events fired at various points during a > test run. Plugins can register event handler functions that will be > called with an event object. Multiple functions may be registered to > handle an event and event handlers can also be removed. > > Over the top of this is a ``Plugin`` class that simplifies building > plugins on top of this mechanism. This is described in a separate > section. > > The events live on the ``unittest2.events.hooks`` class. Handlers are > added using ``+=`` and removed using ``-=``, a syntax borrowed from > the .NET system. > > For example adding a handler for the ``startTestRun`` event:: > > from unittest2.events import hooks > > def startTestRun(event): > print 'test run started at %s' % event.startTime > > hooks.startTestRun += startTestRun > > Handlers are called with an Event object specific to the event. Each > event provides different information on its event objects as > attributes. For example the attributes available on > ``StartTestRunEvent`` objects are: > > * ``suite`` - the test suite for the full test run > * ``runner`` - the test runner > * ``result`` - the test result > * ``startTime`` > > The name of events, whether any should be added or removed, and what > information is available on the event objects are all valid topics for > discussion. Specific events and the information available to them is > covered in a section below. > > An example plugin using events directly is the ``doctestloader`` plugin. > > Framework authors who re-implement parts of unittest, for example > custom test runners, may want to add calling these events in > appropriate places. This is very simple. For example the > ``pluginsLoaded`` event is fired with a ``PluginsLoadedEvent`` object > that is instantiated without parameters:: > > from unittest2.events import hooks, PluginsLoadedEvent > > hooks.pluginsLoaded(PluginsLoadedEvent()) > > > Why use event objects and not function parameters? > -------------------------------------------------- > > There are several reasons to use event objects instead of function > parameters. The *disadvantage* of this is that the information > available to an event is not obvious from the signature of a handler. > There are several compelling advantages however: > > * the signature of all handler functions is identical and therefore > easy to remember > > * backwards compatibility - new attributes can be added to event > objects (and parameters deprecated) without breaking existing plugins. > Changing the way a function is called (unless all handlers have a > ``**kw`` signature) is much harder. > > * several of the events have a lot of information available. This > would make the signature of handlers huge. With an event object > handlers only need to be aware of attributes they are interested in > and ignore information they aren't interested in ("only pay for what > you eat"). > > * some of the attributes are mutable - the event object is shared > between all handlers, this would be less obvious if function > parameters were used > > * calling multiple handlers and still returning a value (see the > handled pattern below) > > > The handled pattern > -------------------- > > Several events can be used to *override* the default behaviour. For > example the 'matchregexp' plugin uses the ``matchPath`` event to > replace the default way of matching files for loading as tests during > test discovery. The handler signals that it is handling this event, > and the default implementation should not be run, by setting > ``event.handled = True``:: > > def matchRegexp(event): > pattern = event.pattern > name = event.name > event.handled = True > path = event.path > if matchFullPath: > return re.match(pattern, path) > return re.match(pattern, name) > > Where the default implementation returns a value, for example creating > a test suite, or in the case of ``matchPath`` deciding if a path > matches a file that should be loaded as a test, the handler can return > a result. > > If an event sets handled on an event then no more handlers will be > called for that event. Which events can be handled, and which not, is > discussed in the events section. > > > The Plugin Class > ================ > > A sometimes-more-convenient way of creating plugins is to subclass the > ``unittest2.events.Plugin`` class. By default subclassing ``Plugin`` > will auto-instantiate the plugin and store the instance in a list of > loaded plugins. > > Each plugin has a ``register()`` method that auto-hooks up all methods > whose names correspond to events. Plugin classes may also provide > ``configSection`` and ``commandLineSwitch`` class attributes which > simplifies enabling the plugin through the command line and making > available a section from the configuration file(s). > > A simple plugin using this is the 'debugger' plugin that starts > ``pdb`` when the ``onTestFail`` event fires:: > > from unittest2.events import Plugin > > import pdb > import sys > > class Debugger(Plugin): > > configSection = 'debugger' > commandLineSwitch = ('D', 'debugger', 'Enter pdb on test fail or error') > > def __init__(self): > self.errorsOnly = self.config.as_bool('errors-only', default=False) > > def onTestFail(self, event): > value, tb = event.exc_info[1:] > test = event.test > if self.errorsOnly and isinstance(value, test.failureException): > return > original = sys.stdout > sys.stdout = sys.__stdout__ > try: > pdb.post_mortem(tb) > finally: > sys.stdout = original > > A plugin that doesn't want to be auto-instantiated (for example a base > class used for several plugins) can set ``autoCreate = False`` as a > class attribute. (This attribute is only looked for on the class > directly and so isn't inherited by subclasses.) If a plugin is > auto-instantiated then the instance will be set as the ``instance`` > attribute on the plugin class. > > ``configSection`` and ``commandLineSwitch`` are described in the > `configuration system`_ and `command line integration`_ sections. > > Plugin instances also have an ``unregister`` method that unhooks all > events. It doesn't exactly correspond to the ``register`` method (it > undoes some of the work done when a plugin is instantiated) and so can > only be called once. > > Plugins to be loaded are specified in configuration files. For > frameworks not using the unittest test runner and configuration system > APIs for loading plugins are available in the form of the > ``loadPlugins`` function (which uses the configuration system to load > plugins) and ``loadPlugin`` which loads an individual plugin by module > name. Loading plugins just means importing the module containing the > plugin. > > > > Configuration system > ==================== > > By default the unittest2 test runner (triggered by the unit2 script or > for unittest ``python -m unittest``) loads two configuration files to > determine which plugins to load. > > A user configuration file, ~/unittest.cfg (alternative name and > location would be possible), can specify plugins that will always be > loaded. A per-project configuration file, unittest.cfg which should be > located in the current directory when unit2 is launched, can specify > plugins for individual projects. > > To support this system several command line options have been added to > the test runner:: > > --config=CONFIGLOCATIONS > Specify local config file location > --no-user-config Don't use user config file > --no-plugins Disable all plugins > > Several config files can be specified using ``--config``. If the user > config is being loaded then it will be loaded first (if it exists), > followed by the project config (if it exists) *or* any config files > specified by ``--config``. ``--config`` can point to specific files, > or to a directory containing a ``unittest.cfg``. > > Config files loaded later are merged into already loaded ones. Where a > *key* is in both the later key overrides the earlier one. Where a > section is in both but with different keys they are merged. (The > exception to keys overriding is the 'plugins' key in the unittest > section - these are combined to create a full list of plugins. Perhaps > multiline values in config files could also be merged?) > > plugins to be loaded are specified in the ``plugins`` key of the > ``unittest`` section:: > > [unittest] > plugins = > unittest2.plugins.checker > unittest2.plugins.doctestloader > unittest2.plugins.matchregexp > unittest2.plugins.moduleloading > unittest2.plugins.debugger > unittest2.plugins.testcoverage > unittest2.plugins.growl > unittest2.plugins.filtertests > > The plugins are simply module names. They either hook themselves up > manually on import or are created by virtue of subclassing ``Plugin``. > A list of all loaded plugins is available as > ``unittest2.events.loadedPlugins`` (a list of strings). > > For accessing config values there is a ``getConfig(sectionName=None)`` > function. By default it returns the whole config data-structure but it > an also return individual sections by name. If the section doesn't > exist an empty section will be returned. The config data-structure is > not read-only but there is no mechanism for persisting changes. > > The config is a dictionary of ``Section`` objects, where a section is > a dictionary subclass with some convenience methods for accessing > values:: > > section = getConfig(sectionName) > > integer = section.as_int('foo', default=3) > number = section.as_float('bar', default=0.0) > > # as_list returns a list with empty lines and comment lines removed > items = section.as_list('items', default=[]) > > # as_bool allows 'true', '1', 'on', 'yes' for True (matched > case-insensitively) and > # 'false', 'off', '0', 'no', '' for False > value = section.as_bool('value', default=True) > > If a plugin specifies a ``configSection`` as a class attribute then > that section will be fetched and set as the ``config`` attribute on > instances. > > By convention plugins should use the 'always-on' key in their config > section to specify that the plugin should be switched on by default. > If 'always-on' exists and is set to 'True' then the ``register()`` > method will be called on the plugin to hook up all events. If you > don't want a plugin to be auto-registered you should fetch the config > section yourself rather than using ``configSection``. > > If the plugin is configured to be 'always-on', and is auto-registered, > then it doesn't need a command line switch to turn it on (although it > may add other command line switches or options) and > ``commandLineSwitch`` will be ignored. > > > Command Line Interface > ====================== > > Plugins may add command line options, either switches with a callback > function or options that take values and will be added to a list. > There are two functions that do this: ``unittest2.events.addOption`` > and ``unittest2.events.addDiscoveryOption``. Some of the events are > only applicable to test discovery (``matchPath`` is the only one > currently I think), options that use these events should use > ``addDiscoveryOption`` which will only be used if test discovery is > invoked. > > Both functions have the same signature:: > > addDiscoveryOption(callback, opt=None, longOpt=None, help=None) > > addOption(plugin.method, 'X', '--extreme', 'Run tests in extreme mode') > > * ``callback`` is a callback function (taking no arguments) to be > invoked if the option is on *or* a list indicating that this is an > option that takes arguments, values passed in at the command line will > be added to the list > * ``opt`` is a short option for the command (or None) not including > the leading '-' > * ``longopt`` a long option for the command (or None) not including > the leading '--' > * ``help`` is optional help text for the option, to be displayed by > ``unit2 -h`` > > Lowercase short options are reserved for use by unittest2 internally. > Plugins may only add uppercase short options. > > If a plugin needs a simple command line switch (on/off) then it can > set the ``commandLineSwitch`` class attribute to a tuple of ``(opt, > longOpt, help)``. The ``register()`` method will be used as the > callback function, automatically hooking the plugin up to events if it > is switched on. > > > The Events > ========== > > This section details the events implemented so far, the order they are > called in, what attributes are available on the event objects, whether > the event is 'handleable' (and what that means for the event), plus > the intended use case for the event. > > Events in rough order are: > > * ``pluginsLoaded`` > * ``handleFile`` > * ``matchPath`` > * ``loadTestsFromNames`` > * ``loadTestsFromName`` > * ``loadTestsFromModule`` > * ``loadTestsFromTestCase`` > * ``getTestCaseNames`` > * ``runnerCreated`` > * ``startTestRun`` > * ``startTest`` > * ``onTestFail`` > * ``stopTest`` > * ``stopTestRun`` > > > pluginsLoaded > ------------- > > This event is useful for plugin initialisation. It is fired after all > plugins have been loaded, the config file has been read and command > line options processed. > > The ``PluginsLoadedEvent`` has one attribute: ``loadedPlugins`` which > is a list of strings referring to all plugin modules that have been > loaded. > > > handleFile > ---------- > > This event is fired when a file is looked at in test discovery or a > *filename* is passed at the command line. It can be used for loading > tests from non-Python files, like doctests from text files, or adding > tests for a file like pep8 and pyflakes checks. > > A ``HandleFileEvent`` object has the following attributes: > > * ``extraTests`` - a list, extend this with tests to *add* tests that > will be loaded from this file without preventing the default test loading > * ``name`` - the name of the file > * ``path`` - the full path of the file being looked at > * ``loader`` - the ``TestLoader`` in use > * ``pattern`` - the pattern being used to match files, or None if not > called during test discovery > * ``top_level_directory`` - the top level directory of the project > tests are being loaded from, or the current working directory if not > called during test discovery > > This event *can* be handled. If it is handled then the handler should > return a test suite or None. Returning None means no tests will be > loaded from this file. If any plugin has created any ``extraTests`` > then these will be used even if a handler handles the event and > returns None. > > If this event is not handled then it will be matched against the > pattern (test discovery only) and either be rejected or go through for > standard test loading. > > > matchPath > --------- > > ``matchPath`` is called to determine if a file should be loaded as a > test module. This event only fires during test discovery. > > ``matchPath`` is only fired if the filename can be converted to a > valid python module name, this is because tests are loaded by > importing. If you want to load tests from files whose paths don't > translate to valid python identifiers then you should use > ``handleFile`` instead. > > A ``MatchPathEvent`` has the following attributes: > > * ``path`` - full path to the file > * ``name`` - filename only > * ``pattern`` - pattern being used for discovery > > This event *can* be handled. If it is handled then the handler should > return True or False to indicate whether or not test loading should be > attempted from this file. If this event is not handled then the > pattern supplied to test discovery will be used as a glob pattern to > match the filename. > > > loadTestsFromNames > ------------------ > > This event is fired when ``TestLoader.loadTestsFromNames`` is called. > > Attributes on the ``LoadFromNamesEvent`` object are: > > * ``loader`` - the test loader > * ``names`` - a list of the names being loaded > * ``module`` - the module passed to ``loader.loadTestsFromNames(...)`` > * ``extraTests`` - a list of extra tests to be added to the suites > loaded from the names > > This event can be handled. If it is handled then the handler should > return a list of suites or None. Returning None means no tests will be > loaded from these names. If any plugin has created any ``extraTests`` > then these will be used even if a handler handles the event and > returns None. > > If this event is not handled then ``loader.loadTestFromName`` will be > called for each name to build up the list of suites. > > > loadTestsFromName > ----------------- > > This event is fired when ``TestLoader.loadTestsFromName`` is called. > > Attributes on the ``LoadFromNameEvent`` object are: > > * ``loader`` - the test loader > * ``name`` - the name being loaded > * ``module`` - the module passed to ``loader.loadTestsFromName(...)`` > * ``extraTests`` - a suite of extra tests to be added to the suite > loaded from the name > > This event can be handled. If it is handled then the handler should > return a TestSuite or None. Returning None means no tests will be > loaded from this name. If any plugin has created any ``extraTests`` > then these will be used even if a handler handles the event and > returns None. > > If the event is not handled then each name will be resolved and tests > loaded from it, which may mean calling ``loader.loadTestsFromModule`` > or ``loader.loadTestsFromTestCase``. > > > > loadTestsFromModule > ------------------- > > This event is fired when ``TestLoader.loadTestsFromModule`` is called. > It can be used to customise the loading of tests from a module, for > example loading tests from functions as well as from TestCase classes. > > Attributes on the ``LoadFromModuleEvent`` object are: > > * ``loader`` - the test loader > * ``module`` - the module object tests > * ``extraTests`` - a suite of extra tests to be added to the suite > loaded from the module > > This event can be handled. If it is handled then the handler should > return a TestSuite or None. Returning None means no tests will be > loaded from this module. If any plugin has created any ``extraTests`` > then these will be used even if a handler handles the event and > returns None. > > If the event is not handled then ``loader.loadTestsFromTestCase`` will > be called for every TestCase in the module. > > Event if the event is handled, if the module defines a ``load_tests`` > function then it *will* be called for the module. This removes the > responsibility for implementing the ``load_tests`` protocol from > plugin authors. > > > loadTestsFromTestCase > --------------------- > > This event is fired when ``TestLoader.loadTestsFromTestCase`` is > called. It could be used to customise the loading of tests from a > TestCase, for example loading tests with an alternative prefix or > created generative / parameterized tests. > > Attributes on the ``LoadFromTestCaseEvent`` object are: > > * ``loader`` - the test loader > * ``testCase`` - the test case class being loaded > * ``extraTests`` - a suite of extra tests to be added to the suite > loaded from the TestCase > > This event can be handled. If it is handled then the handler should > return a TestSuite or None. Returning None means no tests will be > loaded from this module. If any plugin has created any ``extraTests`` > then these will be used even if a handler handles the event and > returns None > > If the event is not handled then ``loader.getTestCaseNames`` will be > called to get method names from the test case and a suite will be > created by instantiating the TestCase class with each name it returns. > > > getTestCaseNames > ---------------- > > This event is fired when ``TestLoader.getTestCaseNames`` is called. It > could be used to customise the method names used to load tests from a > TestCase, for example loading tests with an alternative prefix from > the default or filtering for specific names. > > Attributes on the ``GetTestCaseNamesEvent`` object are: > > * ``loader`` - the test loader > * ``testCase`` - the test case class that tests are being loaded from > * ``testMethodPrefix`` - set to None, modify this attribute to > *change* the prefix being used for this class > * ``extraNames`` - a list of extra names to use for this test case as > well as the default ones > * ``excludedNames`` - a list of names to exclude from loading from > this class > > This event can be handled. If it is handled it should return a list of > strings. Note that if this event returns an empty list (or None which > will be replaced with an empty list then ``loadTestsFromTestCase`` > will still check to see if the TestCase has a ``runTest`` method. > > Even if the event is handled ``extraNames`` will still be added to the > list, however *excludedNames`` won't be removed as they are filtered > out by the default implementation which looks for all attributes that > are methods (or callable) whose name begins with > ``loader.testMethodPrefix`` (or ``event.testMethodPrefix`` if that is > set) and aren't in the list of excluded names (converted to a set > first for efficient lookup). > > The list of names will also be sorted using > ``loader.sortTestMethodsUsing``. > > > runnerCreated > ------------- > > This event is fired when the ``TextTestRunner`` is instantiated. It > can be used to customize the test runner, for example replace the > stream and result class, without needing to write a custom test > harness. This should allow the default test runner script (``unit2`` > or ``python -m untitest``) to be suitable for a greater range of > projects. Projects that want to use custom test reporting should be > able to do it through a plugin rather than having to rebuild the > runner and result machinery, which also requires writing custom test > collection too. > > The ``RunnerCreatedEvent`` object only has one attribute; ``runner`` > which is the runner instance. > > > startTestRun > ------------ > > This event is fired when the test run is started. This is used, for > example, by the growl notifier that displays a growl notification when > a test run begins. It can also be used for filtering tests after they > have all been loaded or for taking over the test run machinery > altogether, for distributed testing for example. > > The ``StartTestRunEvent`` object has the following attributes: > > * ``test`` - the full suite of all tests to be run (may be modified in > place) > * ``result`` - the result object > * ``startTime`` - the time the test run started > > Currently this event can be handled. This prevents the normal test run > from executing, allowing an alternative implementation, but the return > value is unused. Handling this event (as with handling any event) > prevents other plugins from executing. This means that the it wouldn't > be possible to safely combine a distributed test runner with a plugin > that filters the suite. Fixing this issue is one of the open issues > with the plugin system. > > > startTest > --------- > > This event is fired immediately before a test is executed (inside > ``TestCase.run(...)``). > > The ``StartTestEvent`` object has the following attributes: > > * ``test`` - the test to be run > * ``result`` - the result object > * ``startTime`` - the time the test starts execution > > This event cannot be handled. > > > onTestFail > ---------- > > This event is fired when a test setUp, a test, a tearDown or a cleanUp > fails or errors. It is currently used by the debugger plugin. It is > *not* currently called for 'internal' unittest exceptions like > ``SkipTest`` or expected failures and unexpected successes. > > Attributes on the ``TestFailEvent`` are: > > * ``test`` - the test > * ``result`` - the result > * ``exc_info`` - the result of ``sys.exc_info()`` after the error / fail > * ``when`` - one of 'setUp', 'call', 'tearDown', or 'cleanUp' > > This event cannot be handled. Should this event be able to suppress > raised exceptions? It should also be able to modify the traceback so > that bare asserts could be used but still provide useful diagnostic > information. Should this event be fired for test skips? > > > stopTest > -------- > > This event is fired when a test execution is completed. It includes a > great deal of information about the test and could be used to > completely replace test reporting, making the test result potentially > obsolete. It will be used by the junit-xml plugin to generate the xml > reports describing the test run. > > If there are errors during a tearDown or clean up functions then this > event may be fired several times for a test. For each call the > ``stage`` will be different, although there could be several errors > during clean up functions. > > Attributes on the ``StopTestEvent`` are: > > * ``test`` - the test > * ``result`` - the result > * ``exc_info`` - the result of ``sys.exc_info()`` after an error / > fail or None for success > * ``stopTime``- time the test stopped, including tear down and clean > up functions > * ``timeTaken`` - total time for test execution from setUp to clean up > functions > * ``stage`` - one of setUp, call, tearDown, cleanUp, or None for success > * ``outcome`` - one of passed, failed, error, skipped, > unexpectedSuccess, expectedFailure > > The outcomes all correspond to an attribute that will be set to True > or False depending on outcome: > > * ``passed`` > * ``failed`` > * ``error`` > * ``skipped`` > * ``unexpectedSuccess`` > * ``expectedFailure`` > > In addition there is a ``skipReason`` that will be None unless the > test was skipped, in which case it will be a string containing the > reason. > > This event cannot be handled. > > > stopTestRun > ----------- > > This event is fired when the test run completes. It is useful for > reporting tools. > > The ``StopTestRunEvent`` event objects have the following attributes: > > * ``runner`` - the test runner > * ``result`` - the test result > * ``stopTime`` - the time the test run completes > * ``timeTaken`` - total time taken by the test run > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From benjamin at python.org Fri Jul 30 01:40:32 2010 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 29 Jul 2010 17:40:32 -0600 Subject: [Python-Dev] Thoughts fresh after EuroPython In-Reply-To: References: <4C4B7FA9.2010001@holdenweb.com> <201007251416.37181.steve@pearwood.info> Message-ID: 2010/7/29 Nick Coghlan : > On Thu, Jul 29, 2010 at 8:57 AM, Jesse Noller wrote: >> I thought at the last two pycons, we've all discussed that we should >> have a system in place for marking tests *and* modules within the >> stdlib as "will only work on FooPython". I suspect that it's waiting >> on the shared-stdlib effort, which is waiting on mercurial (and time). > > @skipIf, @cpython_only and @test_impl_detail have been getting > sprinkled fairly liberally throughout the test suite, so that part of > the effort is already in progress. Note that as I port PyPy 2.7, I'm marking tests. -- Regards, Benjamin From benjamin at python.org Fri Jul 30 01:41:41 2010 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 29 Jul 2010 17:41:41 -0600 Subject: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2 In-Reply-To: <7DE1C1BA-B9FC-4EC1-896B-D9902FD0DFB7@gmail.com> References: <55ACAB6D-3D92-4643-B7A5-5599037BA952@gmail.com> <7DE1C1BA-B9FC-4EC1-896B-D9902FD0DFB7@gmail.com> Message-ID: 2010/7/29 Raymond Hettinger : > >>> When you proposed the idea at EuroPython, it seemed reasonable >>> but we didn't go into the pros and cons. ?The downsides include >>> breaking tests, changing the output of report generating scripts >>> that aren't using string formatting, and it introduces another >>> inter-version incompatibility. >> >> Yes, I agree that the change has potential for breakage; ?it's a >> change that probably would have been unacceptable for Python 2.7; ?for >> Python 3.2 I think there's a little more scope, since 3.x has fewer >> users. > > +1 for making the change to 3.2 > +0 for 2.7 -1 for 2.7. -- Regards, Benjamin From victor.stinner at haypocalc.com Fri Jul 30 02:19:53 2010 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Fri, 30 Jul 2010 02:19:53 +0200 Subject: [Python-Dev] Full unicode support for the import machinery In-Reply-To: <201007090211.35828.victor.stinner@haypocalc.com> References: <201007090211.35828.victor.stinner@haypocalc.com> Message-ID: <201007300219.53879.victor.stinner@haypocalc.com> Le vendredi 09 juillet 2010 02:11:35, Victor Stinner a ?crit : > I'm trying to fix Python to support undecodable bytes in the Python path > (...) My work is mostly done. I posted a patch on Rietveld and opened an issue. http://bugs.python.org/issue9425 http://codereview.appspot.com/1874048 -- Victor Stinner http://www.haypocalc.com/ From guido at python.org Fri Jul 30 02:12:00 2010 From: guido at python.org (Guido van Rossum) Date: Thu, 29 Jul 2010 17:12:00 -0700 Subject: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2 In-Reply-To: <7DE1C1BA-B9FC-4EC1-896B-D9902FD0DFB7@gmail.com> References: <55ACAB6D-3D92-4643-B7A5-5599037BA952@gmail.com> <7DE1C1BA-B9FC-4EC1-896B-D9902FD0DFB7@gmail.com> Message-ID: On Thu, Jul 29, 2010 at 3:30 PM, Raymond Hettinger wrote: > It also really calls into question whether there are good > reasons for other types to have a __str__ that is different > than their __repr__. Maybe, but there is tons of 3rd party code that uses this distinction. -- --Guido van Rossum (python.org/~guido) From alexander.belopolsky at gmail.com Fri Jul 30 02:34:20 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 29 Jul 2010 20:34:20 -0400 Subject: [Python-Dev] Proposal: make float.__str__ identical to float__repr__ in Python 3.2 In-Reply-To: <7DE1C1BA-B9FC-4EC1-896B-D9902FD0DFB7@gmail.com> References: <55ACAB6D-3D92-4643-B7A5-5599037BA952@gmail.com> <7DE1C1BA-B9FC-4EC1-896B-D9902FD0DFB7@gmail.com> Message-ID: On Thu, Jul 29, 2010 at 6:30 PM, Raymond Hettinger wrote: .. > It also really calls into question whether there are good > reasons for other types to have a __str__ that is different > than their __repr__. For strings, the distinction is very useful. In this and many other cases unifying str and repr would mean making a choice between readability and parseability. From ziade.tarek at gmail.com Fri Jul 30 12:09:30 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 30 Jul 2010 12:09:30 +0200 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <4C5206F6.7090808@voidspace.org.uk> References: <4C5206F6.7090808@voidspace.org.uk> Message-ID: On Fri, Jul 30, 2010 at 12:55 AM, Michael Foord wrote: ... > > The Plugin Class > ================ > > A sometimes-more-convenient way of creating plugins is to subclass the > ``unittest2.events.Plugin`` class. By default subclassing ``Plugin`` will > auto-instantiate the plugin and store the instance in a list of loaded > plugins. > > Each plugin has a ``register()`` method that auto-hooks up all methods whose > names correspond to events. Plugin classes may also provide > ``configSection`` and ``commandLineSwitch`` class attributes which > simplifies enabling the plugin through the command line and making available > a section from the configuration file(s). What about using ABCs here instead ? The big advantage is that you don't force people to subclass your Plugin base class, and you get a registration mechanism for free. They would be able to explicitly register any class using : from unittest2.events import Plugin Plugin.register(MySuperPlugin) Regards Tarek -- Tarek Ziad? | http://ziade.org From fuzzyman at voidspace.org.uk Fri Jul 30 12:38:50 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 30 Jul 2010 11:38:50 +0100 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: References: <4C5206F6.7090808@voidspace.org.uk> Message-ID: <4C52ABBA.7080601@voidspace.org.uk> On 30/07/2010 11:09, Tarek Ziad? wrote: > On Fri, Jul 30, 2010 at 12:55 AM, Michael Foord > wrote: > ... > >> The Plugin Class >> ================ >> >> A sometimes-more-convenient way of creating plugins is to subclass the >> ``unittest2.events.Plugin`` class. By default subclassing ``Plugin`` will >> auto-instantiate the plugin and store the instance in a list of loaded >> plugins. >> >> Each plugin has a ``register()`` method that auto-hooks up all methods whose >> names correspond to events. Plugin classes may also provide >> ``configSection`` and ``commandLineSwitch`` class attributes which >> simplifies enabling the plugin through the command line and making available >> a section from the configuration file(s). >> > What about using ABCs here instead ? The big advantage is that you don't force > people to subclass your Plugin base class, Is that an advantage? Subclassing Plugin provides functionality rather than particularly an interface. > and you get a registration > mechanism for free. > > Actually it looks like it would be *extra* code. (The registration mechanism is already "free" to the programmer when they subclass.) I will investigate this, but if the only benefit it provides is "you don't have to subclass" then I'm not sure there is any point. Note that the Plugin class is only "sugar" over hooking up to the events directly *anyway*, so there is no *need* to use Plugin to write extensions. I'm going to read your blog entry on the topic to evaluate it properly though: https://tarekziade.wordpress.com/2009/05/01/basic-plugin-system-using-abcs-and-the-extensions-package/ Michael > They would be able to explicitly register any class using : > > from unittest2.events import Plugin > > Plugin.register(MySuperPlugin) > > > Regards > Tarek > > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From durga.disc at gmail.com Fri Jul 30 14:18:45 2010 From: durga.disc at gmail.com (Durga D) Date: Fri, 30 Jul 2010 17:48:45 +0530 Subject: [Python-Dev] Basic Information about Python Message-ID: Hi All, I am new to python based application developement. I am using Windows XP. 1. Can I create desktop application (just hello world program) with Python language like exe in VC++? 2. If First statement is Yes, Can I include this application with my existing setup(assume 10 MB) for windows? 3. If Second statement is Yes, What will be the setup size? Thank in advance. Regards, Durga. -------------- next part -------------- An HTML attachment was scrubbed... URL: From phd at phd.pp.ru Fri Jul 30 14:35:46 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Fri, 30 Jul 2010 16:35:46 +0400 Subject: [Python-Dev] Basic Information about Python In-Reply-To: References: Message-ID: <20100730123546.GA1017@phd.pp.ru> Hello. We are sorry but we cannot help you. This mailing list is to work on developing Python (adding new features to Python itself and fixing bugs); if you're having problems learning, understanding or using Python, please find another forum. Probably python-list/comp.lang.python mailing list/news group is the best place; there are Python developers who participate in it; you may get a faster, and probably more complete, answer there. See http://www.python.org/community/ for other lists/news groups/fora. Thank you for understanding. On Fri, Jul 30, 2010 at 05:48:45PM +0530, Durga D wrote: > Hi All, > > I am new to python based application developement. I am using Windows XP. > > 1. Can I create desktop application (just hello world program) with Python > > language like exe in VC++? > > 2. If First statement is Yes, Can I include this application with my > existing setup(assume 10 MB) for windows? > > 3. If Second statement is Yes, What will be the setup size? > > Thank in advance. > > Regards, > Durga. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From durga.disc at gmail.com Fri Jul 30 14:44:26 2010 From: durga.disc at gmail.com (Durga D) Date: Fri, 30 Jul 2010 18:14:26 +0530 Subject: [Python-Dev] Basic Information about Python In-Reply-To: <20100730123546.GA1017@phd.pp.ru> References: <20100730123546.GA1017@phd.pp.ru> Message-ID: Thank you for your response. Regards, Durga. On Fri, Jul 30, 2010 at 6:05 PM, Oleg Broytman wrote: > Hello. > > We are sorry but we cannot help you. This mailing list is to work on > developing Python (adding new features to Python itself and fixing bugs); > if you're having problems learning, understanding or using Python, please > find another forum. Probably python-list/comp.lang.python mailing list/news > group is the best place; there are Python developers who participate in it; > you may get a faster, and probably more complete, answer there. See > http://www.python.org/community/ for other lists/news groups/fora. Thank > you for understanding. > > On Fri, Jul 30, 2010 at 05:48:45PM +0530, Durga D wrote: > > Hi All, > > > > I am new to python based application developement. I am using Windows > XP. > > > > 1. Can I create desktop application (just hello world program) with > Python > > > > language like exe in VC++? > > > > 2. If First statement is Yes, Can I include this application with my > > existing setup(assume 10 MB) for windows? > > > > 3. If Second statement is Yes, What will be the setup size? > > > > Thank in advance. > > > > Regards, > > Durga. > > Oleg. > -- > Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru > Programmers don't die, they just GOSUB without RETURN. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Fri Jul 30 15:23:14 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 30 Jul 2010 14:23:14 +0100 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <4C5206F6.7090808@voidspace.org.uk> References: <4C5206F6.7090808@voidspace.org.uk> Message-ID: For those of you who found this document perhaps just a little bit too long, I've written up a *much* shorter intro to the plugin system (including how to get the prototype) on my blog: http://www.voidspace.org.uk/python/weblog/arch_d7_2010_07_24.shtml#e1186 Michael On 29 July 2010 23:55, Michael Foord wrote: > Hello all, > > My apologies in advance if email mangles whitespace in the code examples. I > can reformulate as a PEP if that is deemed useful and this document can be > found online at: > > http://hg.python.org/unittest2/file/tip/description.txt > > (Please excuse errors and omissions - but do feel free to point them out.) > > This is a description, and request for feedback, of the unittest plugin > system that I am currently prototyping in the plugins branch of unittest2_. > My goal is to merge the plugin system back into unittest itself in Python > 3.2. > > .. _unittest2: http://hg.python.org/unittest2 > > As part of the prototype I have been implementing some example plugins (in > unittest2/plugins/) so I can develop the mechanism against real rather than > imagined use cases. Jason Pellerin, creator of nose, has been providing me > with feedback and has been trying it out by porting some of the nose plugins > to unittest [#]_. He comments on the system "it looks very flexible and > clean". ;-) > > Example plugins available and included: > > * a pep8 and pyflakes checker > * a debugger plugin that drops you into pdb on test fail / error > * a doctest loader (looks for doctests in all text files in the project) > * use a regex for matching files in test discovery instead of a glob > * growl notifications on test run start and stop > * filter individual *test methods* using a regex > * load test functions from modules as well as TestCases > * integration with the coverage module for coverage reporting > > In addition I intend to create a plugin that outputs junit compatible xml > from a test run (for integration with tools like the hudson continuous > integration server) and a test runner that runs tests in parallel using > multiprocessing. > > Not all of these will be included in the merge to unittest. Which ones will > is still an open question. > > I'd like feedback on the proposal, and hopefully approval to port it into > unittest after discussion / amendment / completion. In particular I'd like > feedback on the basic system, plus which events should be available and what > information should be available in them. Note that the system is *not* > complete in the prototype. Enough is implemented to get "the general idea" > and to formalise the full system. It still needs extensive tests and the > extra work in TestProgram makes it abundantly clear that refactoring there > is well overdue... > > In the details below open questions and todos are noted. I *really* value > feedback (but will ignore bikeshedding ;-) > > .. note:: > > Throughout this document I refer to the prototype implementation using > names like ``unittest2.events.hooks``. Should this proposal be accepted then > the names will live in the unittest package instead of unittest2. > > The core classes for the event system live in the current implementation > in the ``unittest2.events`` namespace. > > > Abstract > ======== > > unittest lacks a standard way of extending it to provide commonly requested > functionality, other than subclassing and overriding (and reimplementing) > parts of its behaviour. This document describes a plugin system already > partially prototyped in unittest2. > > Aspects of the plugin system include: > > * an events mechanism where handlers can be registered and called during a > test run > * a Plugin class built over the top of this for easy creation of plugins > * a configuration system for specifying which plugins should be loaded and > for configuring individual plugins > * command line integration > * the specific set of events and the information / actions available to > them > > As the plugin system essentially just adds event calls to key places it has > few backwards compatibility issues. Unfortunately existing extensions that > override the parts of unittest that call these events will not be compatible > with plugins that use them. Framework authors who re-implement parts of > unittest, for example custom test runners, may want to add calling these > events in appropriate places. > > > Rationale > ========= > > Why a plugin system for unittest? > > unittest is the standard library test framework for Python but in recent > years has been eclipsed in functionality by frameworks like nose and > py.test. Among the reasons for this is that these frameworks are easier to > extend with plugins than unittest. unittest makes itself particularly > difficult to extend by using subclassing as its basic extension mechanism. > You subclass and override behaviour in its core classes like the loader, > runner and result classes. > > This means that where you have more than one "extension" working in the > same area it is very hard for them to work together. Whilst various > extensions to unittest do exist (e.g. testtools, zope.testrunner etc) they > don't tend to work well together. In contrast the plugin system makes > creating extensions to unittest much simpler and less likely that extensions > will clash with each other. > > nose itself exists as a large system built over the top of unittest. > Extending unittest in this way was very painful for the creators of nose, > and every release of Python breaks nose in some way due to changes in > unittest. One of the goals of the extension mechanism is to allow nose2 to > be a much thinner set of plugins over unittest(2) that is much simpler to > maintain [#]_. The early indications are that the proposed system is a good > fit for this goal. > ... > -- http://www.voidspace.org.uk -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Fri Jul 30 16:04:00 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 30 Jul 2010 10:04:00 -0400 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <4C52ABBA.7080601@voidspace.org.uk> References: <4C5206F6.7090808@voidspace.org.uk> <4C52ABBA.7080601@voidspace.org.uk> Message-ID: <20100730100400.3fad0c7d@heresy> On Jul 30, 2010, at 11:38 AM, Michael Foord wrote: >I'm going to read your blog entry on the topic to evaluate it properly >though: > >https://tarekziade.wordpress.com/2009/05/01/basic-plugin-system-using-abcs- >and-the-extensions-package/ Very interesting. For Mailman 3, I have YAPS (yet another plugin system) based on zope.interface and setuptools. Bazaar has its own plugin system which is different still. I bet there are as many plugin APIs as there are Python frameworks. (And isn't everything a framework these days? :) I'm glad to see discussion begin to focus on providing consolidation in the world of plugins for Python frameworks, and to begin to refactor basic functionality into common tools. I'd love to see a blessed plugin API promoted to the stdlib for Python 3.2. I think it has to address a number of issues: * Registration - How do third party plugins declare themselves to exist, and be enabled? Part of this seems to me to include interface declarations too. Is installation of the plugin enough to register it? How do end users enable and disable plugins that me be registered on their system? How do plugins describe themselves (provide short and log descriptions, declare options, hook into command line interfaces, etc.)? * Installation - How are plugins installed on the system? Do they have to appear in a special directory on the file system? Do they need special setup.py magic to write extra files? Do they need to live in a pre-defined namespace? * Discoverability - How do frameworks discover all the plugins that are available? Which available plugins claim to support a particular plugin-point? How to do strict type checking on plugins? Which plugins are enabled? I'm sure there are more. As always, I'd like to see simple APIs on both sides that cover the common 80%. Both Tarek's and Michael's posts and proto-peps are great starts. You guys should definitely write up a plugin PEP! -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From fuzzyman at voidspace.org.uk Fri Jul 30 16:34:23 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 30 Jul 2010 15:34:23 +0100 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <20100730100400.3fad0c7d@heresy> References: <4C5206F6.7090808@voidspace.org.uk> <4C52ABBA.7080601@voidspace.org.uk> <20100730100400.3fad0c7d@heresy> Message-ID: <4C52E2EF.3020401@voidspace.org.uk> On 30/07/2010 15:04, Barry Warsaw wrote: > On Jul 30, 2010, at 11:38 AM, Michael Foord wrote: > > >> I'm going to read your blog entry on the topic to evaluate it properly >> though: >> >> https://tarekziade.wordpress.com/2009/05/01/basic-plugin-system-using-abcs- >> and-the-extensions-package/ >> > Very interesting. For Mailman 3, I have YAPS (yet another plugin system) > based on zope.interface and setuptools. Bazaar has its own plugin system > which is different still. I bet there are as many plugin APIs as there are > Python frameworks. (And isn't everything a framework these days? :) > > I'm glad to see discussion begin to focus on providing consolidation in the > world of plugins for Python frameworks, and to begin to refactor basic > functionality into common tools. I'd love to see a blessed plugin API > promoted to the stdlib for Python 3.2. I think it has to address a number of > issues: > > * Registration - How do third party plugins declare themselves to exist, and > be enabled? Part of this seems to me to include interface declarations > too. Is installation of the plugin enough to register it? How do end users > enable and disable plugins that me be registered on their system? How do > plugins describe themselves (provide short and log descriptions, declare > options, hook into command line interfaces, etc.)? > > * Installation - How are plugins installed on the system? Do they have to > appear in a special directory on the file system? Do they need special > setup.py magic to write extra files? Do they need to live in a pre-defined > namespace? > > * Discoverability - How do frameworks discover all the plugins that are > available? Which available plugins claim to support a particular > plugin-point? How to do strict type checking on plugins? Which plugins are > enabled? > > I'm sure there are more. As always, I'd like to see simple APIs on both sides > that cover the common 80%. Both Tarek's and Michael's posts and proto-peps > are great starts. You guys should definitely write up a plugin PEP! > Whilst in principle I agree with you... the plugin requirements for unittest(2) and disutils2 are very different. The biggest advantage of using ABCs in Tarek's post is around interfaces - you can ensure that registered plugins have the required interface. unittest doesn't *have* a required interface for plugins, which may optionally implement handlers for *any* of the unittest events and the rest of the functionality (configuration and command line interface integration) is provided by virtue of the subclassing. Explicit registration over implicit registration by subclassing is an interesting discussion, but I like the simplicity provided by just subclassing. unittest allows any namespace to provide a plugin but has no discoverability built in to it. Users specify which plugins they want to use (per project and per user), plugins are then activated by importing. Framework authors can load whichever plugins they want - it is probable that discoverability would be useful here. Automatic discoverability, a-la setuptools entry points, is not without its problems though. Tarek outlines some of these in a more recent blog post: https://tarekziade.wordpress.com/2010/07/25/plugins-system-thoughts-for-an-entry-points-replacement/ Again I think the *needs* of unittest and distutils are different, so I wonder if a single system can usefully suit both our needs (let alone universally support other systems). Definitely an area worth exploring though. Michael > -Barry > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziade.tarek at gmail.com Fri Jul 30 16:37:39 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 30 Jul 2010 16:37:39 +0200 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <20100730100400.3fad0c7d@heresy> References: <4C5206F6.7090808@voidspace.org.uk> <4C52ABBA.7080601@voidspace.org.uk> <20100730100400.3fad0c7d@heresy> Message-ID: On Fri, Jul 30, 2010 at 4:04 PM, Barry Warsaw wrote: .. > * Registration - How do third party plugins declare themselves to exist, and > ?be enabled? ?Part of this seems to me to include interface declarations > ?too. ?Is installation of the plugin enough to register it? ?How do end users > ?enable and disable plugins that me be registered on their system? ?How do > ?plugins describe themselves (provide short and log descriptions, declare > ?options, hook into command line interfaces, etc.)? > > * Installation - How are plugins installed on the system? ?Do they have to > ?appear in a special directory on the file system? ?Do they need special > ?setup.py magic to write extra files? ?Do they need to live in a pre-defined > ?namespace? FWIW We are thinking about adding in distutils2 a system quite similar to the entry points setuptools has, but with extra abilities for the end user : - activate / deactivate plugins without having to remove the project that added them - configure globally if plugins are implicitely activated or not -- and maybe allow the distutils2 installer to ask the user when a plugin is detected if he wants it activate or not - provide a tool to browse them This will be done through files added in the dist-info/ dirs, with the new PEP 376 api we are adding to pkgutil The idea is that the end user should be able to have a full control on what's activated in his system, without relying on the developer choices Cheers Tarek -- Tarek Ziad? | http://ziade.org From ziade.tarek at gmail.com Fri Jul 30 16:40:06 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 30 Jul 2010 16:40:06 +0200 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <20100730100400.3fad0c7d@heresy> References: <4C5206F6.7090808@voidspace.org.uk> <4C52ABBA.7080601@voidspace.org.uk> <20100730100400.3fad0c7d@heresy> Message-ID: On Fri, Jul 30, 2010 at 4:04 PM, Barry Warsaw wrote: > You guys should definitely write up a plugin PEP! I am all for it, I am pretty sure we can come up with a generic tool that can be useful for many packages in the stdlib Starting this... -- Tarek Ziad? | http://ziade.org From marty at martyalchin.com Fri Jul 30 16:41:29 2010 From: marty at martyalchin.com (Marty Alchin) Date: Fri, 30 Jul 2010 10:41:29 -0400 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <20100730100400.3fad0c7d@heresy> References: <4C5206F6.7090808@voidspace.org.uk> <4C52ABBA.7080601@voidspace.org.uk> <20100730100400.3fad0c7d@heresy> Message-ID: This is my first post to python-dev, so for those who might not know me, I'm the author of Pro Django and more recently, Pro Python. I haven't looked at the plugin landscape in a while, but I was very disappointed the last time I looked at how complex typical systems were in this regard. There seems to be entirely too much time spend worrying about type checking, interface requirements and the like. Python is based on simple ideas providing simple solutions, and I would argue that duck typing is a perfectly acceptable route to take. I'd suggest that any plugin system proposal start off simple and work up as needs arise, rather than trying to cover as many bases as possible at the outset. I know it doesn't cover all the bases yet (particularly with regard to discoverability), so I'm not throwing it in as a proper suggestion, but I'll point out that I wrote up a dead-simple plugin system a while back[1] that may provide a useful starting point for discussion. I haven't read the existing proposals yet, but I just wanted to breathe a word of caution into this before it gets too far into interfaces and whatnot. I'll be glad to help with any discussion that takes place on this, though. It'll be a good way to ease into the mailing list, since it's an area where I've already spent a good deal of thought. -Marty [1] http://martyalchin.com/2008/jan/10/simple-plugin-framework/ From ziade.tarek at gmail.com Fri Jul 30 16:59:00 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 30 Jul 2010 16:59:00 +0200 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <4C52E2EF.3020401@voidspace.org.uk> References: <4C5206F6.7090808@voidspace.org.uk> <4C52ABBA.7080601@voidspace.org.uk> <20100730100400.3fad0c7d@heresy> <4C52E2EF.3020401@voidspace.org.uk> Message-ID: On Fri, Jul 30, 2010 at 4:34 PM, Michael Foord wrote: ... > Again I think the *needs* of unittest and distutils are different, so I > wonder if a single system can usefully suit both our needs (let alone > universally support other systems). Definitely an area worth exploring > though. Yes, even if they do if for different needs, both projects want to do the same thing: to be easily extensible by third party project. If we can provide a generic tool for both unittest and distutils extensions use cases, that's great and that will probably be useful for other projects. If we fail, I doubt we will ever have a generic plugin system in the stdlib. But that worth trying I think, and having different use cases is good for this imho From fuzzyman at voidspace.org.uk Fri Jul 30 17:01:01 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 30 Jul 2010 16:01:01 +0100 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: References: <4C5206F6.7090808@voidspace.org.uk> <4C52ABBA.7080601@voidspace.org.uk> <20100730100400.3fad0c7d@heresy> Message-ID: <4C52E92D.50909@voidspace.org.uk> On 30/07/2010 15:37, Tarek Ziad? wrote: > On Fri, Jul 30, 2010 at 4:04 PM, Barry Warsaw wrote: > .. > >> * Registration - How do third party plugins declare themselves to exist, and >> be enabled? Part of this seems to me to include interface declarations >> too. Is installation of the plugin enough to register it? How do end users >> enable and disable plugins that me be registered on their system? How do >> plugins describe themselves (provide short and log descriptions, declare >> options, hook into command line interfaces, etc.)? >> >> * Installation - How are plugins installed on the system? Do they have to >> appear in a special directory on the file system? Do they need special >> setup.py magic to write extra files? Do they need to live in a pre-defined >> namespace? >> > FWIW We are thinking about adding in distutils2 a system quite similar > to the entry points > setuptools has, but with extra abilities for the end user : > > - activate / deactivate plugins without having to remove the project > that added them > - configure globally if plugins are implicitely activated or not -- > and maybe allow the distutils2 installer to ask the user > when a plugin is detected if he wants it activate or not > - provide a tool to browse them > > This will be done through files added in the dist-info/ dirs, with the > new PEP 376 api we are > adding to pkgutil > > The idea is that the end user should be able to have a full control on > what's activated in his system, > without relying on the developer choices > This system sounds great. unittest could certainly use it for discovering plugins provided by other packages. The question then is still how to decide which ones should be active for any individual project (just because a plugin is available doesn't mean you want it used for every project). A configuration system is still good for this, but that kind of negates the advantage of discovery if the user still has to configure the plugin *anyway*. For framework authors not using the default test runner ("python -m unittest" or "unit2") this would be very useful. Michael > Cheers > Tarek > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From fuzzyman at voidspace.org.uk Fri Jul 30 17:04:27 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 30 Jul 2010 16:04:27 +0100 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: References: <4C5206F6.7090808@voidspace.org.uk> <4C52ABBA.7080601@voidspace.org.uk> <20100730100400.3fad0c7d@heresy> Message-ID: <4C52E9FB.3000400@voidspace.org.uk> On 30/07/2010 15:41, Marty Alchin wrote: > This is my first post to python-dev, so for those who might not know > me, I'm the author of Pro Django and more recently, Pro Python. > > I haven't looked at the plugin landscape in a while, but I was very > disappointed the last time I looked at how complex typical systems > were in this regard. There seems to be entirely too much time spend > worrying about type checking, interface requirements and the like. > Python is based on simple ideas providing simple solutions, and I > would argue that duck typing is a perfectly acceptable route to take. > There is no type checking or interface requirements in my plugin proposal for unittest. It is essentially an event based system. > I'd suggest that any plugin system proposal start off simple and work > up as needs arise, rather than trying to cover as many bases as > possible at the outset. I know it doesn't cover all the bases yet > (particularly with regard to discoverability), so I'm not throwing it > in as a proper suggestion, but I'll point out that I wrote up a > dead-simple plugin system a while back[1] that may provide a useful > starting point for discussion. > > Tarek and I will look at what common ground there is between our plugin needs and see if *that* can be usefully abstracted out. FWIW I think PEP 376 (as amended - needs updating I *believe*) for plugin installation and discovery probably covers most of the common ground anyway. For the actual plugins our two systems are very different. > I haven't read the existing proposals yet, If you want to help us this may be a good place to start... ;-) Michael > but I just wanted to > breathe a word of caution into this before it gets too far into > interfaces and whatnot. I'll be glad to help with any discussion that > takes place on this, though. It'll be a good way to ease into the > mailing list, since it's an area where I've already spent a good deal > of thought. > > -Marty > > [1] http://martyalchin.com/2008/jan/10/simple-plugin-framework/ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From merwok at netwok.org Fri Jul 30 17:28:28 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Fri, 30 Jul 2010 17:28:28 +0200 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <4C52E9FB.3000400@voidspace.org.uk> References: <4C5206F6.7090808@voidspace.org.uk> <4C52ABBA.7080601@voidspace.org.uk> <20100730100400.3fad0c7d@heresy> <4C52E9FB.3000400@voidspace.org.uk> Message-ID: <4C52EF9C.9040701@netwok.org> Le 30/07/2010 17:04, Michael Foord a ?crit : > There is no type checking or interface requirements in my plugin > proposal for unittest. It is essentially an event based system. Event-based sounds good. unittest2 does have an interface IMO: configuration loading, plugin registration/activation, signal names. As you said, the common ground may be little more than an API to discover or register plugins, activate and configure them, and the design decision of using events and callbacks, but even this small subset is very much worth a PEP. Regards From fuzzyman at voidspace.org.uk Fri Jul 30 17:35:59 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 30 Jul 2010 16:35:59 +0100 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <4C52EF9C.9040701@netwok.org> References: <4C5206F6.7090808@voidspace.org.uk> <4C52ABBA.7080601@voidspace.org.uk> <20100730100400.3fad0c7d@heresy> <4C52E9FB.3000400@voidspace.org.uk> <4C52EF9C.9040701@netwok.org> Message-ID: <4C52F15F.3020604@voidspace.org.uk> On 30/07/2010 16:28, ?ric Araujo wrote: > Le 30/07/2010 17:04, Michael Foord a ?crit : > >> There is no type checking or interface requirements in my plugin >> proposal for unittest. It is essentially an event based system. >> > Event-based sounds good. unittest2 does have an interface IMO: > configuration loading, plugin registration/activation, signal names. > > It has an API, but the plugins are not interface based (so interface requirements don't need to be part of the plugin system). > As you said, the common ground may be little more than an API to > discover or register plugins, activate and configure them, and the > design decision of using events and callbacks, but even this small > subset is very much worth a PEP. > PEP 376 is *very* promising for this part - well at least if the PLUGINS stuff becomes part of PEP 376, it doesn't seem to be mentioned at all at the moment. Based on what you and tarek are saying it *sounds* good though... (Metadata for packages to declare that they provide plugins for another package and an API for frameworks to discover installed plugins.) Michael > Regards > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From merwok at netwok.org Fri Jul 30 17:54:17 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Fri, 30 Jul 2010 17:54:17 +0200 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <4C52F15F.3020604@voidspace.org.uk> References: <4C5206F6.7090808@voidspace.org.uk> <4C52ABBA.7080601@voidspace.org.uk> <20100730100400.3fad0c7d@heresy> <4C52E9FB.3000400@voidspace.org.uk> <4C52EF9C.9040701@netwok.org> <4C52F15F.3020604@voidspace.org.uk> Message-ID: <4C52F5A9.8000707@netwok.org> > It has an API, but the plugins are not interface based (so interface > requirements don't need to be part of the plugin system). Oh, I see. With duck-typing and ABCs, I don?t make a difference between protocols and interfaces in my head :) Also, the I in API does mean interface, but not imply bondage-and-discipline interface systems. > PEP 376 is *very* promising for this part - well at least if the PLUGINS > stuff becomes part of PEP 376, it doesn't seem to be mentioned at all at > the moment. The dist-info directory introduced by PEP 376 will certainly be reused. Not sure if Tarek will want to edit PEP 376 (already accepted) to add plugins in metadata (provides-dist is in metadata, so having plugins here can make sense), or start a new PEP about a new concept in a new file. Regards From da.waterworth at gmail.com Fri Jul 30 08:26:26 2010 From: da.waterworth at gmail.com (Daniel Waterworth) Date: Fri, 30 Jul 2010 07:26:26 +0100 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: References: Message-ID: On 29 July 2010 07:32, Daniel Waterworth wrote: > Hi, > > I'm not sure if this is a bug or not, I certainly didn't expect it. If > you create a file called test.py with the following contents, > > class Test: > ? ?pass > > def test_1(): > ? ?import test > ? ?print Test == test.Test > > if __name__ == '__main__': > ? ?test_1() > > and then run it ($ python test.py), it'll print False. Now try: > > $python > import test > test.test_1() > > and it'll print True. Is this behaviour expected? What was the > rationale for it if is? > > Thanks, > > Daniel > > -- > active-thought.com > @Oleg: I know this list is plagued by people who should be on comp.lang.python, but I assure you I'm not looking to learn to program in python, in fact I've been programming competently in python for many years. This is purely CPython bug-fixing/the discussion of implementation choices. @ Nick: In terms of backward compatibility, it would only break someone's code if they were relying on having the same module imported twice as different instances. Could this behaviour be added to python3.2? I'm not sure how far you are through the release cycle. Or even just a warning as Michael suggested? @Michael: Yes, I guessed as much. In fact adding, import sys, os if globals().get("__file__") and __name__=='__main__': base = os.path.basename(__file__) ext = base.rfind('.') if ext > 0: main_name = base[:ext] else: main_name = base sys.modules[main_name] = __import__('__main__') to the beginning of a file solves the problem, but seems more than a little hacky and I think I've missed edge cases with packages. Thanks for your answers, Daniel -- active-thought.com From status at bugs.python.org Fri Jul 30 18:08:42 2010 From: status at bugs.python.org (Python tracker) Date: Fri, 30 Jul 2010 18:08:42 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20100730160842.913ED7812B@psf.upfronthosting.co.za> ACTIVITY SUMMARY (2010-07-23 - 2010-07-30) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2804 open (+45) / 18443 closed (+27) / 21247 total (+72) Open issues with patches: 1151 Average duration of open issues: 689 days. Median duration of open issues: 481 days. Open Issues Breakdown open 2760 (+45) languishing 16 ( +0) pending 27 ( +0) Issues Created Or Reopened (79) _______________________________ Add named tuple reader to CSV module 2010-07-25 http://bugs.python.org/issue1818 reopened rhettinger patch MatchObject is not a hyperlink in the 're' module documentatio 2010-07-28 CLOSED http://bugs.python.org/issue6538 reopened tjreedy patch, easy SysLogHandler can't handle Unicode 2010-07-26 http://bugs.python.org/issue7077 reopened georg.brandl Extraneous newlines with csv.writer on Windows 2010-07-28 CLOSED http://bugs.python.org/issue7198 reopened skip.montanaro Create a bytes version of os.environ and getenvb() 2010-07-29 http://bugs.python.org/issue8603 reopened ezio.melotti patch Add annotation option to pickletools.dis 2010-07-26 http://bugs.python.org/issue9268 reopened belopolsky patch, easy urllib.quote(None) returns None in 2.7 (raised TypeError befor 2010-07-27 http://bugs.python.org/issue9301 reopened flox patch Improve queue tests 2010-07-23 CLOSED http://bugs.python.org/issue9357 created bbrazil patch Document fnmatch.translate's lack of support for escaping 2010-07-23 CLOSED http://bugs.python.org/issue9358 created bbrazil patch Misspelled exception 2010-07-23 CLOSED http://bugs.python.org/issue9359 created kwadrat patch nntplib cleanup 2010-07-23 http://bugs.python.org/issue9360 created Dmitry.Jemerov patch Tests for leapdays in calendar.py module 2010-07-23 http://bugs.python.org/issue9361 created John.Chandler patch Make exit/quit hint more novice friendly 2010-07-23 http://bugs.python.org/issue9362 created belopolsky easy data_files are not installed relative to sys.prefix 2010-07-23 http://bugs.python.org/issue9363 created rioch some problems with the documentation of pydoc 2010-07-24 http://bugs.python.org/issue9364 created eli.bendersky Installing a distro without sqlite3 will require a reinstall o 2010-07-24 CLOSED http://bugs.python.org/issue9365 created antlong Reference leak for local new style class 2010-07-24 CLOSED http://bugs.python.org/issue9366 created kayhayen test_getgroups in test_posix fails 2010-07-24 CLOSED http://bugs.python.org/issue9367 created DazWorrall patch Const-Correctness for Method Calls 2010-07-24 CLOSED http://bugs.python.org/issue9368 created aronacher patch const char* for PyObject_CallMethod and PyObject_CallFunction 2010-07-24 http://bugs.python.org/issue9369 created neXyon patch Add reader redirect from test package docs to unittest module 2010-07-24 http://bugs.python.org/issue9370 created ncoghlan easy pulldom doesn't provide END_DOCUMENT or COMMENT nodes. 2010-07-24 http://bugs.python.org/issue9371 created mark.smith pulldom.DOMEventStream.__getitem__ is broken 2010-07-24 http://bugs.python.org/issue9372 created mark.smith pulldom has low code coverage 2010-07-24 http://bugs.python.org/issue9373 created mark.smith urlparse should parse query and fragment for arbitrary schemes 2010-07-24 http://bugs.python.org/issue9374 created Nick.Welch ElementPath parser in ElementTree 1.3 does not reject "element 2010-07-25 http://bugs.python.org/issue9375 created scoder Refer to gnuwin32 diff util on development FAQ 2010-07-25 CLOSED http://bugs.python.org/issue9376 created BreamoreBoy easy socket, PEP 383: Mishandling of non-ASCII bytes in host/domain 2010-07-25 http://bugs.python.org/issue9377 created baikie patch Make python -m pickle do something useful 2010-07-25 CLOSED http://bugs.python.org/issue9378 created belopolsky patch, easy random.randrange behaviour problems 2010-07-25 http://bugs.python.org/issue9379 created lukasz.langa Allow popping of n elements at once from a deque 2010-07-26 CLOSED http://bugs.python.org/issue9380 created ipatrol syntax error in Unicode C API docs 2010-07-26 CLOSED http://bugs.python.org/issue9381 created cvrebert os.popen referenced but not documented in Python 3.x 2010-07-26 http://bugs.python.org/issue9382 created cvrebert PIL Bug with split 2010-07-26 CLOSED http://bugs.python.org/issue9383 created Arrnaud.Fabre Tkinter windows pop under the terminal in OSX 2010-07-26 http://bugs.python.org/issue9384 created belopolsky _ctypes module uses 'rwx' mmap() calls 2010-07-26 http://bugs.python.org/issue9385 created radegand Bad indentation in urllib import fixer with multiple imports 2010-07-26 http://bugs.python.org/issue9386 created dmalcolm patch Make python -m tkinter run tkinter demo 2010-07-26 CLOSED http://bugs.python.org/issue9387 created belopolsky patch, easy, needs review locale documentation describes non-existing ERA_YEAR constant 2010-07-27 CLOSED http://bugs.python.org/issue9388 created belopolsky easy Traceback: Exception Shows Code that's On-Disk (Not in memory) 2010-07-28 CLOSED http://bugs.python.org/issue9389 reopened Guy Error in sys.excepthook on windows when redirecting output of 2010-07-27 http://bugs.python.org/issue9390 created sorin Allow docstrings on dicts and named tuples outside of function 2010-07-27 CLOSED http://bugs.python.org/issue9391 created antlong easy 2.7 framework install doesn't create 2to3-2.7 2010-07-27 CLOSED http://bugs.python.org/issue9392 created ronaldoussoren shelve.open/bsddb.hashopen exception with unicode paths 2010-07-28 http://bugs.python.org/issue9393 created wjm251 patch shelve.open/bsddb.hashopen raise bsddb.db.DBNoSuchFileError: ( 2010-07-28 CLOSED http://bugs.python.org/issue9394 created wjm251 clean does not remove all temp files 2010-07-28 http://bugs.python.org/issue9395 created merwok Standardise (and publish?) cache handling in standard library 2010-07-28 http://bugs.python.org/issue9396 created ncoghlan Remove references to the missing dbm.bsd module 2010-07-28 CLOSED http://bugs.python.org/issue9397 created haypo Unify sys.settrace and sys.setprofile tests 2010-07-28 http://bugs.python.org/issue9398 created belopolsky Provide a 'print' action for argparse 2010-07-28 http://bugs.python.org/issue9399 created travistouchdown patch multiprocessing.pool.AsyncResult.get() messes up exceptions 2010-07-28 http://bugs.python.org/issue9400 created Nikratio automatically try forward operations when reverse operations a 2010-07-28 CLOSED http://bugs.python.org/issue9401 created Michael.Gilbert pyexpat: replace PyObject_DEL() by Py_DECREF() to fix a crash 2010-07-28 http://bugs.python.org/issue9402 created haypo patch cElementTree: replace PyObject_DEL() by Py_DECREF() to fix a c 2010-07-28 http://bugs.python.org/issue9403 created haypo patch IDLE won't launch on XP 2010-07-28 http://bugs.python.org/issue9404 created Chris.Leaf crash when calling urllib.getproxies() under OSX with subproce 2010-07-28 http://bugs.python.org/issue9405 created zzzeek ConfigParser exceptions do not subclass StandardError 2010-07-28 CLOSED http://bugs.python.org/issue9406 created jtatum patch ConfigParser documentation does not mention ConfigParser.Error 2010-07-28 CLOSED http://bugs.python.org/issue9407 created pekkle patch curses: Link against libncursesw instead of libncurses 2010-07-28 http://bugs.python.org/issue9408 created haypo doctest in python2.7 can't handle non-ascii characters 2010-07-29 http://bugs.python.org/issue9409 created hugo Add Unladden Swallow's optimizations to Python 3's pickle. 2010-07-29 http://bugs.python.org/issue9410 created alexandre.vassalotti patch configparser doesn't support specifying encoding in read() 2010-07-29 CLOSED http://bugs.python.org/issue9411 created lukasz.langa patch test_smtpd leaks references 2010-07-29 CLOSED http://bugs.python.org/issue9412 created pitrou asyncore.close_all() should call x.handle_close() instead of x 2010-07-29 http://bugs.python.org/issue9413 created me.myself infinite loop with specific regexp 2010-07-29 CLOSED http://bugs.python.org/issue9414 created gnuthor SSL issues on "Ubuntu i386" buildbots 2010-07-29 http://bugs.python.org/issue9415 created pitrou complex formatting incorrectly omits a negative zero real part 2010-07-29 http://bugs.python.org/issue9416 created mark.dickinson patch Declaring a class creates circular references 2010-07-29 http://bugs.python.org/issue9417 created candrea Move _formatter_* methods from string type into _string module 2010-07-29 http://bugs.python.org/issue9418 created georg.brandl patch RUNSHARED needs LDFLAGS 2010-07-29 http://bugs.python.org/issue9419 created phaering patch gdbm with /usr/include/ndbm.h 2010-07-29 http://bugs.python.org/issue9420 created phaering patch configparser.ConfigParser's getint, getboolean and getfloat do 2010-07-29 http://bugs.python.org/issue9421 created lukasz.langa Memory leak when reinitializing a Struct object 2010-07-29 CLOSED http://bugs.python.org/issue9422 created Winston451 patch Error in urllib2.do_open(self, http_class, req) 2010-07-29 http://bugs.python.org/issue9423 created lyle.ross deprecate unittest.TestCase.assertEquals 2010-07-30 http://bugs.python.org/issue9424 created michael.foord Rewrite import machinery to work with unicode paths 2010-07-30 http://bugs.python.org/issue9425 created haypo Explicitly state lack of support for keyword arguments in buil 2010-07-30 CLOSED http://bugs.python.org/issue9426 created lukasz.langa logging.error('...', exc_info=True) should display upper frame 2010-07-30 http://bugs.python.org/issue9427 created guettli profile.py bug with the main file being profiled 2010-07-30 http://bugs.python.org/issue9428 created belopolsky patch, easy Issues Now Closed (108) _______________________ Build issues in DragonFly BSD 807 days http://bugs.python.org/issue2796 BreamoreBoy patch Make the left sidebar in the doc collapsible 771 days http://bugs.python.org/issue3143 georg.brandl Direct calls to PyObject_Del/PyObject_DEL are broken for --wit 752 days http://bugs.python.org/issue3299 haypo patch bytes.tohex method 715 days http://bugs.python.org/issue3532 BreamoreBoy patch cookielib doesn't handle URLs with / in parameters 697 days http://bugs.python.org/issue3704 gregory.p.smith patch, needs review documentation bug: HTMLParser needs to document unknown_decl 682 days http://bugs.python.org/issue3874 tjreedy patch Win32ErrorTests from test_os.py 670 days http://bugs.python.org/issue3966 merwok robotparser.py fail when more than one User-Agent: * is presen 655 days http://bugs.python.org/issue4108 georg.brandl pdb: Allow the "list" command to return to the currently debug 645 days http://bugs.python.org/issue4179 georg.brandl patch Remove use of private attributes in smtpd 639 days http://bugs.python.org/issue4184 richard tokenize.detect_encoding() and Mac newline 615 days http://bugs.python.org/issue4377 haypo patch getopt should not accept no_argument that ends with '=' 590 days http://bugs.python.org/issue4629 haypo patch Exceptions in ConfigParser don't set .args 586 days http://bugs.python.org/issue4686 michael.foord patch binascii module, inconsistent behavior: some functions accept 575 days http://bugs.python.org/issue4770 flox patch On OS-X the directories searched by setup.py for Tk are in the 567 days http://bugs.python.org/issue4813 ronaldoussoren Pdb cannot access doctest source in postmortem 562 days http://bugs.python.org/issue4938 georg.brandl Duplicate UTF-16 BOM if a file is open in append mode 554 days http://bugs.python.org/issue5006 flox patch pdb "break" command messes up "continue" 528 days http://bugs.python.org/issue5294 georg.brandl Socket is closed prematurely in httplib, if server sends respo 488 days http://bugs.python.org/issue5542 haypo test_fileio fails on windows MSVC Assertion 491 days http://bugs.python.org/issue5544 BreamoreBoy doctest pdb readline broken 477 days http://bugs.python.org/issue5727 georg.brandl patch zipfile: Extracting a directory that already exists generates 437 days http://bugs.python.org/issue6050 ezio.melotti patch Error compiling valid regex 424 days http://bugs.python.org/issue6156 georg.brandl patch Incremental encoder incompatibility between 2.x and py3k 417 days http://bugs.python.org/issue6213 haypo patch robotparser doesn't handle URL's with query strings 401 days http://bugs.python.org/issue6325 orsenthil patch code.runsource() parsing bug 380 days http://bugs.python.org/issue6480 georg.brandl infinite recursion from calling builtins.open() 377 days http://bugs.python.org/issue6504 georg.brandl docs for unittest.expectedFailure do not syntactically show it 375 days http://bugs.python.org/issue6522 georg.brandl optparse required field for Options 373 days http://bugs.python.org/issue6535 georg.brandl patch, easy MatchObject is not a hyperlink in the 're' module documentatio 1 days http://bugs.python.org/issue6538 georg.brandl patch, easy string.Template custom pattern not working 360 days http://bugs.python.org/issue6630 georg.brandl pdb messes up when debugging an non-ascii program 347 days http://bugs.python.org/issue6719 georg.brandl Snow Leopard python program fails because _PyType_Modified is 326 days http://bugs.python.org/issue6812 ronaldoussoren test_macostools fails on os x 10.6 295 days http://bugs.python.org/issue7041 ronaldoussoren ConfigParser load speedup 287 days http://bugs.python.org/issue7113 brian.curtin patch ElementTree: Incorrect serialization of end-of-line characters 283 days http://bugs.python.org/issue7139 BreamoreBoy Extraneous newlines with csv.writer on Windows 1 days http://bugs.python.org/issue7198 skip.montanaro IGNORE_EXCEPTION_DETAIL should ignore the module name 228 days http://bugs.python.org/issue7490 r.david.murray patch ConfigParser can't read files with BOM markers 222 days http://bugs.python.org/issue7519 BreamoreBoy unicode exceptions terminate pdb.pm() loop 224 days http://bugs.python.org/issue7539 georg.brandl patch Improve 19.5. xml.dom.minidom doc 202 days http://bugs.python.org/issue7637 akuchling test_threadsignals leaks references 177 days http://bugs.python.org/issue7825 flox patch posix.getgroups() failure on Mac OS X 164 days http://bugs.python.org/issue7900 ronaldoussoren patch "-m pdb" SyntaxError for "\r\n" formatted py files 161 days http://bugs.python.org/issue7964 georg.brandl patch Add pure Python implementation of datetime module to CPython 151 days http://bugs.python.org/issue7989 belopolsky patch pdb "commands" command throws you into postmortem if you enter 156 days http://bugs.python.org/issue8015 georg.brandl patch doctest assumes sys.displayhook hasn't been touched 149 days http://bugs.python.org/issue8048 georg.brandl patch Wrong arguments in sqlite3.connect() documentation 139 days http://bugs.python.org/issue8129 haypo Improve support of PEP 383 (surrogates) in Python3: meta-issue 125 days http://bugs.python.org/issue8242 haypo patch Use locale encoding to encode command line arguments (subproce 65 days http://bugs.python.org/issue8775 loewis patch tkinter build variants on OSX 61 days http://bugs.python.org/issue8798 ronaldoussoren Promote SafeConfigParser and warn about ConfigParser 52 days http://bugs.python.org/issue8888 BreamoreBoy ctypes: remove implicit conversion between unicode and bytes 47 days http://bugs.python.org/issue8966 haypo patch PyArg_Parse*() functions: reject discontinious buffers 44 days http://bugs.python.org/issue8991 haypo patch xmlrpc: Transport.request() should also catch socket.error(EPI 35 days http://bugs.python.org/issue9032 haypo patch zipfile.extractall always raises an OSError after successfully 24 days http://bugs.python.org/issue9172 BreamoreBoy pop multiple elements of a list at once 13 days http://bugs.python.org/issue9218 tjreedy pdb.Pdb.checkline() throws AttributeError if called before sta 18 days http://bugs.python.org/issue9230 georg.brandl patch document that the 'test' package is for core developers only 10 days http://bugs.python.org/issue9255 tjreedy easy Python 2.7 breaks assigned __exit__s 12 days http://bugs.python.org/issue9259 dmalcolm test_struct failure on ARM 8 days http://bugs.python.org/issue9277 mark.dickinson patch, buildbot buggy repr for os.environ 11 days http://bugs.python.org/issue9283 haypo patch Dead code in Objects/object.c 9 days http://bugs.python.org/issue9294 pitrou patch, easy LC_CTYPE system setting not respected by setlocale() 3 days http://bugs.python.org/issue9335 haypo string.letters should display locale based equivalent of a-Z 1 days http://bugs.python.org/issue9336 r.david.murray file_wrapper fails to provide getsockopt() 5 days http://bugs.python.org/issue9354 georg.brandl patch Improved tests and style for fnmatch 0 days http://bugs.python.org/issue9356 brett.cannon patch Improve queue tests 0 days http://bugs.python.org/issue9357 brett.cannon patch Document fnmatch.translate's lack of support for escaping 0 days http://bugs.python.org/issue9358 brett.cannon patch Misspelled exception 0 days http://bugs.python.org/issue9359 ezio.melotti patch Installing a distro without sqlite3 will require a reinstall o 0 days http://bugs.python.org/issue9365 r.david.murray Reference leak for local new style class 5 days http://bugs.python.org/issue9366 georg.brandl test_getgroups in test_posix fails 0 days http://bugs.python.org/issue9367 r.david.murray patch Const-Correctness for Method Calls 0 days http://bugs.python.org/issue9368 merwok patch Refer to gnuwin32 diff util on development FAQ 1 days http://bugs.python.org/issue9376 brian.curtin easy Make python -m pickle do something useful 2 days http://bugs.python.org/issue9378 belopolsky patch, easy Allow popping of n elements at once from a deque 0 days http://bugs.python.org/issue9380 rhettinger syntax error in Unicode C API docs 0 days http://bugs.python.org/issue9381 georg.brandl PIL Bug with split 0 days http://bugs.python.org/issue9383 ezio.melotti Make python -m tkinter run tkinter demo 1 days http://bugs.python.org/issue9387 belopolsky patch, easy, needs review locale documentation describes non-existing ERA_YEAR constant 2 days http://bugs.python.org/issue9388 georg.brandl easy Traceback: Exception Shows Code that's On-Disk (Not in memory) 0 days http://bugs.python.org/issue9389 r.david.murray Allow docstrings on dicts and named tuples outside of function 0 days http://bugs.python.org/issue9391 rhettinger easy 2.7 framework install doesn't create 2to3-2.7 2 days http://bugs.python.org/issue9392 ronaldoussoren shelve.open/bsddb.hashopen raise bsddb.db.DBNoSuchFileError: ( 0 days http://bugs.python.org/issue9394 merwok Remove references to the missing dbm.bsd module 1 days http://bugs.python.org/issue9397 haypo automatically try forward operations when reverse operations a 1 days http://bugs.python.org/issue9401 georg.brandl ConfigParser exceptions do not subclass StandardError 0 days http://bugs.python.org/issue9406 r.david.murray patch ConfigParser documentation does not mention ConfigParser.Error 1 days http://bugs.python.org/issue9407 georg.brandl patch configparser doesn't support specifying encoding in read() 0 days http://bugs.python.org/issue9411 georg.brandl patch test_smtpd leaks references 0 days http://bugs.python.org/issue9412 georg.brandl infinite loop with specific regexp 0 days http://bugs.python.org/issue9414 georg.brandl Memory leak when reinitializing a Struct object 0 days http://bugs.python.org/issue9422 Winston451 patch Explicitly state lack of support for keyword arguments in buil 0 days http://bugs.python.org/issue9426 merwok <> in attrs in sgmllib not handled 2619 days http://bugs.python.org/issue745002 georg.brandl minidom.py -- TypeError: object doesn't support slice assig 2557 days http://bugs.python.org/issue777884 akuchling Improve pdb breakpoint feedback 2505 days http://bugs.python.org/issue809887 georg.brandl patch, easy document docs.python.org in PEP-101 2248 days http://bugs.python.org/issue965065 georg.brandl Defaults in ConfigParser.get overrides section values 2045 days http://bugs.python.org/issue1090076 georg.brandl "continue" in .pdbrc has no effect 1618 days http://bugs.python.org/issue1437051 georg.brandl easy test_minidom.py fails for Python-2.4.3 on SUSE 9.3 1575 days http://bugs.python.org/issue1463043 akuchling pdb 'run' crashes when the it's first argument is non-string 1564 days http://bugs.python.org/issue1472251 georg.brandl patch W3C <-> Python DOM type mapping docs need updating 1522 days http://bugs.python.org/issue1495229 akuchling easy Pdb doesn't call flush on its stdout file descriptor 1512 days http://bugs.python.org/issue1503502 georg.brandl Add "compose" function to the functools 1500 days http://bugs.python.org/issue1506122 rhettinger ConfigParser: accept leading whitespace on options+comments 1469 days http://bugs.python.org/issue1524825 tjreedy patch, easy ConfigParser support for alt delimiters 1228 days http://bugs.python.org/issue1682942 georg.brandl patch Remove references to _xmlrpclib from xmlrpclib.py 1081 days http://bugs.python.org/issue1773632 georg.brandl patch Top Issues Most Discussed (10) ______________________________ 23 Make exit/quit hint more novice friendly 7 days open http://bugs.python.org/issue9362 17 Sum() doc and behavior mismatch 237 days open http://bugs.python.org/issue7447 16 The trace module lacks unit tests 10 days open http://bugs.python.org/issue9315 14 IDLE won't launch on XP 2 days open http://bugs.python.org/issue9404 14 os.makedirs(): Add a keyword argument to suppress "File exists" 11 days open http://bugs.python.org/issue9299 13 Declaring a class creates circular references 1 days open http://bugs.python.org/issue9417 13 Pydoc interactive browsing enhancement 909 days open http://bugs.python.org/issue2001 13 ConfigParser support for alt delimiters 1228 days closed http://bugs.python.org/issue1682942 10 shlex have problems with parsing unicode 1047 days open http://bugs.python.org/issue1170 9 extend configparser to support mapping access(__*item__) 514 days open http://bugs.python.org/issue5412 From raymond.hettinger at gmail.com Fri Jul 30 18:19:34 2010 From: raymond.hettinger at gmail.com (Raymond Hettinger) Date: Fri, 30 Jul 2010 09:19:34 -0700 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: References: <4C5206F6.7090808@voidspace.org.uk> Message-ID: <479D0787-9DE6-4E70-A74B-64A7CFB839BB@gmail.com> On Jul 30, 2010, at 6:23 AM, Michael Foord wrote: > For those of you who found this document perhaps just a little bit too long, I've written up a *much* shorter intro to the plugin system (including how to get the prototype) on my blog: > > http://www.voidspace.org.uk/python/weblog/arch_d7_2010_07_24.shtml#e1186 Nice work. Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: From phd at phd.pp.ru Fri Jul 30 18:59:45 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Fri, 30 Jul 2010 20:59:45 +0400 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: References: Message-ID: <20100730165945.GA25784@phd.pp.ru> On Fri, Jul 30, 2010 at 07:26:26AM +0100, Daniel Waterworth wrote: > @Oleg: ... > This is purely CPython bug-fixing/the discussion of > implementation choices. I am not sure it's a bug. By manipulating sys.path (or symlinks in the FS) one can import the same file as different modules as many times as [s]he wants. Should this be fixed for __main__? I doubt it. Instead of making __main__ a special case follow the rule: don't import the same module under different paths/names. Make your script simply from test import main main() Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From fuzzyman at voidspace.org.uk Fri Jul 30 19:32:10 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 30 Jul 2010 18:32:10 +0100 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: <20100730165945.GA25784@phd.pp.ru> References: <20100730165945.GA25784@phd.pp.ru> Message-ID: <4C530C9A.3030204@voidspace.org.uk> On 30/07/2010 17:59, Oleg Broytman wrote: > On Fri, Jul 30, 2010 at 07:26:26AM +0100, Daniel Waterworth wrote: > >> @Oleg: ... >> This is purely CPython bug-fixing/the discussion of >> implementation choices. >> > I am not sure it's a bug. It isn't a bug but it's a very common *cause* of bugs, even for relatively experienced Python programmers (this exchange being another case in point). Michael > By manipulating sys.path (or symlinks in the > FS) one can import the same file as different modules as many times as > [s]he wants. Should this be fixed for __main__? I doubt it. Instead of > making __main__ a special case follow the rule: don't import the same > module under different paths/names. > Make your script simply > > from test import main > main() > > Oleg. > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From ziade.tarek at gmail.com Fri Jul 30 20:19:14 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 30 Jul 2010 20:19:14 +0200 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <4C52F5A9.8000707@netwok.org> References: <4C5206F6.7090808@voidspace.org.uk> <4C52ABBA.7080601@voidspace.org.uk> <20100730100400.3fad0c7d@heresy> <4C52E9FB.3000400@voidspace.org.uk> <4C52EF9C.9040701@netwok.org> <4C52F15F.3020604@voidspace.org.uk> <4C52F5A9.8000707@netwok.org> Message-ID: On Fri, Jul 30, 2010 at 5:54 PM, ?ric Araujo wrote: >> It has an API, but the plugins are not interface based (so interface >> requirements don't need to be part of the plugin system). > > Oh, I see. With duck-typing and ABCs, I don?t make a difference between > protocols and interfaces in my head :) Also, the I in API does mean > interface, but not imply bondage-and-discipline interface systems. > >> PEP 376 is *very* promising for this part - well at least if the PLUGINS >> stuff becomes part of PEP 376, it doesn't seem to be mentioned at all at >> the moment. > > The dist-info directory introduced by PEP 376 will certainly be reused. > Not sure if Tarek will want to edit PEP 376 (already accepted) to add > plugins in metadata (provides-dist is in metadata, so having plugins > here can make sense), or start a new PEP about a new concept in a new file. What we want to do I think is to make dist-info a place when any file can be added, besides the ones described in PEP 376, then build the plugins proposal on this > > Regards > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com > -- Tarek Ziad? | http://ziade.org From lukasz at langa.pl Fri Jul 30 21:24:09 2010 From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=) Date: Fri, 30 Jul 2010 21:24:09 +0200 Subject: [Python-Dev] configparser: `__name__` special key is mad Message-ID: <18A297B0-4C7D-4F64-9309-436002E4C1E1@langa.pl> Hello guys, I want to sum up all strange things about the behaviour of `__name__`, a special key present in every section of a parser instance. Hopefully it will be removed at once or in the worst case oficially deprecated and removed in 3.4. 1. There is a special `__name__` key in every section. 2. Except for the DEFAULTSECT. 3. `__name__` key is set for every section read from a file. 4. and not when adding by `add_section()`. 5. if `__name__` does exist, it's not visible in `parser.options('section')` 6. but it is here: `parser.has_option('section', '__name__') == True` 7. and can be obtained by `parser.get('section', '__name__')` 8. and can be changed by `parser.set('section', '__name__', 'ANY VALUE')` 9. and can be removed by `parser.remove_option('section', '__name__')` 10. even if the value is changed by `parser.set()`, it won't be written back to a file with `parser.write()` All this looks like a feature that was not particularly complete and well defined when it was first created. Or possibly, it became rotten with time and now nobody is using it anyway. That way or the other, I couldn't come up with a valid use case for `__name__` with the current implementation. It doesn't serve any internal purpose and the *only* way you can actually get it is to `parser.get('section', '__name__')` which returns 'section' anyway. About as useless as it gets. Of course, one can go and take the internal parser._sections data structure of the parser but that's evil. If I were to say what to do with all this, I'd simply remove all mentions of a special `__name__` key in configparser.py. I know that your default answer for that kind of idea is "no, backwards compatibility for the win". And I agree with that. In this case however we have a concept that is so broken that you can't actually use it. Many of you are here for quite a long time, maybe someone can shed some light on this. Maybe I'm missing something. -- Best regards, ?ukasz Langa tel. +48 791 080 144 WWW http://lukasz.langa.pl/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From da.waterworth at gmail.com Fri Jul 30 20:46:44 2010 From: da.waterworth at gmail.com (Daniel Waterworth) Date: Fri, 30 Jul 2010 19:46:44 +0100 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: <4C530C9A.3030204@voidspace.org.uk> References: <20100730165945.GA25784@phd.pp.ru> <4C530C9A.3030204@voidspace.org.uk> Message-ID: On 30 July 2010 18:32, Michael Foord wrote: > On 30/07/2010 17:59, Oleg Broytman wrote: >> >> On Fri, Jul 30, 2010 at 07:26:26AM +0100, Daniel Waterworth wrote: >> >>> >>> @Oleg: ... >>> This is purely CPython bug-fixing/the discussion of >>> implementation choices. >>> >> >> ? ?I am not sure it's a bug. > > It isn't a bug but it's a very common *cause* of bugs, even for relatively > experienced Python programmers (this exchange being another case in point). > > Michael > >> By manipulating sys.path (or symlinks in the >> FS) one can import the same file as different modules as many times as >> [s]he wants. Should this be fixed for __main__? I doubt it. Instead of >> making __main__ a special case follow the rule: don't import the same >> module under different paths/names. >> ? ?Make your script simply >> >> from test import main >> main() >> >> Oleg. >> > > > -- > http://www.ironpythoninaction.com/ > http://www.voidspace.org.uk/blog > > READ CAREFULLY. By accepting and reading this email you agree, on behalf of > your employer, to release me from all obligations and waivers arising from > any and all NON-NEGOTIATED agreements, licenses, terms-of-service, > shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, > non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have > entered into with your employer, its partners, licensors, agents and > assigns, in perpetuity, without prejudice to my ongoing rights and > privileges. You further represent that you have the authority to release me > from any BOGUS AGREEMENTS on behalf of your employer. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/da.waterworth%40googlemail.com > Having thought it through thoroughly, my preference is for a warning. I don't think it's a good practise to import the __main__ module by filename, as renaming the file will break the code. I got stung after, having dropped into a python interpreter shell and imported the module, I executed a function that uses isinstance. If a warning showed up after importing the module, explaining the problem and suggested that I use __import__('__main__') instead, I would have saved myself a fair amount of time debugging code. This is another case of "Explicit is better than implicit.". It also means that code that relies on the current behaviour will not be broken. @Oleg: yes, but in the __main__ case, it's more difficult to tell that you are importing something under a different name. I suppose the proof is in the pudding, can anyone think of a case where someone has been annoyed that, having imported that same module twice via symlinks, they have had problems relating to modules being independent instances? Thanks, Daniel -- active-thought.com From phd at phd.pp.ru Fri Jul 30 21:50:42 2010 From: phd at phd.pp.ru (Oleg Broytman) Date: Fri, 30 Jul 2010 23:50:42 +0400 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: References: <20100730165945.GA25784@phd.pp.ru> <4C530C9A.3030204@voidspace.org.uk> Message-ID: <20100730195042.GA8050@phd.pp.ru> On Fri, Jul 30, 2010 at 07:46:44PM +0100, Daniel Waterworth wrote: > can anyone think of a case where someone has > been annoyed that, having imported that same module twice via > symlinks, they have had problems relating to modules being independent > instances? I've had problems with two instances of the same module imported after sys.path manipulations. Never had a problem with reimported scripts. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From barry at python.org Fri Jul 30 22:03:10 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 30 Jul 2010 16:03:10 -0400 Subject: [Python-Dev] Bazaar branch of Python 2.7 Message-ID: <20100730160310.31e76666@heresy> Hello all! For those of you who use the Bazaar mirrors on Launchpad of the Python Subversion branches (dvcs ftw! :), I've just registered the Python 2.7 maintenance branch. It will take a little while to complete the import, but when it's done you can grab it with: % bzr branch lp:python/2.7 Series details are here: https://launchpad.net/python/2.7 Branch details are here: https://code.launchpad.net/~python-dev/python/2.7 Enjoy, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Fri Jul 30 22:26:08 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 30 Jul 2010 16:26:08 -0400 Subject: [Python-Dev] Is it intentional that "sys.__debug__ = 1" is illegal in Python 2.7? Message-ID: <20100730162608.7315c3a5@heresy> In working on making Python 2.7 available in Debian and Ubuntu, we ran into two packages that fail to byte compile against Python 2.7, where they are fine in Python 2.6. The Debian bug tracker issues are: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=590821 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=590822 Specifically, what fails now is assignment to the __debug__ attribute of a module, e.g. sys -- *not* builtin __debug__. % python2.6 Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> __debug__ = 1 File "", line 1 SyntaxError: can not assign to __debug__ >>> import sys >>> sys.__debug__ = 1 >>> % python2.7 Python 2.7.0+ (release27-maint:83294, Jul 30 2010, 15:49:51) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> __debug__ = 1 File "", line 1 SyntaxError: cannot assign to __debug__ >>> import sys >>> sys.__debug__ = 1 File "", line 1 SyntaxError: cannot assign to __debug__ >>> I think it's actually somewhat questionable that this is now illegal. It certainly breaks at least two packages. More disturbing though is that I cannot find mention of this change in Misc/NEWS or in the tracker, so I don't actually know if this was intentional or not. It looks like Benjamin's change in r67171 was the relevant diff. Thoughts? Either way I will file a bug. IOW, if this change should *not* have been made, I think it should be reverted for Python 2.7.1 and I can patch our version of Python in the meantime. If this change is intentional, I'll file a bug against (or just fix) Misc/NEWS and the What's New. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From gboutsioukis at gmail.com Fri Jul 30 22:28:14 2010 From: gboutsioukis at gmail.com (George Boutsioukis) Date: Fri, 30 Jul 2010 20:28:14 +0000 (UTC) Subject: [Python-Dev] Speeding up 2to3: Results from a GSOC Project Message-ID: Hi everyone, I haven't had a chance to introduce myself yet. I'm George Boutsioukis, a CS student from Greece and I'm currently enrolled as a GSOC student for the PSF. The task I was involved with for the past few weeks was speeding up the 2to3 tool. For those who are not aware of 2to3's internals, the tool matches a series of tree patterns to a python syntax tree and applies a code transformation on each match. The real bottleneck of this tool is the tree pattern matching algorithm, since the current version traverses all nodes of the tree, top-to-bottom, and checks each node against a set of tree patterns. The new algorithm I've implemented reduces the candidate nodes for matching. The process works in three steps: 1) Each pattern tree is reduced to the most 'characteristic' path, i.e. the rarest path you'll encounter on a real code tree. (this is simpler than it sounds) 2) From the set of the above linear patterns an Aho-Corasick automaton is constructed that is capable of inclusively matching all patterns(meaning that it may mark a wrong candidate but will never miss an instance) 3) The automaton is run on each leaf until the root of the tree is reached and the resulting match set is returned. And finally we apply each fixer for each node in the match set. Of course the process is a bit more complicated since we have to recheck the transformed code for new matches etc. Since it is quite a hard concept to illustrate in a post, tell me if you have any questions or need more info. The benefit of this somewhat complex process is substantial; the total run time is reduced by roughly 50%. The new module is still experimental code, but mature enough to pass all tests. I propose to include the new module with the next version of Python, perhaps as an explicit option for 2to3 until it can be thoroughly tested. The code repository can be found here: https://code.google.com/p/2to3-speedup2/source/browse/ (the code is a fork from py3k's 2to3 and was tested with version 3.1.2) The blog I've been using for GSOC(containing more details on the matching algorithm): http://george-gsoc.blogspot.com/ Regards, George From guido at python.org Fri Jul 30 22:42:37 2010 From: guido at python.org (Guido van Rossum) Date: Fri, 30 Jul 2010 13:42:37 -0700 Subject: [Python-Dev] Is it intentional that "sys.__debug__ = 1" is illegal in Python 2.7? In-Reply-To: <20100730162608.7315c3a5@heresy> References: <20100730162608.7315c3a5@heresy> Message-ID: On Fri, Jul 30, 2010 at 1:26 PM, Barry Warsaw wrote: > In working on making Python 2.7 available in Debian and Ubuntu, we ran into > two packages that fail to byte compile against Python 2.7, where they are fine > in Python 2.6. ?The Debian bug tracker issues are: > > ? ?http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=590821 > ? ?http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=590822 > > Specifically, what fails now is assignment to the __debug__ attribute of a > module, e.g. sys -- *not* builtin __debug__. > > % python2.6 > Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) > [GCC 4.4.3] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> __debug__ = 1 > ?File "", line 1 > SyntaxError: can not assign to __debug__ >>>> import sys >>>> sys.__debug__ = 1 >>>> > % python2.7 > Python 2.7.0+ (release27-maint:83294, Jul 30 2010, 15:49:51) > [GCC 4.4.3] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> __debug__ = 1 > ?File "", line 1 > SyntaxError: cannot assign to __debug__ >>>> import sys >>>> sys.__debug__ = 1 > ?File "", line 1 > SyntaxError: cannot assign to __debug__ >>>> > > I think it's actually somewhat questionable that this is now illegal. ?It > certainly breaks at least two packages. ?More disturbing though is that I > cannot find mention of this change in Misc/NEWS or in the tracker, so I don't > actually know if this was intentional or not. Well it is a reserved name so those packages that were setting it should have known that they were using undefined behavior that could change at any time. > It looks like Benjamin's change in r67171 was the relevant diff. > > Thoughts? ?Either way I will file a bug. ?IOW, if this change should *not* > have been made, I think it should be reverted for Python 2.7.1 and I can patch > our version of Python in the meantime. ?If this change is intentional, I'll > file a bug against (or just fix) Misc/NEWS and the What's New. -- --Guido van Rossum (python.org/~guido) From guido at python.org Fri Jul 30 22:43:47 2010 From: guido at python.org (Guido van Rossum) Date: Fri, 30 Jul 2010 13:43:47 -0700 Subject: [Python-Dev] Speeding up 2to3: Results from a GSOC Project In-Reply-To: References: Message-ID: Great! CS FTW! --Guido On Fri, Jul 30, 2010 at 1:28 PM, George Boutsioukis wrote: > Hi everyone, I haven't had a chance to introduce myself yet. I'm George > Boutsioukis, a CS student from Greece and I'm currently enrolled as a GSOC > student for the PSF. The task I was involved with for the past few weeks was > speeding up the 2to3 tool. > > For those who are not aware of 2to3's internals, the tool matches a series > of tree patterns to a python syntax tree and applies a code transformation > on each match. The real bottleneck of this tool is the tree pattern matching > algorithm, since the current version traverses all nodes of the tree, > top-to-bottom, and checks each node against a set of tree patterns. > > The new algorithm I've implemented reduces the candidate nodes for matching. > The process works in three steps: > > 1) Each pattern tree is reduced to the most 'characteristic' path, i.e. the > rarest path you'll encounter on a real code tree. (this is simpler than it > sounds) > > 2) From the set of the above linear patterns an Aho-Corasick automaton is > constructed that is capable of inclusively matching all patterns(meaning > that it may mark a wrong candidate but will never miss an instance) > > 3) The automaton is run on each leaf until the root of the tree is reached > and the resulting match set is returned. And finally we apply each fixer > for each node in the match set. Of course the process is a bit more > complicated since we have to recheck the transformed code for new matches etc. > > Since it is quite a hard concept to illustrate in a post, tell me if you > have any questions or need more info. > > The benefit of this somewhat complex process is substantial; the total run > time is reduced by roughly 50%. The new module is still experimental code, > but mature enough to pass all tests. I propose to include the new module > with the next version of Python, perhaps as an explicit option for 2to3 > until it can be thoroughly tested. > > The code repository can be found here: > https://code.google.com/p/2to3-speedup2/source/browse/ > (the code is a fork from py3k's 2to3 and was tested with version 3.1.2) > > The blog I've been using for GSOC(containing more details on the matching > algorithm): http://george-gsoc.blogspot.com/ > > Regards, > George > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) From barry at python.org Fri Jul 30 22:53:25 2010 From: barry at python.org (Barry Warsaw) Date: Fri, 30 Jul 2010 16:53:25 -0400 Subject: [Python-Dev] Is it intentional that "sys.__debug__ = 1" is illegal in Python 2.7? In-Reply-To: References: <20100730162608.7315c3a5@heresy> Message-ID: <20100730165325.49414a21@heresy> On Jul 30, 2010, at 01:42 PM, Guido van Rossum wrote: >Well it is a reserved name so those packages that were setting it >should have known that they were using undefined behavior that could >change at any time. Shouldn't it be described here then? http://docs.python.org/reference/lexical_analysis.html#identifiers -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From fuzzyman at voidspace.org.uk Fri Jul 30 22:56:02 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 30 Jul 2010 21:56:02 +0100 Subject: [Python-Dev] Is it intentional that "sys.__debug__ = 1" is illegal in Python 2.7? In-Reply-To: <20100730165325.49414a21@heresy> References: <20100730162608.7315c3a5@heresy> <20100730165325.49414a21@heresy> Message-ID: <4C533C62.9030206@voidspace.org.uk> On 30/07/2010 21:53, Barry Warsaw wrote: > On Jul 30, 2010, at 01:42 PM, Guido van Rossum wrote: > > >> Well it is a reserved name so those packages that were setting it >> should have known that they were using undefined behavior that could >> change at any time. >> > Shouldn't it be described here then? > > http://docs.python.org/reference/lexical_analysis.html#identifiers > And raise a DeprecationWarning one release before becoming invalid syntax? Michael > -Barry > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pje at telecommunity.com Fri Jul 30 22:56:44 2010 From: pje at telecommunity.com (P.J. Eby) Date: Fri, 30 Jul 2010 16:56:44 -0400 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <4C52E2EF.3020401@voidspace.org.uk> References: <4C5206F6.7090808@voidspace.org.uk> <4C52ABBA.7080601@voidspace.org.uk> <20100730100400.3fad0c7d@heresy> <4C52E2EF.3020401@voidspace.org.uk> Message-ID: <20100730205641.ED73E3A40DF@sparrow.telecommunity.com> At 03:34 PM 7/30/2010 +0100, Michael Foord wrote: >Automatic discoverability, a-la setuptools entry points, is not >without its problems though. Tarek outlines some of these in a more >recent blog post: FWIW, it's not discovery that's the problem, but configuring *which* plugins you wish to have active. Entry points support access by name, and it's up to the application using them to decide *which* ones to load. The underlying idea is that entry points expose a hook; it's up to the app to decide which ones it should actually import and use. An application also can list the available plugins and ask the user, etc. (For example, setuptools only loads "setup() argument" entry points for specified arguments, and command entry points only for the commands a user explicitly invokes.) IOW, entry points provide access to plugins, not policy or configuration for *which* plugins you wish to use. This was an intentional decision since applications vary widely in what sort of configuration mechanism they use. In the simplest cases (e.g. single-app environments like Chandler), simply making the plugin available on sys.path (e.g. via a special plugins directory) is configuration enough. In more complex use cases, an app might have to import plugins in order to get more information about them. From fuzzyman at voidspace.org.uk Fri Jul 30 23:02:44 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 30 Jul 2010 22:02:44 +0100 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <20100730205641.ED73E3A40DF@sparrow.telecommunity.com> References: <4C5206F6.7090808@voidspace.org.uk> <4C52ABBA.7080601@voidspace.org.uk> <20100730100400.3fad0c7d@heresy> <4C52E2EF.3020401@voidspace.org.uk> <20100730205641.ED73E3A40DF@sparrow.telecommunity.com> Message-ID: <4C533DF4.80908@voidspace.org.uk> On 30/07/2010 21:56, P.J. Eby wrote: > At 03:34 PM 7/30/2010 +0100, Michael Foord wrote: >> Automatic discoverability, a-la setuptools entry points, is not >> without its problems though. Tarek outlines some of these in a more >> recent blog post: > > FWIW, it's not discovery that's the problem, but configuring *which* > plugins you wish to have active. Entry points support access by name, > and it's up to the application using them to decide *which* ones to load. > > The underlying idea is that entry points expose a hook; it's up to the > app to decide which ones it should actually import and use. An > application also can list the available plugins and ask the user, etc. > (For example, setuptools only loads "setup() argument" entry points > for specified arguments, and command entry points only for the > commands a user explicitly invokes.) > > IOW, entry points provide access to plugins, not policy or > configuration for *which* plugins you wish to use. This was an > intentional decision since applications vary widely in what sort of > configuration mechanism they use. In the simplest cases (e.g. > single-app environments like Chandler), simply making the plugin > available on sys.path (e.g. via a special plugins directory) is > configuration enough. In more complex use cases, an app might have to > import plugins in order to get more information about them. > Right (and thanks), and in the unittest plugin system the user decides which plugins should be active by listing them explicitly in the configuration file. Discovery could be useful for a tool that tells the user which plugins are available and modify the config file *for them* to switch plugins on and off. Useful metadata would then be which config options a plugin supports (and their defaults) so they can be added to the config file too when a plugin is activated. Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From pje at telecommunity.com Fri Jul 30 23:03:20 2010 From: pje at telecommunity.com (P.J. Eby) Date: Fri, 30 Jul 2010 17:03:20 -0400 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: <20100730195042.GA8050@phd.pp.ru> References: <20100730165945.GA25784@phd.pp.ru> <4C530C9A.3030204@voidspace.org.uk> <20100730195042.GA8050@phd.pp.ru> Message-ID: <20100730210311.F10EB3A40DF@sparrow.telecommunity.com> At 11:50 PM 7/30/2010 +0400, Oleg Broytman wrote: >On Fri, Jul 30, 2010 at 07:46:44PM +0100, Daniel Waterworth wrote: > > can anyone think of a case where someone has > > been annoyed that, having imported that same module twice via > > symlinks, they have had problems relating to modules being independent > > instances? > > I've had problems with two instances of the same module imported after >sys.path manipulations. Never had a problem with reimported scripts. I have. The "unittest" module used to have this problem, when used as a script. From pje at telecommunity.com Fri Jul 30 23:04:19 2010 From: pje at telecommunity.com (P.J. Eby) Date: Fri, 30 Jul 2010 17:04:19 -0400 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) Message-ID: <20100730210410.8C70E3A4105@sparrow.telecommunity.com> At 04:37 PM 7/30/2010 +0200, Tarek Ziad? wrote: >On Fri, Jul 30, 2010 at 4:04 PM, Barry Warsaw wrote: >.. > > * Registration - How do third party plugins declare themselves to > exist, and > > be enabled? Part of this seems to me to include interface declarations > > too. Is installation of the plugin enough to register it? How > do end users > > enable and disable plugins that me be registered on their system? How do > > plugins describe themselves (provide short and log descriptions, declare > > options, hook into command line interfaces, etc.)? > > > > * Installation - How are plugins installed on the system? Do they have to > > appear in a special directory on the file system? Do they need special > > setup.py magic to write extra files? Do they need to live in a > pre-defined > > namespace? > >FWIW We are thinking about adding in distutils2 a system quite similar >to the entry points >setuptools has, but with extra abilities for the end user : > >- activate / deactivate plugins without having to remove the project >that added them >- configure globally if plugins are implicitely activated or not -- >and maybe allow the distutils2 installer to ask the user > when a plugin is detected if he wants it activate or not >- provide a tool to browse them Note, by the way, that none of these are mutually exclusive to the entry point mechanism; it is simply up to an application developer to decide which of those features he/she wishes to provide. A library that provides common implementations of such features on top of entry points would be a good idea. pkg_resources already supplies one such tool, btw: the "find_plugins()" API for locating projects in one or more "plugin directories" that *could* be added to sys.path to provide plugins for an application. It's then up to the application to filter this list further (e.g. via its own configuration). From ggpolo at gmail.com Fri Jul 30 23:06:15 2010 From: ggpolo at gmail.com (Guilherme Polo) Date: Fri, 30 Jul 2010 18:06:15 -0300 Subject: [Python-Dev] Is it intentional that "sys.__debug__ = 1" is illegal in Python 2.7? In-Reply-To: <20100730165325.49414a21@heresy> References: <20100730162608.7315c3a5@heresy> <20100730165325.49414a21@heresy> Message-ID: 2010/7/30 Barry Warsaw : > On Jul 30, 2010, at 01:42 PM, Guido van Rossum wrote: > >>Well it is a reserved name so those packages that were setting it >>should have known that they were using undefined behavior that could >>change at any time. > > Shouldn't it be described here then? > > http://docs.python.org/reference/lexical_analysis.html#identifiers > Doesn't the section http://docs.python.org/reference/lexical_analysis.html#reserved-classes-of-identifiers make this clear enough ? > -Barry > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com > > -- -- Guilherme H. Polo Goncalves From alexandre at peadrop.com Fri Jul 30 23:12:27 2010 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Fri, 30 Jul 2010 14:12:27 -0700 Subject: [Python-Dev] Speeding up 2to3: Results from a GSOC Project In-Reply-To: References: Message-ID: Love it! BTW, it's not a good idea to have an import statement under 3 level of loops: https://code.google.com/p/2to3-speedup2/source/browse/trunk/lib2to3/refactor.py#427 -- Alexandre From guido at python.org Fri Jul 30 23:44:42 2010 From: guido at python.org (Guido van Rossum) Date: Fri, 30 Jul 2010 14:44:42 -0700 Subject: [Python-Dev] Is it intentional that "sys.__debug__ = 1" is illegal in Python 2.7? In-Reply-To: <20100730165325.49414a21@heresy> References: <20100730162608.7315c3a5@heresy> <20100730165325.49414a21@heresy> Message-ID: On Fri, Jul 30, 2010 at 1:53 PM, Barry Warsaw wrote: > On Jul 30, 2010, at 01:42 PM, Guido van Rossum wrote: > >>Well it is a reserved name so those packages that were setting it >>should have known that they were using undefined behavior that could >>change at any time. > > Shouldn't it be described here then? > > http://docs.python.org/reference/lexical_analysis.html#identifiers No, since it is covered here: http://docs.python.org/reference/lexical_analysis.html#reserved-classes-of-identifiers -- --Guido van Rossum (python.org/~guido) From esnow at verio.net Fri Jul 30 23:23:22 2010 From: esnow at verio.net (Eric Snow) Date: Fri, 30 Jul 2010 17:23:22 -0400 Subject: [Python-Dev] Is it intentional that "sys.__debug__ = 1" isillegal in Python 2.7? In-Reply-To: References: <20100730162608.7315c3a5@heresy> <20100730165325.49414a21@heresy> Message-ID: First appeared in docs for 2.6 (October 02, 2008). Not sure if that is when it first because constrained this way. http://docs.python.org/library/constants.html?highlight=__debug__#__debug__ -eric -----Original Message----- From: python-dev-bounces+esnow=verio.net at python.org [mailto:python-dev-bounces+esnow=verio.net at python.org] On Behalf Of Guilherme Polo Sent: Friday, July 30, 2010 3:06 PM To: Barry Warsaw Cc: Guido van Rossum; Python-Dev Dev Subject: Re: [Python-Dev] Is it intentional that "sys.__debug__ = 1" isillegal in Python 2.7? 2010/7/30 Barry Warsaw : > On Jul 30, 2010, at 01:42 PM, Guido van Rossum wrote: > >>Well it is a reserved name so those packages that were setting it >>should have known that they were using undefined behavior that could >>change at any time. > > Shouldn't it be described here then? > > http://docs.python.org/reference/lexical_analysis.html#identifiers > Doesn't the section http://docs.python.org/reference/lexical_analysis.html#reserved-classes-of-identifiers make this clear enough ? > -Barry > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com > > -- -- Guilherme H. Polo Goncalves _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/esnow%40verio.net This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio, Inc. makes no warranty that this email is error or virus free. Thank you. From steve at pearwood.info Sat Jul 31 00:32:59 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 31 Jul 2010 08:32:59 +1000 Subject: [Python-Dev] Is it intentional that "sys.__debug__ = 1" is illegal in Python 2.7? In-Reply-To: References: <20100730162608.7315c3a5@heresy> <20100730165325.49414a21@heresy> Message-ID: <201007310833.00914.steve@pearwood.info> On Sat, 31 Jul 2010 07:44:42 am Guido van Rossum wrote: > On Fri, Jul 30, 2010 at 1:53 PM, Barry Warsaw wrote: > > On Jul 30, 2010, at 01:42 PM, Guido van Rossum wrote: > >>Well it is a reserved name so those packages that were setting it > >>should have known that they were using undefined behavior that > >> could change at any time. > > > > Shouldn't it be described here then? > > > > http://docs.python.org/reference/lexical_analysis.html#identifiers > > No, since it is covered here: > > http://docs.python.org/reference/lexical_analysis.html#reserved-class >es-of-identifiers I have a small concern about the wording of that, specifically this: "System-defined names. These names are defined by the interpreter and its implementation (including the standard library); applications SHOULD NOT EXPECT TO DEFINE additional names using this convention. The set of names of this class defined by Python may be extended in future versions." [emphasis added] This implies to me that at some time in the future, Python may make it illegal to assign to any __*__ name apart from those in a list of "approved" methods. Is that the intention? I have always understood that if you create your own __*__ names, you risk clashing with a special method, but otherwise it is allowed, if disapproved off. I would not like to see it become forbidden. -- Steven D'Aprano From g.brandl at gmx.net Sat Jul 31 00:45:05 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 31 Jul 2010 00:45:05 +0200 Subject: [Python-Dev] pdb mini-sprint report and questions Message-ID: Hi, to warm up for tomorrow's 3.2alpha1 release, I did a mini-sprint on pdb issues today. I'm pleased to report that 14 issues could be closed, and pdb got a range of small new features, such as commands on the command line, "until " or "longlist" showing all the code for the current function (the latter courtesy of Antonio Cuni's pdb++). One issue that's not yet closed is #7245, which adds a (very nice IMO) feature: when you press Ctrl-C while the program being debugged runs, you will not get a traceback but execution is suspended, and you can debug from the current point of execution -- just like in gdb. However, there were apparently issues with some of the buildbots when the patch was applied for a short time. I also don't know how and if it works on Windows, so I'd need some helpful people testing it. Another question is about a feature of pdb++ that I personally would like, but imagine would make others unhappy: one-letter abbreviations of commands such as c(ontinue) or l(ist) are also often-used variable names, so they are frequently typed without the required "!" or "print" that would distinguish them from the command, and the command is actually executed. The feature in question would default to printing the variable in cases where one exists -- handy enough or too inconsistent? Also, are there any other features you would like to see? One feature of pdb++ that is general enough and has no dependencies would be watch expressions... Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From guido at python.org Sat Jul 31 01:01:49 2010 From: guido at python.org (Guido van Rossum) Date: Fri, 30 Jul 2010 16:01:49 -0700 Subject: [Python-Dev] pdb mini-sprint report and questions In-Reply-To: References: Message-ID: On Fri, Jul 30, 2010 at 3:45 PM, Georg Brandl wrote: > to warm up for tomorrow's 3.2alpha1 release, I did a mini-sprint on > pdb issues today. ?I'm pleased to report that 14 issues could be closed, > and pdb got a range of small new features, such as commands on the > command line, "until " or "longlist" showing all the code for the > current function (the latter courtesy of Antonio Cuni's pdb++). > > One issue that's not yet closed is #7245, which adds a (very nice IMO) > feature: when you press Ctrl-C while the program being debugged runs, > you will not get a traceback but execution is suspended, and you can > debug from the current point of execution -- just like in gdb. > > However, there were apparently issues with some of the buildbots when > the patch was applied for a short time. ?I also don't know how and if > it works on Windows, so I'd need some helpful people testing it. > > Another question is about a feature of pdb++ that I personally would > like, but imagine would make others unhappy: ?one-letter abbreviations > of commands such as c(ontinue) or l(ist) are also often-used variable > names, so they are frequently typed without the required "!" or "print" > that would distinguish them from the command, and the command is > actually executed. ?The feature in question would default to printing > the variable in cases where one exists -- handy enough or too > inconsistent? Warning: 'p' is *not* the same as 'print'. 'p' is a built-in command that prints the expression's repr(). 'print' OTOH falls back through the interpreter and executes a print statement or function, which prints the str(). For me personally, the problem this feature tries to address is never a problem (of course I am probably unique as far as pdb users go :-) so I'd like a way to turn it off. > Also, are there any other features you would like to see? ?One feature > of pdb++ that is general enough and has no dependencies would be watch > expressions... Cool. -- --Guido van Rossum (python.org/~guido) From g.brandl at gmx.net Sat Jul 31 01:07:47 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 31 Jul 2010 01:07:47 +0200 Subject: [Python-Dev] pdb mini-sprint report and questions In-Reply-To: References: Message-ID: Am 31.07.2010 01:01, schrieb Guido van Rossum: > On Fri, Jul 30, 2010 at 3:45 PM, Georg Brandl wrote: >> to warm up for tomorrow's 3.2alpha1 release, I did a mini-sprint on >> pdb issues today. I'm pleased to report that 14 issues could be closed, >> and pdb got a range of small new features, such as commands on the >> command line, "until " or "longlist" showing all the code for the >> current function (the latter courtesy of Antonio Cuni's pdb++). >> >> One issue that's not yet closed is #7245, which adds a (very nice IMO) >> feature: when you press Ctrl-C while the program being debugged runs, >> you will not get a traceback but execution is suspended, and you can >> debug from the current point of execution -- just like in gdb. >> >> However, there were apparently issues with some of the buildbots when >> the patch was applied for a short time. I also don't know how and if >> it works on Windows, so I'd need some helpful people testing it. >> >> Another question is about a feature of pdb++ that I personally would >> like, but imagine would make others unhappy: one-letter abbreviations >> of commands such as c(ontinue) or l(ist) are also often-used variable >> names, so they are frequently typed without the required "!" or "print" >> that would distinguish them from the command, and the command is >> actually executed. The feature in question would default to printing >> the variable in cases where one exists -- handy enough or too >> inconsistent? > > Warning: 'p' is *not* the same as 'print'. 'p' is a built-in command > that prints the expression's repr(). 'print' OTOH falls back through > the interpreter and executes a print statement or function, which > prints the str(). Not in py3k -- 'print' is an alias for 'p' there, otherwise 'print x' would be a syntax error. > For me personally, the problem this feature tries to address is never > a problem (of course I am probably unique as far as pdb users go :-) > so I'd like a way to turn it off. > >> Also, are there any other features you would like to see? One feature >> of pdb++ that is general enough and has no dependencies would be watch >> expressions... > > Cool. I take it you'd like to see them? ;) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From guido at python.org Sat Jul 31 01:14:47 2010 From: guido at python.org (Guido van Rossum) Date: Fri, 30 Jul 2010 16:14:47 -0700 Subject: [Python-Dev] pdb mini-sprint report and questions In-Reply-To: References: Message-ID: On Fri, Jul 30, 2010 at 4:07 PM, Georg Brandl wrote: > Am 31.07.2010 01:01, schrieb Guido van Rossum: >> Warning: 'p' is *not* the same as 'print'. 'p' is a built-in command >> that prints the expression's repr(). 'print' OTOH falls back through >> the interpreter and executes a print statement or function, which >> prints the str(). > > Not in py3k -- 'print' is an alias for 'p' there, otherwise 'print x' > would be a syntax error. Ah, ok, even better. >> For me personally, the problem this feature tries to address is never >> a problem (of course I am probably unique as far as pdb users go :-) >> so I'd like a way to turn it off. >> >>> Also, are there any other features you would like to see? ?One feature >>> of pdb++ that is general enough and has no dependencies would be watch >>> expressions... >> >> Cool. > > I take it you'd like to see them? ;) Sorry, yes, I meant that as a "kewl" cool, not "lukewarm" cool. IOW +1, not +0. :) -- --Guido van Rossum (python.org/~guido) From cournape at gmail.com Sat Jul 31 02:51:45 2010 From: cournape at gmail.com (David Cournapeau) Date: Sat, 31 Jul 2010 09:51:45 +0900 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: References: <4C5206F6.7090808@voidspace.org.uk> Message-ID: On Fri, Jul 30, 2010 at 10:23 PM, Michael Foord wrote: > For those of you who found this document perhaps just a little bit too long, > I've written up a *much* shorter intro to the plugin system (including how > to get the prototype) on my blog: > > ??? http://www.voidspace.org.uk/python/weblog/arch_d7_2010_07_24.shtml#e1186 This looks nice and simple, but I am a bit worried about the configuration file for registration. My experience is that end users don't like editing files much. I understand that may be considered as bikesheding, but have you considered a system analog to bzr instead ? A plugin is a directory somewhere, which means that disabling it is just removing a directory. In my experience, it is more reliable from a user POV than e.g. the hg way of doing things. The plugin system of bzr is one of the thing that I still consider the best in its category, even though I stopped using bzr for quite some time. The registration was incredibly robust and easy to use from a user and developer POV, David From ncoghlan at gmail.com Sat Jul 31 03:06:42 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 31 Jul 2010 11:06:42 +1000 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: References: <20100730165945.GA25784@phd.pp.ru> <4C530C9A.3030204@voidspace.org.uk> Message-ID: On Sat, Jul 31, 2010 at 4:46 AM, Daniel Waterworth wrote: > Having thought it through thoroughly, my preference is for a warning. That's actually harder than it sounds. Inserting "__main__" into sys.modules under its normal name as well as "__main__" is actually pretty easy (for both direct execution and -m). Making it generate a warning when accessing __main__ under its normal name is trickier (akin to the various "lazy module loading" hacks that are available in assorted packages on pypi). Extending this to work for arbitrary objects is very hard to do efficiently (you'd almost certainly need an additional index from __file__ values to sys.modules keys that would complain whenever the list of associated key entries for a given __file__ value contained more than 1 entry). That's a lot of infrastructure just to try and detect a fairly rare kind of bug. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From alexander.belopolsky at gmail.com Sat Jul 31 03:21:05 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Fri, 30 Jul 2010 21:21:05 -0400 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: References: <20100730165945.GA25784@phd.pp.ru> <4C530C9A.3030204@voidspace.org.uk> Message-ID: On Fri, Jul 30, 2010 at 2:46 PM, Daniel Waterworth wrote: .. > Having thought it through thoroughly, my preference is for a warning. > > I don't think it's a good practise to import the __main__ module by > filename, as renaming the file will break the code. I got stung after, > having dropped into a python interpreter shell and imported the > module, I executed a function that uses isinstance. > > If a warning showed up after importing the module, explaining the > problem and suggested that I use __import__('__main__') instead, I > would have saved myself a fair amount of time debugging code. This is > another case of "Explicit is better than implicit.". You can easily disallow importing __main__ module by filename by simply giving your script a name that does not end with .py or by using say '-' character in the filename. No change to python itself is needed. From ncoghlan at gmail.com Sat Jul 31 03:35:29 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 31 Jul 2010 11:35:29 +1000 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <4C52E2EF.3020401@voidspace.org.uk> References: <4C5206F6.7090808@voidspace.org.uk> <4C52ABBA.7080601@voidspace.org.uk> <20100730100400.3fad0c7d@heresy> <4C52E2EF.3020401@voidspace.org.uk> Message-ID: On Sat, Jul 31, 2010 at 12:34 AM, Michael Foord wrote: > Explicit registration over implicit registration by subclassing is an > interesting discussion, but I like the simplicity provided by just > subclassing. Note that ABCs are deliberately designed to let *users* choose to do either. Subclassing gets you better implementation support (since you pick up all the concrete method implementations "for free"), but you can still use explicit registration if you have an existing class that provides the API but isn't a subclass of the ABC. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From matthieu.brucher at gmail.com Sat Jul 31 08:08:43 2010 From: matthieu.brucher at gmail.com (Matthieu Brucher) Date: Sat, 31 Jul 2010 08:08:43 +0200 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: References: <4C5206F6.7090808@voidspace.org.uk> Message-ID: 2010/7/31 David Cournapeau : > On Fri, Jul 30, 2010 at 10:23 PM, Michael Foord > wrote: >> For those of you who found this document perhaps just a little bit too long, >> I've written up a *much* shorter intro to the plugin system (including how >> to get the prototype) on my blog: >> >> ??? http://www.voidspace.org.uk/python/weblog/arch_d7_2010_07_24.shtml#e1186 > > This looks nice and simple, but I am a bit worried about the > configuration file for registration. My experience is that end users > don't like editing files much. I understand that may be considered as > bikesheding, but have you considered a system analog to bzr instead ? > A plugin is a directory somewhere, which means that disabling it is > just removing a directory. In my experience, it is more reliable from > a user POV than e.g. the hg way of doing things. The plugin system of > bzr is one of the thing that I still consider the best in its > category, even though I stopped using bzr for quite some time. The > registration was incredibly robust and easy to use from a user and > developer POV, > > David Hi David, I think the point Michael tries to make is to be able to activate (not register) some plugins for some projects. In this case, even if bzr system is used (and I agree, it's a really good system), how will you activate only some of them for your project? I don't think it is geared towards end users, much more towards developers (at least for the unittest plugins) in which case adding a .cfg file to your vcs is not much work to do ;) Matthieu -- Information System Engineer, Ph.D. Blog: http://matt.eifelle.com LinkedIn: http://www.linkedin.com/in/matthieubrucher From da.waterworth at gmail.com Sat Jul 31 07:57:33 2010 From: da.waterworth at gmail.com (Daniel Waterworth) Date: Sat, 31 Jul 2010 06:57:33 +0100 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: References: <20100730165945.GA25784@phd.pp.ru> <4C530C9A.3030204@voidspace.org.uk> Message-ID: On 31 July 2010 02:21, Alexander Belopolsky wrote: > On Fri, Jul 30, 2010 at 2:46 PM, Daniel Waterworth > wrote: > .. >> Having thought it through thoroughly, my preference is for a warning. >> >> I don't think it's a good practise to import the __main__ module by >> filename, as renaming the file will break the code. I got stung after, >> having dropped into a python interpreter shell and imported the >> module, I executed a function that uses isinstance. >> >> If a warning showed up after importing the module, explaining the >> problem and suggested that I use __import__('__main__') instead, I >> would have saved myself a fair amount of time debugging code. This is >> another case of "Explicit is better than implicit.". > > > You can easily disallow importing __main__ module by filename by > simply giving your script a name that does not end with .py or by > using say '-' character in the filename. ? No change to python itself > is needed. > My problem is not that I'm likely to fall for the same trap twice. It's that I want to prevent other people from doing so. @Nick: I suppose the simplest way to detect re-importation in the general case, is to store a set of hashes of files that have been imported. When a user tries to import a file where it's hash is already in the set, a warning is generated. It's simpler than trying to figure out all the different ways that a file can be imported, and will also detect copied files. This is less infrastructure than you were suggesting, but it's not a perfect solution. Thanks, Daniel -- active-thought.com From fuzzyman at voidspace.org.uk Sat Jul 31 13:46:05 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sat, 31 Jul 2010 12:46:05 +0100 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: References: <4C5206F6.7090808@voidspace.org.uk> Message-ID: <4C540CFD.9030308@voidspace.org.uk> On 31/07/2010 01:51, David Cournapeau wrote: > On Fri, Jul 30, 2010 at 10:23 PM, Michael Foord > wrote: > >> For those of you who found this document perhaps just a little bit too long, >> I've written up a *much* shorter intro to the plugin system (including how >> to get the prototype) on my blog: >> >> http://www.voidspace.org.uk/python/weblog/arch_d7_2010_07_24.shtml#e1186 >> > This looks nice and simple, but I am a bit worried about the > configuration file for registration. My experience is that end users > don't like editing files much. I understand that may be considered as > bikesheding, but have you considered a system analog to bzr instead ? > A plugin is a directory somewhere, which means that disabling it is > just removing a directory. In my experience, it is more reliable from > a user POV than e.g. the hg way of doing things. The plugin system of > bzr is one of the thing that I still consider the best in its > category, even though I stopped using bzr for quite some time. The > registration was incredibly robust and easy to use from a user and > developer POV, > > Definitely not bikeshedding, a useful suggestion David. As Matthieu says in his reply, individual projects need to be able to enable (and configure) individual plugins that their tests depend on - potentially even shipping the plugin with the project. The other side of this is generally useful plugins that developers may want to have permanently active (like the debugger plugin), so that it is always available to them (via a command line switch). The proposed system allows this with a user configuration file plus a per-project configuration file. I take your point about users not liking configuration files though. I've looked a little bit at the bzr plugin system and I like the plugins subcommand. If PEP 376 goes ahead then we could keep the user plugin and use the PEP 376 metadata, in concert with a user config file, to discover all plugins *available*. A plugins subcommand could then activate / deactivate individual plugins by editing (or creating) the config file for the user. This could be bolted *on top* of the config file solution once PEP 376 is in place. It *doesn't* handle the problem of configuring plugins. So long as metadata is available about what configuration options plugins have (through a plugins API) then the plugins subcommand could also handle configuration. Installation of plugins would still be done through the standard distutils(2) machinery. (Using PEP 376 would depend on distutils2. I would be fine with this.) Another possibility would be to have a zero-config plugin installation solution *as well* as the config files. Create a plugins directory (in the user home directory?) and automatically activate plugins in this directory. This violates TOOWTDI though. As it happens adding a plugin directory would be easy to implement as a plugin... All the best, Michael > David > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From fuzzyman at voidspace.org.uk Sat Jul 31 13:57:09 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sat, 31 Jul 2010 12:57:09 +0100 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <4C540CFD.9030308@voidspace.org.uk> References: <4C5206F6.7090808@voidspace.org.uk> <4C540CFD.9030308@voidspace.org.uk> Message-ID: <4C540F95.9030606@voidspace.org.uk> On 31/07/2010 12:46, Michael Foord wrote: > [snip...] > If PEP 376 goes ahead then we could keep the user plugin I meant "keep the user config file". Michael > and use the PEP 376 metadata, in concert with a user config file, to > discover all plugins *available*. A plugins subcommand could then > activate / deactivate individual plugins by editing (or creating) the > config file for the user. This could be bolted *on top* of the config > file solution once PEP 376 is in place. It *doesn't* handle the > problem of configuring plugins. So long as metadata is available about > what configuration options plugins have (through a plugins API) then > the plugins subcommand could also handle configuration. > > Installation of plugins would still be done through the standard > distutils(2) machinery. (Using PEP 376 would depend on distutils2. I > would be fine with this.) > > Another possibility would be to have a zero-config plugin installation > solution *as well* as the config files. Create a plugins directory (in > the user home directory?) and automatically activate plugins in this > directory. This violates TOOWTDI though. As it happens adding a plugin > directory would be easy to implement as a plugin... > > All the best, > > Michael > >> David > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From ncoghlan at gmail.com Sat Jul 31 17:07:40 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 1 Aug 2010 01:07:40 +1000 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: References: <20100730165945.GA25784@phd.pp.ru> <4C530C9A.3030204@voidspace.org.uk> Message-ID: On Sat, Jul 31, 2010 at 3:57 PM, Daniel Waterworth wrote: > @Nick: I suppose the simplest way to detect re-importation in the > general case, is to store a set of hashes of files that have been > imported. When a user tries to import a file where it's hash is > already in the set, a warning is generated. It's simpler than trying > to figure out all the different ways that a file can be imported, and > will also detect copied files. This is less infrastructure than you > were suggesting, but it's not a perfect solution. Hashing every file on import would definitely be more overhead than just checking __file__ values (since we already calculate the latter, and regardless of how a file is imported, it needs to end up in sys.modules eventually). Besides, importing the same code under different names happens in several places in our own test suite (we use it to check that code behaviour doesn't change just because we import it differently), so we can hardly disable that behaviour. That said, I really don't think catching such a rare error is worth *any* runtime overhead. Just making "__main__" and the real module name refer to the same object in sys.modules is a different matter, but I'm not confident enough that I fully grasp the implications to do it without gathering feedback from a wider audience. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From fuzzyman at voidspace.org.uk Sat Jul 31 17:14:07 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sat, 31 Jul 2010 16:14:07 +0100 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: References: <20100730165945.GA25784@phd.pp.ru> <4C530C9A.3030204@voidspace.org.uk> Message-ID: <4C543DBF.5030506@voidspace.org.uk> On 31/07/2010 16:07, Nick Coghlan wrote: > On Sat, Jul 31, 2010 at 3:57 PM, Daniel Waterworth > wrote: > >> @Nick: I suppose the simplest way to detect re-importation in the >> general case, is to store a set of hashes of files that have been >> imported. When a user tries to import a file where it's hash is >> already in the set, a warning is generated. It's simpler than trying >> to figure out all the different ways that a file can be imported, and >> will also detect copied files. This is less infrastructure than you >> were suggesting, but it's not a perfect solution. >> > Hashing every file on import would definitely be more overhead than > just checking __file__ values (since we already calculate the latter, > and regardless of how a file is imported, it needs to end up in > sys.modules eventually). Besides, importing the same code under > different names happens in several places in our own test suite (we > use it to check that code behaviour doesn't change just because we > import it differently), so we can hardly disable that behaviour. > > That said, I really don't think catching such a rare error is worth > *any* runtime overhead. Just making "__main__" and the real module > name refer to the same object in sys.modules is a different matter, > but I'm not confident enough that I fully grasp the implications to do > it without gathering feedback from a wider audience. > > Some people workaround the potential for bugs caused by __main__ reimporting itself by doing it *deliberately*. Glyf even recommends it as good practise. ;-) http://glyf.livejournal.com/60326.html So - the fix you suggest would *break* this code. Raising a warning wouldn't... (and would eventually make this workaround unnecessary.) Michael > Cheers, > Nick. > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From alexander.belopolsky at gmail.com Sat Jul 31 17:23:27 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Sat, 31 Jul 2010 11:23:27 -0400 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: References: <20100730165945.GA25784@phd.pp.ru> <4C530C9A.3030204@voidspace.org.uk> Message-ID: On Sat, Jul 31, 2010 at 11:07 AM, Nick Coghlan wrote: .. > That said, I really don't think catching such a rare error is worth > *any* runtime overhead. Just making "__main__" and the real module > name refer to the same object in sys.modules is a different matter, > but I'm not confident enough that I fully grasp the implications to do > it without gathering feedback from a wider audience. If you make sys.module['__main__'] and sys.module['modname'] the same (let's call it mod), what will be the value of mod.__name__? From ncoghlan at gmail.com Sat Jul 31 17:30:32 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 1 Aug 2010 01:30:32 +1000 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: <4C543DBF.5030506@voidspace.org.uk> References: <20100730165945.GA25784@phd.pp.ru> <4C530C9A.3030204@voidspace.org.uk> <4C543DBF.5030506@voidspace.org.uk> Message-ID: On Sun, Aug 1, 2010 at 1:14 AM, Michael Foord wrote: > Some people workaround the potential for bugs caused by __main__ reimporting > itself by doing it *deliberately*. Glyf even recommends it as good practise. > ;-) > > http://glyf.livejournal.com/60326.html > > So - the fix you suggest would *break* this code. Raising a warning > wouldn't... (and would eventually make this workaround unnecessary.) With my change, that code would work just fine. "from myproject.gizmo import main" and "from __main__ import main" would just return the same object, whereas currently they return something different. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From fuzzyman at voidspace.org.uk Sat Jul 31 17:36:13 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sat, 31 Jul 2010 16:36:13 +0100 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: References: <20100730165945.GA25784@phd.pp.ru> <4C530C9A.3030204@voidspace.org.uk> <4C543DBF.5030506@voidspace.org.uk> Message-ID: <4C5442ED.9020504@voidspace.org.uk> On 31/07/2010 16:30, Nick Coghlan wrote: > On Sun, Aug 1, 2010 at 1:14 AM, Michael Foord wrote: > >> Some people workaround the potential for bugs caused by __main__ reimporting >> itself by doing it *deliberately*. Glyf even recommends it as good practise. >> ;-) >> >> http://glyf.livejournal.com/60326.html >> >> So - the fix you suggest would *break* this code. Raising a warning >> wouldn't... (and would eventually make this workaround unnecessary.) >> > With my change, that code would work just fine. "from myproject.gizmo > import main" and "from __main__ import main" would just return the > same object, whereas currently they return something different. > Have you looked at the code in that example? I don't think it would work... Michael > Cheers, > Nick. > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From ncoghlan at gmail.com Sat Jul 31 17:36:27 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 1 Aug 2010 01:36:27 +1000 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: References: <20100730165945.GA25784@phd.pp.ru> <4C530C9A.3030204@voidspace.org.uk> Message-ID: On Sun, Aug 1, 2010 at 1:23 AM, Alexander Belopolsky wrote: > On Sat, Jul 31, 2010 at 11:07 AM, Nick Coghlan wrote: > .. >> That said, I really don't think catching such a rare error is worth >> *any* runtime overhead. Just making "__main__" and the real module >> name refer to the same object in sys.modules is a different matter, >> but I'm not confident enough that I fully grasp the implications to do >> it without gathering feedback from a wider audience. > > If you make sys.module['__main__'] and sys.module['modname'] the same > (let's call it mod), what will be the value of mod.__name__? "__main__", so pickling would remain broken. unpickling would at least work correctly under this regime though. The only way to fix pickling is to avoid monkeying with __name__ at all (e.g. something along the lines of PEP 299, or a special "__is_main__" flag). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ncoghlan at gmail.com Sat Jul 31 17:50:02 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 1 Aug 2010 01:50:02 +1000 Subject: [Python-Dev] unexpected import behaviour In-Reply-To: <4C5442ED.9020504@voidspace.org.uk> References: <20100730165945.GA25784@phd.pp.ru> <4C530C9A.3030204@voidspace.org.uk> <4C543DBF.5030506@voidspace.org.uk> <4C5442ED.9020504@voidspace.org.uk> Message-ID: On Sun, Aug 1, 2010 at 1:36 AM, Michael Foord wrote: > On 31/07/2010 16:30, Nick Coghlan wrote: >> With my change, that code would work just fine. "from myproject.gizmo >> import main" and "from __main__ import main" would just return the >> same object, whereas currently they return something different. >> > > Have you looked at the code in that example? I don't think it would work... Ah, I see what you mean - yes, there would need to be some additional work done to detect the case of direct execution from within a package directory in order to set __main__.__package__ accordingly (as if the command line had been "python -m myproject.gizmo" rather than "python myproject/gizmo.py"). Even then, the naming problem would remain. Still, this kind of the thing is the reason I'm reluctant to arbitrarily change the existing semantics - as irritating as they can be at times (with pickling/unpickling problems being the worst of it, as pickling in particular depends on the value in __name__ being correct), people have all sorts of workarounds kicking around that need to be accounted for if we're going to make any changes. I kind of regret PEP 366 being accepted in the __package__ form now. At one point I considered proposing something like __module_name__ instead, but I didn't actually need that extra information to solve the relative import issue, and nobody else mentioned the pickling problem at the time. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From ziade.tarek at gmail.com Sat Jul 31 18:22:56 2010 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 31 Jul 2010 18:22:56 +0200 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <4C540CFD.9030308@voidspace.org.uk> References: <4C5206F6.7090808@voidspace.org.uk> <4C540CFD.9030308@voidspace.org.uk> Message-ID: On Sat, Jul 31, 2010 at 1:46 PM, Michael Foord wrote: ... > > Installation of plugins would still be done through the standard > distutils(2) machinery. (Using PEP 376 would depend on distutils2. I would > be fine with this.) Note that the PEP 376 implementation is mainly done in pkgutil. A custom version lives in distutils2 but when ready, will be pushed independently in pkgutil Regards Tarek From fuzzyman at voidspace.org.uk Sat Jul 31 18:35:07 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sat, 31 Jul 2010 17:35:07 +0100 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: References: <4C5206F6.7090808@voidspace.org.uk> <4C540CFD.9030308@voidspace.org.uk> Message-ID: <4C5450BB.7050703@voidspace.org.uk> On 31/07/2010 17:22, Tarek Ziad? wrote: > On Sat, Jul 31, 2010 at 1:46 PM, Michael Foord > wrote: > ... > >> Installation of plugins would still be done through the standard >> distutils(2) machinery. (Using PEP 376 would depend on distutils2. I would >> be fine with this.) >> > Note that the PEP 376 implementation is mainly done in pkgutil. A > custom version lives in distutils2 but > when ready, will be pushed independently in pkgutil > Ok. It would be helpful for unittest2 (the backport) if it was *still* available in distutils2 even after the merge into pkgutil (for use by earlier versions of Python). I guess you will do this anyway for the distutils2 backport itself anyway... (?) All the best, Michael Foord > Regards > Tarek > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. From barry at python.org Sat Jul 31 18:36:25 2010 From: barry at python.org (Barry Warsaw) Date: Sat, 31 Jul 2010 12:36:25 -0400 Subject: [Python-Dev] Is it intentional that "sys.__debug__ = 1" is illegal in Python 2.7? In-Reply-To: <201007310833.00914.steve@pearwood.info> References: <20100730162608.7315c3a5@heresy> <20100730165325.49414a21@heresy> <201007310833.00914.steve@pearwood.info> Message-ID: <20100731123625.3eeb018f@heresy> On Jul 31, 2010, at 08:32 AM, Steven D'Aprano wrote: >On Sat, 31 Jul 2010 07:44:42 am Guido van Rossum wrote: >> On Fri, Jul 30, 2010 at 1:53 PM, Barry Warsaw >wrote: >> > On Jul 30, 2010, at 01:42 PM, Guido van Rossum wrote: >> >>Well it is a reserved name so those packages that were setting it >> >>should have known that they were using undefined behavior that >> >> could change at any time. >> > >> > Shouldn't it be described here then? >> > >> > http://docs.python.org/reference/lexical_analysis.html#identifiers >> >> No, since it is covered here: >> >> http://docs.python.org/reference/lexical_analysis.html#reserved-class >>es-of-identifiers > > >I have a small concern about the wording of that, specifically this: > >"System-defined names. These names are defined by the interpreter and >its implementation (including the standard library); applications >SHOULD NOT EXPECT TO DEFINE additional names using this convention. >The set of names of this class defined by Python may be extended in >future versions." [emphasis added] > >This implies to me that at some time in the future, Python may make it >illegal to assign to any __*__ name apart from those in a list >of "approved" methods. Is that the intention? I have always understood >that if you create your own __*__ names, you risk clashing with a >special method, but otherwise it is allowed, if disapproved off. I >would not like to see it become forbidden. I'm with Steven on this one. I've always understood the rules on double-underscore names to mean that Python reserves the use of those names for its own purposes, and is free to break your code if you define your own. That's very different than saying it's forbidden to use double-underscore names for your own purposes or assign to them, which is I think what's going on with the sys.__debug__ example. If that's the rule, I'd want to make this section of the documentation much stronger about the prohibitions. I've just never considered Python's rule here to be that strong. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Sat Jul 31 18:41:22 2010 From: barry at python.org (Barry Warsaw) Date: Sat, 31 Jul 2010 12:41:22 -0400 Subject: [Python-Dev] Is it intentional that "sys.__debug__ = 1" isillegal in Python 2.7? In-Reply-To: References: <20100730162608.7315c3a5@heresy> <20100730165325.49414a21@heresy> Message-ID: <20100731124122.67890e85@heresy> On Jul 30, 2010, at 05:23 PM, Eric Snow wrote: >First appeared in docs for 2.6 (October 02, 2008). Not sure if that >is when it first because constrained this way. > >http://docs.python.org/library/constants.html?highlight=__debug__#__debug__ Thanks Eric, this is probably the right section of the docs to reference on the issue. I want to add two clarifications to this section: * Be more explicit that assigments to None and __debug__ are illegal even when used as attributes. IOW it's not just assignment to the built-in names that are illegal. * Add a "Changed in 2.7" to __debug__ stating that assignments to __debug__ as an attribute became illegal. From this though, I think it's clear that Benjamin's change was intentional. I will also add this to the NEWS and What's New files for 2.7. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at python.org Sat Jul 31 19:02:03 2010 From: barry at python.org (Barry Warsaw) Date: Sat, 31 Jul 2010 13:02:03 -0400 Subject: [Python-Dev] pdb mini-sprint report and questions In-Reply-To: References: Message-ID: <20100731130203.652b3970@heresy> On Jul 31, 2010, at 12:45 AM, Georg Brandl wrote: >to warm up for tomorrow's 3.2alpha1 release, I did a mini-sprint on >pdb issues today. I'm pleased to report that 14 issues could be >closed, and pdb got a range of small new features, such as commands on >the command line, "until " or "longlist" showing all the code >for the current function (the latter courtesy of Antonio Cuni's pdb++). I haven't played with pdb++ (I might have to do something about that) but it's awesome that you're giving pdb some love. >One issue that's not yet closed is #7245, which adds a (very nice IMO) >feature: when you press Ctrl-C while the program being debugged runs, >you will not get a traceback but execution is suspended, and you can >debug from the current point of execution -- just like in gdb. That *would* be nice. >Another question is about a feature of pdb++ that I personally would >like, but imagine would make others unhappy: one-letter abbreviations >of commands such as c(ontinue) or l(ist) are also often-used variable >names, so they are frequently typed without the required "!" or "print" >that would distinguish them from the command, and the command is >actually executed. The feature in question would default to printing >the variable in cases where one exists -- handy enough or too >inconsistent? Not that important to me... >Also, are there any other features you would like to see? One feature >of pdb++ that is general enough and has no dependencies would be watch >expressions... ...but watch expressions - and the equivalent of gdb's 'display' command - would be very cool. `interact` would also be useful and probably pretty easy to add. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From merwok at netwok.org Sat Jul 31 19:24:44 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sat, 31 Jul 2010 19:24:44 +0200 Subject: [Python-Dev] proto-pep: plugin proposal (for unittest) In-Reply-To: <4C5450BB.7050703@voidspace.org.uk> References: <4C5206F6.7090808@voidspace.org.uk> <4C540CFD.9030308@voidspace.org.uk> <4C5450BB.7050703@voidspace.org.uk> Message-ID: <4C545C5C.6010005@netwok.org> >> Note that the PEP 376 implementation is mainly done in pkgutil. A >> custom version lives in distutils2 but >> when ready, will be pushed independently in pkgutil > > Ok. It would be helpful for unittest2 (the backport) if it was *still* > available in distutils2 even after the merge into pkgutil (for use by > earlier versions of Python). I guess you will do this anyway for the > distutils2 backport itself anyway... (?) Yes. Even if the goal is to have distutils2 in the stdlib for 3.2 or 3.3, there will still be a standalone release on PyPI for Python 2.4-3.1. You?ll just have to write such compat code: try: from pkgutil import shiny_new_function except ImportError: from distutils2._backport.pkgutil import shiny_new_function Regards From guido at python.org Sat Jul 31 23:02:38 2010 From: guido at python.org (Guido van Rossum) Date: Sat, 31 Jul 2010 14:02:38 -0700 Subject: [Python-Dev] Is it intentional that "sys.__debug__ = 1" isillegal in Python 2.7? In-Reply-To: <20100731124122.67890e85@heresy> References: <20100730162608.7315c3a5@heresy> <20100730165325.49414a21@heresy> <20100731124122.67890e85@heresy> Message-ID: >>On Sat, 31 Jul 2010 07:44:42 am Guido van Rossum wrote: >>> http://docs.python.org/reference/lexical_analysis.html#reserved-classes-of-identifiers > On Jul 31, 2010, at 08:32 AM, Steven D'Aprano wrote: >>I have a small concern about the wording of that, specifically this: >> >>"System-defined names. These names are defined by the interpreter and >>its implementation (including the standard library); applications >>SHOULD NOT EXPECT TO DEFINE additional names using this convention. >>The set of names of this class defined by Python may be extended in >>future versions." [emphasis added] >> >>This implies to me that at some time in the future, Python may make it >>illegal to assign to any __*__ name apart from those in a list >>of "approved" methods. Is that the intention? I have always understood >>that if you create your own __*__ names, you risk clashing with a >>special method, but otherwise it is allowed, if disapproved off. I >>would not like to see it become forbidden. The key phrase is "system-defined names". Since this is in the section on lexical analysis, it does not limit the contexts in which such names are reserved for the system; they are potentially special *everywhere* (as variables, builtins, classes, function, methods, attributes, any other use of names in the language). The phrase "define additional names" should not be intended to imply that using __*__ names that already have a defined meaning (like __debug__) in new contexts is fair game -- to the contrary, I would think that since __debug__ is a system-defined name (and one with pretty deep implications) doing things not explicitly allowed, like setting sys.__debug__, is really like playing with fire. On Sat, Jul 31, 2010 at 9:36 AM, Barry Warsaw wrote: > I'm with Steven on this one. I've always understood the rules on > double-underscore names to mean that Python reserves the use of those names > for its own purposes, and is free to break your code if you define your own. Or if you use the ones reserved by Python in undocumented ways. > That's very different than saying it's forbidden to use double-underscore > names for your own purposes or assign to them, which is I think what's going > on with the sys.__debug__ example. A blanket prohibition of assigning to or defining any __*__ names in any context (besides the documented ones in documented contexts) would clearly break a lot of code, but I don't think implementations are required or expected to avoid occasional such breakages at all cost. The occasional introduction of new __*__ names with new special meanings is clearly allowed, and if the language were to introduce a bunch of new keywords of this form (keywords meaning that they become syntactically illegal everywhere except where the syntax explicitly allows them) that would be totally within the rules. > If that's the rule, I'd want to make this section of the documentation much > stronger about the prohibitions. I've just never considered Python's rule > here to be that strong. I have. I have also occasionally ignored this rule, but I've always felt that I was taking a calculated risk and would not have a leg to stand on if my code would be broken. On Sat, Jul 31, 2010 at 9:41 AM, Barry Warsaw wrote: > On Jul 30, 2010, at 05:23 PM, Eric Snow wrote: > >>First appeared in docs for 2.6 (October 02, 2008). ?Not sure if that >>is when it first because constrained this way. >> >>http://docs.python.org/library/constants.html?highlight=__debug__#__debug__ > > Thanks Eric, this is probably the right section of the docs to reference on > the issue. ?I want to add two clarifications to this section: > > * Be more explicit that assigments to None and __debug__ are illegal even when > ?used as attributes. ?IOW it's not just assignment to the built-in names that > ?are illegal. Well None is a reserved word in Py3k (as are True and False). But yes, the docs should clarify that *any* use of __*__ names, in *any* context, that does not follow explicitly documented use, is subject to breakage without warning. > * Add a "Changed in 2.7" to __debug__ stating that assignments to __debug__ as > ?an attribute became illegal. > > From this though, I think it's clear that Benjamin's change was intentional. > I will also add this to the NEWS and What's New files for 2.7. Thanks! -- --Guido van Rossum (python.org/~guido)