From martin at v.loewis.de Sat Mar 1 00:03:27 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 01 Mar 2008 00:03:27 +0100 Subject: [Python-Dev] Fixing buildbot/external(-amd64).bat files on Windows In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E1657C617@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E1657C60B@EXMBX04.exchhosting.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E1657C617@EXMBX04.exchhosting.com> Message-ID: <47C88F3F.6010806@v.loewis.de> >>> I'm going through the motions of getting my newly added build >>> slave >> in a half decent state. >> >> I think the buildbot should have a name different from 'x86 XP'. >> (Martin, Neal?) >> > Yeah, I've dropped Martin a note regarding this. The community bots > refer to Windows Server 2003 boxes as just that, so perhaps a rename > to 'x86 Windows Server 2008' is appropriate. FWIW as it's a 64-bit > box, I'm hoping to get a slave set up for 'x64 Windows Server 2008' > as well. I (now) see what you mean. I renamed it to "x86 W2k8". > (As far as I can see, the x64/x86 nature of the slave is dictated by > the master, correct? i.e. I can't tweak/clone this myself?) Right. I can set up another slave, but would prefer to do so only after you got the first one running. Regards, Martin From martin at v.loewis.de Sat Mar 1 00:30:07 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 01 Mar 2008 00:30:07 +0100 Subject: [Python-Dev] Fixing buildbot/external(-amd64).bat files on Windows In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E1657C60B@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E1657C60B@EXMBX04.exchhosting.com> Message-ID: <47C8957F.4030803@v.loewis.de> > I'm going through the motions of getting my newly added build slave > in a half decent state. The external.bat and external-amd64.bat > files needed the following in order to build db-4.4.20: I've fixed that in a different way. devenv.exe should not be used, as it is unavailable in VS Express. Regards, Martin From martin at v.loewis.de Sat Mar 1 00:31:54 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 01 Mar 2008 00:31:54 +0100 Subject: [Python-Dev] Fixing buildbot/external(-amd64).bat files on Windows In-Reply-To: References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E1657C60B@EXMBX04.exchhosting.com> Message-ID: <47C895EA.3040903@v.loewis.de> >> - vcbuild db-4.4.20\build_win32\Berkeley_DB.sln /build Debug /project db_static >> + devenv /upgrade db-4.4.20\build_win32\Berkeley_DB.sln >> + devenv db-4.4.20\build_win32\Berkeley_DB.sln /build Debug /project db_static > > The upgrade is requires only once. It probably belongs next to the > checkout or svn up and not in the build section. I came up with yet another solution: I changed the repository to contain already the converted files (or, rather, made a copy, as the 2.5 branch is still using the 7.1 project files):. Regards, Martin From martin at v.loewis.de Sat Mar 1 00:34:08 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 01 Mar 2008 00:34:08 +0100 Subject: [Python-Dev] Fixing buildbot/external(-amd64).bat files on Windows In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDD5@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E1657C60B@EXMBX04.exchhosting.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E1657C67C@EXMBX04.exchhosting.com> , <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDD5@EXMBX04.exchhosting.com> Message-ID: <47C89670.7010209@v.loewis.de> > I don't know how the existing vcbuild line ever worked, given the following output from vcbuild /?: > > Usage: vcbuild [options] [project|solution] [config|$ALL] It didn't. All the existing slaves already had a checkout when we switched to VS 2008, so a fresh checkout was never tested. That might also explain the buildbot crashes - they still were using object files from VS 2003, for db-4.4.20. Regards, Martin From barry at python.org Sat Mar 1 05:38:32 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Feb 2008 23:38:32 -0500 Subject: [Python-Dev] No releases tonight Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I tried, I really did. Python 2.6 is nearly ready, I'm mostly trying to figure out how to build the web pages properly. I haven't started on 3.0, but huge thanks go to Brett Cannon, Neal Norwitz, Mark Dickinson, and Fred Drake for helping out tonight with everything, including fixing the outstanding bugs in 3.0. Python 2.6a1 is tagged and built, so feel free to commit changes to the trunk. Python 3.0 is not tagged yet so if you can hold off from committing to that branch for a little while longer, that would be appreciated. I will cut 3.0a3 tomorrow (Saturday) as early as possible. time-to-start-drinking-ly y'rs, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8jdyXEjvBPtnXfVAQJSowQAgIqqYKYPvzEMtvtAHiNC+OkMpO3vxelh 9kcAgXClhCS47wAMc9viJsb5Uo12f7TlOURkEjuMZ13jrBri+HCFtrAGjjHj/qHk KyH9ua+q0dCtpg0P0CzvxznGr7k2XV5LiZit4G+UwuSokJr/F/dUDQV3jkSgWEvh xTRj8ZZisQw= =Hg6S -----END PGP SIGNATURE----- From steve at holdenweb.com Sat Mar 1 06:36:58 2008 From: steve at holdenweb.com (Steve Holden) Date: Sat, 01 Mar 2008 00:36:58 -0500 Subject: [Python-Dev] No releases tonight In-Reply-To: References: Message-ID: Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I tried, I really did. > > Python 2.6 is nearly ready, I'm mostly trying to figure out how to > build the web pages properly. I haven't started on 3.0, but huge > thanks go to Brett Cannon, Neal Norwitz, Mark Dickinson, and Fred > Drake for helping out tonight with everything, including fixing the > outstanding bugs in 3.0. > > Python 2.6a1 is tagged and built, so feel free to commit changes to > the trunk. > > Python 3.0 is not tagged yet so if you can hold off from committing to > that branch for a little while longer, that would be appreciated. I > will cut 3.0a3 tomorrow (Saturday) as early as possible. > > time-to-start-drinking-ly y'rs, Barry: If you can document the web stuff you have to do I will formalize it as a procedure for use in future releases. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From barry at python.org Sat Mar 1 15:11:34 2008 From: barry at python.org (Barry Warsaw) Date: Sat, 1 Mar 2008 09:11:34 -0500 Subject: [Python-Dev] No releases tonight In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 1, 2008, at 12:36 AM, Steve Holden wrote: > If you can document the web stuff you have to do I will formalize it > as > a procedure for use in future releases. Hi Steve, In this case, there was a lot more work to do because 2.6 wasn't tied in at all. Add to the fact that I didn't have any experience with the website infrastructure made things a bit more difficult the first time out. I still don't quite have the 2.6 links working correctly in my local fs. So the biggest problem is really: what steps do you take when you need to expose a new major release on the website? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8lkF3EjvBPtnXfVAQLjtQP/cyIu6i/3hL7RMyJjwi5UWaVelq+6+DjX c8P6aLT0Gq2jIIacwt2P5lWYGx5D2nahoLCkLoA4M3avHM6UiTaIW45nFrnffItx F/ib/UY/j+gsudlg5GsZQrBTzvrso6BFDuIr9VISuzf3e6QRr7sAhUfXzgIETXjj DPT3y474bDs= =Kxv7 -----END PGP SIGNATURE----- From barry at python.org Sat Mar 1 17:30:58 2008 From: barry at python.org (Barry Warsaw) Date: Sat, 1 Mar 2008 11:30:58 -0500 Subject: [Python-Dev] [Python-3000] No releases tonight In-Reply-To: References: Message-ID: <506EC4C0-11C1-4032-BB18-16C65E1EEC5B@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 1, 2008, at 10:38 AM, Steve Holden wrote: > Barry Warsaw wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Mar 1, 2008, at 12:36 AM, Steve Holden wrote: >> >>> If you can document the web stuff you have to do I will formalize it >>> as >>> a procedure for use in future releases. >> >> Hi Steve, >> >> In this case, there was a lot more work to do because 2.6 wasn't tied >> in at all. Add to the fact that I didn't have any experience with >> the >> website infrastructure made things a bit more difficult the first >> time >> out. I still don't quite have the 2.6 links working correctly in my >> local fs. So the biggest problem is really: what steps do you take >> when you need to expose a new major release on the website? >> > I guess the thing to do is to look at your diffs once you have > committed > the changes - I presume this'll all be dropped in as a single > revision? That would be great Steve, thanks! r11294. I'm going to build Python 3.0 now. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8mEw3EjvBPtnXfVAQKm9gP+PdoDvqTmA4kuqVrNmQJdHSsQDVcB7/sV BRxeYH0S1TNo05NCzv6qyJ6nxe2CVI6he+chhoCbtCRqp6c5LQOXtaSHUqoSNzP9 FSw9YnQ3DHmFRX3BfNyZ7d9FS2Fs5irsLuAc+WUFNR0AWsvbwXR6qlT0qNQGP666 V46DiUdNhUA= =ybmk -----END PGP SIGNATURE----- From barry at python.org Sat Mar 1 19:51:38 2008 From: barry at python.org (Barry Warsaw) Date: Sat, 1 Mar 2008 13:51:38 -0500 Subject: [Python-Dev] RELEASED Python 2.6a1 and 3.0a3 Message-ID: <6E72CEB8-D3BF-4440-A1EA-1A3D545CC8DB@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On behalf of the Python development team and the Python community, I'm happy to announce the first alpha release of Python 2.6, and the third alpha release of Python 3.0. Python 2.6 is not only the next advancement in the Python 2 series, it is also a transitionary release, helping developers begin to prepare their code for Python 3.0. As such, many features are being backported from Python 3.0 to 2.6. It makes sense to release both versions in at the same time, the precedence for this having been set with the Python 1.6 and 2.0 releases. During the alpha testing cycle we will be releasing both versions in lockstep, on a monthly release cycle. The releases will happen on the last Friday of every month. If this schedule works well, we will continue releasing in lockstep during the beta program. See PEP 361 for schedule details: http://www.python.org/dev/peps/pep-0361/ Please note that these are alpha releases, and as such are not suitable for production environments. We continue to strive for a high degree of quality, but there are still some known problems and the feature sets have not been finalized. These alphas are being released to solicit feedback and hopefully discover bugs, as well as allowing you to determine how changes in 2.6 and 3.0 might impact you. If you find things broken or incorrect, please submit a bug report at http://bugs.python.org For more information and downloadable distributions, see the Python 2.6 web site: http://www.python.org/download/releases/2.6/ and the Python 3.0 web site: http://www.python.org/download/releases/3.0/ We are planning a number of additional alpha releases, with the final release schedule still to be determined. Enjoy, - -Barry Barry Warsaw barry at python.org Python 2.6/3.0 Release Manager (on behalf of the entire python-dev team) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8mlu3EjvBPtnXfVAQKePAQAgx6w9wztfJaSWkbKrbwur2U6t6o5aIY5 pyMa00CZWY06p8099BztcSjgp5rKrd6/9V7cJ0NP7NLZ+tz20uRfyI8uqoIYBIWC ibJay6SSnzgOQM3PRIJV/K/m0dVPPPVD1LDnoEvuu+cKUpV434yHdgWkMPswsxUd fLydrXABlOM= =l6aj -----END PGP SIGNATURE----- From lists at cheimes.de Sat Mar 1 19:56:28 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 01 Mar 2008 19:56:28 +0100 Subject: [Python-Dev] No releases tonight In-Reply-To: References: Message-ID: <47C9A6DC.9070708@cheimes.de> Barry Warsaw wrote: > In this case, there was a lot more work to do because 2.6 wasn't tied > in at all. Add to the fact that I didn't have any experience with the > website infrastructure made things a bit more difficult the first time > out. I still don't quite have the 2.6 links working correctly in my > local fs. So the biggest problem is really: what steps do you take > when you need to expose a new major release on the website? Starting with the first betas of 2.6 and 3.0 we should also work on official texts for the press. Other projects like PHP are drawing lots of attention with their releases, even with bug fix and security releases. Bad news are better than no news - a beta release is *good* news. When 3.0a2 was released I contacted two larger German IT news sites. Non of them even bother to reply. :/ I propose that we provide two official texts for the press. A shorter text which explains Python and the most important changes since the last version in a few paragraphs and a longer, more detailed text like Martin's text for the 2.5.2 release. I also propose translations of the shorter text to important languages like French, German, Japanese, Portuguese and Spanish. I'm willing to help with the German translation. Christian From barry at python.org Sat Mar 1 20:07:49 2008 From: barry at python.org (Barry Warsaw) Date: Sat, 1 Mar 2008 14:07:49 -0500 Subject: [Python-Dev] The release process Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I just announced the 2.6a1 and 3.0a3 releases, and am thawing both branches. Just some quick feedback in case anybody is interested. First, huge thanks go to Brett Cannon, Neal Norwitz, Mark Dickinson and Fred Drake for their help last night. Apologies also to them for my drunken rants on jabber :). Also thanks to Martin von Loewis for the Windows msi's for Python 2.6. I'm sure Martin will soon provide msi's for 3.0, but these are not yet available. Some other random notes: Brett fixed test_profile in 3.0 last night but test_cProfile was still broken. I disabled the test via a TestSkipped and set this to expected in regrtest.py. This test should be fixed and the expected skip removed. I will definitely need help keeping the various NEWS files up to date. I don't see any way that I'll be able to spend time on these when I'm cutting a release. Python 2.6 NEWS was simply impossible to proofread because of its sheer size and the fact that it was the first alpha of the series. PEP 101 describes 4 news files: Misc/NEWS and Lib/idlelib/NEWS.txt for both 2.6 and 3.0. I am urgently requesting that when people commit newsworthy items to the Python releases that they keep the NEWS files up-to-date. This is especially tricky for code merged between the two versions. Thanks to Neal for looking over 3.0's NEWS file last night. As RM, I am going to operate on the assumption that the NEWS files are up-to-date. I'd be thrilled if someone volunteered to be the "NEWS czar" -- we all know when the next alpha release is coming (Friday March 25), so this czar would be responsible for watching commits and making sure that NEWS was updated as appropriate, or harassing the committer into updating NEWS to describe their new feature. If you'd like to be this NEWS czar, please let me know. With apologies to Anthony, welease is crack. I made pretty good progress once I ditched it and starting doing things manually. Between now and the next alpha I intend to work on a command line script to help with releases. If you're interested in helping, let me know. PEP 101 is sorely out of date, especially with regards to updating web content and the Python documentation. I think I now know how to update the python.org web site, but the new Python documentation format is still a mystery to me. If someone would like to help update PEP 101 for the documentation steps, please let me know. PEP 101 also describes some steps for updating the distutils version numbers. These instructions seemed stale too. If you know anything about distutils version numbers and the process for updating them, please contact me. There's no Misc/RPM/python-3.0.spec file so I skipped that step too. Sean, do you know anything about that? That's it. See you again next time :). Let me know if you notice anything broken about the releases. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8mphXEjvBPtnXfVAQKPcwQAqQVP+IWO60/m1Rm1OKpcGfpS+BZILKvj LkLJamZ6gvupFeJj1kCr6eAl62Mqaec2Z29jsnXK9TfAogGGVcW21a98rgcQUong fRh34dt1YGVMcqw4r8G60kqYQG4caGJ9tS5oKEXq+lYWPfirLZ7mC1SkkfnJ9mVd Cscr0ZAYayI= =nnlY -----END PGP SIGNATURE----- From barry at python.org Sat Mar 1 20:11:18 2008 From: barry at python.org (Barry Warsaw) Date: Sat, 1 Mar 2008 14:11:18 -0500 Subject: [Python-Dev] No releases tonight In-Reply-To: <47C9A6DC.9070708@cheimes.de> References: <47C9A6DC.9070708@cheimes.de> Message-ID: <3B6D7A5F-2426-485E-BF5F-0BC16DF26A74@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 1, 2008, at 1:56 PM, Christian Heimes wrote: > Barry Warsaw wrote: >> In this case, there was a lot more work to do because 2.6 wasn't tied >> in at all. Add to the fact that I didn't have any experience with >> the >> website infrastructure made things a bit more difficult the first >> time >> out. I still don't quite have the 2.6 links working correctly in my >> local fs. So the biggest problem is really: what steps do you take >> when you need to expose a new major release on the website? > > Starting with the first betas of 2.6 and 3.0 we should also work on > official texts for the press. Other projects like PHP are drawing lots > of attention with their releases, even with bug fix and security > releases. Bad news are better than no news - a beta release is > *good* news. Great idea, and I agree. I won't be the person spearheading this though, but since it'll probably be me making the announcement, I'd like to coordinate with this effort. > When 3.0a2 was released I contacted two larger German IT news sites. > Non > of them even bother to reply. :/ > > I propose that we provide two official texts for the press. A shorter > text which explains Python and the most important changes since the > last > version in a few paragraphs and a longer, more detailed text like > Martin's text for the 2.5.2 release. > > I also propose translations of the shorter text to important languages > like French, German, Japanese, Portuguese and Spanish. I'm willing to > help with the German translation. Cool, thanks. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8mqVnEjvBPtnXfVAQIlkAP/U5z0xSaaDKbXxArncL49NpRTU5O431Wi +qdlnJQbApMtWMJyw14jXD0ovDlAFFK/71fGSUW7IxBvd+sWy9xJJpwydNz5xUVJ Ze3GYX0pWF0zDp9IIX9o3W9uotm9156lWe8Ahbsa0TWXN2AXyuRjyccIS7v2mU55 mHY6niZ8SbE= =OJl/ -----END PGP SIGNATURE----- From lists at cheimes.de Sat Mar 1 20:43:24 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 01 Mar 2008 20:43:24 +0100 Subject: [Python-Dev] No releases tonight In-Reply-To: References: <47C9A6DC.9070708@cheimes.de> Message-ID: <47C9B1DC.5070201@cheimes.de> Steve Holden wrote: > PyCon is using a PR team to help with publicity. Maybe we can ask them > for assistance on how to get the word out? That's a *very* good idea! Let's ask some professionals rather than writing something on our own. Christian From stefan_ml at behnel.de Sat Mar 1 20:37:25 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 01 Mar 2008 20:37:25 +0100 Subject: [Python-Dev] C-API status of Python 3? Message-ID: Hi all, I would like to know how stable the C-API of Python 3 is, or what the expected release level (beta?) would be at which I can expect it to stabilise. What is the plan here? The background is Cython, which will need to support Python 3 one day or another, so I wanted to know from which point on it will make sense to start thinking about a migration plan. Thanks, Stefan From lists at cheimes.de Sat Mar 1 21:04:27 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 01 Mar 2008 21:04:27 +0100 Subject: [Python-Dev] The release process In-Reply-To: References: Message-ID: <47C9B6CB.1060909@cheimes.de> Barry Warsaw wrote: > I will definitely need help keeping the various NEWS files up to > date. I don't see any way that I'll be able to spend time on these > when I'm cutting a release. Python 2.6 NEWS was simply impossible to > proofread because of its sheer size and the fact that it was the first > alpha of the series. > > PEP 101 describes 4 news files: Misc/NEWS and Lib/idlelib/NEWS.txt for > both 2.6 and 3.0. I am urgently requesting that when people commit > newsworthy items to the Python releases that they keep the NEWS files > up-to-date. This is especially tricky for code merged between the two > versions. Thanks to Neal for looking over 3.0's NEWS file last > night. As RM, I am going to operate on the assumption that the NEWS > files are up-to-date. I'd be thrilled if someone volunteered to be > the "NEWS czar" -- we all know when the next alpha release is coming > (Friday March 25), so this czar would be responsible for watching > commits and making sure that NEWS was updated as appropriate, or > harassing the committer into updating NEWS to describe their new > feature. If you'd like to be this NEWS czar, please let me know. I *never* sync changes from trunk Misc/NEWS to py3k Misc/NEWS. From my point of view it doesn't make sense to put Python 2.6 changes in the same section as Python 3.0 changes. Moving changes from to the right section would put a large and unnecessary burden on me. In general every change of the 2.6 source tree makes it into 3.0. Exceptions to the rule is stuff that makes no sense like 3.0 compatibility and warnings. Thumb rule: Changes, bug fixes and new features in 2.6 are also in 3.0 except they are outruled by a Python 3.0 feature. Several people including me and Guido himself are watching the cvs lists. We make sure everybody adds an entry to Misc/NEWS whenever a bug is fixed or a new feature is added. Otherwise we crack the whip ^H^H^H contact the committer. You can be sure that at least 98% of all closed bug reports, feature request and important changes have an entry in Misc/NEWS. So in general Misc/NEWS isn't an issue but Docs/whatsnew/ is. Only a couple of people - mostly Georg and Andrew - are updating the files. Christian From lists at cheimes.de Sat Mar 1 21:14:03 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 01 Mar 2008 21:14:03 +0100 Subject: [Python-Dev] C-API status of Python 3? In-Reply-To: References: Message-ID: Stefan Behnel wrote: > I would like to know how stable the C-API of Python 3 is, or what the expected > release level (beta?) would be at which I can expect it to stabilise. What is > the plan here? > > The background is Cython, which will need to support Python 3 one day or > another, so I wanted to know from which point on it will make sense to start > thinking about a migration plan. The 3.0 API isn't stable yet. I plan to rename some of the functions before the first beta is released. Currently the naming schema is too confusing: PyUnicode - str PyString - bytes PyBytes - bytearray See? :) The documentation for the PyString functions is outdated and IIRC the PyBytes docs are non existing. Christian From martin at v.loewis.de Sat Mar 1 23:26:10 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 01 Mar 2008 23:26:10 +0100 Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6a1 and 3.0a3 In-Reply-To: References: <6E72CEB8-D3BF-4440-A1EA-1A3D545CC8DB@python.org> Message-ID: <47C9D802.2090205@v.loewis.de> > As of 4:50 PM EST, the links to Windows installers give 404 File Not > Found. > > I gather that they are still in process, > and notice that there is no public c.l.p. announcement. I just fixed that. The files were there; just the links were wrong. Regards, Martin From barry at python.org Sat Mar 1 23:29:39 2008 From: barry at python.org (Barry Warsaw) Date: Sat, 1 Mar 2008 17:29:39 -0500 Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6a1 and 3.0a3 In-Reply-To: <47C9D802.2090205@v.loewis.de> References: <6E72CEB8-D3BF-4440-A1EA-1A3D545CC8DB@python.org> <47C9D802.2090205@v.loewis.de> Message-ID: <05D00F06-8B87-4B94-8878-42E1F3F50F90@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 1, 2008, at 5:26 PM, Martin v. L?wis wrote: >> As of 4:50 PM EST, the links to Windows installers give 404 File Not >> Found. >> >> I gather that they are still in process, >> and notice that there is no public c.l.p. announcement. > > I just fixed that. The files were there; just the links were wrong. Thanks for fixing these Martin! - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8nY1HEjvBPtnXfVAQJ3YgP/TYr0X5vRqvVDEMgsHxHuiSuYZCIr8y36 ibAh3RAGeLLK7C7NiOyAfxkesf91HCbL1in0TcnD06QZy52O8elBa927JOomP3mc Y6K4Y49JhxonBrmGcmasnc9PFjbhXtGdWLREinuzpB5itLpRv+SevMhxP27Fp8qr df173TV4hpk= =nf32 -----END PGP SIGNATURE----- From martin at v.loewis.de Sat Mar 1 23:37:05 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 01 Mar 2008 23:37:05 +0100 Subject: [Python-Dev] [Python-3000] The release process In-Reply-To: References: Message-ID: <47C9DA91.9000105@v.loewis.de> > With apologies to Anthony, welease is crack. I made pretty good > progress once I ditched it and starting doing things manually. > Between now and the next alpha I intend to work on a command line > script to help with releases. If you're interested in helping, let me > know. I guess every release manager is free to come up with his own set of tools, but I feel you've given up too quickly (or started too late - perhaps a test release run a few days before the release would have helped). In any case, I'm willing to help with welease, but not with yet another release tool. If you primarily complain about the GUIness of welease, I could help with a command line version of it. Regards, Martin From p.f.moore at gmail.com Sat Mar 1 23:39:41 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 1 Mar 2008 22:39:41 +0000 Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6a1 and 3.0a3 In-Reply-To: <47C9D802.2090205@v.loewis.de> References: <6E72CEB8-D3BF-4440-A1EA-1A3D545CC8DB@python.org> <47C9D802.2090205@v.loewis.de> Message-ID: <79990c6b0803011439n583bc2b2w5c544b620a91fa61@mail.gmail.com> On 01/03/2008, "Martin v. L?wis" wrote: > > As of 4:50 PM EST, the links to Windows installers give 404 File Not > > Found. > > > > I gather that they are still in process, > > and notice that there is no public c.l.p. announcement. > > > I just fixed that. The files were there; just the links were wrong. The 2.6a1 x86 MSI is there, but the 3.0a3 x86 MSI is still giving a 404. Paul. From barry at python.org Sat Mar 1 23:46:31 2008 From: barry at python.org (Barry Warsaw) Date: Sat, 1 Mar 2008 17:46:31 -0500 Subject: [Python-Dev] The release process In-Reply-To: <47C9B6CB.1060909@cheimes.de> References: <47C9B6CB.1060909@cheimes.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 1, 2008, at 3:04 PM, Christian Heimes wrote: > I *never* sync changes from trunk Misc/NEWS to py3k Misc/NEWS. From my > point of view it doesn't make sense to put Python 2.6 changes in the > same section as Python 3.0 changes. Moving changes from to the right > section would put a large and unnecessary burden on me. In general > every > change of the 2.6 source tree makes it into 3.0. Exceptions to the > rule > is stuff that makes no sense like 3.0 compatibility and warnings. > > Thumb rule: Changes, bug fixes and new features in 2.6 are also in 3.0 > except they are outruled by a Python 3.0 feature. > > Several people including me and Guido himself are watching the cvs > lists. We make sure everybody adds an entry to Misc/NEWS whenever a > bug > is fixed or a new feature is added. Otherwise we crack the whip ^H^H^H > contact the committer. You can be sure that at least 98% of all > closed > bug reports, feature request and important changes have an entry in > Misc/NEWS. Great, this is all I'm really asking for! The point of my unconscionable rant :) was that I think it's unfeasible to update the NEWS files at release time. Knowing that you, Guido and others are keeping an eye on commits and an iron hand on the NEWS files makes me as the RM rest comfortably. :) > So in general Misc/NEWS isn't an issue but Docs/whatsnew/ is. Only a > couple of people - mostly Georg and Andrew - are updating the files. I think it's okay if these lag behind during the alphas, but it would be good to start whipping these in shape by the time we start releasing betas. Thanks, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8ncx3EjvBPtnXfVAQJeNQP+L2wK4CmGfwzgsQUHQISniaJ2rREBhJua sJwqGNpBhnf6Uc8jJz+JRiexPdCW4AlH34FtHkyRkw2ZIVWwd6sO+7ixQPk0A/TP +gGk1uST/sjPG3q8T5u9OElri5SoTqJzEgWMkTiGhwYouSvOjpW/GFFREySU68Tk h9XGzJFZex0= =aXqC -----END PGP SIGNATURE----- From martin at v.loewis.de Sat Mar 1 23:49:42 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 01 Mar 2008 23:49:42 +0100 Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6a1 and 3.0a3 In-Reply-To: <05D00F06-8B87-4B94-8878-42E1F3F50F90@python.org> References: <6E72CEB8-D3BF-4440-A1EA-1A3D545CC8DB@python.org> <47C9D802.2090205@v.loewis.de> <05D00F06-8B87-4B94-8878-42E1F3F50F90@python.org> Message-ID: <47C9DD86.4040300@v.loewis.de> > Thanks for fixing these Martin! I have now also uploaded signed MSI files for 3.0a3. I have not tested them on a machine which doesn't have the VS 2008 CRT installed (as all the machines I can access right now do have it); please report what works and what doesn't. Regards, Martin From martin at v.loewis.de Sat Mar 1 23:51:36 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 01 Mar 2008 23:51:36 +0100 Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6a1 and 3.0a3 In-Reply-To: <79990c6b0803011439n583bc2b2w5c544b620a91fa61@mail.gmail.com> References: <6E72CEB8-D3BF-4440-A1EA-1A3D545CC8DB@python.org> <47C9D802.2090205@v.loewis.de> <79990c6b0803011439n583bc2b2w5c544b620a91fa61@mail.gmail.com> Message-ID: <47C9DDF8.30000@v.loewis.de> > The 2.6a1 x86 MSI is there, but the 3.0a3 x86 MSI is still giving a 404. Please try again - *those* files weren't actually there when I sent my last message; I just built them. Regards, Martin From barry at python.org Sun Mar 2 00:03:35 2008 From: barry at python.org (Barry Warsaw) Date: Sat, 1 Mar 2008 18:03:35 -0500 Subject: [Python-Dev] [Python-3000] The release process In-Reply-To: <47C9DA91.9000105@v.loewis.de> References: <47C9DA91.9000105@v.loewis.de> Message-ID: <10EE358F-8120-4EAC-9413-BDE5B812639F@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 1, 2008, at 5:37 PM, Martin v. L?wis wrote: >> With apologies to Anthony, welease is crack. I made pretty good >> progress once I ditched it and starting doing things manually. >> Between now and the next alpha I intend to work on a command line >> script to help with releases. If you're interested in helping, let >> me know. > > I guess every release manager is free to come up with his own set of > tools, but I feel you've given up too quickly (or started too late - > perhaps a test release run a few days before the release would have > helped). Maybe. > In any case, I'm willing to help with welease, but not with yet > another > release tool. If you primarily complain about the GUIness of welease, > I could help with a command line version of it. The dependency on gtk is unnecessary and means it can effectively only be run on Linux. Specifically it means I can't do releases on OS X. I don't see much benefit in having a gui tool for doing releases. Some of the problems I had include having to run glade3 in order to update the menus which allow you to choose which release you were going to make. It only new about py24 and 25 maintenance. No knock on Anthony there, since those are the releases he had to make, but I shouldn't have to edit the interface description files to add new release versions. Also, the scheme to compare IDLE versions seemed way off. Maybe that's another thing that makes sense for py24 and py25 but it definitely didn't work for py30. The much more serious problem, and where I stopped, is that welease broke on code containing the with statement. I don't remember the details because at that point I was pretty tired and hadn't made any progress on the releases, but I /think/ the problem is that welease runs on a different version of Python than it's checking so it can't handle syntactic differences and such. The kind of tool I think we need is a fairly straightforward command line tool, but one that would not just check that things are done, but also do them. E.g. the tool would keep track of all the little places where version numbers and copyright years need updating. The tool would actually make those changes, and using $EDITOR would show them to you for confirmation. It would pause at steps that require coordination, such as when things need to be committed or signed, or when you're waiting from input from others. It would have a dry-run mode and it would fairly closely follow PEP 101. Anyway, that's the kind of tool I plan on building (or perhaps with help from others -- hi Benjamin) for the next alpha round. Cheers, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8ngyHEjvBPtnXfVAQKYaAP+JzIj8HiwVYIJLMyxh+Glq57YQozOh2bB XILPBwUyppBBkezcT6IWAnawo5YUGXwg1tJjS/OsqSnIoajnRQCzzR6X896qXUAn asgXKTydmf553iSD03OG4K38UsdeD6uPUWN9zg/bceKaH2GM72p6md3Wepof4DuE UdTGgXENXOI= =uKmC -----END PGP SIGNATURE----- From lists at cheimes.de Sun Mar 2 00:24:45 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 02 Mar 2008 00:24:45 +0100 Subject: [Python-Dev] [Python-3000] The release process In-Reply-To: <47C9DA91.9000105@v.loewis.de> References: <47C9DA91.9000105@v.loewis.de> Message-ID: <47C9E5BD.6070507@cheimes.de> Martin v. L?wis wrote: > I guess every release manager is free to come up with his own set of > tools, but I feel you've given up too quickly (or started too late - > perhaps a test release run a few days before the release would have > helped). > > In any case, I'm willing to help with welease, but not with yet another > release tool. If you primarily complain about the GUIness of welease, > I could help with a command line version of it. [python dev only] It may sound like a dumb question by why do we need a release tool at all? I was involved in the release process of 3.0a2. Almost every step of the build process required human interaction. I don't want to diminish the effort that was put into welease though. But maybe (!) the same time spent for fixing some bugs would have helped the RM more. Christian From martin at v.loewis.de Sun Mar 2 00:53:30 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 02 Mar 2008 00:53:30 +0100 Subject: [Python-Dev] [Python-3000] The release process In-Reply-To: <47C9E5BD.6070507@cheimes.de> References: <47C9DA91.9000105@v.loewis.de> <47C9E5BD.6070507@cheimes.de> Message-ID: <47C9EC7A.2090701@v.loewis.de> > It may sound like a dumb question by why do we need a release tool at > all? I was involved in the release process of 3.0a2. Almost every step > of the build process required human interaction. I don't want to > diminish the effort that was put into welease though. But maybe (!) the > same time spent for fixing some bugs would have helped the RM more. I think you underestimate the number of changes that the RM needs to make manually, the the ease at which some of these steps get forgotten. Just take a look at the changes in r60911 and r60913, or r61144, r61147, r61150, r61151. Would you have found and remembered all the changes necessary? It helps *tremendously* if a tool tells you that you didn't miss any of the mechanic edits. Then, it also helps if the tool performs some of the mechanic steps, like: - tagging the tree (would you get the svn command line right the first time?) - exporting the tree, and creating the tar files (would you remember to touch the AST files that need more recent time stamps than their respective sources?) - uploading the tar files to dinsdale (do you remember the path on dinsdale the files need to go to?) Barry apparently wants it to go even further, making many of the edits for you. Creating the Windows installer is comparatively much less error-prone (although I do sometimes forget to update Python/sysmodule.c when I switch my sandbox to the release tag). Regards, Martin From skip at pobox.com Sun Mar 2 01:28:07 2008 From: skip at pobox.com (skip at pobox.com) Date: Sat, 1 Mar 2008 18:28:07 -0600 Subject: [Python-Dev] [Python-3000] The release process In-Reply-To: <10EE358F-8120-4EAC-9413-BDE5B812639F@python.org> References: <47C9DA91.9000105@v.loewis.de> <10EE358F-8120-4EAC-9413-BDE5B812639F@python.org> Message-ID: <18377.62615.194234.88761@montanaro-dyndns-org.local> Barry> The dependency on gtk is unnecessary and means it can effectively Barry> only be run on Linux. Specifically it means I can't do releases Barry> on OS X. I don't see much benefit in having a gui tool for doing Barry> releases. Gtk and Glade are available through MacPorts, at least according to "port search ...". Skip From aleaxit at gmail.com Sun Mar 2 03:00:11 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Sat, 1 Mar 2008 18:00:11 -0800 Subject: [Python-Dev] [Python-3000] No releases tonight In-Reply-To: <3B6D7A5F-2426-485E-BF5F-0BC16DF26A74@python.org> References: <47C9A6DC.9070708@cheimes.de> <3B6D7A5F-2426-485E-BF5F-0BC16DF26A74@python.org> Message-ID: On Sat, Mar 1, 2008 at 11:11 AM, Barry Warsaw wrote: ... > > I also propose translations of the shorter text to important languages > > like French, German, Japanese, Portuguese and Spanish. I'm willing to > > help with the German translation. > > Cool, thanks. I'd like to volunteer for Italian (and we, the Italian Python community, do have reasonably good connections to the Italian technical press, which is covering e.g. the upcoming Pycon Due conference), and although my French is VERY rusty I can give it a try if no native French speaker is forthcoming. Alex From aleaxit at gmail.com Sun Mar 2 03:14:18 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Sat, 1 Mar 2008 18:14:18 -0800 Subject: [Python-Dev] C-API status of Python 3? In-Reply-To: References: Message-ID: On Sat, Mar 1, 2008 at 12:14 PM, Christian Heimes wrote: ... > The 3.0 API isn't stable yet. I plan to rename some of the functions > before the first beta is released. Currently the naming schema is too > confusing: > > PyUnicode - str > PyString - bytes > PyBytes - bytearray > > See? :) Yep, but please do keep the PyUnicode for str and PyString for bytes (as macros/synonnyms of PyStr and PyBytes if you want!-) to help the task of porting existing extensions... the bytearray functions should no doubt be PyBytearray, though. Alex From stephen at xemacs.org Sun Mar 2 04:29:28 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 02 Mar 2008 12:29:28 +0900 Subject: [Python-Dev] [Python-3000] The release process In-Reply-To: <47C9E5BD.6070507@cheimes.de> References: <47C9DA91.9000105@v.loewis.de> <47C9E5BD.6070507@cheimes.de> Message-ID: <87od9xkdlz.fsf@uwakimon.sk.tsukuba.ac.jp> Christian Heimes writes: > It may sound like a dumb question by why do we need a release tool > at all? I was involved in the release process of 3.0a2. Almost > every step of the build process required human interaction. Interaction, yes, but often it can be reduced to "Abort, retry, fail?" > I don't want to diminish the effort that was put into welease > though. But maybe (!) the same time spent for fixing some bugs > would have helped the RM more. Which RM? Barry was hoping to get some useful process support at very low cost; that apparently didn't work out. But welease "is" Anthony's RM process, and surely it he considered it an investment in on-going quality or efficiency, or he wouldn't have done it. The fact that Barry found Anthony's process unusable is IMO not a reflection on either Barry or Anthony's code. Release processes seem to be highly personal, even within the same project. My own project (XEmacs) has 3 concurrent processes going at any one time (stable core, unstable core, stdlib). In my time with the project, stable core has seen two RM successions, unstable core has seen four, and stdlib has seen two. In no case did the new RM adopt the tools of any of his predecessors, but in two cases one person was a successor twice, and in both cases they reverted to their old tools. All processes seem to have been of roughly the same quality (my opinion, there are no metrics available). Been-there-done-that-shredded-the-T-shirt-in-the-process-ly y'rs, From stefan_ml at behnel.de Sun Mar 2 06:44:47 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 02 Mar 2008 06:44:47 +0100 Subject: [Python-Dev] C-API status of Python 3? In-Reply-To: References: Message-ID: Christian Heimes wrote: > Stefan Behnel wrote: >> I would like to know how stable the C-API of Python 3 is, or what the expected >> release level (beta?) would be at which I can expect it to stabilise. What is >> the plan here? The release schedule in PEP 3000 says "August 2008" for 3.0 final, is that still the current goal? Can I expect the C-API to stabilise by June, then? That's where we are planning a Cython workshop with a couple of sprints. Py3k support might be worth targeting - if we can rely on a fixed target by then. >> The background is Cython, which will need to support Python 3 one day or >> another, so I wanted to know from which point on it will make sense to start >> thinking about a migration plan. > > The 3.0 API isn't stable yet. I plan to rename some of the functions > before the first beta is released. Currently the naming schema is too > confusing: > > PyUnicode - str > PyString - bytes > PyBytes - bytearray > > See? :) I see. :) I actually expect the string semantics to be amongst the harder changes (at least, it's the most visible from a C-API point of view). However, names are not a big problem if you generate code anyway. Behaviour is what matters most for Cython. And we're already trying to adapt Cython's syntax to Py3k's, although that's not a requirement in all cases, as Cython lives with a couple of differences already. Keeping old syntax around and mapping it to the new C-API makes it easier to migrate existing Cython code. Hmmm, I even guess that the biggest problem might be porting Cython itself... > The documentation for the PyString functions is outdated and IIRC the > PyBytes docs are non existing. Ok, so I guess it would at least be a good idea to wait for the docs to be fixed, then. Thanks, Stefan From stefan_ml at behnel.de Sun Mar 2 07:21:25 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 02 Mar 2008 07:21:25 +0100 Subject: [Python-Dev] RELEASED Python 2.6a1 and 3.0a3 In-Reply-To: <6E72CEB8-D3BF-4440-A1EA-1A3D545CC8DB@python.org> References: <6E72CEB8-D3BF-4440-A1EA-1A3D545CC8DB@python.org> Message-ID: Barry Warsaw wrote: > On behalf of the Python development team and the Python community, I'm > happy to announce the first alpha release of Python 2.6, and the third > alpha release of Python 3.0. Cool! :) But how comes the release notes for Python 3a3 on the download site are the same as for 3a2? Stefan From Pavel.Vinogradov at nixdev.net Sun Mar 2 07:51:30 2008 From: Pavel.Vinogradov at nixdev.net (Pavel Vinogradov) Date: Sun, 2 Mar 2008 11:51:30 +0500 Subject: [Python-Dev] No releases tonight In-Reply-To: <47C9A6DC.9070708@cheimes.de> References: <47C9A6DC.9070708@cheimes.de> Message-ID: <9a305fc50803012251p4166f6c2m9121e119c4180420@mail.gmail.com> On 3/1/08, Christian Heimes wrote: > When 3.0a2 was released I contacted two larger German IT news sites. Non > of them even bother to reply. :/ > > I propose that we provide two official texts for the press. A shorter > text which explains Python and the most important changes since the last > version in a few paragraphs and a longer, more detailed text like > Martin's text for the 2.5.2 release. > > I also propose translations of the shorter text to important languages > like French, German, Japanese, Portuguese and Spanish. I'm willing to > help with the German translation. I'm ready to translate this documents in Russian and submit it to two or three Russian it-related news site. -- Pavel Vinogradov NixDev.Net, Linux Developer From g.brandl at gmx.net Sun Mar 2 07:54:08 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 02 Mar 2008 07:54:08 +0100 Subject: [Python-Dev] The release process In-Reply-To: References: Message-ID: Barry Warsaw schrieb: > PEP 101 is sorely out of date, especially with regards to updating web > content and the Python documentation. I think I now know how to > update the python.org web site, but the new Python documentation > format is still a mystery to me. If someone would like to help update > PEP 101 for the documentation steps, please let me know. Well, it's not very hard to guess who this someone is, is it? I've updated PEP 101, except for the part where the docs are uploaded to the website, I've no idea if the FTP paths etc. are still valid. For the next releases, if you want to do documentation packages, please feel free to contact me, I'll be happy to help! Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From hasan.diwan at gmail.com Sun Mar 2 07:57:50 2008 From: hasan.diwan at gmail.com (Hasan Diwan) Date: Sat, 1 Mar 2008 22:57:50 -0800 Subject: [Python-Dev] No releases tonight In-Reply-To: References: Message-ID: <2cda2fc90803012257i3a63bd1etd176948ebc4ba048@mail.gmail.com> I'll volunteer to do a French translation of the release. -- Cheers, Hasan Diwan From asmodai at in-nomine.org Sun Mar 2 09:49:35 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Sun, 2 Mar 2008 09:49:35 +0100 Subject: [Python-Dev] No releases tonight In-Reply-To: <47C9A6DC.9070708@cheimes.de> References: <47C9A6DC.9070708@cheimes.de> Message-ID: <20080302084935.GA62047@nexus.in-nomine.org> -On [20080301 19:57], Christian Heimes (lists at cheimes.de) wrote: >I also propose translations of the shorter text to important languages >like French, German, Japanese, Portuguese and Spanish. I'm willing to >help with the German translation. I can probably get a translation done in Japanese, Chinese and Korean. For Portuguese I guess we could ask the ASync guys in Brazil, they're huge Python users with their work for Canonical/Ubuntu. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ There can be no justice so long as laws are absolute... From lists at cheimes.de Sun Mar 2 14:47:53 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 02 Mar 2008 14:47:53 +0100 Subject: [Python-Dev] C-API status of Python 3? In-Reply-To: References: Message-ID: <47CAB009.3010606@cheimes.de> Alex Martelli wrote: > Yep, but please do keep the PyUnicode for str and PyString for bytes > (as macros/synonnyms of PyStr and PyBytes if you want!-) to help the > task of porting existing extensions... the bytearray functions should > no doubt be PyBytearray, though. Yeah, we've already planed to keep PyUnicode as prefix for str type functions. It makes perfectly sense, not only from the historical point of view. But for PyString I planed to rename the prefix to PyBytes. In my opinion we are going to regret it, when we keep too many legacy names from 2.x. In order to make the migration process easier I can add a header file that provides PyString_* functions as aliases for PyBytes_* Comments? Christian From mal at egenix.com Sun Mar 2 14:59:24 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Sun, 02 Mar 2008 14:59:24 +0100 Subject: [Python-Dev] C-API status of Python 3? In-Reply-To: <47CAB009.3010606@cheimes.de> References: <47CAB009.3010606@cheimes.de> Message-ID: <47CAB2BC.4070305@egenix.com> On 2008-03-02 14:47, Christian Heimes wrote: > Alex Martelli wrote: >> Yep, but please do keep the PyUnicode for str and PyString for bytes >> (as macros/synonnyms of PyStr and PyBytes if you want!-) to help the >> task of porting existing extensions... the bytearray functions should >> no doubt be PyBytearray, though. > > Yeah, we've already planed to keep PyUnicode as prefix for str type > functions. It makes perfectly sense, not only from the historical point > of view. > > But for PyString I planed to rename the prefix to PyBytes. In my opinion > we are going to regret it, when we keep too many legacy names from 2.x. > In order to make the migration process easier I can add a header file > that provides PyString_* functions as aliases for PyBytes_* > > Comments? +1 Why not also make unicode() the default type constructor and only keep str() as alias to simplify porting (perhaps with a warning) ? The term "string" is just too overloaded with all kinds of misinterpretations. The term "string" just refers to a string of bytes - a variable length array so to speak. However, depending on the application space, "string" is used as synonym for "text string" just as well as "data string". Removing the term "string" altogether would make it easier for people to understand that Py3k only has unicode (for text data) and bytes (for binary data). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 02 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From greg at krypto.org Sun Mar 2 19:39:26 2008 From: greg at krypto.org (Gregory P. Smith) Date: Sun, 2 Mar 2008 10:39:26 -0800 Subject: [Python-Dev] C-API status of Python 3? In-Reply-To: <47CAB009.3010606@cheimes.de> References: <47CAB009.3010606@cheimes.de> Message-ID: <52dc1c820803021039s50690468k4d60d0baeb97d3df@mail.gmail.com> On 3/2/08, Christian Heimes wrote: > > Alex Martelli wrote: > > Yep, but please do keep the PyUnicode for str and PyString for bytes > > (as macros/synonnyms of PyStr and PyBytes if you want!-) to help the > > task of porting existing extensions... the bytearray functions should > > no doubt be PyBytearray, though. > > > Yeah, we've already planed to keep PyUnicode as prefix for str type > functions. It makes perfectly sense, not only from the historical point > of view. > > But for PyString I planed to rename the prefix to PyBytes. In my opinion > we are going to regret it, when we keep too many legacy names from 2.x. > In order to make the migration process easier I can add a header file > that provides PyString_* functions as aliases for PyBytes_* +1 on only doing this via a header that must be explicitly included by modules wanting the compatibility names. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080302/3d37f586/attachment.htm From aleaxit at gmail.com Sun Mar 2 20:06:20 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Sun, 2 Mar 2008 11:06:20 -0800 Subject: [Python-Dev] C-API status of Python 3? In-Reply-To: <52dc1c820803021039s50690468k4d60d0baeb97d3df@mail.gmail.com> References: <47CAB009.3010606@cheimes.de> <52dc1c820803021039s50690468k4d60d0baeb97d3df@mail.gmail.com> Message-ID: On Sun, Mar 2, 2008 at 10:39 AM, Gregory P. Smith wrote: > > On 3/2/08, Christian Heimes wrote: > > Alex Martelli wrote: > > > Yep, but please do keep the PyUnicode for str and PyString for bytes > > > (as macros/synonnyms of PyStr and PyBytes if you want!-) to help the > > > task of porting existing extensions... the bytearray functions should > > > no doubt be PyBytearray, though. > > > > Yeah, we've already planed to keep PyUnicode as prefix for str type > > functions. It makes perfectly sense, not only from the historical point > > of view. > > > > But for PyString I planed to rename the prefix to PyBytes. In my opinion > > we are going to regret it, when we keep too many legacy names from 2.x. > > In order to make the migration process easier I can add a header file > > that provides PyString_* functions as aliases for PyBytes_* > > +1 on only doing this via a header that must be explicitly included by > modules wanting the compatibility names. OK, as long as it's also supplied (and presumably empty) for 2.6 -- my key concern is faciitating the maintenance of a single codebase for C-coded Python extensions that can be compiled for both 2.6 and 3.0. (I'm also thinking of SWIG and similar utilities, but those can probably best be tweaked to emit rather different C code for the two cases; still, that C code will also include some C snippets hand-coded by the extension author/maintainer, e.g. via SWIG typemaps &c, so easing the "single codebase" approach may help there too). I don't think we want to go the route of code translators/generators for C-coded Python extensions (the way we do for Python code via 2to3), and the fewer #if's and #define's C extension authors/maintainers are required to devise (in order to support both 2.6 and 3.0), the likelier it is that we'll see 3.0 support in popular C-coded Python extensions sooner rather than later. Alex From janssen at parc.com Sun Mar 2 20:39:24 2008 From: janssen at parc.com (Bill Janssen) Date: Sun, 2 Mar 2008 11:39:24 PST Subject: [Python-Dev] C-API status of Python 3? In-Reply-To: <47CAB2BC.4070305@egenix.com> References: <47CAB009.3010606@cheimes.de> <47CAB2BC.4070305@egenix.com> Message-ID: <08Mar2.113928pst."58696"@synergy1.parc.xerox.com> > Why not also make unicode() the default type constructor and only > keep str() as alias to simplify porting (perhaps with a warning) ? > > The term "string" is just too overloaded with all kinds of > misinterpretations. The term "string" just refers to a string of > bytes - a variable length array so to speak. However, depending > on the application space, "string" is used as synonym for > "text string" just as well as "data string". > > Removing the term "string" altogether would make it easier for > people to understand that Py3k only has unicode (for text data) > and bytes (for binary data). I agree that "string" is very overloaded, but calling it "unicode" is sort of like calling integers "int32" -- that is, you're talking about the implementation rather than the type. In most programming languages that aren't at the machine level (like C is), "string" really is a sequence of text characters, not a "string of bytes", and that's probably the term that should be used for Python going forward, despite the legacy issues it involves. Personally, I feel that "string" (for text) and "bytes" (for binary data represented as a sequence of bytes) are appropriate terms for Python. Keep "unicode" for a release or two as an alias for "string". But isn't all this in a PEP somewhere already? Bill From phil at riverbankcomputing.com Sun Mar 2 20:33:41 2008 From: phil at riverbankcomputing.com (Phil Thompson) Date: Sun, 2 Mar 2008 19:33:41 +0000 Subject: [Python-Dev] C-API status of Python 3? In-Reply-To: References: <52dc1c820803021039s50690468k4d60d0baeb97d3df@mail.gmail.com> Message-ID: <200803021933.41163.phil@riverbankcomputing.com> On Sunday 02 March 2008, Alex Martelli wrote: > On Sun, Mar 2, 2008 at 10:39 AM, Gregory P. Smith wrote: > > On 3/2/08, Christian Heimes wrote: > > > Alex Martelli wrote: > > > > Yep, but please do keep the PyUnicode for str and PyString for bytes > > > > (as macros/synonnyms of PyStr and PyBytes if you want!-) to help the > > > > task of porting existing extensions... the bytearray functions should > > > > no doubt be PyBytearray, though. > > > > > > Yeah, we've already planed to keep PyUnicode as prefix for str type > > > functions. It makes perfectly sense, not only from the historical point > > > of view. > > > > > > But for PyString I planed to rename the prefix to PyBytes. In my > > > opinion we are going to regret it, when we keep too many legacy names > > > from 2.x. In order to make the migration process easier I can add a > > > header file that provides PyString_* functions as aliases for PyBytes_* > > > > +1 on only doing this via a header that must be explicitly included by > > modules wanting the compatibility names. > > OK, as long as it's also supplied (and presumably empty) for 2.6 -- my > key concern is faciitating the maintenance of a single codebase for > C-coded Python extensions that can be compiled for both 2.6 and 3.0. > (I'm also thinking of SWIG and similar utilities, but those can > probably best be tweaked to emit rather different C code for the two > cases; still, that C code will also include some C snippets hand-coded > by the extension author/maintainer, e.g. via SWIG typemaps &c, so > easing the "single codebase" approach may help there too). > > I don't think we want to go the route of code translators/generators > for C-coded Python extensions (the way we do for Python code via > 2to3), and the fewer #if's and #define's C extension > authors/maintainers are required to devise (in order to support both > 2.6 and 3.0), the likelier it is that we'll see 3.0 support in popular > C-coded Python extensions sooner rather than later. Speaking for myself, this isn't going to make any difference as pre-2.6 versions of Python still need to be supported. More of a pain is if 2.6 introduces source level incompatibilities with 2.5 (as 2.5 did with 2.4). Phil From martin at v.loewis.de Sun Mar 2 22:05:14 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 02 Mar 2008 22:05:14 +0100 Subject: [Python-Dev] [ANN] Python 2.3.7 and 2.4.5, release candidate 1 Message-ID: <47CB168A.103@v.loewis.de> On behalf of the Python development team and the Python community, I'm happy to announce the release candidates of Python 2.4.5 and 2.4.5. Both releases include only security fixes. Python 2.5 is the latest version of Python, we're making this release for people who are still running Python 2.3 or 2.4. See the release notes at the website (also available as Misc/NEWS in the source distribution) for details of bugs fixed; most of them prevent interpreter crashes (and now cause proper Python exceptions in cases where the interprerter may have crashed before). Assuming no major problems crop up, a final release of Python 2.4.4 will follow in about a week's time. For more information on Python 2.3.7 and 2.4.5, including download links for various platforms, release notes, and known issues, please see: http://www.python.org/2.3.7 http://www.python.org/2.4.5 Highlights of the previous major Python releases are available from the Python 2.4 page, at http://www.python.org/2.3/highlights.html http://www.python.org/2.4/highlights.html Enjoy this release, Martin Martin v. Loewis martin at v.loewis.de Python Release Manager (on behalf of the entire python-dev team) From lists at cheimes.de Sun Mar 2 22:11:00 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 02 Mar 2008 22:11:00 +0100 Subject: [Python-Dev] C-API status of Python 3? In-Reply-To: References: Message-ID: <47CB17E4.9000306@cheimes.de> Stefan Behnel wrote: > The release schedule in PEP 3000 says "August 2008" for 3.0 final, is that > still the current goal? Can I expect the C-API to stabilise by June, then? > That's where we are planning a Cython workshop with a couple of sprints. Py3k > support might be worth targeting - if we can rely on a fixed target by then. Yes, August 2008 is still our goal. I still think it's a realistic goal. The C API is mostly stabilized around May when we target the first beta. > I actually expect the string semantics to be amongst the harder changes (at > least, it's the most visible from a C-API point of view). > > However, names are not a big problem if you generate code anyway. Behaviour is > what matters most for Cython. And we're already trying to adapt Cython's > syntax to Py3k's, although that's not a requirement in all cases, as Cython > lives with a couple of differences already. Keeping old syntax around and > mapping it to the new C-API makes it easier to migrate existing Cython code. The semantics are easier in Python 3.x than in the 2.x series. Old style classes are gone, longs are gone and integers are PyLong based, the distinction between bytes and text is much easier ... Christian From greg.ewing at canterbury.ac.nz Sun Mar 2 23:11:08 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 03 Mar 2008 11:11:08 +1300 Subject: [Python-Dev] C-API status of Python 3? In-Reply-To: <47CAB2BC.4070305@egenix.com> References: <47CAB009.3010606@cheimes.de> <47CAB2BC.4070305@egenix.com> Message-ID: <47CB25FC.9010409@canterbury.ac.nz> M.-A. Lemburg wrote: > Why not also make unicode() the default type constructor and only > keep str() as alias to simplify porting (perhaps with a warning) ? -1 on making us type 7 characters instead of 3 all over the place. > The term "string" is just too overloaded with all kinds of > misinterpretations. The term "string" just refers to a string of > bytes - a variable length array so to speak. I disagree -- "string" has come to mean "string of characters" unless otherwise qualified. Using one to hold non-characters is just an aberration that was necessary in Python 2 because there wasn't much alternative. -- Greg From mal at egenix.com Mon Mar 3 00:13:39 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 03 Mar 2008 00:13:39 +0100 Subject: [Python-Dev] C-API status of Python 3? In-Reply-To: <08Mar2.113928pst."58696"@synergy1.parc.xerox.com> References: <47CAB009.3010606@cheimes.de> <47CAB2BC.4070305@egenix.com> <08Mar2.113928pst."58696"@synergy1.parc.xerox.com> Message-ID: <47CB34A3.4060306@egenix.com> On 2008-03-02 20:39, Bill Janssen wrote: >> Why not also make unicode() the default type constructor and only >> keep str() as alias to simplify porting (perhaps with a warning) ? >> >> The term "string" is just too overloaded with all kinds of >> misinterpretations. The term "string" just refers to a string of >> bytes - a variable length array so to speak. However, depending >> on the application space, "string" is used as synonym for >> "text string" just as well as "data string". >> >> Removing the term "string" altogether would make it easier for >> people to understand that Py3k only has unicode (for text data) >> and bytes (for binary data). > > I agree that "string" is very overloaded, but calling it "unicode" is > sort of like calling integers "int32" -- that is, you're talking about > the implementation rather than the type. Hmm in that case, we'd have to call it "ucs2" or "ucs4" depending on how Python was compiled ;-) > In most programming > languages that aren't at the machine level (like C is), "string" > really is a sequence of text characters, not a "string of bytes", and > that's probably the term that should be used for Python going forward, > despite the legacy issues it involves. I'm not bound to "unicode" at all, just don't think using "string" for text data will really make people think twice often enough and then you end up having binary data in a "string" again - with the only difference that it's now using the Unicode type internally. My personal favorite is "text" for text data. > Personally, I feel that "string" (for text) and "bytes" (for binary > data represented as a sequence of bytes) are appropriate terms for > Python. Keep "unicode" for a release or two as an alias for "string". > But isn't all this in a PEP somewhere already? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 03 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From mal at egenix.com Mon Mar 3 00:26:15 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 03 Mar 2008 00:26:15 +0100 Subject: [Python-Dev] C-API status of Python 3? In-Reply-To: <47CB25FC.9010409@canterbury.ac.nz> References: <47CAB009.3010606@cheimes.de> <47CAB2BC.4070305@egenix.com> <47CB25FC.9010409@canterbury.ac.nz> Message-ID: <47CB3797.7050209@egenix.com> On 2008-03-02 23:11, Greg Ewing wrote: > M.-A. Lemburg wrote: >> Why not also make unicode() the default type constructor and only >> keep str() as alias to simplify porting (perhaps with a warning) ? > > -1 on making us type 7 characters instead of > 3 all over the place. Oh well... how about "text()" ? >> The term "string" is just too overloaded with all kinds of >> misinterpretations. The term "string" just refers to a string of >> bytes - a variable length array so to speak. > > I disagree -- "string" has come to mean "string of > characters" unless otherwise qualified. Using one > to hold non-characters is just an aberration that > was necessary in Python 2 because there wasn't much > alternative. Buffer objects have been around for years and for exactly this purpose. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 03 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From gnewsg at gmail.com Mon Mar 3 00:19:22 2008 From: gnewsg at gmail.com (Giampaolo Rodola') Date: Sun, 2 Mar 2008 15:19:22 -0800 (PST) Subject: [Python-Dev] Py3k and asyncore/asynchat In-Reply-To: <47B5FE8E.2060901@cornell.edu> References: <95d2b6e8-2524-4464-b5a9-c4d56e5f4318@t16g2000hsc.googlegroups.com> <20080215142414.GA6805@amk-desktop.matrixgroup.net> <20080215142832.GC13291@snowy.squish.net> <08Feb15.123656pst."58696"@synergy1.parc.xerox.com> <47B5FE8E.2060901@cornell.edu> Message-ID: I've discussed a lot with Josiah via e-mail and I provided an updated version of the patch proposed in bug #1736190 including a fix for the two issues raised by me in the bug report. The update has been needed also because the original patch has been out-dated by some commits after r53734 involving the test suite and the documentation, which I both took off this patch. I also re-added simple_producer and fifo classes in asynchat.py since it seems they are needed for passing tests. The test suite has passed on Windows XP using Python 2.6 alpha 1. I have also successfully run the test suite of a project of mine based on asynchat which includes over 40 tests. From josiah.carlson at gmail.com Mon Mar 3 00:59:42 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Sun, 2 Mar 2008 15:59:42 -0800 Subject: [Python-Dev] Py3k and asyncore/asynchat In-Reply-To: References: <20080215142414.GA6805@amk-desktop.matrixgroup.net> <20080215142832.GC13291@snowy.squish.net> <47B5FE8E.2060901@cornell.edu> Message-ID: As far as I can tell, the asyncore.py, asynchat.py, and updated test_asyncore.py are good. I have been using earlier variants in my own projects (prior to their updating to pass the test suite) for quite a few months now. The updated modules provide better performance, features, and support for real-world async socket servers, never mind fixing more than a half dozen outstanding bugs. - Josiah On Sun, Mar 2, 2008 at 3:19 PM, Giampaolo Rodola' wrote: > I've discussed a lot with Josiah via e-mail and I provided an updated > version of the patch proposed in bug #1736190 including a fix for the > two issues raised by me in the bug report. > The update has been needed also because the original patch has been > out-dated by some commits after r53734 involving the test suite > and the documentation, which I both took off this patch. > I also re-added simple_producer and fifo classes in asynchat.py since > it seems they are needed for passing tests. > > The test suite has passed on Windows XP using Python 2.6 alpha 1. > I have also successfully run the test suite of a project of mine based > on asynchat which includes over 40 tests. > From fdrake at acm.org Mon Mar 3 01:43:33 2008 From: fdrake at acm.org (Fred Drake) Date: Sun, 2 Mar 2008 19:43:33 -0500 Subject: [Python-Dev] [ANN] Python 2.3.7 and 2.4.5, release candidate 1 In-Reply-To: <47CB168A.103@v.loewis.de> References: <47CB168A.103@v.loewis.de> Message-ID: <88D43806-B25D-4538-8F7D-D17EB979C40E@acm.org> On Mar 2, 2008, at 4:05 PM, Martin v. L?wis wrote: > Assuming no major problems crop up, a final release of Python 2.4.4 > will > follow in about a week's time. I do suppose you mean 2.4.5. 2.4.5 won't build for me from the svn checkout on Mac OS X 10.5.2: gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused- madd -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -I./Include - DPy_BUILD_CORE -c ./Modules/posixmodule.c -o Modules/posixmodule.o ./Modules/posixmodule.c: In function ?posix_setpgrp?: ./Modules/posixmodule.c:3145: error: too few arguments to function ?setpgrp? make: *** [Modules/posixmodule.o] Error 1 I can only presume I'm doing something wrong at this point, since I don't consider myself a Mac OS X developer. -Fred -- Fred Drake From fdrake at acm.org Mon Mar 3 01:52:50 2008 From: fdrake at acm.org (Fred Drake) Date: Sun, 2 Mar 2008 19:52:50 -0500 Subject: [Python-Dev] [ANN] Python 2.3.7 and 2.4.5, release candidate 1 In-Reply-To: <88D43806-B25D-4538-8F7D-D17EB979C40E@acm.org> References: <47CB168A.103@v.loewis.de> <88D43806-B25D-4538-8F7D-D17EB979C40E@acm.org> Message-ID: <592F38CC-7EFD-4ABA-BC93-CE8BB50C9CB4@acm.org> On Mar 2, 2008, at 7:43 PM, Fred Drake wrote: > 2.4.5 won't build for me from the svn checkout on Mac OS X 10.5.2: Neither does 2.3.7 now that I've tried that: gcc -u __dummy -u _PyMac_Error -framework System -framework CoreServices -framework Foundation -o python.exe \ Modules/python.o \ libpython2.3.a -ldl Undefined symbols: "__dummy", referenced from: ld: symbol(s) not found collect2: ld returned 1 exit status make: *** [python.exe] Error 1 Of course, I wasn't using an earlier 2.3.x version on this box. I would really like to be able to use 2.4.5, since I've been using 2.4.4 for work for a while now. -Fred -- Fred Drake From brett at python.org Mon Mar 3 01:58:35 2008 From: brett at python.org (Brett Cannon) Date: Sun, 2 Mar 2008 16:58:35 -0800 Subject: [Python-Dev] [ANN] Python 2.3.7 and 2.4.5, release candidate 1 In-Reply-To: <592F38CC-7EFD-4ABA-BC93-CE8BB50C9CB4@acm.org> References: <47CB168A.103@v.loewis.de> <88D43806-B25D-4538-8F7D-D17EB979C40E@acm.org> <592F38CC-7EFD-4ABA-BC93-CE8BB50C9CB4@acm.org> Message-ID: On Sun, Mar 2, 2008 at 4:52 PM, Fred Drake wrote: > On Mar 2, 2008, at 7:43 PM, Fred Drake wrote: > > 2.4.5 won't build for me from the svn checkout on Mac OS X 10.5.2: > > > Neither does 2.3.7 now that I've tried that: > > gcc -u __dummy -u _PyMac_Error -framework System -framework > CoreServices -framework Foundation -o python.exe \ > Modules/python.o \ > libpython2.3.a -ldl > Undefined symbols: > "__dummy", referenced from: > ld: symbol(s) not found > collect2: ld returned 1 exit status > make: *** [python.exe] Error 1 > > Of course, I wasn't using an earlier 2.3.x version on this box. I > would really like to be able to use 2.4.5, since I've been using 2.4.4 > for work for a while now. For me on OS X 10.5.2 (gcc 4.0.1) for 2.37 I am getting a ton of: sem_post: Bad file descriptor sem_init: Function not implemented sem_trywait: Bad file descriptor -Brett From guido at python.org Mon Mar 3 02:14:19 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 2 Mar 2008 17:14:19 -0800 Subject: [Python-Dev] C-API status of Python 3? In-Reply-To: <47CB3797.7050209@egenix.com> References: <47CAB009.3010606@cheimes.de> <47CAB2BC.4070305@egenix.com> <47CB25FC.9010409@canterbury.ac.nz> <47CB3797.7050209@egenix.com> Message-ID: On Sun, Mar 2, 2008 at 3:26 PM, M.-A. Lemburg wrote: > On 2008-03-02 23:11, Greg Ewing wrote: > > M.-A. Lemburg wrote: > >> Why not also make unicode() the default type constructor and only > >> keep str() as alias to simplify porting (perhaps with a warning) ? > > > > -1 on making us type 7 characters instead of > > 3 all over the place. > > Oh well... how about "text()" ? Sorry, this was discussed and decided long ago. I'm not going to change this now. The type is called string or some variation thereof in most other popular languages. > >> The term "string" is just too overloaded with all kinds of > >> misinterpretations. The term "string" just refers to a string of > >> bytes - a variable length array so to speak. > > > > I disagree -- "string" has come to mean "string of > > characters" unless otherwise qualified. Using one > > to hold non-characters is just an aberration that > > was necessary in Python 2 because there wasn't much > > alternative. Historically that's incorrect. In 1990, when Unicode hadn't even been invented, 'str' was very intentionally designed to hold text and data equally well. > Buffer objects have been around for years and for exactly > this purpose. No, buffer objects were not invented to *hold* binary data. The buffer API was invented to *reference* bytes that were owned by 3rd party C libraries. Its descendant in Py3k, 'memoryview' (see PEP 3118) has the same purpose without having the same bugs. For *holding* bytes in Py3k we'll use bytes (immutable) or bytearray (mutable). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From kevin at bud.ca Mon Mar 3 02:35:21 2008 From: kevin at bud.ca (Kevin Teague) Date: Sun, 2 Mar 2008 17:35:21 -0800 Subject: [Python-Dev] [ANN] Python 2.3.7 and 2.4.5, release candidate 1 In-Reply-To: References: <47CB168A.103@v.loewis.de> <88D43806-B25D-4538-8F7D-D17EB979C40E@acm.org> <592F38CC-7EFD-4ABA-BC93-CE8BB50C9CB4@acm.org> Message-ID: "It has to do with the MACOSX_DEPLOYMENT_TARGET. If it's set to 10.4, the legacy version of setpgrp is used (with args), it it's 10.5, setpgrp expects no arguments. It seems configure won't detect the difference." http://bugs.python.org/issue1358 This issue was fixed for Python 2.5. As the issue notes, you can work around it with: ./configure MACOSX_DEPLOYMENT_TARGET=10.5 But it would be really nice if the configure fix for 2.5 was backported to 2.4.5 since Zope is still on 2.4 and Mac OS X skipped system builds for 2.4 going direct from 2.3 -> 2.5. From fdrake at acm.org Mon Mar 3 03:22:27 2008 From: fdrake at acm.org (Fred Drake) Date: Sun, 2 Mar 2008 21:22:27 -0500 Subject: [Python-Dev] [ANN] Python 2.3.7 and 2.4.5, release candidate 1 In-Reply-To: References: <47CB168A.103@v.loewis.de> <88D43806-B25D-4538-8F7D-D17EB979C40E@acm.org> <592F38CC-7EFD-4ABA-BC93-CE8BB50C9CB4@acm.org> Message-ID: On Mar 2, 2008, at 8:35 PM, Kevin Teague wrote: > This issue was fixed for Python 2.5. As the issue notes, you can > work around it with: > > ./configure MACOSX_DEPLOYMENT_TARGET=10.5 Indeed, that works wonderfully for me for 2.4.5. > But it would be really nice if the configure fix for 2.5 was > backported to 2.4.5 since Zope is still on 2.4 and Mac OS X skipped > system builds for 2.4 going direct from 2.3 -> 2.5. Yes, it would be very nice if this worked out of the box on Mac OS X 10.5.2. It's definitely a surprise for those of us who built our 2.4.4 on Mac OS X 10.4.x. -Fred -- Fred Drake From martin at v.loewis.de Mon Mar 3 07:44:57 2008 From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 03 Mar 2008 07:44:57 +0100 Subject: [Python-Dev] [ANN] Python 2.3.7 and 2.4.5, release candidate 1 In-Reply-To: <88D43806-B25D-4538-8F7D-D17EB979C40E@acm.org> References: <47CB168A.103@v.loewis.de> <88D43806-B25D-4538-8F7D-D17EB979C40E@acm.org> Message-ID: <47CB9E69.9000709@v.loewis.de> >> Assuming no major problems crop up, a final release of Python 2.4.4 will >> follow in about a week's time. > > I do suppose you mean 2.4.5. Oops, yes. > 2.4.5 won't build for me from the svn checkout on Mac OS X 10.5.2: > > gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp > -mno-fused-madd -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. > -I./Include -DPy_BUILD_CORE -c ./Modules/posixmodule.c -o > Modules/posixmodule.o > ./Modules/posixmodule.c: In function ?posix_setpgrp?: > ./Modules/posixmodule.c:3145: error: too few arguments to function > ?setpgrp? > make: *** [Modules/posixmodule.o] Error 1 > > I can only presume I'm doing something wrong at this point, since I > don't consider myself a Mac OS X developer. No. 2.4.5 just won't compile on OSX 10.5.2. This bug has been fixed for 2.5 (IIUC), but the fix was not backported (nor should it be, as it is not relevant for security). Use OS X 10.4 if you want to use Python 2.4. Regards, Martin From martin at v.loewis.de Mon Mar 3 07:48:28 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 03 Mar 2008 07:48:28 +0100 Subject: [Python-Dev] [ANN] Python 2.3.7 and 2.4.5, release candidate 1 In-Reply-To: References: <47CB168A.103@v.loewis.de> <88D43806-B25D-4538-8F7D-D17EB979C40E@acm.org> <592F38CC-7EFD-4ABA-BC93-CE8BB50C9CB4@acm.org> Message-ID: <47CB9F3C.9030202@v.loewis.de> >> But it would be really nice if the configure fix for 2.5 was >> backported to 2.4.5 since Zope is still on 2.4 and Mac OS X skipped >> system builds for 2.4 going direct from 2.3 -> 2.5. > > > Yes, it would be very nice if this worked out of the box on Mac OS X > 10.5.2. It's definitely a surprise for those of us who built our 2.4.4 > on Mac OS X 10.4.x. I can put a notice in the release notes, but I definitely won't change it to work out of the box. If 2.4.4 compiled out of the box on this box, it would have been a regression and would have to be fixed. IIUC, 2.4.4 won't compile on 10.5, either, and Python 2.4.5 will have no code to port it to new platforms. Regards, Martin From barry at python.org Mon Mar 3 13:17:00 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 3 Mar 2008 07:17:00 -0500 Subject: [Python-Dev] [ANN] Python 2.3.7 and 2.4.5, release candidate 1 In-Reply-To: <47CB9F3C.9030202@v.loewis.de> References: <47CB168A.103@v.loewis.de> <88D43806-B25D-4538-8F7D-D17EB979C40E@acm.org> <592F38CC-7EFD-4ABA-BC93-CE8BB50C9CB4@acm.org> <47CB9F3C.9030202@v.loewis.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 3, 2008, at 1:48 AM, Martin v. L?wis wrote: >>> But it would be really nice if the configure fix for 2.5 was >>> backported to 2.4.5 since Zope is still on 2.4 and Mac OS X skipped >>> system builds for 2.4 going direct from 2.3 -> 2.5. >> >> >> Yes, it would be very nice if this worked out of the box on Mac OS X >> 10.5.2. It's definitely a surprise for those of us who built our >> 2.4.4 >> on Mac OS X 10.4.x. > > I can put a notice in the release notes, but I definitely won't change > it to work out of the box. If 2.4.4 compiled out of the box on this > box, > it would have been a regression and would have to be fixed. IIUC, > 2.4.4 > won't compile on 10.5, either, and Python 2.4.5 will have no code to > port it to new platforms. Can you also add a note to the 2.3 and 2.4 web pages? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8vsPHEjvBPtnXfVAQJVpgP5AbRU+BENEa7fv7vGUykjtQRftaF6ATQz yTo9018UiQZ20bFv2PIvgHltsET1ksTuieSdDjGbQ3rGu3vo1tldiGYxUQJgi++C q8ntOyLUo+nHSlKm11TTyMiNX4igl+X0bes5PlgJZbWOnw0vBvWbVRrwgMUsJqfi ox/d8+2jWc4= =HLja -----END PGP SIGNATURE----- From aahz at pythoncraft.com Mon Mar 3 15:16:36 2008 From: aahz at pythoncraft.com (Aahz) Date: Mon, 3 Mar 2008 06:16:36 -0800 Subject: [Python-Dev] Python XML Validator In-Reply-To: References: Message-ID: <20080303141636.GB4942@panix.com> On Thu, Feb 28, 2008, Medhat Gayed wrote: > > I tested and tried a few XML validators but none of them is able to > successfully validate a string of xml (not a file just a string) to > programatically be able to validate messages of xml that flow in and > out of the different systems. Teh validators I used were XSV, oNVDL > and lxml, can we implement a pure python module for validating strings > of xml using XML Schema (not DTD). We certainly "can", for values of "we" that include "you". ;-) IOW, please write it yourself and post it to PyPI. Or find someone else to do the work, but in any event, python-dev is not an appropriate place to discuss it. Try comp.lang.python, perhaps, or a Python/XML mailing list. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson From facundobatista at gmail.com Mon Mar 3 18:03:06 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Mon, 3 Mar 2008 15:03:06 -0200 Subject: [Python-Dev] [Python-3000] No releases tonight In-Reply-To: <47C9A6DC.9070708@cheimes.de> References: <47C9A6DC.9070708@cheimes.de> Message-ID: 2008/3/1, Christian Heimes : > I also propose translations of the shorter text to important languages > like French, German, Japanese, Portuguese and Spanish. I'm willing to > help with the German translation. /me raises his hand while saying "Spanish, Spanish!". Which is the official text? -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From barry at python.org Mon Mar 3 21:41:22 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 3 Mar 2008 15:41:22 -0500 Subject: [Python-Dev] RELEASED Python 2.6a1 and 3.0a3 In-Reply-To: References: <6E72CEB8-D3BF-4440-A1EA-1A3D545CC8DB@python.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 2, 2008, at 1:21 AM, Stefan Behnel wrote: > Barry Warsaw wrote: >> On behalf of the Python development team and the Python community, >> I'm >> happy to announce the first alpha release of Python 2.6, and the >> third >> alpha release of Python 3.0. > > Cool! :) > > But how comes the release notes for Python 3a3 on the download site > are the > same as for 3a2? Cut-and-paste mostly. I'll try to fix those. Thanks, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8xidHEjvBPtnXfVAQKTLwP/Q8Jpfdw5sR6bRBrAcGJcMsU2bUGhGYgV OpTxOZ7hO0f7JhxK2/mCuCOKqXsbSjPreHkztbKSvJf/nBeNI24UaHLTyxhoczeS XCpDdQm9fefiYYgw4NWdI/KUGdKWbbrmm2bFuRbddXt9hysCaFq5PMWU/og0MqUC UJa7/sojkUU= =9Yo0 -----END PGP SIGNATURE----- From g.brandl at gmx.net Mon Mar 3 22:24:36 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 03 Mar 2008 22:24:36 +0100 Subject: [Python-Dev] _abcoll Callable bug Message-ID: The Callable abc has a __contains__ but no __call__ method. I'd fix this, but am unsure which args it should get. Georg From barry at python.org Mon Mar 3 23:40:54 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 3 Mar 2008 17:40:54 -0500 Subject: [Python-Dev] Upcoming 2.4.5 and 2.3.7 releases In-Reply-To: <5E6A9FAF-F3BA-4281-9053-3BD986B79C69@acm.org> References: <479CDB1F.60809@v.loewis.de> <0124660D-3CF9-48BF-8DB1-B71763B1D2EC@python.org> <479E4778.7010500@v.loewis.de> <479E4D76.1080205@python.org> <5E6A9FAF-F3BA-4281-9053-3BD986B79C69@acm.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 28, 2008, at 4:56 PM, Fred Drake wrote: > On Jan 28, 2008, at 4:47 PM, Barry Warsaw wrote: >> The problem is that I make separate releases of the standalone email >> package from these branches, so that means that email 3.0.3 or 2.5.10 >> will have regressions. > > Releasing the email package from the Python maintenance branches is > probably insane. > > This kind of thing would be less of a problem if the standard > library were smaller, and the email package only available > separately. It's also why some of us want a smaller standard library. Yep. We've solved this problem for the old releases. I still think it makes sense to use the maintenance branch for the current stable release, but as soon as 2.6 is released, I'll fiddle the sandbox to contain a separate copy, since 2.5 will be in security-only mode. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8x+dnEjvBPtnXfVAQJa3AP/VZQoT+b+tRBLLXtBZ0kLxMGwpuvTdDae aD8X3zu4SaUJOIw47NUb/Re3xTOWA0cjBfoEZLYVtNBUICUt0AhFvP7s26Or2jlM NINAqiq1gnzyVhwfhUMHoaX66PsJvA8OMq4owoKIUJDQiAnnz5/Gw2t6/0I4PPrt wunxcXvfOdg= =zf3e -----END PGP SIGNATURE----- From barry at python.org Mon Mar 3 23:43:57 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 3 Mar 2008 17:43:57 -0500 Subject: [Python-Dev] [Python-3000] The release process In-Reply-To: <18377.62615.194234.88761@montanaro-dyndns-org.local> References: <47C9DA91.9000105@v.loewis.de> <10EE358F-8120-4EAC-9413-BDE5B812639F@python.org> <18377.62615.194234.88761@montanaro-dyndns-org.local> Message-ID: <614DA72A-2900-4BA6-AA39-48B1E8C3CA98@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 1, 2008, at 7:28 PM, skip at pobox.com wrote: > > Barry> The dependency on gtk is unnecessary and means it can > effectively > Barry> only be run on Linux. Specifically it means I can't do > releases > Barry> on OS X. I don't see much benefit in having a gui tool > for doing > Barry> releases. > > Gtk and Glade are available through MacPorts, at least according to > "port > search ...". True, but it's just more moving parts to break, especially when I don't really feel a u/i buys you much[*]. - -Barry [*] Skip knows me as a diehard XEmacser so that statement can pretty much define my standard answer to all such questions. :) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8x/LXEjvBPtnXfVAQKnFAP/RDKzhQsIDuc8joSR3G2mrKv7jH+6tl04 Fq3d++1jFjE5cBiY9uDL2/799wqUKvevx2QnGrON1ins9xuYGh5/xY9w4JvbI8aX zLq/RjNFzPl/YnMCX7OSUBjbQoE3sr+dgUpLurImsJxWaFcjqufzQuno6N0sqvWg sevJTwfBkOE= =HfWd -----END PGP SIGNATURE----- From guido at python.org Mon Mar 3 23:46:25 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 3 Mar 2008 14:46:25 -0800 Subject: [Python-Dev] _abcoll Callable bug In-Reply-To: References: Message-ID: On Mon, Mar 3, 2008 at 1:24 PM, Georg Brandl wrote: > The Callable abc has a __contains__ but no __call__ method. > I'd fix this, but am unsure which args it should get. Oops, bad copy/paste. :-( def __call__(self, *args, **kwds): return None I don't expect it'll have many users. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Mon Mar 3 23:49:38 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 3 Mar 2008 17:49:38 -0500 Subject: [Python-Dev] [Python-3000] The release process In-Reply-To: <87od9xkdlz.fsf@uwakimon.sk.tsukuba.ac.jp> References: <47C9DA91.9000105@v.loewis.de> <47C9E5BD.6070507@cheimes.de> <87od9xkdlz.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 1, 2008, at 10:29 PM, Stephen J. Turnbull wrote: > The fact that Barry found Anthony's process unusable is IMO not a > reflection on either Barry or Anthony's code. Release processes seem > to be highly personal, even within the same project. My own project > (XEmacs) has 3 concurrent processes going at any one time (stable > core, unstable core, stdlib). In my time with the project, stable > core has seen two RM successions, unstable core has seen four, and > stdlib has seen two. In no case did the new RM adopt the tools of any > of his predecessors, but in two cases one person was a successor > twice, and in both cases they reverted to their old tools. All > processes seem to have been of roughly the same quality (my opinion, > there are no metrics available). I agree with all of this, and I definitely never meant to impugn Anthony's hack. However, I would categorize every release tool I've ever used (both bought and built, in a commercial or FLOSS environment) as "a hack". Releasing something as complex as Python is just going to be a PITA, so all the RM is looking for is a hack that fits his hands better and does just enough to lower his threshold of pain, or ToP (tm), to a level where he doesn't want to spend his time waiting for the next step by plunging ice picks into his earholes. I'm sure when Anthony releases Python 2.9 he'll curse my release tool too. Or he'll do the smart thing and as Stephen suggests, just ditch my pile of crap and resurrect welease. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8yAg3EjvBPtnXfVAQIW2QP/e0Ny+l8mYGrOzmVJ1zCDZp9cdvBVgEXB fBWc0UPjyBRhmVBoeZ773R5j/IMlsLCetp2VKDkDCutq4PRo9z78ZjrYE2M2+RZP rigMxReSvv5Nw83kOXRy99jQva0ptjnYw2Gdpd1nhtlVSrRmEXaLnVF52Z2hLgul Q9JkBpg7kr8= =PEaE -----END PGP SIGNATURE----- From barry at python.org Mon Mar 3 23:53:08 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 3 Mar 2008 17:53:08 -0500 Subject: [Python-Dev] The release process In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 2, 2008, at 1:54 AM, Georg Brandl wrote: > Barry Warsaw schrieb: > >> PEP 101 is sorely out of date, especially with regards to updating >> web >> content and the Python documentation. I think I now know how to >> update the python.org web site, but the new Python documentation >> format is still a mystery to me. If someone would like to help >> update >> PEP 101 for the documentation steps, please let me know. > > Well, it's not very hard to guess who this someone is, is it? Actually, I honestly didn't know... sorry Georg! > I've updated PEP 101, except for the part where the docs are uploaded > to the website, I've no idea if the FTP paths etc. are still valid. Thanks, this is a big help. I'll double check the ftp paths, but I think they haven't changed. > For the next releases, if you want to do documentation packages, > please feel free to contact me, I'll be happy to help! Great, thanks! - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8yBVHEjvBPtnXfVAQJemgP/aE/a54SaKc7Ls9osae+g57zcYB7d95FX O/HcE3/QFJCQeBquTfwbOV8doyAYHFDDIw8U2GIvgppLjPEqHAJzKfBqeQIILM3a uZM/4yUUcnnI8UQYhi4u+maztv6YwQxyKj9sLuK9GFncxk1B1z7zW4WcWISoet3H j3XC4eFVvjM= =SDdJ -----END PGP SIGNATURE----- From barry at python.org Tue Mar 4 00:06:21 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 3 Mar 2008 18:06:21 -0500 Subject: [Python-Dev] [Python-3000] No releases tonight In-Reply-To: References: <47C9A6DC.9070708@cheimes.de> <3B6D7A5F-2426-485E-BF5F-0BC16DF26A74@python.org> Message-ID: <1D91070A-1387-4CFD-A6AC-0BA84626EE62@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 1, 2008, at 9:00 PM, Alex Martelli wrote: > On Sat, Mar 1, 2008 at 11:11 AM, Barry Warsaw > wrote: > ... >>> I also propose translations of the shorter text to important >>> languages >>> like French, German, Japanese, Portuguese and Spanish. I'm willing >>> to >>> help with the German translation. >> >> Cool, thanks. > > I'd like to volunteer for Italian (and we, the Italian Python > community, do have reasonably good connections to the Italian > technical press, which is covering e.g. the upcoming Pycon Due > conference), and although my French is VERY rusty I can give it a try > if no native French speaker is forthcoming. So how should we handle this? Is it something you need input from me from, or can you just take my announcement, do the translations and post them to the appropriate forums? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8yEbXEjvBPtnXfVAQLU9QP/Tq3NsOHmKijNZQ2Gq8y4iLjznRAMS5ld gkI+onjtjHO42pJ40p7UJMuUkp5V6+WzFSh1JqhxfyTLSYuMv90Jr+SKo8emtyg1 kJQjLfjtPu4JY/RIVhmbBD2IBIAchNBH2HWUjFY7Odp7TapuG78KRUFMsbt0GDP8 7XDl9xYXalg= =w0Uo -----END PGP SIGNATURE----- From lists at cheimes.de Tue Mar 4 01:41:58 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 04 Mar 2008 01:41:58 +0100 Subject: [Python-Dev] [Python-3000] No releases tonight In-Reply-To: <1D91070A-1387-4CFD-A6AC-0BA84626EE62@python.org> References: <47C9A6DC.9070708@cheimes.de> <3B6D7A5F-2426-485E-BF5F-0BC16DF26A74@python.org> <1D91070A-1387-4CFD-A6AC-0BA84626EE62@python.org> Message-ID: <47CC9AD6.90504@cheimes.de> Barry Warsaw wrote: > So how should we handle this? Is it something you need input from me > from, or can you just take my announcement, do the translations and post > them to the appropriate forums? Your announcements are fine for Python's mailing lists and forums. However I had a differnt target in mind, mainly large IT news sites with broader audience. It'd be great if we can mobilize some professional PR people to write an announcement for the press. Christian From skip at pobox.com Tue Mar 4 04:30:48 2008 From: skip at pobox.com (skip at pobox.com) Date: Mon, 3 Mar 2008 21:30:48 -0600 Subject: [Python-Dev] [Python-3000] The release process In-Reply-To: <614DA72A-2900-4BA6-AA39-48B1E8C3CA98@python.org> References: <47C9DA91.9000105@v.loewis.de> <10EE358F-8120-4EAC-9413-BDE5B812639F@python.org> <18377.62615.194234.88761@montanaro-dyndns-org.local> <614DA72A-2900-4BA6-AA39-48B1E8C3CA98@python.org> Message-ID: <18380.49768.250393.961583@montanaro-dyndns-org.local> Barry> True, but it's just more moving parts to break, especially when I Barry> don't really feel a u/i buys you much[*]. Barry> [*] Skip knows me as a diehard XEmacser so that statement can Barry> pretty much define my standard answer to all such questions. :) So if we put together a "gui" which uses Johan Vroman's defunct Emacs Lisp forms package you'd be okay with that? Skip From ncoghlan at gmail.com Tue Mar 4 13:35:42 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 04 Mar 2008 22:35:42 +1000 Subject: [Python-Dev] Documentation for ability to execute zipfiles & directories Message-ID: <47CD421E.2010402@gmail.com> A few months ago, 2.6 & 3.0 gained the ability to execute zipfiles and directories containing a __main__.py file (see [1] for details). The idea is that a whole application can be bundled into a zipfile containing a __main__.py module in its root directory, and then passed directly to the interpreter for execution, with the zipfile being inserted as the first entry on sys.path to allow easy access to the rest of the application code. It is inspired by Java's JAR option, but not needing an explicit interpreter option makes it more shebang friendly on *nix systems (it can also be mapped more easily to the existing Python file type handling on Windows). The ability to also execute directories containing a __main__.py was something of a side effect of the implementation technique, but was also considered valuable as it makes it much easier to develop such bundled applications (using a directory most of the time, and then bundling into a single zipfile prior to release). The part I'm struggling with now is where to document the way this feature works. Currently, the only real documentation we have of the command line invocation is in section 2.1 of the tutorial, and the idea of packaging whole applications as zipfiles seems far too esoteric to be covering it there. It doesn't really seem to fit in section 6 (covering modules and packages) either. Do we need a new appendix to the tutorial which goes into detail about the CPython interpreter's command line options, environment variables and details on what can be executed? Cheers, Nick. [1] http://bugs.python.org/issue1739468 -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From p.f.moore at gmail.com Tue Mar 4 13:55:39 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 4 Mar 2008 12:55:39 +0000 Subject: [Python-Dev] Documentation for ability to execute zipfiles & directories In-Reply-To: <47CD421E.2010402@gmail.com> References: <47CD421E.2010402@gmail.com> Message-ID: <79990c6b0803040455q7ce211d0y78956b3a395c3bad@mail.gmail.com> On 04/03/2008, Nick Coghlan wrote: > Do we need a new appendix to the tutorial which goes into detail about > the CPython interpreter's command line options, environment variables > and details on what can be executed? There is a Python man page, which covers the command line usage. However, it's separate from the documentation, and it isn't bundled with the Windows installers - both of which are a real pain (for me, at least). I'd suggest taking the man page, adding the information about executing zip files and directories, and putting the whole lot into the formal documentation. The big problem is that there isn't really anywhere in the docs which is formally CPython-specific. My preference would be to put it in the language reference, as a new chapter (between the current chapters 1 and 2) called "Invoking the Python Interpreter". You could also make the manpage a new document, called "Invoking Python", but it's a bit small to warrant a ful document. An appendix to the Tutorial is OK, I guess, but personally I never think of looking at the tutorial (I've been using Python too long to feel that I need a tutorial any more, although the quality of my code probably says otherwise :-)) Paul. From phd at phd.pp.ru Tue Mar 4 14:22:07 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Tue, 4 Mar 2008 16:22:07 +0300 Subject: [Python-Dev] Documentation for ability to execute zipfiles & directories In-Reply-To: <47CD421E.2010402@gmail.com> References: <47CD421E.2010402@gmail.com> Message-ID: <20080304132207.GA3080@phd.pp.ru> On Tue, Mar 04, 2008 at 10:35:42PM +1000, Nick Coghlan wrote: > not needing an explicit interpreter option makes it more shebang friendly Sorry, I missed something here. How does one combine a zipfile with a shebang script?! Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From steve at holdenweb.com Tue Mar 4 14:58:57 2008 From: steve at holdenweb.com (Steve Holden) Date: Tue, 04 Mar 2008 08:58:57 -0500 Subject: [Python-Dev] Documentation for ability to execute zipfiles & directories In-Reply-To: <79990c6b0803040455q7ce211d0y78956b3a395c3bad@mail.gmail.com> References: <47CD421E.2010402@gmail.com> <79990c6b0803040455q7ce211d0y78956b3a395c3bad@mail.gmail.com> Message-ID: Paul Moore wrote: > On 04/03/2008, Nick Coghlan wrote: >> Do we need a new appendix to the tutorial which goes into detail about >> the CPython interpreter's command line options, environment variables >> and details on what can be executed? > > There is a Python man page, which covers the command line usage. > However, it's separate from the documentation, and it isn't bundled > with the Windows installers - both of which are a real pain (for me, > at least). > > I'd suggest taking the man page, adding the information about > executing zip files and directories, and putting the whole lot into > the formal documentation. > > The big problem is that there isn't really anywhere in the docs which > is formally CPython-specific. My preference would be to put it in the > language reference, as a new chapter (between the current chapters 1 > and 2) called "Invoking the Python Interpreter". > > You could also make the manpage a new document, called "Invoking > Python", but it's a bit small to warrant a ful document. > > An appendix to the Tutorial is OK, I guess, but personally I never > think of looking at the tutorial (I've been using Python too long to > feel that I need a tutorial any more, although the quality of my code > probably says otherwise :-)) > While I hesitate to suggest a change of such magnitude, there's something to recommend the old IBM mainframe approach of separating out "Principles of Operation" (which would be the reference manuals, in Python's case the Language and Library refs) from "Users' Guide" which contains the practical stuff you need to actually make use of a product. I've always found it rather counter-intuitive that you have to go to the Library Reference manual to find information about Python's built-in types, for example. I though the whole point of libraries was that they *aren't* built in, and represent baggage that should only be carried on necessary trips. And let's not get started on the import statement. I have just spent some time trying to work out how we might get rid of the embarrassing "XXX can't be bothered to spell this out right now ..." mess, and have come to the conclusion that a thorough review and a complete rewrite is the only thing that will do the topic justice. I can only hope that Brett plans to make a start on this as a part of his rework of the import code (and if you're reading, Brett, I'd like to help). It doesn't help my motivation that the import mechanism is about to change yet again, though I am happy that it will be more regular and easier to understand after the next change. I believe with 3.0 the biggest improvement we could make to the language for newcomers would be to reorganize our documentation so that things live in the places they belong rather than the place they landed and got stuck over time. Please note this isn't a rant, in the sense that I believe there are perfectly comprehensible reasons for how the docs got to be the shape they are. But I fear that we are possibly blinded to their inappropriate nature by our closeness to and familiarity with them, and I think a major effort to revise their structure (and to a lesser degree their content) could pay itself back many times over in increased user friendliness. Georg's recent revision of the technology puts us in a better position to prepare for this, but it would still be a major project. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From ncoghlan at gmail.com Tue Mar 4 15:14:04 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 05 Mar 2008 00:14:04 +1000 Subject: [Python-Dev] Documentation for ability to execute zipfiles & directories In-Reply-To: <20080304132207.GA3080@phd.pp.ru> References: <47CD421E.2010402@gmail.com> <20080304132207.GA3080@phd.pp.ru> Message-ID: <47CD592C.6040103@gmail.com> Oleg Broytmann wrote: > On Tue, Mar 04, 2008 at 10:35:42PM +1000, Nick Coghlan wrote: >> not needing an explicit interpreter option makes it more shebang friendly > > Sorry, I missed something here. How does one combine a zipfile with > a shebang script?! Very carefully ;) As a more helpful answer, the ZIP spec allows additional data to be included in the file before the ZIP header. A more common way of using this is to add a zip file on to the end of an ELF executable while still using normal zipfile utilities to read the data in the zip file section and ignore the executable part. It turns out you can actually use the same trick to prepend a shebang line like "/usr/bin/env python" and a newline character - the whole zip file is still a binary file, but that doesn't prevent the shell from reading that first line of text and handing the file over to Python for execution. The fact that this actually works was also news to me when the issue I linked in my previous post was first brought to my attention :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From phd at phd.pp.ru Tue Mar 4 15:40:51 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Tue, 4 Mar 2008 17:40:51 +0300 Subject: [Python-Dev] Documentation for ability to execute zipfiles & directories In-Reply-To: <47CD592C.6040103@gmail.com> References: <47CD421E.2010402@gmail.com> <20080304132207.GA3080@phd.pp.ru> <47CD592C.6040103@gmail.com> Message-ID: <20080304144051.GC3080@phd.pp.ru> On Wed, Mar 05, 2008 at 12:14:04AM +1000, Nick Coghlan wrote: > As a more helpful answer, the ZIP spec allows additional data to be > included in the file before the ZIP header. A more common way of using > this is to add a zip file on to the end of an ELF executable while still > using normal zipfile utilities to read the data in the zip file section > and ignore the executable part. > > It turns out you can actually use the same trick to prepend a shebang > line like "/usr/bin/env python" and a newline character That's what I thought, too. > - the whole zip > file is still a binary file, but that doesn't prevent the shell from > reading that first line of text and handing the file over to Python for > execution. Unix doesn't distinguish text and binary files. (-: > The fact that this actually works was also news to me when the issue I > linked in my previous post was first brought to my attention :) So it really works? Amazing! Thank you! Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From tnelson at onresolve.com Tue Mar 4 15:58:36 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Tue, 4 Mar 2008 06:58:36 -0800 Subject: [Python-Dev] Windows buildbot test_bsddb3 problems (was RE: Buildbots for trunk are all red) Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDE2@EXMBX04.exchhosting.com> Spent some time on my buildbot (x86 2k8 trunk) this morning trying to track down why test_bsddb3 is failing (trunk with db-4.4.20). The first test that fails is this: test01_GetsAndPuts (bsddb.test.test_basics.BasicBTreeWithEnvTestCase) ... ERROR That's slightly misleading though as the test runs fine -- the actual exception is being thrown in test_basics.BasicTestCase.tearDown() when os.remove() is called against the first db file (i.e. '__db.001'): WindowsError: [Error 5] Access is denied: 'c:\users\trent~1.nel\appdata\local\temp\2\db_home2808\__db.001 This isn't surprising, given that the python_d.exe process still seems to have __db.001, __db.002 and __db.003 open at the time os.remove() is called. The aforementioned tearDown() method looks like this: def tearDown(self): self.d.close() if self.env is not None: test_support.rmtree(self.homeDir) self.env.close() ## Make a new DBEnv to remove the env files from the home dir. ## (It can't be done while the env is open, nor after it has been ## closed, so we make a new one to do it.) #e = db.DBEnv() #e.remove(self.homeDir) #os.remove(os.path.join(self.homeDir, self.filename)) else: os.remove(self.filename) If I switch the order of statements such that self.env.close() is called before test_suppot.rmtree(self.homeDir), this particular test and a host of others that were also failing now pass (a runtime abort is no longer raised by the CRT half way into the tests either). (Note that the order was switched to that shown above by Neal in r61052 on Feb 24th, which is when these issues started occurring.) That said, there are still a lot more test failures down the track once this change has been made, either via the access denied WindowsError, or a DBInvalidArgError, e.g.: ERROR: test02_WithSource (bsddb.test.test_recno.SimpleRecnoTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "S:\src\svn\svn.python.org\projects\python\trunk\lib\bsddb\test\test_recno.py", line 33, in tearDown test_support.rmtree(self.homeDir) File "S:\src\svn\svn.python.org\projects\python\trunk\lib\test\test_support.py", line 70, in rmtree shutil.rmtree(path) File "S:\src\svn\svn.python.org\projects\python\trunk\lib\shutil.py", line 184, in rmtree onerror(os.remove, fullname, sys.exc_info()) File "S:\src\svn\svn.python.org\projects\python\trunk\lib\shutil.py", line 182, in rmtree os.remove(fullname) WindowsError: [Error 5] Access is denied: 'c:\\users\\trent~1.nel\\appdata\\local\\temp\\2\\db_home4656\\tmp04_knk' ====================================================================== ERROR: test01_1WriterMultiReaders (bsddb.test.test_thread.BTreeConcurrentDataStore) ---------------------------------------------------------------------- Traceback (most recent call last): File "S:\src\svn\svn.python.org\projects\python\trunk\lib\bsddb\test\test_thread.py", line 62, in setUp self.env.open(homeDir, self.envflags | db.DB_CREATE) DBInvalidArgError: (22, 'Invalid argument -- configured environment flags incompatible with existing environment') The DBInvalidArgError exception is only raised after a previous WindowsError is encountered, so I assume it's a side-effect of the tearDown() method not cleaning the environment correctly. It seems this comment in tearDown() is quite pertinent to our situation: ## Make a new DBEnv to remove the env files from the home dir. ## (It can't be done while the env is open, nor after it has been ## closed, so we make a new one to do it.) #e = db.DBEnv() #e.remove(self.homeDir) #os.remove(os.path.join(self.homeDir, self.filename)) Not sure why this is commented out -- quick search of svn history indicates it's been like that for at least the last year and a half. Will have some more time this evening to spend on this, however, work calls at the moment. Regards, Trent. ________________________________________ From: python-dev-bounces+tnelson=onresolve.com at python.org [python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of Facundo Batista [facundobatista at gmail.com] Sent: 26 February 2008 06:22 To: Thomas Herv? Cc: python-dev at python.org Subject: Re: [Python-Dev] Buildbots for trunk are all red 2008/2/25, Thomas Herv? : > I've worked on that problem during the bug day. I've open a ticket with > a patch at http://bugs.python.org/issue2168. Most of the buildbots are green now!!! Thank you all! This community is as awesome as Python itself, ;) Three remains in red, though: - Alpha Tru64: test_smtplib.py is flaky, and _ssl.c is not compiled correctly. Neil is hunting this, I think. - X86 XP-3: seems to crash after test_bsddb3.py. - X86 XP-4: idem. For this two, how can be tried if the bsddb lib in those windows is correctly installed? Thanks again. -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/tnelson%40onresolve.com From pje at telecommunity.com Tue Mar 4 17:06:59 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 04 Mar 2008 11:06:59 -0500 Subject: [Python-Dev] Documentation for ability to execute zipfiles & directories In-Reply-To: <20080304144051.GC3080@phd.pp.ru> References: <47CD421E.2010402@gmail.com> <20080304132207.GA3080@phd.pp.ru> <47CD592C.6040103@gmail.com> <20080304144051.GC3080@phd.pp.ru> Message-ID: <20080304160643.4DA343A40A5@sparrow.telecommunity.com> At 05:40 PM 3/4/2008 +0300, Oleg Broytmann wrote: >On Wed, Mar 05, 2008 at 12:14:04AM +1000, Nick Coghlan wrote: > > As a more helpful answer, the ZIP spec allows additional data to be > > included in the file before the ZIP header. A more common way of using > > this is to add a zip file on to the end of an ELF executable while still > > using normal zipfile utilities to read the data in the zip file section > > and ignore the executable part. > > > > It turns out you can actually use the same trick to prepend a shebang > > line like "/usr/bin/env python" and a newline character > > That's what I thought, too. > > > - the whole zip > > file is still a binary file, but that doesn't prevent the shell from > > reading that first line of text and handing the file over to Python for > > execution. > > Unix doesn't distinguish text and binary files. (-: > > > The fact that this actually works was also news to me when the issue I > > linked in my previous post was first brought to my attention :) > > So it really works? Amazing! Setuptools has been distributed this way for some time: http://pypi.python.org/pypi/setuptools#cygwin-mac-os-x-linux-other It actually contains an entire shell script prefix that launches Python and invokes an entry point inside the egg. With the new interpreter capability, this would've been a *lot* simpler to implement. From nnorwitz at gmail.com Tue Mar 4 17:24:53 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 4 Mar 2008 08:24:53 -0800 Subject: [Python-Dev] Windows buildbot test_bsddb3 problems (was RE: Buildbots for trunk are all red) In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDE2@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDE2@EXMBX04.exchhosting.com> Message-ID: Trent, thanks for working on the buildbot. I fixed the first case you mentioned in r61233 wrt removing the directory before closing the file. It would be great if you could submit a patch when you are able to fix the remaining problems. Cheers, n On Tue, Mar 4, 2008 at 6:58 AM, Trent Nelson wrote: > Spent some time on my buildbot (x86 2k8 trunk) this morning trying to track down why test_bsddb3 is failing (trunk with db-4.4.20). The first test that fails is this: > > test01_GetsAndPuts (bsddb.test.test_basics.BasicBTreeWithEnvTestCase) ... ERROR > > That's slightly misleading though as the test runs fine -- the actual exception is being thrown in test_basics.BasicTestCase.tearDown() when os.remove() is called against the first db file (i.e. '__db.001'): > > WindowsError: [Error 5] Access is denied: 'c:\users\trent~1.nel\appdata\local\temp\2\db_home2808\__db.001 > > This isn't surprising, given that the python_d.exe process still seems to have __db.001, __db.002 and __db.003 open at the time os.remove() is called. The aforementioned tearDown() method looks like this: > > def tearDown(self): > self.d.close() > if self.env is not None: > test_support.rmtree(self.homeDir) > self.env.close() > ## Make a new DBEnv to remove the env files from the home dir. > ## (It can't be done while the env is open, nor after it has been > ## closed, so we make a new one to do it.) > #e = db.DBEnv() > #e.remove(self.homeDir) > #os.remove(os.path.join(self.homeDir, self.filename)) > else: > os.remove(self.filename) > > If I switch the order of statements such that self.env.close() is called before test_suppot.rmtree(self.homeDir), this particular test and a host of others that were also failing now pass (a runtime abort is no longer raised by the CRT half way into the tests either). (Note that the order was switched to that shown above by Neal in r61052 on Feb 24th, which is when these issues started occurring.) > > That said, there are still a lot more test failures down the track once this change has been made, either via the access denied WindowsError, or a DBInvalidArgError, e.g.: > > ERROR: test02_WithSource (bsddb.test.test_recno.SimpleRecnoTestCase) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "S:\src\svn\svn.python.org\projects\python\trunk\lib\bsddb\test\test_recno.py", line 33, in tearDown > test_support.rmtree(self.homeDir) > File "S:\src\svn\svn.python.org\projects\python\trunk\lib\test\test_support.py", line 70, in rmtree > shutil.rmtree(path) > File "S:\src\svn\svn.python.org\projects\python\trunk\lib\shutil.py", line 184, in rmtree > onerror(os.remove, fullname, sys.exc_info()) > File "S:\src\svn\svn.python.org\projects\python\trunk\lib\shutil.py", line 182, in rmtree > os.remove(fullname) > WindowsError: [Error 5] Access is denied: 'c:\\users\\trent~1.nel\\appdata\\local\\temp\\2\\db_home4656\\tmp04_knk' > ====================================================================== > ERROR: test01_1WriterMultiReaders (bsddb.test.test_thread.BTreeConcurrentDataStore) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "S:\src\svn\svn.python.org\projects\python\trunk\lib\bsddb\test\test_thread.py", line 62, in setUp > self.env.open(homeDir, self.envflags | db.DB_CREATE) > DBInvalidArgError: (22, 'Invalid argument -- configured environment flags incompatible with existing environment') > > The DBInvalidArgError exception is only raised after a previous WindowsError is encountered, so I assume it's a side-effect of the tearDown() method not cleaning the environment correctly. > > It seems this comment in tearDown() is quite pertinent to our situation: > ## Make a new DBEnv to remove the env files from the home dir. > ## (It can't be done while the env is open, nor after it has been > ## closed, so we make a new one to do it.) > #e = db.DBEnv() > #e.remove(self.homeDir) > #os.remove(os.path.join(self.homeDir, self.filename)) > > Not sure why this is commented out -- quick search of svn history indicates it's been like that for at least the last year and a half. > > Will have some more time this evening to spend on this, however, work calls at the moment. > > Regards, > > Trent. > > ________________________________________ > From: python-dev-bounces+tnelson=onresolve.com at python.org [python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of Facundo Batista [facundobatista at gmail.com] > Sent: 26 February 2008 06:22 > To: Thomas Herv? > Cc: python-dev at python.org > Subject: Re: [Python-Dev] Buildbots for trunk are all red > > 2008/2/25, Thomas Herv? : > > > I've worked on that problem during the bug day. I've open a ticket with > > a patch at http://bugs.python.org/issue2168. > > Most of the buildbots are green now!!! > > Thank you all! This community is as awesome as Python itself, ;) > > Three remains in red, though: > > - Alpha Tru64: test_smtplib.py is flaky, and _ssl.c is not compiled > correctly. Neil is hunting this, I think. > > - X86 XP-3: seems to crash after test_bsddb3.py. > > - X86 XP-4: idem. For this two, how can be tried if the bsddb lib in > those windows is correctly installed? > > Thanks again. > > -- > . Facundo > > Blog: http://www.taniquetil.com.ar/plog/ > PyAr: http://www.python.org/ar/ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/tnelson%40onresolve.com > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > From facundobatista at gmail.com Tue Mar 4 17:51:20 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Tue, 4 Mar 2008 14:51:20 -0200 Subject: [Python-Dev] Windows buildbot test_bsddb3 problems (was RE: Buildbots for trunk are all red) In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDE2@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDE2@EXMBX04.exchhosting.com> Message-ID: 2008/3/4, Trent Nelson : > Spent some time on my buildbot (x86 2k8 trunk) this morning trying to track down why test_bsddb3 is failing (trunk with db-4.4.20). The first test that fails is this: Thank you very much!! Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From amk at amk.ca Tue Mar 4 18:36:05 2008 From: amk at amk.ca (A.M. Kuchling) Date: Tue, 4 Mar 2008 12:36:05 -0500 Subject: [Python-Dev] Documentation for ability to execute zipfiles & directories In-Reply-To: References: <47CD421E.2010402@gmail.com> <79990c6b0803040455q7ce211d0y78956b3a395c3bad@mail.gmail.com> Message-ID: <20080304173605.GA9119@amk-desktop.matrixgroup.net> On Tue, Mar 04, 2008 at 08:58:57AM -0500, Steve Holden wrote: > While I hesitate to suggest a change of such magnitude, there's > something to recommend the old IBM mainframe approach of separating out > "Principles of Operation" (which would be the reference manuals, in > Python's case the Language and Library refs) from "Users' Guide" which > contains the practical stuff you need to actually make use of a product. Good suggestion. Using the debugger and profiler could also be covered in the User's Guide. Would splitting up the docs make them more useful for IronPython/Jython? For example, Jython could eventually take the 2.6 language docs as-is, but modify the library reference to remove unsupported modules and add Jython-specific ones. --amk From g.brandl at gmx.net Tue Mar 4 20:21:52 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 04 Mar 2008 20:21:52 +0100 Subject: [Python-Dev] Documentation for ability to execute zipfiles & directories In-Reply-To: References: <47CD421E.2010402@gmail.com> <79990c6b0803040455q7ce211d0y78956b3a395c3bad@mail.gmail.com> Message-ID: Steve Holden schrieb: > Paul Moore wrote: >> On 04/03/2008, Nick Coghlan wrote: >>> Do we need a new appendix to the tutorial which goes into detail about >>> the CPython interpreter's command line options, environment variables >>> and details on what can be executed? >> >> There is a Python man page, which covers the command line usage. >> However, it's separate from the documentation, and it isn't bundled >> with the Windows installers - both of which are a real pain (for me, >> at least). >> >> I'd suggest taking the man page, adding the information about >> executing zip files and directories, and putting the whole lot into >> the formal documentation. Look no further: http://docs.python.org/dev/using/cmdline.html There's even more platform-specific stuff at http://docs.python.org/dev/using/. >> The big problem is that there isn't really anywhere in the docs which >> is formally CPython-specific. My preference would be to put it in the >> language reference, as a new chapter (between the current chapters 1 >> and 2) called "Invoking the Python Interpreter". The "Using Python" documentation section could be marked as CPython specific very well. >> You could also make the manpage a new document, called "Invoking >> Python", but it's a bit small to warrant a ful document. >> >> An appendix to the Tutorial is OK, I guess, but personally I never >> think of looking at the tutorial (I've been using Python too long to >> feel that I need a tutorial any more, although the quality of my code >> probably says otherwise :-)) >> > While I hesitate to suggest a change of such magnitude, there's > something to recommend the old IBM mainframe approach of separating out > "Principles of Operation" (which would be the reference manuals, in > Python's case the Language and Library refs) from "Users' Guide" which > contains the practical stuff you need to actually make use of a product. > > I've always found it rather counter-intuitive that you have to go to the > Library Reference manual to find information about Python's built-in > types, for example. I though the whole point of libraries was that they > *aren't* built in, and represent baggage that should only be carried on > necessary trips. You speak my mind. For ages I've wanted to put the builtins together with the language reference into a new document called "Python Core Language". I've just never had the time to draft a serious proposal. > I believe with 3.0 the biggest improvement we could make to the language > for newcomers would be to reorganize our documentation so that things > live in the places they belong rather than the place they landed and got > stuck over time. I fully agree. Georg From mwm at mired.org Mon Mar 3 05:07:08 2008 From: mwm at mired.org (Mike Meyer) Date: Sun, 2 Mar 2008 23:07:08 -0500 Subject: [Python-Dev] Python XML Validator In-Reply-To: References: Message-ID: <20080302230708.260fa4a9@bhuda.mired.org> On Thu, 28 Feb 2008 23:42:49 +0000 (UTC) Medhat Gayed wrote: > lxml is good but not written in python and difficult to install and didn't work > on MacOS X. lxml is built on top of libxml2/libxslt, which are bundled with most Unix-like OS's (including Mac OS X), or available in their package systems. Trying to install it from the repository is a PITA, because it uses both the easyinstall and Pyrex (later Cython) packages - which aren't bundled with anything. On the other hand, if it's in the package system (I no longer have macports installed anywhere, but believe it was there at one time), that solves all those problems. I believe they've excised the easyinstall source dependencies, though. Using lxml on OS X Tiger was problematical, because the versions of python, libxml2 and libxslt provided with Tiger were pretty much all older than lxml supported; I built python from macports, including current versions of libxml2, libxslt and lxml, and everything worked with no problems. (I later stopped working with this on the Mac because I need cx_Oracle as well, which doesn't exist for intel macs). On Leopard, Python is up to date, but libxml/libxslt seems a bit behind for lxml 2.0.x (no schematron support being the obvious problem). I went back to the 1.3.6 source tarball (which is what I'm using everywhere anyway), and "python setup.py install" worked like a charm. (So it looks the easyinstall dependency is gone). Of course, the real issue here is that, while Python may come with "batteries included" you only get the common sizes like A, C and D. If you need a B cell, you're on your own. In XML land, validation is one such case. Me, I'd like complete xpath support, and xslt as well. But this happens with other subsystems, like doing client-side SSL support, but not server-side (at least, not as of 2.4; I haven't checked 2.5). If you just want an xml module in the standard library that's more complete, I'd vote for the source distribution of lxml, as that's C + Python and built on top of commonly available libraries. The real issue would be making current lxml work with the "outdated" versions of those libraries found in current OS distributions. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. From tnelson at onresolve.com Tue Mar 4 21:36:04 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Tue, 4 Mar 2008 12:36:04 -0800 Subject: [Python-Dev] Windows buildbot test_bsddb3 problems (was RE: Buildbots for trunk are all red) In-Reply-To: References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDE2@EXMBX04.exchhosting.com>, Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDE5@EXMBX04.exchhosting.com> > Trent, thanks for working on the buildbot. I fixed the first case you > mentioned in r61233 wrt removing the directory before closing the > file. It would be great if you could submit a patch when you are able > to fix the remaining problems. Nod, found a few more things now that test_bsddb3 isn't causing a CRT abortion. tmpfile() needs to be reworked on Windows, see http://bugs.python.org/issue2232. Going to spend some more time on it this evening. I'm determined to see a flippin' green build/test status for my slave if it kills me :> Trent. From stephen at xemacs.org Tue Mar 4 23:13:28 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 05 Mar 2008 07:13:28 +0900 Subject: [Python-Dev] Documentation reorganization [was: ... for ability to execute zipfiles & directories] In-Reply-To: References: <47CD421E.2010402@gmail.com> <79990c6b0803040455q7ce211d0y78956b3a395c3bad@mail.gmail.com> Message-ID: <87mypew31z.fsf@uwakimon.sk.tsukuba.ac.jp> Georg Brandl writes: > You speak my mind. For ages I've wanted to put the builtins together with > the language reference into a new document called "Python Core Language". > I've just never had the time to draft a serious proposal. I think that combination is reasonable, but I would like to see the clear division between the language (ie, the syntax) and the built-in functionality maintained. I'm not sure I like the proposed title for that reason. From bkline at rksystems.com Tue Mar 4 23:07:51 2008 From: bkline at rksystems.com (Bob Kline) Date: Tue, 04 Mar 2008 17:07:51 -0500 Subject: [Python-Dev] Compiler used to build Python for Windows Message-ID: <47CDC837.8000104@rksystems.com> I know this is a topic which has been discussed before (more than once). I'm just adding one more data point. Python.org currently uses VS2003's compiler for building the distributed Windows binaries for Python. Unfortunately, there's a nasty bug in the runtime libraries that support this compiler which causes Python to give the wrong answer when you ask what time it is in most US time zones for four weeks of the year (daylight saving time is observed in that country for four more weeks than those Microsoft's libraries think it is). Even more unfortunate, the patch Microsoft offers is not for the redistributable libraries directly, but for Visual Studio. Worse, Microsoft advises against applying the patch at all, implying that insufficient testing has taken place, urging instead that customers wait for a service pack for Visual Studio which corrects the bug. We've been waiting more than a year, without any indication from MS that this service pack will ever materialize, and it doesn't apply to non-development machines anyway. Every indication seems to point to effective abandonment by Microsoft of those who still use this version of the compiler. http://support.microsoft.com/kb/932299 We don't have conclusive evidence that later versions of Visual Studio are not affected by this bug, but the only KB article we have found for the bug is specific to C runtime library for the 2003 compiler. Any possibility of revisiting this question (upgrading to a more recent compiler for Windows builds of Python)? -- Bob Kline http://www.rksystems.com mailto:bkline at rksystems.com From rhamph at gmail.com Tue Mar 4 23:24:13 2008 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 4 Mar 2008 15:24:13 -0700 Subject: [Python-Dev] Documentation reorganization [was: ... for ability to execute zipfiles & directories] In-Reply-To: <87mypew31z.fsf@uwakimon.sk.tsukuba.ac.jp> References: <47CD421E.2010402@gmail.com> <79990c6b0803040455q7ce211d0y78956b3a395c3bad@mail.gmail.com> <87mypew31z.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Tue, Mar 4, 2008 at 3:13 PM, Stephen J. Turnbull wrote: > Georg Brandl writes: > > You speak my mind. For ages I've wanted to put the builtins together with > > the language reference into a new document called "Python Core Language". > > I've just never had the time to draft a serious proposal. > > I think that combination is reasonable, but I would like to see the > clear division between the language (ie, the syntax) and the built-in > functionality maintained. I'm not sure I like the proposed title for > that reason. Such a division would make it unnecessarily hard to find documentation on True, False, None, etc. They've become keywords for pragmatic purposes (to prevent accidental modification), not because we think they ideally should be syntax instead of builtins. -- Adam Olsen, aka Rhamphoryncus From lists at cheimes.de Tue Mar 4 23:47:31 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 04 Mar 2008 23:47:31 +0100 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: <47CDC837.8000104@rksystems.com> References: <47CDC837.8000104@rksystems.com> Message-ID: <47CDD183.2040201@cheimes.de> Bob Kline wrote: > Any possibility of revisiting this question (upgrading to a more recent > compiler for Windows builds of Python)? The latest alphas of Python 2.6 and 3.0 are build with VS 2088. I've spent some time to get the new build system ready for Python 3.0a2. Is VS 2008 recent enought for you? :] Christian From martin at v.loewis.de Tue Mar 4 23:53:18 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 04 Mar 2008 23:53:18 +0100 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: <47CDC837.8000104@rksystems.com> References: <47CDC837.8000104@rksystems.com> Message-ID: <47CDD2DE.7050405@v.loewis.de> > Any possibility of revisiting this question (upgrading to a more recent > compiler for Windows builds of Python)? Python 2.6 is built with Visual Studio 2008. Regards, Martin From bkline at rksystems.com Wed Mar 5 00:01:43 2008 From: bkline at rksystems.com (Bob Kline) Date: Tue, 04 Mar 2008 18:01:43 -0500 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: <47CDD183.2040201@cheimes.de> References: <47CDC837.8000104@rksystems.com> <47CDD183.2040201@cheimes.de> Message-ID: <47CDD4D7.4050905@rksystems.com> Christian Heimes wrote: > Bob Kline wrote: > >> Any possibility of revisiting this question (upgrading to a more recent >> compiler for Windows builds of Python)? >> > > The latest alphas of Python 2.6 and 3.0 are build with VS 2088. I've > spent some time to get the new build system ready for Python 3.0a2. > > Is VS 2008 recent enought for you? :] > Yes, thanks! I would hope Microsoft has fixed that bug by now. :-) -- Bob Kline http://www.rksystems.com mailto:bkline at rksystems.com From bkline at rksystems.com Wed Mar 5 00:16:04 2008 From: bkline at rksystems.com (Bob Kline) Date: Tue, 04 Mar 2008 18:16:04 -0500 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: <47CDD4D7.4050905@rksystems.com> References: <47CDC837.8000104@rksystems.com> <47CDD183.2040201@cheimes.de> <47CDD4D7.4050905@rksystems.com> Message-ID: <47CDD834.3040405@rksystems.com> Bob Kline wrote: > Christian Heimes wrote: > >> Is VS 2008 recent enought for you? :] >> >> > > Yes, thanks! I would hope Microsoft has fixed that bug by now. :-) > And yes, indeed, the bug is gone in Python 2.6. -- Bob Kline http://www.rksystems.com mailto:bkline at rksystems.com From greg.ewing at canterbury.ac.nz Wed Mar 5 00:42:57 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 05 Mar 2008 12:42:57 +1300 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: <47CDD183.2040201@cheimes.de> References: <47CDC837.8000104@rksystems.com> <47CDD183.2040201@cheimes.de> Message-ID: <47CDDE81.4090801@canterbury.ac.nz> Christian Heimes wrote: > The latest alphas of Python 2.6 and 3.0 are build with VS 2088. ^^^^ Wow, that must be a very, very pre-alpha release... Or has someone at Redmond stolen Guido's time machine? -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Wed Mar 5 00:45:06 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 05 Mar 2008 12:45:06 +1300 Subject: [Python-Dev] Documentation reorganization [was: ... for ability to execute zipfiles & directories] In-Reply-To: References: <47CD421E.2010402@gmail.com> <79990c6b0803040455q7ce211d0y78956b3a395c3bad@mail.gmail.com> <87mypew31z.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <47CDDF02.7060800@canterbury.ac.nz> Adam Olsen wrote: > Such a division would make it unnecessarily hard to find documentation > on True, False, None, etc. They've become keywords for pragmatic > purposes (to prevent accidental modification), not because we think > they ideally should be syntax instead of builtins. Maybe the solution is to rename the Library Reference to the Class and Module Reference or something like that. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From jcea at argo.es Wed Mar 5 00:46:31 2008 From: jcea at argo.es (Jesus Cea) Date: Wed, 05 Mar 2008 00:46:31 +0100 Subject: [Python-Dev] BSDDB3 (was: Re: Buildbots for trunk are all red) In-Reply-To: <20080227141322.GA5871@amk-desktop.matrixgroup.net> References: <47C32D2D.1030206@free.fr> <47C4813F.2080403@v.loewis.de> <52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com> <20080227141322.GA5871@amk-desktop.matrixgroup.net> Message-ID: <47CDDF57.1020006@argo.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 A.M. Kuchling wrote: | Doing a code search finds a fair number of users of the module: Zope's | BDBStorage, Mailman 2.x's archiver, 4Suite, PyTone, OuterSpace, | Chandler, BioPython. I'm pybsddb maintainer since late January. Maintainer transfering ownership is going a bit slow, since both Greg and I are busy guys. Also, I have no commit access to python svn, and I'm barely familiar with practices here (just reading PEP are not enough); managing by proxy is a bit... difficult. That said, it is my aim to keep bsddb in stdlib, providing a stable and featureful module. I think keeping bsddb development inside python svn is not appropiate. Currently (I could change idea), my approach will be keeping pybssdb as a separate project and sync with python SVN from time to time. Mainly to take advantage of buildbot architecture and, of course, to be able to release python with current bindings. Since I have no python commit access, this seems a sensible approach. And I could do frequent pybssdb releases (let say, every couple of months) without waiting for a full python release (current approach). I see a lot of complains about bsddb3 in stdlib, spitting buildbot red status, and such. I want to help there. Just remember that I have no direct control over python svn and, although I spend last 28 years breathing computer programming (python since 1998), I'm a newbie in python-dev policies and procedures. PS: I have tried to sign the Python Contributor Agreement, but I am not sure about current pybsddb license terms. Help welcomed. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBR83fUZlgi5GaxT1NAQIx5AP/VR6HD3HBGbYBl+OeMkL+cHUFgBljIZgl T/N1T2CnertBc6EqEJ2ejEfMuVJF/icCyzwmldXbQWsVjd9XFq7gVGe7h9r7RTc5 DaiBrZk2ZBVEXk5Vp+QcWiyObMU4dYYF5JRyjRl4hCdKPRXo1TLTkQchoc945ubc VEgLgCHROcE= =1I5Z -----END PGP SIGNATURE----- From jcea at argo.es Wed Mar 5 00:49:29 2008 From: jcea at argo.es (Jesus Cea) Date: Wed, 05 Mar 2008 00:49:29 +0100 Subject: [Python-Dev] Buildbots for trunk are all red In-Reply-To: <47C712F6.9040809@cheimes.de> References: <47C32D2D.1030206@free.fr> <47C4813F.2080403@v.loewis.de> <52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com> <20080227141322.GA5871@amk-desktop.matrixgroup.net> <47C712F6.9040809@cheimes.de> Message-ID: <47CDE009.7010300@argo.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Christian Heimes wrote: | Oh yeah ... ZODB4 and BDBStorage ... a dark chapter starting with high | hopes and ending in tragedy ... Several projects like Zope and | Subversion worked hard on a a Berkeley DB backend but in the end all | projects had the same fate. | | A few years ago in G?teborg/SE during lunch Jim explained me the reasons | for the cancellation. As far as I remember the conversation he used some | words I dare not to repeat in public. Some kids may read the Python dev | list. :) I would like to know. Private email is fine to me :-). - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBR83gCZlgi5GaxT1NAQJXKgQAkq6E9zmLRPPINh0rfDy/cOQTY74IS8BP SnBbx2/317nTxe7aNMY4HUAMptA5cae5LKOW+b7+WuHn+F2DMmfbKxvq4PHop2b3 zx9Q3lv4qk6p0ZYL6KLuis5sYivpejuxbRgwFI2m8gkdFsRh5a0MgmVG0b4768TW 6c9ELDY/Xr0= =7pRA -----END PGP SIGNATURE----- From nad at acm.org Wed Mar 5 00:44:32 2008 From: nad at acm.org (Ned Deily) Date: Tue, 04 Mar 2008 15:44:32 -0800 Subject: [Python-Dev] Python XML Validator References: <20080302230708.260fa4a9@bhuda.mired.org> Message-ID: In article <20080302230708.260fa4a9 at bhuda.mired.org>, Mike Meyer wrote: > On Thu, 28 Feb 2008 23:42:49 +0000 (UTC) Medhat Gayed > wrote: > > lxml is good but not written in python and difficult to install and didn't > > work > > on MacOS X. > lxml is built on top of libxml2/libxslt, which are bundled with most > Unix-like OS's (including Mac OS X), or available in their package > systems. Trying to install it from the repository is a PITA, because > it uses both the easyinstall and Pyrex (later Cython) packages - which > aren't bundled with anything. On the other hand, if it's in the > package system (I no longer have macports installed anywhere, but > believe it was there at one time), that solves all those problems. I > believe they've excised the easyinstall source dependencies, though. [...] > If you just want an xml module in the standard library that's more > complete, I'd vote for the source distribution of lxml, as that's C + > Python and built on top of commonly available libraries. The real > issue would be making current lxml work with the "outdated" versions > of those libraries found in current OS distributions. I'm not sure what you perceive to be the problems with easy_install on OSX; I find it makes life *much* simpler for managing python packages. Be that as it may, since the release of lxml 2.0, the project has updated the lxml website with useful information about source installations and, in particular, OSX source installations: IIRC, here's what worked for me on Leopard (10.5.2) using the python.org 2.5.2, though it should work fine with the Apple-supplied 2.5.1: 1. Download and build source tarballs of recent libxml2 (at the moment 2.6.31 is the latest, OSX 10.5.2 has 2.6.16) and libxlst (1.1.22 vs 1.1.12) from xmlsoft.org and then install them to /usr/local/. (Don't bother with the python bindings unless they're needed for some other package.) 2. As noted on the lxml source page, Cython is not needed to install lxml and can complicate matters so, as suggested, if you have Cython (or Pyrex, for that matter) installed in the Python path you're going to install to, temporarily remove it(/them). 3. Using the lxml 2.0.x source tarball: /path/to/python setup.py install \ --with-xslt-config=/usr/local/bin/xslt-config 4. Verify installation: $ which python /usr/local/bin/python $ python Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53) [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from lxml import etree >>> print etree.LXML_VERSION (2, 0, 2, 0) >>> print etree.LIBXML_VERSION (2, 6, 31) >>> print etree.LIBXML_COMPILED_VERSION (2, 6, 31) >>> print etree.LIBXSLT_VERSION (1, 1, 22) >>> print etree.LIBXSLT_COMPILED_VERSION (1, 1, 22) >>> Clearly there are other ways to do this but HTH. -- Ned Deily, nad at acm.org From jcea at argo.es Wed Mar 5 00:54:20 2008 From: jcea at argo.es (Jesus Cea) Date: Wed, 05 Mar 2008 00:54:20 +0100 Subject: [Python-Dev] Buildbots for trunk are all red In-Reply-To: References: <47C32D2D.1030206@free.fr> <47C4813F.2080403@v.loewis.de> <52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com> <20080227141322.GA5871@amk-desktop.matrixgroup.net> <47C712F6.9040809@cheimes.de> Message-ID: <47CDE12C.1060402@argo.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Guido van Rossum wrote: | Correction: the subversion BerkeleyDB backend is still very much alive | and kicking. There were some early issues (they did things that | SleepyCat told them not to do :-) but it was corrected and it's still | working fine for several large users. I use BerkeleyDB everyday in a lot of application critical environments, like mailbox storage and such, specially combined with Durus persistence engine. I have medical clients with hundred of terabytes of data under Durus+BerkeleyDB, happy and glad to have met me :). PS: Yes I tried also ZODB+BerkeleyDB. It was ugly and unusable (sigh!), but I don't know why you suppose BerkeleyDB was the one to blame. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBR83hLJlgi5GaxT1NAQJJ0wQAhjmGh5CWxaiEEkduXuJJFTl1kr4mg8Rr i5Fy+xuquYXScGlv8p0O2G7B+bqji+46cuhk6htuFfXpfIBXBW7QSTBDS4KlsD2+ 8dHxZvKg7irH1rISD6mOfeeAkqFWcVFOBgLOGbwU1EPD+Jfqppaa2dPZ0XemHQon ve3ZAC6wWyg= =xecT -----END PGP SIGNATURE----- From greg.ewing at canterbury.ac.nz Wed Mar 5 00:56:04 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 05 Mar 2008 12:56:04 +1300 Subject: [Python-Dev] Python 2.6 and 3.0 ...and applink.c? In-Reply-To: <9418DB6C0B9D434190E54A78E931C3D1036F8BED@XCH-NW-7V1.nw.nos.boeing.com> References: <47C3DAEB.1050102@cheimes.de> <9418DB6C0B9D434190E54A78E931C3D1036F8BE7@XCH-NW-7V1.nw.nos.boeing.com> <9418DB6C0B9D434190E54A78E931C3D1036F8BE8@XCH-NW-7V1.nw.nos.boeing.com> <08Feb26.101423pst.58696@synergy1.parc.xerox.com> <47C459B1.60806@cheimes.de> <47C4801C.2000105@v.loewis.de> <9418DB6C0B9D434190E54A78E931C3D1036F8BED@XCH-NW-7V1.nw.nos.boeing.com> Message-ID: <47CDE194.4070600@canterbury.ac.nz> Bugbee, Larry wrote: > Is there a reason why Python.exe cannot be built using gcc instead > of Visual Studio? > > It seems building everything with gcc would get around the problem > of having to match VS versions. As I understand it, the problem isn't the compiler so much as the stdio library being used. As long as there is no single standard C library used by everyone for all Windows applications, the problem will exist in one form or another whichever compiler is used. If the standard Python were build with gcc using the GNU libc, everyone would be required to compile extensions against *that* library instead. If it were compiled with gcc but using one of the Microsoft libraries, the same situation would exist as before. The entire mess is all Microsoft's fault for not picking one version of libc and sticking to it. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From jcea at argo.es Wed Mar 5 00:59:40 2008 From: jcea at argo.es (Jesus Cea) Date: Wed, 05 Mar 2008 00:59:40 +0100 Subject: [Python-Dev] Buildbots for trunk are all red In-Reply-To: <47C71BA3.6030107@cheimes.de> References: <47C32D2D.1030206@free.fr> <47C4813F.2080403@v.loewis.de> <52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com> <20080227141322.GA5871@amk-desktop.matrixgroup.net> <47C712F6.9040809@cheimes.de> <20080228201940.GA23104@phd.pp.ru> <47C71BA3.6030107@cheimes.de> Message-ID: <47CDE26C.1050003@argo.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Christian Heimes wrote: | More than 5 years ago Zope Corp was working on a Berkeley DB backend for | ZODB. It was more of a marketing decision to show large companies that | ZODB is using a well known database instead of a self made one. The | project failed due to problems with the Berkeley db. I vaguely remember | something with transactions and speed ... BerkeleyDB transaction performance, like any other ACID database, is limited to disk performance (unless you have a non volatile ram, of course). Using ext2 and such, you are happy if you get 30-50 transactions per second. Using nice Solaris ZFS, I do a transaction per rotation; that is, 120 transactions per second with a consumer level 7200RPM disk. PS: Until Oracle takeover, BerkeleyDB was the heart of MySQL transactional engine. I don't like mysql, but not for its use of BerkeleyDB :-). - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBR83ia5lgi5GaxT1NAQJ5NwP/bIBXEDO3158cnAQhfz2TH70aPK7T3YDg SD3FQMoALNSKb5JP8MsMicRB9JcQUdmznEmi3L2C5TrhvLv/Rb2GBZVyJqqGrFn/ ceBYwIzbV2utniWkk7J7bUWisutZQp72//6CNZFNqM6kPA/MLSngm732V1ZX6JRC vzhsWQhO9i8= =2AQ8 -----END PGP SIGNATURE----- From steve at holdenweb.com Wed Mar 5 01:04:40 2008 From: steve at holdenweb.com (Steve Holden) Date: Tue, 04 Mar 2008 19:04:40 -0500 Subject: [Python-Dev] Documentation reorganization [was: ... for ability to execute zipfiles & directories] In-Reply-To: <47CDDF02.7060800@canterbury.ac.nz> References: <47CD421E.2010402@gmail.com> <79990c6b0803040455q7ce211d0y78956b3a395c3bad@mail.gmail.com> <87mypew31z.fsf@uwakimon.sk.tsukuba.ac.jp> <47CDDF02.7060800@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Adam Olsen wrote: >> Such a division would make it unnecessarily hard to find documentation >> on True, False, None, etc. They've become keywords for pragmatic >> purposes (to prevent accidental modification), not because we think >> they ideally should be syntax instead of builtins. > > Maybe the solution is to rename the Library Reference > to the Class and Module Reference or something like > that. > Although DRY is fine as a programming principle, it fails for pedagogic purposes. We should therefore be prepared to repeat the same material in different contexts (hopefully by including some common documentation source rather than laborious and error-prone copy-and-paste). Document things where people expect to find them. (Now *there's* a usability study screaming to be done ... and SoC is coming up). regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From rbp at isnomore.net Wed Mar 5 01:01:04 2008 From: rbp at isnomore.net (Rodrigo Bernardo Pimentel) Date: Tue, 4 Mar 2008 21:01:04 -0300 Subject: [Python-Dev] [Python-3000] No releases tonight In-Reply-To: <20080302084935.GA62047@nexus.in-nomine.org> References: <47C9A6DC.9070708@cheimes.de> <20080302084935.GA62047@nexus.in-nomine.org> Message-ID: <20080305000103.GA17165@isnomore.net> On Sun, Mar 02 2008 at 05:49:35AM BRT, Jeroen Ruigrok van der Werven wrote: > -On [20080301 19:57], Christian Heimes (lists at cheimes.de) wrote: > >I also propose translations of the shorter text to important languages > >like French, German, Japanese, Portuguese and Spanish. I'm willing to > >help with the German translation. > > I can probably get a translation done in Japanese, Chinese and Korean. > > For Portuguese I guess we could ask the ASync guys in Brazil, they're huge > Python users with their work for Canonical/Ubuntu. I could write the Portuguese translation. rbp From greg.ewing at canterbury.ac.nz Wed Mar 5 01:01:14 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 05 Mar 2008 13:01:14 +1300 Subject: [Python-Dev] Python XML Validator In-Reply-To: <20080302230708.260fa4a9@bhuda.mired.org> References: <20080302230708.260fa4a9@bhuda.mired.org> Message-ID: <47CDE2CA.2080003@canterbury.ac.nz> Mike Meyer wrote: > Trying to install it from the repository is a PITA, because > it uses both the easyinstall and Pyrex It shouldn't depend on Pyrex as long as it's distributed with the generated C files. If it's not, that's an oversight on the part of the distributor. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From steve at holdenweb.com Wed Mar 5 01:10:49 2008 From: steve at holdenweb.com (Steve Holden) Date: Tue, 04 Mar 2008 19:10:49 -0500 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: <47CDD183.2040201@cheimes.de> References: <47CDC837.8000104@rksystems.com> <47CDD183.2040201@cheimes.de> Message-ID: <47CDE509.8080801@holdenweb.com> Christian Heimes wrote: > Bob Kline wrote: >> Any possibility of revisiting this question (upgrading to a more recent >> compiler for Windows builds of Python)? > > The latest alphas of Python 2.6 and 3.0 are build with VS 2088. I've ^^^^ Been out in the time machine, eh? > spent some time to get the new build system ready for Python 3.0a2. > > Is VS 2008 recent enought for you? :] > It would be really nice if we could maintain two parallel Windows versions, one with an open source tool chain like Mingw and one with the latest and greatest MS production line. Unfortunately the problem seems to be developer effort. For a long time only Martin and Tim were serious about the Windows platform, and I can appreciate (some of) the extra effort that parallel development platforms would require. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From rhamph at gmail.com Wed Mar 5 01:18:17 2008 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 4 Mar 2008 17:18:17 -0700 Subject: [Python-Dev] Documentation reorganization [was: ... for ability to execute zipfiles & directories] In-Reply-To: References: <47CD421E.2010402@gmail.com> <79990c6b0803040455q7ce211d0y78956b3a395c3bad@mail.gmail.com> <87mypew31z.fsf@uwakimon.sk.tsukuba.ac.jp> <47CDDF02.7060800@canterbury.ac.nz> Message-ID: On Tue, Mar 4, 2008 at 5:04 PM, Steve Holden wrote: > Greg Ewing wrote: > > Adam Olsen wrote: > >> Such a division would make it unnecessarily hard to find documentation > >> on True, False, None, etc. They've become keywords for pragmatic > >> purposes (to prevent accidental modification), not because we think > >> they ideally should be syntax instead of builtins. > > > > Maybe the solution is to rename the Library Reference > > to the Class and Module Reference or something like > > that. > > > Although DRY is fine as a programming principle, it fails for pedagogic > purposes. We should therefore be prepared to repeat the same material in > different contexts (hopefully by including some common documentation > source rather than laborious and error-prone copy-and-paste). > > Document things where people expect to find them. (Now *there's* a > usability study screaming to be done ... and SoC is coming up). Python's usage of import makes it clear when something is imported from a library, as opposed to being an integral part of the language. To me, this is an obvious basis on whether to look in the language reference or in the stdlib reference. That said, it would be useful to also have a link for major builtin types in the stdlib section, if only for people who learned to look for them there. -- Adam Olsen, aka Rhamphoryncus From lists at cheimes.de Wed Mar 5 01:32:52 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 05 Mar 2008 01:32:52 +0100 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: <47CDDE81.4090801@canterbury.ac.nz> References: <47CDC837.8000104@rksystems.com> <47CDD183.2040201@cheimes.de> <47CDDE81.4090801@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Christian Heimes wrote: >> The latest alphas of Python 2.6 and 3.0 are build with VS 2088. > ^^^^ > Wow, that must be a very, very pre-alpha release... > > Or has someone at Redmond stolen Guido's time machine? DA-LEK ... EX-TER-MI-NATE *scnr* :) Christian From steven.bethard at gmail.com Wed Mar 5 01:54:27 2008 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 4 Mar 2008 17:54:27 -0700 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: <47CDE509.8080801@holdenweb.com> References: <47CDC837.8000104@rksystems.com> <47CDD183.2040201@cheimes.de> <47CDE509.8080801@holdenweb.com> Message-ID: On Tue, Mar 4, 2008 at 5:10 PM, Steve Holden wrote: > Christian Heimes wrote: > > Bob Kline wrote: > >> Any possibility of revisiting this question (upgrading to a more recent > >> compiler for Windows builds of Python)? > > > > The latest alphas of Python 2.6 and 3.0 are build with VS 2088. I've > ^^^^ > Been out in the time machine, eh? > > > spent some time to get the new build system ready for Python 3.0a2. > > > > Is VS 2008 recent enought for you? :] > > > It would be really nice if we could maintain two parallel Windows > versions, one with an open source tool chain like Mingw and one with the > latest and greatest MS production line. Is this mainly a request to use more open source tools? Because if the concern is just cost, Python 2.6 and 3.0 compile with the free Microsoft Visual Studio 2008 Express editions. Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From steve at holdenweb.com Wed Mar 5 02:01:36 2008 From: steve at holdenweb.com (Steve Holden) Date: Tue, 04 Mar 2008 20:01:36 -0500 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: References: <47CDC837.8000104@rksystems.com> <47CDD183.2040201@cheimes.de> <47CDE509.8080801@holdenweb.com> Message-ID: Steven Bethard wrote: > On Tue, Mar 4, 2008 at 5:10 PM, Steve Holden wrote: >> Christian Heimes wrote: >> > Bob Kline wrote: >> >> Any possibility of revisiting this question (upgrading to a more recent >> >> compiler for Windows builds of Python)? >> > >> > The latest alphas of Python 2.6 and 3.0 are build with VS 2088. I've >> ^^^^ >> Been out in the time machine, eh? >> >>> spent some time to get the new build system ready for Python 3.0a2. >> > >> > Is VS 2008 recent enought for you? :] >> > >> It would be really nice if we could maintain two parallel Windows >> versions, one with an open source tool chain like Mingw and one with the >> latest and greatest MS production line. > > Is this mainly a request to use more open source tools? Because if > the concern is just cost, Python 2.6 and 3.0 compile with the free > Microsoft Visual Studio 2008 Express editions. > It was maily a request to use more open source tools, but the decision has to be made by the developers supporting the build chain. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From tnelson at onresolve.com Wed Mar 5 03:01:51 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Tue, 4 Mar 2008 18:01:51 -0800 Subject: [Python-Dev] Windows buildbot test_bsddb3 problems (was RE: Buildbots for trunk are all red) In-Reply-To: References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDE2@EXMBX04.exchhosting.com> Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C00@EXMBX04.exchhosting.com> > Trent, thanks for working on the buildbot. I fixed the first case you > mentioned in r61233 wrt removing the directory before closing the > file. It would be great if you could submit a patch when you are able > to fix the remaining problems. % svn diff Index: test_dbshelve.py =================================================================== --- test_dbshelve.py (revision 61233) +++ test_dbshelve.py (working copy) @@ -267,8 +267,8 @@ def tearDown(self): + self.do_close() test_support.rmtree(self.homeDir) - self.do_close() class EnvBTreeShelveTestCase(BasicEnvShelveTestCase): Index: test_thread.py =================================================================== --- test_thread.py (revision 61233) +++ test_thread.py (working copy) @@ -73,9 +73,9 @@ self.d.open(self.filename, self.dbtype, self.dbopenflags|db.DB_CREATE) def tearDown(self): - test_support.rmtree(self.homeDir) self.d.close() self.env.close() + test_support.rmtree(self.homeDir) def setEnvOpts(self): pass I'm getting 100% success rate with test_bsddb3 on Windows now with this patch. Yay! Trent. -- http://www.onresolve.com From tnelson at onresolve.com Wed Mar 5 03:29:27 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Tue, 4 Mar 2008 18:29:27 -0800 Subject: [Python-Dev] signal.alarm(3) in trunk test_socketserver.py Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C06@EXMBX04.exchhosting.com> r61099 added the following to trunk/Lib/test/test_socketserver.py: if __name__ == "__main__": test_main() + signal.alarm(3) # Shutdown shouldn't take more than 3 seconds. ....which breaks platforms that don't have signal.alarm, like, say, !unix ;-) Trent. -- http://www.onresolve.com From stephen at xemacs.org Wed Mar 5 04:03:39 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 05 Mar 2008 12:03:39 +0900 Subject: [Python-Dev] Documentation reorganization In-Reply-To: References: <47CD421E.2010402@gmail.com> <79990c6b0803040455q7ce211d0y78956b3a395c3bad@mail.gmail.com> <87mypew31z.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87ejapx46s.fsf@uwakimon.sk.tsukuba.ac.jp> Adam Olsen writes: > On Tue, Mar 4, 2008 at 3:13 PM, Stephen J. Turnbull wrote: > > I would like to see the clear division between the language (ie, > > the syntax) and the built-in functionality maintained. I'm not > > sure I like the proposed title for that reason. > > Such a division would make it unnecessarily hard to find documentation > on True, False, None, etc. They've become keywords for pragmatic > purposes (to prevent accidental modification), not because we think > they ideally should be syntax instead of builtins. This is Python; of course practicality beats purity. I have no problem with putting some keywords in the "built-in functionality" section, or even (boggle) duplicate them across the two sections. I too was put off by the separation of syntax from built-in functionality when I first started using the documentation, but later I came to appreciate it. I'm a relatively casual user of Python, and having a spare "syntax" section has made it much easier to learn new syntax such as comprehensions and generators. I suspect it will make it a lot easier to learn the differences between Python 2 and Python 3, too. I do not want to lose that. I don't pretend to be speaking for anyone else, but I'd be surprised if I were unique. From stephen at xemacs.org Wed Mar 5 04:10:26 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 05 Mar 2008 12:10:26 +0900 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: <47CDDE81.4090801@canterbury.ac.nz> References: <47CDC837.8000104@rksystems.com> <47CDD183.2040201@cheimes.de> <47CDDE81.4090801@canterbury.ac.nz> Message-ID: <87d4q9x3vh.fsf@uwakimon.sk.tsukuba.ac.jp> Greg Ewing writes: > Christian Heimes wrote: > > The latest alphas of Python 2.6 and 3.0 are build with VS 2088. > ^^^^ > Wow, that must be a very, very pre-alpha release... Nah, it's a version optimized for 8/16-bit segmented architectures. From tnelson at onresolve.com Wed Mar 5 04:25:15 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Tue, 4 Mar 2008 19:25:15 -0800 Subject: [Python-Dev] signal.alarm(3) in trunk test_socketserver.py In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C06@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C06@EXMBX04.exchhosting.com> Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C18@EXMBX04.exchhosting.com> > r61099 added the following to trunk/Lib/test/test_socketserver.py: > > if __name__ == "__main__": > test_main() > + signal.alarm(3) # Shutdown shouldn't take more than 3 seconds. > Actually, signal.alarm() was introduced all over the place in that revision. I understand the intent of this commit was to speed up the runtime of this test (something like 28s -> 0.3s was quoted in the commit log). FWIW, runtime of the test with the following patch on Windows is 0.125s: Index: test_socketserver.py =================================================================== --- test_socketserver.py (revision 61233) +++ test_socketserver.py (working copy) @@ -28,6 +28,9 @@ HAVE_UNIX_SOCKETS = hasattr(socket, "AF_UNIX") HAVE_FORKING = hasattr(os, "fork") and os.name != "os2" +def signal_alarm(n): + if hasattr(signal, 'alarm'): + signal.alarm(n) def receive(sock, n, timeout=20): r, w, x = select.select([sock], [], [], timeout) @@ -99,7 +102,7 @@ """Test all socket servers.""" def setUp(self): - signal.alarm(20) # Kill deadlocks after 20 seconds. + signal_alarm(20) # Kill deadlocks after 20 seconds. self.port_seed = 0 self.test_files = [] @@ -112,7 +115,7 @@ except os.error: pass self.test_files[:] = [] - signal.alarm(0) # Didn't deadlock. + signal_alarm(0) # Didn't deadlock. def pickaddr(self, proto): if proto == socket.AF_INET: @@ -267,4 +270,4 @@ if __name__ == "__main__": test_main() - signal.alarm(3) # Shutdown shouldn't take more than 3 seconds. + signal_alarm(3) # Shutdown shouldn't take more than 3 seconds. From tnelson at onresolve.com Wed Mar 5 04:37:52 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Tue, 4 Mar 2008 19:37:52 -0800 Subject: [Python-Dev] Patch for trunk test_winsound.py (fixes my buildbot) Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C1C@EXMBX04.exchhosting.com> winsound.Beep fails for me on the 'x86 2k8 trunk' build slave, which is a virtual Windows Server 2008 instance running under Hyper-V. Not surprisingly, there's not a single audio-related device on this system. The attached patch to test_winsound.py incorporates the _have_soundcard() checks to the BeepTest class, which fixes the problem for me. (I've also tested the patch on a Vista system (that does have a soundcard) and everything works as expected.) Trent. -------------- next part -------------- A non-text attachment was scrubbed... Name: test_winsound.py.patch Type: application/octet-stream Size: 1677 bytes Desc: test_winsound.py.patch Url : http://mail.python.org/pipermail/python-dev/attachments/20080304/2566d8c5/attachment.obj From jyasskin at gmail.com Wed Mar 5 04:40:18 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Tue, 4 Mar 2008 19:40:18 -0800 Subject: [Python-Dev] signal.alarm(3) in trunk test_socketserver.py In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C18@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C06@EXMBX04.exchhosting.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C18@EXMBX04.exchhosting.com> Message-ID: <5d44f72f0803041940ie22885ajb044b02599cd04df@mail.gmail.com> On Tue, Mar 4, 2008 at 7:25 PM, Trent Nelson wrote: > > r61099 added the following to trunk/Lib/test/test_socketserver.py: > > > > if __name__ == "__main__": > > test_main() > > + signal.alarm(3) # Shutdown shouldn't take more than 3 seconds. > > > > Actually, signal.alarm() was introduced all over the place in that revision. I understand the intent of this commit was to speed up the runtime of this test (something like 28s -> 0.3s was quoted in the commit log). FWIW, runtime of the test with the following patch on Windows is 0.125s: > Yep, the alarm is only there to prevent what would be deadlocks from running forever. Sorry for breaking !unix. Your patch looks fine to me. Do you want to submit it or shall I? > Index: test_socketserver.py > =================================================================== > --- test_socketserver.py (revision 61233) > +++ test_socketserver.py (working copy) > @@ -28,6 +28,9 @@ > HAVE_UNIX_SOCKETS = hasattr(socket, "AF_UNIX") > HAVE_FORKING = hasattr(os, "fork") and os.name != "os2" > > +def signal_alarm(n): > + if hasattr(signal, 'alarm'): > + signal.alarm(n) > > def receive(sock, n, timeout=20): > r, w, x = select.select([sock], [], [], timeout) > @@ -99,7 +102,7 @@ > """Test all socket servers.""" > > def setUp(self): > - signal.alarm(20) # Kill deadlocks after 20 seconds. > + signal_alarm(20) # Kill deadlocks after 20 seconds. > self.port_seed = 0 > self.test_files = [] > > @@ -112,7 +115,7 @@ > except os.error: > pass > self.test_files[:] = [] > - signal.alarm(0) # Didn't deadlock. > + signal_alarm(0) # Didn't deadlock. > > def pickaddr(self, proto): > if proto == socket.AF_INET: > @@ -267,4 +270,4 @@ > > > if __name__ == "__main__": > test_main() > - signal.alarm(3) # Shutdown shouldn't take more than 3 seconds. > + signal_alarm(3) # Shutdown shouldn't take more than 3 seconds. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jyasskin%40gmail.com > -- Namast?, Jeffrey Yasskin http://jeffrey.yasskin.info/ From tnelson at onresolve.com Wed Mar 5 04:46:03 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Tue, 4 Mar 2008 19:46:03 -0800 Subject: [Python-Dev] signal.alarm(3) in trunk test_socketserver.py In-Reply-To: <5d44f72f0803041940ie22885ajb044b02599cd04df@mail.gmail.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C06@EXMBX04.exchhosting.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C18@EXMBX04.exchhosting.com> <5d44f72f0803041940ie22885ajb044b02599cd04df@mail.gmail.com> Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C1D@EXMBX04.exchhosting.com> > Yep, the alarm is only there to prevent what would be deadlocks from > running forever. Sorry for breaking !unix. Your patch looks fine to > me. Do you want to submit it or shall I? I'm not a committer, so it's all yours. Thanks for the quick turnaround! Trent. From rhamph at gmail.com Wed Mar 5 05:11:45 2008 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 4 Mar 2008 21:11:45 -0700 Subject: [Python-Dev] Documentation reorganization In-Reply-To: <87ejapx46s.fsf@uwakimon.sk.tsukuba.ac.jp> References: <47CD421E.2010402@gmail.com> <79990c6b0803040455q7ce211d0y78956b3a395c3bad@mail.gmail.com> <87mypew31z.fsf@uwakimon.sk.tsukuba.ac.jp> <87ejapx46s.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Tue, Mar 4, 2008 at 8:03 PM, Stephen J. Turnbull wrote: > Adam Olsen writes: > > On Tue, Mar 4, 2008 at 3:13 PM, Stephen J. Turnbull wrote: > > > > I would like to see the clear division between the language (ie, > > > the syntax) and the built-in functionality maintained. I'm not > > > sure I like the proposed title for that reason. > > > > Such a division would make it unnecessarily hard to find documentation > > on True, False, None, etc. They've become keywords for pragmatic > > purposes (to prevent accidental modification), not because we think > > they ideally should be syntax instead of builtins. > > This is Python; of course practicality beats purity. I have no > problem with putting some keywords in the "built-in functionality" > section, or even (boggle) duplicate them across the two sections. -1 on duplicating anything. Provide links to a single location instead. Otherwise you end up with two explanations of the same thing, with different wording and subtle differences or missing details. > I too was put off by the separation of syntax from built-in > functionality when I first started using the documentation, but later > I came to appreciate it. I'm a relatively casual user of Python, and > having a spare "syntax" section has made it much easier to learn new > syntax such as comprehensions and generators. I suspect it will make > it a lot easier to learn the differences between Python 2 and Python > 3, too. I do not want to lose that. I learned them through third-party docs. Even now I have a hard time finding list comprehension/generator expression in the docs. Apparently they're in the Expression section, under "Displays for lists, sets and dictionaries", and neither that nor anything with list comprehension or generator expression is in the index. The term "Displays" is pretty obscure as well, not something I've seen used besides on this list or right there in the documentation. > I don't pretend to be speaking for anyone else, but I'd be surprised > if I were unique. Your experiences *shouldn't* be unique, but I'm afraid they might be. Another example is the use of BNF, which although dominant in its field, it provides a steep learning curve for most programmers. I'm afraid this has turned into a rant, but it should be seen as the experiences of someone who relies on the documentation a great deal. Is there a better way to channel feedback on the documentation? -- Adam Olsen, aka Rhamphoryncus From tnelson at onresolve.com Wed Mar 5 05:33:10 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Tue, 4 Mar 2008 20:33:10 -0800 Subject: [Python-Dev] Windows buildbots randomly die with twisted ConnectionLost errors? Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C31@EXMBX04.exchhosting.com> I've started to see my build slave dying every so often with a twisted error half way through tests: ... test_htmlparser test_httplib remoteFailed: [Failure instance: Traceback (failure with no frames): twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion. ] Examples: http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/46/step-test/0 http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/36/step-test/0 I'm not sure if I should read into the fact that it's occurring after networking-oriented tests like test_httplib and test_ftplib. Running rt.bat on the resulting build manually doesn't indicate any errors in these tests. Have other Windows buildbot owners seen this? Trent. From nnorwitz at gmail.com Wed Mar 5 06:31:58 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 4 Mar 2008 21:31:58 -0800 Subject: [Python-Dev] BSDDB3 (was: Re: Buildbots for trunk are all red) In-Reply-To: <47CDDF57.1020006@argo.es> References: <47C32D2D.1030206@free.fr> <47C4813F.2080403@v.loewis.de> <52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com> <20080227141322.GA5871@amk-desktop.matrixgroup.net> <47CDDF57.1020006@argo.es> Message-ID: On Tue, Mar 4, 2008 at 3:46 PM, Jesus Cea wrote: > > That said, it is my aim to keep bsddb in stdlib, providing a stable and > featureful module. I think keeping bsddb development inside python svn > is not appropiate. Currently (I could change idea), my approach will be > keeping pybssdb as a separate project and sync with python SVN from time > to time. Mainly to take advantage of buildbot architecture and, of > course, to be able to release python with current bindings. > > Since I have no python commit access, this seems a sensible approach. > And I could do frequent pybssdb releases (let say, every couple of > months) without waiting for a full python release (current approach). I think that approach is fine. Hopefully you can keep the changes reasonably small (preferably less than 500 lines per change). That will ensure more people will review your changes. Cheers, n From g.brandl at gmx.net Wed Mar 5 08:16:59 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 05 Mar 2008 08:16:59 +0100 Subject: [Python-Dev] Documentation reorganization In-Reply-To: References: <47CD421E.2010402@gmail.com> <79990c6b0803040455q7ce211d0y78956b3a395c3bad@mail.gmail.com> <87mypew31z.fsf@uwakimon.sk.tsukuba.ac.jp> <87ejapx46s.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: Adam Olsen schrieb: >> I don't pretend to be speaking for anyone else, but I'd be surprised >> if I were unique. > > Your experiences *shouldn't* be unique, but I'm afraid they might be. > Another example is the use of BNF, which although dominant in its > field, it provides a steep learning curve for most programmers. We could of course accompany each BNF-described item with an example. > I'm afraid this has turned into a rant, but it should be seen as the > experiences of someone who relies on the documentation a great deal. > Is there a better way to channel feedback on the documentation? There's of course the doc-SIG, but it's just another mailing list. In any case, the best way to channel feedback is to provide a patch Georg From ncoghlan at gmail.com Wed Mar 5 11:18:41 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 05 Mar 2008 20:18:41 +1000 Subject: [Python-Dev] Patch for trunk test_winsound.py (fixes my buildbot) In-Reply-To: <47CE7248.9070003@gmail.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C1C@EXMBX04.exchhosting.com> <47CE7248.9070003@gmail.com> Message-ID: <47CE7381.5020803@gmail.com> Nick Coghlan wrote: > While the patches are appreciated, please submit them to the tracker at > bugs.python.org rather than mailing them directly to this list. This comment doesn't apply to your recent posts - looks like those have all been checked in already ;) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Mar 5 11:13:28 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 05 Mar 2008 20:13:28 +1000 Subject: [Python-Dev] Patch for trunk test_winsound.py (fixes my buildbot) In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C1C@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C1C@EXMBX04.exchhosting.com> Message-ID: <47CE7248.9070003@gmail.com> Trent Nelson wrote: > winsound.Beep fails for me on the 'x86 2k8 trunk' build slave, which > is a virtual Windows Server 2008 instance running under Hyper-V. Not > surprisingly, there's not a single audio-related device on this > system. The attached patch to test_winsound.py incorporates the > _have_soundcard() checks to the BeepTest class, which fixes the > problem for me. (I've also tested the patch on a Vista system (that > does have a soundcard) and everything works as expected.) Trent, While the patches are appreciated, please submit them to the tracker at bugs.python.org rather than mailing them directly to this list. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From facundobatista at gmail.com Wed Mar 5 12:56:57 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Wed, 5 Mar 2008 09:56:57 -0200 Subject: [Python-Dev] [Python-ideas] new super redux (better late than never?) In-Reply-To: References: <47857e720803041836v6796c236v6ad8c675b7fafd75@mail.gmail.com> Message-ID: 2008/3/5, Guido van Rossum : (Bringing this from python-ideas, Guido is talking about PEP 3135) > Ehhh! The PEP's "reference implementation" is useless and probably > doesn't even work. The actual implementation is completely different. > If you want to help, a rewrite of the PEP to match reality would be > most welcome! Guido, I know that in this fight-for-reality some of the PEPs for Py3 are not correct. Which is the plan to handle this? Will the original authors fix them? And if not, will these PEP be marked as "Caution: non compliant with reality" or something? PEPs are a great tool, one of the Python assets, and it's a pity that we may not trust them... Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From ncoghlan at gmail.com Wed Mar 5 15:03:41 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 06 Mar 2008 00:03:41 +1000 Subject: [Python-Dev] [Python-ideas] new super redux (better late than never?) In-Reply-To: References: <47857e720803041836v6796c236v6ad8c675b7fafd75@mail.gmail.com> Message-ID: <47CEA83D.2090600@gmail.com> Facundo Batista wrote: > 2008/3/5, Guido van Rossum : > > (Bringing this from python-ideas, Guido is talking about PEP 3135) > >> Ehhh! The PEP's "reference implementation" is useless and probably >> doesn't even work. The actual implementation is completely different. >> If you want to help, a rewrite of the PEP to match reality would be >> most welcome! > > Guido, I know that in this fight-for-reality some of the PEPs for Py3 > are not correct. > > Which is the plan to handle this? Will the original authors fix them? > And if not, will these PEP be marked as "Caution: non compliant with > reality" or something? > > PEPs are a great tool, one of the Python assets, and it's a pity that > we may not trust them... Some PEPs (252/253 come to mind) already carry such warnings. The wording in those two PEPs suggests that this warning was added by the PEP editors after the fact: [Editor's note: the ideas described in this PEP have been incorporated into Python. The PEP no longer accurately describes the implementation.] In other cases we have gone back and fixed the PEP to match the implementation (e.g. PEP 343 was updated to reflect changes that happened prior to 2.5 release). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Mar 5 15:15:29 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 06 Mar 2008 00:15:29 +1000 Subject: [Python-Dev] Documentation for ability to execute zipfiles & directories In-Reply-To: References: <47CD421E.2010402@gmail.com> <79990c6b0803040455q7ce211d0y78956b3a395c3bad@mail.gmail.com> Message-ID: <47CEAB01.5030008@gmail.com> Georg Brandl wrote: > Steve Holden schrieb: >> Paul Moore wrote: >>> On 04/03/2008, Nick Coghlan wrote: >>>> Do we need a new appendix to the tutorial which goes into detail about >>>> the CPython interpreter's command line options, environment variables >>>> and details on what can be executed? >>> There is a Python man page, which covers the command line usage. >>> However, it's separate from the documentation, and it isn't bundled >>> with the Windows installers - both of which are a real pain (for me, >>> at least). >>> >>> I'd suggest taking the man page, adding the information about >>> executing zip files and directories, and putting the whole lot into >>> the formal documentation. > > Look no further: http://docs.python.org/dev/using/cmdline.html Thanks Georg, that looks like exactly the right place - I'll try to get that updated before the next alpha. >> I've always found it rather counter-intuitive that you have to go to the >> Library Reference manual to find information about Python's built-in >> types, for example. I though the whole point of libraries was that they >> *aren't* built in, and represent baggage that should only be carried on >> necessary trips. > > You speak my mind. For ages I've wanted to put the builtins together with > the language reference into a new document called "Python Core Language". > I've just never had the time to draft a serious proposal. I borrowed the keys to Guido's time machine: http://svn.python.org/view/sandbox/trunk/userref/ It hasn't been updated for a lot of the 2.6/3.0 features as yet, but it may be a decent basis for what you're considering here. (and all credit to this thread for motivating me to actually get those files cleaned up and into the sandbox - I've been thinking about doing it for ages, but never got around to it). (For MS Office users, you may need to get OpenOffice.org or similar in order to read the Open Document Format files) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From theller at ctypes.org Wed Mar 5 16:03:54 2008 From: theller at ctypes.org (Thomas Heller) Date: Wed, 05 Mar 2008 16:03:54 +0100 Subject: [Python-Dev] Windows buildbots randomly die with twisted ConnectionLost errors? In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C31@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C31@EXMBX04.exchhosting.com> Message-ID: Trent Nelson schrieb: > I've started to see my build slave dying every so often with a > twisted error half way through tests: ... test_htmlparser > test_httplib > > remoteFailed: [Failure instance: Traceback (failure with no frames): > twisted.internet.error.ConnectionLost: Connection to the other side > was lost in a non-clean fashion. ] > > Examples: > http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/46/step-test/0 > > http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/36/step-test/0 > > > I'm not sure if I should read into the fact that it's occurring after > networking-oriented tests like test_httplib and test_ftplib. Running > rt.bat on the resulting build manually doesn't indicate any errors in > these tests. Have other Windows buildbot owners seen this? > > Trent. I have not observed this behaviour on my buildbots. Have you looked into the twistd.log logfile? Thomas From mwm at mired.org Wed Mar 5 16:23:06 2008 From: mwm at mired.org (Mike Meyer) Date: Wed, 5 Mar 2008 10:23:06 -0500 Subject: [Python-Dev] Python XML Validator In-Reply-To: References: <20080302230708.260fa4a9@bhuda.mired.org> Message-ID: <20080305102306.0a3db148@mbook-fbsd> On Tue, 04 Mar 2008 15:44:32 -0800 Ned Deily wrote: > In article <20080302230708.260fa4a9 at bhuda.mired.org>, > Mike Meyer wrote: > > On Thu, 28 Feb 2008 23:42:49 +0000 (UTC) Medhat Gayed > > wrote: > > > lxml is good but not written in python and difficult to install and didn't > > > work > > > on MacOS X. > > lxml is built on top of libxml2/libxslt, which are bundled with most > > Unix-like OS's (including Mac OS X), or available in their package > > systems. Trying to install it from the repository is a PITA, because > > it uses both the easyinstall and Pyrex (later Cython) packages - which > > aren't bundled with anything. On the other hand, if it's in the > > package system (I no longer have macports installed anywhere, but > > believe it was there at one time), that solves all those problems. I > > believe they've excised the easyinstall source dependencies, though. > [...] > > If you just want an xml module in the standard library that's more > > complete, I'd vote for the source distribution of lxml, as that's C + > > Python and built on top of commonly available libraries. The real > > issue would be making current lxml work with the "outdated" versions > > of those libraries found in current OS distributions. > > I'm not sure what you perceive to be the problems with easy_install on > OSX; I find it makes life *much* simpler for managing python packages. I don't, but the real issue is that it's been considered - and rejected - for inclusion in the standard library multiple times. The OPs request was for a validating XML parser in the standard library. Any third party code that requires easy_install won't be acceptable. I think lxml is the best Python XML library that meets his requirements, and it would make my life a lot easier if it were part of the standard library. However, the authors tend to require recent versions of libxml2 and libxslt, which means recent versions of lxml won't build and/or work with the libraries bundled with many Unix and Unix-like systems - including OSX. Which means you wind up having to build those yourself if you want a recent version of lxml, even if you're using a system that includes lxml in it's package system. > Be that as it may, since the release of lxml 2.0, the project has > updated the lxml website with useful information about source > installations and, in particular, OSX source installations: > > > > IIRC, here's what worked for me on Leopard (10.5.2) using the python.org > 2.5.2, though it should work fine with the Apple-supplied 2.5.1: This is similar to what I went through with 1.3.6 on Tiger, but I used MacPorts. On Leopard, 1.3.6 builds out of the box. Just do "sudo python setup.py install" and you're done. That's probably the easiest way to get a validating xml parser on OS X at this time. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. From mwm at mired.org Wed Mar 5 16:25:53 2008 From: mwm at mired.org (Mike Meyer) Date: Wed, 5 Mar 2008 10:25:53 -0500 Subject: [Python-Dev] Python XML Validator In-Reply-To: <47CDE2CA.2080003@canterbury.ac.nz> References: <20080302230708.260fa4a9@bhuda.mired.org> <47CDE2CA.2080003@canterbury.ac.nz> Message-ID: <20080305102553.5841f338@mbook-fbsd> On Wed, 05 Mar 2008 13:01:14 +1300 Greg Ewing wrote: > Mike Meyer wrote: > > Trying to install it from the repository is a PITA, because > > it uses both the easyinstall and Pyrex > > It shouldn't depend on Pyrex as long as it's distributed > with the generated C files. If it's not, that's an > oversight on the part of the distributor. Sorry I wasn't clear. "from the repository" means building from sourced checked out of the source repository, not from a distribution. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. From tnelson at onresolve.com Wed Mar 5 16:47:27 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Wed, 5 Mar 2008 07:47:27 -0800 Subject: [Python-Dev] Windows buildbots randomly die with twisted ConnectionLost errors? In-Reply-To: References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C31@EXMBX04.exchhosting.com>, Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDE8@EXMBX04.exchhosting.com> Had a chat with some Twisted/buildbot folk and they can confirm they've seen it as well on Windows. They've given me a few things to look into. Out of interest, how are you running your buildbot? Via the command line in an interactive desktop session, as a service, or as a scheduled task, or some other way? ________________________________________ From: python-dev-bounces+tnelson=onresolve.com at python.org [python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of Thomas Heller [theller at ctypes.org] Sent: 05 March 2008 10:03 To: python-dev at python.org Subject: Re: [Python-Dev] Windows buildbots randomly die with twisted ConnectionLost errors? Trent Nelson schrieb: > I've started to see my build slave dying every so often with a > twisted error half way through tests: ... test_htmlparser > test_httplib > > remoteFailed: [Failure instance: Traceback (failure with no frames): > twisted.internet.error.ConnectionLost: Connection to the other side > was lost in a non-clean fashion. ] > > Examples: > http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/46/step-test/0 > > http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/36/step-test/0 > > > I'm not sure if I should read into the fact that it's occurring after > networking-oriented tests like test_httplib and test_ftplib. Running > rt.bat on the resulting build manually doesn't indicate any errors in > these tests. Have other Windows buildbot owners seen this? > > Trent. I have not observed this behaviour on my buildbots. Have you looked into the twistd.log logfile? Thomas _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/tnelson%40onresolve.com From guido at python.org Wed Mar 5 16:58:22 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 5 Mar 2008 07:58:22 -0800 Subject: [Python-Dev] [Python-ideas] new super redux (better late than never?) In-Reply-To: References: <47857e720803041836v6796c236v6ad8c675b7fafd75@mail.gmail.com> Message-ID: On Wed, Mar 5, 2008 at 3:56 AM, Facundo Batista wrote: > 2008/3/5, Guido van Rossum : > > (Bringing this from python-ideas, Guido is talking about PEP 3135) > > > > Ehhh! The PEP's "reference implementation" is useless and probably > > doesn't even work. The actual implementation is completely different. > > If you want to help, a rewrite of the PEP to match reality would be > > most welcome! > > Guido, I know that in this fight-for-reality some of the PEPs for Py3 > are not correct. > > Which is the plan to handle this? Will the original authors fix them? > And if not, will these PEP be marked as "Caution: non compliant with > reality" or something? > > PEPs are a great tool, one of the Python assets, and it's a pity that > we may not trust them... PEP 3135 is the only one that is grossly wrong. It should be fixed. I originally planned to do so myself (since I wrote the implementation) but find I don't have the time. Volunteers welcome! Regarding the OP's idea that started this thread, it's a waste of time. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From theller at ctypes.org Wed Mar 5 17:01:08 2008 From: theller at ctypes.org (Thomas Heller) Date: Wed, 05 Mar 2008 17:01:08 +0100 Subject: [Python-Dev] Windows buildbots randomly die with twisted ConnectionLost errors? In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDE8@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C31@EXMBX04.exchhosting.com>, <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDE8@EXMBX04.exchhosting.com> Message-ID: Trent Nelson schrieb: > Had a chat with some Twisted/buildbot folk and they can confirm > they've seen it as well on Windows. They've given me a few things to > look into. Out of interest, how are you running your buildbot? Via > the command line in an interactive desktop session, as a service, or > as a scheduled task, or some other way? >From the command line in interactive sessions. Thomas From bkline at rksystems.com Wed Mar 5 16:40:22 2008 From: bkline at rksystems.com (Bob Kline) Date: Wed, 05 Mar 2008 10:40:22 -0500 Subject: [Python-Dev] Python XML Validator In-Reply-To: <20080305102306.0a3db148@mbook-fbsd> References: <20080302230708.260fa4a9@bhuda.mired.org> <20080305102306.0a3db148@mbook-fbsd> Message-ID: <47CEBEE6.7080804@rksystems.com> Mike Meyer wrote: > I think lxml is the best Python XML library that meets his > requirements, and it would make my life a lot easier if it were part > of the standard library. +1 (!) -- Bob Kline http://www.rksystems.com mailto:bkline at rksystems.com From nnorwitz at gmail.com Wed Mar 5 19:11:04 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 5 Mar 2008 10:11:04 -0800 Subject: [Python-Dev] Patch for trunk test_winsound.py (fixes my buildbot) In-Reply-To: <47CE7381.5020803@gmail.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C1C@EXMBX04.exchhosting.com> <47CE7248.9070003@gmail.com> <47CE7381.5020803@gmail.com> Message-ID: On Wed, Mar 5, 2008 at 2:18 AM, Nick Coghlan wrote: > Nick Coghlan wrote: > > While the patches are appreciated, please submit them to the tracker at > > bugs.python.org rather than mailing them directly to this list. > > This comment doesn't apply to your recent posts - looks like those have > all been checked in already ;) Just to follow up on what Nick said, you should submit patches to the tracker. While it's usually pretty easy to submit really small things like this to the mailing list, it has two negative effects: 1) clutters the mailing list with trivial things many people may not care about, 2) it can easily be lost/forgotten. Typically the only reason we post an actual patch on python-dev is to spawn wider discussion. Generally, patches should be posted as links to the tracker rather than inline. But keep the patches coming! Cheers, n From martin at v.loewis.de Wed Mar 5 22:07:08 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 05 Mar 2008 22:07:08 +0100 Subject: [Python-Dev] BSDDB3 In-Reply-To: <47CDDF57.1020006@argo.es> References: <47C32D2D.1030206@free.fr> <47C4813F.2080403@v.loewis.de> <52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com> <20080227141322.GA5871@amk-desktop.matrixgroup.net> <47CDDF57.1020006@argo.es> Message-ID: <47CF0B7C.109@v.loewis.de> > That said, it is my aim to keep bsddb in stdlib, providing a stable and > featureful module. I think keeping bsddb development inside python svn > is not appropiate. I think it would be helpful if you could analyze the crashes that bsddb caused on Windows. Just go back a few revisions in the subversion tree to reproduce the crashes. These were particularly bad since they invoked the CRT abort() inside bsddb, namely inside __db_win32_mutex_unlock, which, in debug mode, would __db_panic with "Win32 unlock failed: lock already unlocked". This problem has now been worked-around, by reformulating the test cases so that the situation doesn't occur anymore, but IMO, it should not be possible for an extension module to cause an interpreter abort. Regards, Martin From martin at v.loewis.de Wed Mar 5 22:31:42 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 05 Mar 2008 22:31:42 +0100 Subject: [Python-Dev] Windows buildbots randomly die with twisted ConnectionLost errors? In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C31@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C3C31@EXMBX04.exchhosting.com> Message-ID: <47CF113E.2070307@v.loewis.de> > I'm not sure if I should read into the fact that it's occurring after > networking-oriented tests like test_httplib and test_ftplib. Running > rt.bat on the resulting build manually doesn't indicate any errors in > these tests. Have other Windows buildbot owners seen this? Notice that it also occurs in other steps, e.g. http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/50/step-compile/0 Please understand that the basic principle of buildbot is that the slaves connect to the master, and that the master relies on the slaves keeping the connection up. Could it be that you are behind some firewall infrastructure which suddenly decides that certain TCP connections are idle/expired/closed? In that case, the firewall might reject further messages from the master, causing the master to believe that the connection was lost. In that case, the slave should send repeated keepalive messages (or the firewall be reconfigured to not close connections to dinsdale:buildbot). It seem buildbot already has support for keepalives, although the frequency might be too low. In any case, you should check the twistd log files around the time of the connection loss whether it shows any problems from the client side as well. Regards, Martin From greg.ewing at canterbury.ac.nz Wed Mar 5 22:49:49 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2008 10:49:49 +1300 Subject: [Python-Dev] Documentation reorganization In-Reply-To: References: <47CD421E.2010402@gmail.com> <79990c6b0803040455q7ce211d0y78956b3a395c3bad@mail.gmail.com> <87mypew31z.fsf@uwakimon.sk.tsukuba.ac.jp> <87ejapx46s.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <47CF157D.8030503@canterbury.ac.nz> Adam Olsen wrote: > The term "Displays" is pretty obscure as well, Hmmm, I'd call them "constructor expressions" or some such. -- Greg From greg.ewing at canterbury.ac.nz Wed Mar 5 22:59:07 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2008 10:59:07 +1300 Subject: [Python-Dev] Documentation reorganization In-Reply-To: References: <47CD421E.2010402@gmail.com> <79990c6b0803040455q7ce211d0y78956b3a395c3bad@mail.gmail.com> <87mypew31z.fsf@uwakimon.sk.tsukuba.ac.jp> <87ejapx46s.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <47CF17AB.6000307@canterbury.ac.nz> Georg Brandl wrote: > Adam Olsen schrieb: > >>Another example is the use of BNF, which although dominant in its >>field, it provides a steep learning curve for most programmers. > > We could of course accompany each BNF-described item with an example. An alternative to BNF would be syntax diagrams. They're just as formal, and can be a lot easier to read. -- Greg From greg.ewing at canterbury.ac.nz Wed Mar 5 22:03:05 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2008 10:03:05 +1300 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: References: <47CDC837.8000104@rksystems.com> <47CDD183.2040201@cheimes.de> <47CDE509.8080801@holdenweb.com> Message-ID: <47CF0A89.2010804@canterbury.ac.nz> Steven Bethard wrote: > Is this mainly a request to use more open source tools? Because if > the concern is just cost, Python 2.6 and 3.0 compile with the free > Microsoft Visual Studio 2008 Express editions. I don't think it's only about cost, it's about not being reliant on tools that appear and disappear at the whim of Microsoft. Their "free" compilers only last until the next version, then they become unavailable. So it can still be a problem finding exactly the right version to match your Python installation, if you missed the window of opportunity to download it. -- Greg From steve at holdenweb.com Thu Mar 6 01:52:57 2008 From: steve at holdenweb.com (Steve Holden) Date: Wed, 05 Mar 2008 19:52:57 -0500 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: <47CF0A89.2010804@canterbury.ac.nz> References: <47CDC837.8000104@rksystems.com> <47CDD183.2040201@cheimes.de> <47CDE509.8080801@holdenweb.com> <47CF0A89.2010804@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Steven Bethard wrote: >> Is this mainly a request to use more open source tools? Because if >> the concern is just cost, Python 2.6 and 3.0 compile with the free >> Microsoft Visual Studio 2008 Express editions. > > I don't think it's only about cost, it's about not > being reliant on tools that appear and disappear at > the whim of Microsoft. Their "free" compilers only > last until the next version, then they become > unavailable. So it can still be a problem finding > exactly the right version to match your Python > installation, if you missed the window of opportunity > to download it. > Yes, it would be nice to be able to avoid "Microsoft Version Churn" without having to spring for the commercial versions of Visual Studio. Mingw tends to be rather more stable (though not itself without the occasional library compatibility issue), and more freely available. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From xchaos12345 at gmail.com Thu Mar 6 03:49:20 2008 From: xchaos12345 at gmail.com (andrew rainwater) Date: Wed, 5 Mar 2008 20:49:20 -0600 Subject: [Python-Dev] Hello Message-ID: Hello to all I am a new member. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080305/a97f07b8/attachment.htm From jyasskin at gmail.com Thu Mar 6 04:35:28 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Wed, 5 Mar 2008 19:35:28 -0800 Subject: [Python-Dev] Optimizing with. Message-ID: <5d44f72f0803051935u10e9f3b9q3149f36075ad5790@mail.gmail.com> I've got a patch in http://bugs.python.org/issue2179 that optimizes the bytecode generated by a with statement by tucking the context_manager.__exit__ method onto the stack. It saves 2 opcodes, 8 bytes, and about .5us for each with block at the cost of an extra stack entry for the duration of the block. Since it's the first time I've touched the core of the compiler and interpreter, I'm hoping that someone can take a look before I check it in. Thanks! Jeffrey From snaury at gmail.com Thu Mar 6 07:50:14 2008 From: snaury at gmail.com (Alexey Borzenkov) Date: Thu, 6 Mar 2008 09:50:14 +0300 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: References: <47CDC837.8000104@rksystems.com> <47CDD183.2040201@cheimes.de> <47CDE509.8080801@holdenweb.com> <47CF0A89.2010804@canterbury.ac.nz> Message-ID: On Thu, Mar 6, 2008 at 3:52 AM, Steve Holden wrote: > Mingw tends to be rather more stable (though not itself without the > occasional library compatibility issue), and more freely available. Not all extensions can be built using mingw (pywin32 comes to mind immediately). And while you can almost easily (but not very easily, since you need to patch libmoldname yourself and then update your spec file) change runtime used by mingw, you cannot do so with Visual Studio. Besides, this would require spec-editing on both, build machines, as well as user machines (otherwise you get extensions that depend on both msvcrt.dll and msvcrXX.dll), this is a complication too. Runtime compatibility could be a potential problem (or maybe not, if WITH_PYMALLOC maybe fixes that?), so official mingw port (as much as I'd like for one to be there) does not seem feasible to me in any future, Python is already too far on the needle of Visual Studio. :( The biggest problem would also be with some extensions' setup.py. Many extensions I've seen hardwire to distutils.msvccompiler in one way or another. If official python would shift to mingw, they would suddenly break. I think the best lesson here is Tcl. Because it uses stubs mechanism, you don't need to depend on tclXX.dll, you don't deal with really direct implementation details, you don't care about runtimes, everything is much easier. Maybe it's possible (and not too late) for Python to somehow embrace such mechanism? From greg at krypto.org Thu Mar 6 08:29:05 2008 From: greg at krypto.org (Gregory P. Smith) Date: Wed, 5 Mar 2008 23:29:05 -0800 Subject: [Python-Dev] BSDDB3 (was: Re: Buildbots for trunk are all red) In-Reply-To: <47CDDF57.1020006@argo.es> References: <47C32D2D.1030206@free.fr> <47C4813F.2080403@v.loewis.de> <52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com> <20080227141322.GA5871@amk-desktop.matrixgroup.net> <47CDDF57.1020006@argo.es> Message-ID: <52dc1c820803052329g1fbc6e45y41db696901ec0b57@mail.gmail.com> On 3/4/08, Jesus Cea wrote: > That said, it is my aim to keep bsddb in stdlib, providing a stable and > featureful module. I think keeping bsddb development inside python svn > is not appropiate. Currently (I could change idea), my approach will be > keeping pybssdb as a separate project and sync with python SVN from time > to time. Mainly to take advantage of buildbot architecture and, of > course, to be able to release python with current bindings. > > Since I have no python commit access, this seems a sensible approach. > And I could do frequent pybssdb releases (let say, every couple of > months) without waiting for a full python release (current approach). That makes sense. I also agree with Neal's comments, merging back into python in reasonable sized chunks is good. Don't worry about commit access for now, I'll do the initial pybsddb back into python trunk merges until we get that setup. I've merged the python trunk changes that others have made back into the pybsddb tree. PS: I have tried to sign the Python Contributor Agreement, but I am not > sure about current pybsddb license terms. Help welcomed. The current bsddb license first and foremost is the Python license. If I read the comments in the _bsddb file correctly I believe you could also call it a MIT style license. Keep things simple, just write "Python License" on your contributor form and submit it. -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080305/6c897baa/attachment.htm From tnelson at onresolve.com Thu Mar 6 08:43:12 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Wed, 5 Mar 2008 23:43:12 -0800 Subject: [Python-Dev] [Python-checkins] r61264 - in python/trunk: Lib/test/test_os.py Misc/NEWS In-Reply-To: <20080306065523.429771E4003@bag.python.org> References: <20080306065523.429771E4003@bag.python.org> Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C4490@EXMBX04.exchhosting.com> Hurrah, 'x86 W2k8 trunk' has just experienced its first green build and test! Thanks to everyone that committed the various patches I sent out in such a timely fashion. Martin, does this mean I can have a slave set up for x64 now? }:> Trent. > -----Original Message----- > From: python-checkins-bounces at python.org [mailto:python-checkins- > bounces at python.org] On Behalf Of martin.v.loewis > Sent: 06 March 2008 01:55 > To: python-checkins at python.org > Subject: [Python-checkins] r61264 - in python/trunk: > Lib/test/test_os.py Misc/NEWS > > Author: martin.v.loewis > Date: Thu Mar 6 07:55:22 2008 > New Revision: 61264 > > Modified: > python/trunk/Lib/test/test_os.py > python/trunk/Misc/NEWS > Log: > Patch #2232: os.tmpfile might fail on Windows if the user has no > permission to create files in the root directory. > Will backport to 2.5. From martin at v.loewis.de Thu Mar 6 08:45:23 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 06 Mar 2008 08:45:23 +0100 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: References: <47CDC837.8000104@rksystems.com> <47CDD183.2040201@cheimes.de> <47CDE509.8080801@holdenweb.com> <47CF0A89.2010804@canterbury.ac.nz> Message-ID: <47CFA113.2080101@v.loewis.de> > I think the best lesson here is Tcl. Because it uses stubs mechanism, > you don't need to depend on tclXX.dll, you don't deal with really > direct implementation details, you don't care about runtimes, > everything is much easier. Maybe it's possible (and not too late) for > Python to somehow embrace such mechanism? It would be possible, but it would be a fairly large project. You would have to remove a lot of things from the Python header files, and that would cause significant breakage in existing extension modules. Regards, Martin From g.brandl at gmx.net Thu Mar 6 08:51:55 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 06 Mar 2008 08:51:55 +0100 Subject: [Python-Dev] Documentation for ability to execute zipfiles & directories In-Reply-To: <47CEAB01.5030008@gmail.com> References: <47CD421E.2010402@gmail.com> <79990c6b0803040455q7ce211d0y78956b3a395c3bad@mail.gmail.com> <47CEAB01.5030008@gmail.com> Message-ID: Nick Coghlan schrieb: > Georg Brandl wrote: >> Steve Holden schrieb: >>> Paul Moore wrote: >>>> On 04/03/2008, Nick Coghlan wrote: >>>>> Do we need a new appendix to the tutorial which goes into detail about >>>>> the CPython interpreter's command line options, environment variables >>>>> and details on what can be executed? >>>> There is a Python man page, which covers the command line usage. >>>> However, it's separate from the documentation, and it isn't bundled >>>> with the Windows installers - both of which are a real pain (for me, >>>> at least). >>>> >>>> I'd suggest taking the man page, adding the information about >>>> executing zip files and directories, and putting the whole lot into >>>> the formal documentation. >> >> Look no further: http://docs.python.org/dev/using/cmdline.html > > Thanks Georg, that looks like exactly the right place - I'll try to get > that updated before the next alpha. Great! Feel free to add anything you think would make it a more complete document. >>> I've always found it rather counter-intuitive that you have to go to the >>> Library Reference manual to find information about Python's built-in >>> types, for example. I though the whole point of libraries was that they >>> *aren't* built in, and represent baggage that should only be carried on >>> necessary trips. >> >> You speak my mind. For ages I've wanted to put the builtins together with >> the language reference into a new document called "Python Core Language". >> I've just never had the time to draft a serious proposal. > > I borrowed the keys to Guido's time machine: > > http://svn.python.org/view/sandbox/trunk/userref/ > > It hasn't been updated for a lot of the 2.6/3.0 features as yet, but it > may be a decent basis for what you're considering here. > > (and all credit to this thread for motivating me to actually get those > files cleaned up and into the sandbox - I've been thinking about doing > it for ages, but never got around to it). Thanks, I'll certainly look at them! Georg From martin at v.loewis.de Thu Mar 6 08:50:50 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 06 Mar 2008 08:50:50 +0100 Subject: [Python-Dev] [Python-checkins] r61264 - in python/trunk: Lib/test/test_os.py Misc/NEWS In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C4490@EXMBX04.exchhosting.com> References: <20080306065523.429771E4003@bag.python.org> <87D3F9C72FBF214DB39FA4E3FE618CDC6E166C4490@EXMBX04.exchhosting.com> Message-ID: <47CFA25A.3030506@v.loewis.de> Trent Nelson wrote: > Hurrah, 'x86 W2k8 trunk' has just experienced its first green build > and test! Thanks to everyone that committed the various patches I > sent out in such a timely fashion. > > Martin, does this mean I can have a slave set up for x64 now? }:> Versprochen ist versprochen :-) I'll send you the details. Martin From greg.ewing at canterbury.ac.nz Thu Mar 6 10:22:06 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 06 Mar 2008 22:22:06 +1300 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: References: <47CDC837.8000104@rksystems.com> <47CDD183.2040201@cheimes.de> <47CDE509.8080801@holdenweb.com> <47CF0A89.2010804@canterbury.ac.nz> Message-ID: <47CFB7BE.7040604@canterbury.ac.nz> Alexey Borzenkov wrote: > I think the best lesson here is Tcl. Because it uses stubs mechanism, > you don't need to depend on tclXX.dll, you don't deal with really > direct implementation details, you don't care about runtimes, How does that solve the problem of an extension using one stdio library and the base interpreter another? > Maybe it's possible (and not too late) for > Python to somehow embrace such mechanism? There would have to be an awful lot of stubs to cover the whole Python/C API... but maybe that isn't seen as a problem. -- Greg From steve at holdenweb.com Thu Mar 6 13:50:39 2008 From: steve at holdenweb.com (Steve Holden) Date: Thu, 06 Mar 2008 07:50:39 -0500 Subject: [Python-Dev] BSDDB3 In-Reply-To: <52dc1c820803052329g1fbc6e45y41db696901ec0b57@mail.gmail.com> References: <47C32D2D.1030206@free.fr> <47C4813F.2080403@v.loewis.de> <52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com> <20080227141322.GA5871@amk-desktop.matrixgroup.net> <47CDDF57.1020006@argo.es> <52dc1c820803052329g1fbc6e45y41db696901ec0b57@mail.gmail.com> Message-ID: Gregory P. Smith wrote: > > On 3/4/08, *Jesus Cea* > wrote: > > That said, it is my aim to keep bsddb in stdlib, providing a stable and > featureful module. I think keeping bsddb development inside python svn > is not appropiate. Currently (I could change idea), my approach will be > keeping pybssdb as a separate project and sync with python SVN from time > to time. Mainly to take advantage of buildbot architecture and, of > course, to be able to release python with current bindings. > > Since I have no python commit access, this seems a sensible approach. > And I could do frequent pybssdb releases (let say, every couple of > months) without waiting for a full python release (current approach). > > > That makes sense. I also agree with Neal's comments, merging back into > python in reasonable sized chunks is good. Don't worry about commit > access for now, I'll do the initial pybsddb back into python trunk > merges until we get that setup. I've merged the python trunk changes > that others have made back into the pybsddb tree. > > PS: I have tried to sign the Python Contributor Agreement, but I am not > sure about current pybsddb license terms. Help welcomed. > > > The current bsddb license first and foremost is the Python license. If > I read the comments in the _bsddb file correctly I believe you could > also call it a MIT style license. Keep things simple, just write > "Python License" on your contributor form and submit it. > Unfortunately that won't do it. At present the PSF can only accept contributions under two initial licenses: either the Academic Free License v 2.1 or the Apache License v 2.0. See http://www.python.org/psf/contrib/ This is because only these licenses have been approved by attorneys as giving the PSF the necessary unencumbered permission to relicense for distribution *under* the Python license. So Jesus was right to be concerned about licensing. I know it's a pain, but there are reasons. I don't see anything in the file to stop _bsddb.c being licensed to the PSF under either approved initial license. The licensing FAQ http://wiki.python.org/moin/PythonSoftwareFoundationLicenseFaq explains why you can't just contribute code "under the PSF license", and the contributor agreement requires that each contribution should include the contributor's valid copyright notice and the phrase "Licensed to PSF under a Contributor Agreement." This condition appears to be more honored in the breach than the observance though, since the phrase currently only seems to appear five times in the whole source unless I need to practice my grep-fu. As to the library that _bsddb wraps, I have not checked its licensing conditions and so cannot say how it is licensed to the PSF for redistribution, nor do I know whether Oracle's acquisition of SleepyCat will affect future versions. Bet it gave the MySQL guys some sleepless nights, though. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From jcea at argo.es Thu Mar 6 15:42:06 2008 From: jcea at argo.es (Jesus Cea) Date: Thu, 06 Mar 2008 15:42:06 +0100 Subject: [Python-Dev] Contributor Agreement (Re: BSDDB3) In-Reply-To: <52dc1c820803052329g1fbc6e45y41db696901ec0b57@mail.gmail.com> References: <47C32D2D.1030206@free.fr> <47C4813F.2080403@v.loewis.de> <52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com> <20080227141322.GA5871@amk-desktop.matrixgroup.net> <47CDDF57.1020006@argo.es> <52dc1c820803052329g1fbc6e45y41db696901ec0b57@mail.gmail.com> Message-ID: <47D002BE.7050101@argo.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Gregory P. Smith wrote: | The current bsddb license first and foremost is the Python license. If | I read the comments in the _bsddb file correctly I believe you could | also call it a MIT style license. Keep things simple, just write | "Python License" on your contributor form and submit it. But Python license is not accepted for Python Contributor Agreement. This is even in the FAQ. MIT is not accepted either. Software patents sucks :p - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBR9ACuJlgi5GaxT1NAQLSNwP/f4x813gpXcG3HhIbav1PVTMnzKhe6PNx 5jDxEwpUNoXFRtRH2aj5Et5ecUghm2B1sbsSX5Mpfb/yKWsc9yvyDe50Z05Tm961 MvPj3o39eztZr5LtHW+FwFJuUfbAO2SMC98aGDZ+9IhwNt3/3emB1hMNCh+90tFF aiPU0nQ14vU= =DIGQ -----END PGP SIGNATURE----- From martin at v.loewis.de Thu Mar 6 21:19:20 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 06 Mar 2008 21:19:20 +0100 Subject: [Python-Dev] Auto-Assignment Message-ID: <47D051C8.1060402@v.loewis.de> I just implemented auto-assignment for the tracker, i.e. a component can be linked to a developer so that issues mentioning the component get assigned to that developer (unless an explicit assignment is made). I have assigned 2to3 to Collin Winter (this was already implemented as a special case), and Documentation and Sphinx to Georg Brandl. Regards, Martin From g.brandl at gmx.net Thu Mar 6 23:16:54 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 06 Mar 2008 23:16:54 +0100 Subject: [Python-Dev] Auto-Assignment In-Reply-To: <47D051C8.1060402@v.loewis.de> References: <47D051C8.1060402@v.loewis.de> Message-ID: Martin v. L?wis schrieb: > I just implemented auto-assignment for the tracker, i.e. a component > can be linked to a developer so that issues mentioning the component > get assigned to that developer (unless an explicit assignment is > made). > > I have assigned 2to3 to Collin Winter (this was already implemented > as a special case), and Documentation and Sphinx to Georg Brandl. Thanks Martin! Georg From collinw at gmail.com Thu Mar 6 23:49:26 2008 From: collinw at gmail.com (Collin Winter) Date: Thu, 6 Mar 2008 14:49:26 -0800 Subject: [Python-Dev] Auto-Assignment In-Reply-To: <47D051C8.1060402@v.loewis.de> References: <47D051C8.1060402@v.loewis.de> Message-ID: <43aa6ff70803061449v74533819n5a4e12fe3c4f9132@mail.gmail.com> On Thu, Mar 6, 2008 at 12:19 PM, "Martin v. L?wis" wrote: > I just implemented auto-assignment for the tracker, i.e. a component > can be linked to a developer so that issues mentioning the component > get assigned to that developer (unless an explicit assignment is > made). > > I have assigned 2to3 to Collin Winter (this was already implemented > as a special case), and Documentation and Sphinx to Georg Brandl. Thanks! From jek-gmane at kleckner.net Fri Mar 7 00:41:33 2008 From: jek-gmane at kleckner.net (Jim Kleckner) Date: Thu, 06 Mar 2008 15:41:33 -0800 Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6a1 and 3.0a3 In-Reply-To: <47C9DD86.4040300@v.loewis.de> References: <6E72CEB8-D3BF-4440-A1EA-1A3D545CC8DB@python.org> <47C9D802.2090205@v.loewis.de> <05D00F06-8B87-4B94-8878-42E1F3F50F90@python.org> <47C9DD86.4040300@v.loewis.de> Message-ID: Martin v. L?wis wrote: >> Thanks for fixing these Martin! > > I have now also uploaded signed MSI files for 3.0a3. > > I have not tested them on a machine which doesn't have the > VS 2008 CRT installed (as all the machines I can access > right now do have it); please report what works and what > doesn't. When I install 2.6a1 onto a Windoze machine without any previous install, I get a dialog: There is a problem with this Windows Installer package. A program run as part of the setup did not finish as expected. Contact your support personnel or package vendor. Note that it didn't have VS2008 or any other new code installed. When running the Python console, running import Tkinter results in an error saying that _tkinter can't be loaded. DLL load failed. The system can't find _tkinter.pyd _tkinter.pyd and tcl84.dll and tk84.dll are in DLLs and appear to have reasonable permissions. I suspect something in the install failed to set something in the registry. Is there any log file of the install to inspect? From jek-gmane at kleckner.net Fri Mar 7 00:56:24 2008 From: jek-gmane at kleckner.net (Jim Kleckner) Date: Thu, 06 Mar 2008 15:56:24 -0800 Subject: [Python-Dev] RELEASED Python 2.6a1 and 3.0a3 In-Reply-To: References: <6E72CEB8-D3BF-4440-A1EA-1A3D545CC8DB@python.org> <47C9D802.2090205@v.loewis.de> <05D00F06-8B87-4B94-8878-42E1F3F50F90@python.org> <47C9DD86.4040300@v.loewis.de> Message-ID: Jim Kleckner wrote: > Martin v. L?wis wrote: >>> Thanks for fixing these Martin! >> I have now also uploaded signed MSI files for 3.0a3. >> >> I have not tested them on a machine which doesn't have the >> VS 2008 CRT installed (as all the machines I can access >> right now do have it); please report what works and what >> doesn't. > > When I install 2.6a1 onto a Windoze machine without any previous > install, I get a dialog: > There is a problem with this Windows Installer package. > A program run as part of the setup did not finish as expected. > Contact your support personnel or package vendor. > > Note that it didn't have VS2008 or any other new code installed. > > When running the Python console, running > import Tkinter > > results in an error saying that _tkinter can't be loaded. > DLL load failed. The system can't find _tkinter.pyd > > _tkinter.pyd and tcl84.dll and tk84.dll are in DLLs and > appear to have reasonable permissions. > > I suspect something in the install failed to set something > in the registry. Is there any log file of the install > to inspect? I forgot to mention that this is on a current-patch XP system and that I also tried python-2.6.13944.msi from the buildbot just in case after uninstalling/reinstalling. From martin at v.loewis.de Fri Mar 7 01:03:15 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 07 Mar 2008 01:03:15 +0100 Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6a1 and 3.0a3 In-Reply-To: References: <6E72CEB8-D3BF-4440-A1EA-1A3D545CC8DB@python.org> <47C9D802.2090205@v.loewis.de> <05D00F06-8B87-4B94-8878-42E1F3F50F90@python.org> <47C9DD86.4040300@v.loewis.de> Message-ID: <47D08643.50207@v.loewis.de> > I suspect something in the install failed to set something > in the registry. Is there any log file of the install > to inspect? You need to run msiexec /i /l*v to generate a logfile for the installation. Regards, Martin From martin at v.loewis.de Fri Mar 7 01:04:24 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 07 Mar 2008 01:04:24 +0100 Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6a1 and 3.0a3 In-Reply-To: References: <6E72CEB8-D3BF-4440-A1EA-1A3D545CC8DB@python.org> <47C9D802.2090205@v.loewis.de> <05D00F06-8B87-4B94-8878-42E1F3F50F90@python.org> <47C9DD86.4040300@v.loewis.de> Message-ID: <47D08688.7010702@v.loewis.de> > When I install 2.6a1 onto a Windoze machine without any previous > install, I get a dialog: > There is a problem with this Windows Installer package. > A program run as part of the setup did not finish as expected. > Contact your support personnel or package vendor. > > Note that it didn't have VS2008 or any other new code installed. > > When running the Python console, running How did you do that if you can't install the file? Regards, Martin From jek-gmane at kleckner.net Fri Mar 7 01:56:26 2008 From: jek-gmane at kleckner.net (Jim Kleckner) Date: Thu, 06 Mar 2008 16:56:26 -0800 Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6a1 and 3.0a3 In-Reply-To: <47D08688.7010702@v.loewis.de> References: <6E72CEB8-D3BF-4440-A1EA-1A3D545CC8DB@python.org> <47C9D802.2090205@v.loewis.de> <05D00F06-8B87-4B94-8878-42E1F3F50F90@python.org> <47C9DD86.4040300@v.loewis.de> <47D08688.7010702@v.loewis.de> Message-ID: Martin v. L?wis wrote: >> When I install 2.6a1 onto a Windoze machine without any previous >> install, I get a dialog: >> There is a problem with this Windows Installer package. >> A program run as part of the setup did not finish as expected. >> Contact your support personnel or package vendor. >> >> Note that it didn't have VS2008 or any other new code installed. >> >> When running the Python console, running > > How did you do that if you can't install the file? It got far enough to have entries in "All Programs/Python 2.6" From martin at v.loewis.de Fri Mar 7 07:40:50 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 07 Mar 2008 07:40:50 +0100 Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6a1 and 3.0a3 In-Reply-To: References: <6E72CEB8-D3BF-4440-A1EA-1A3D545CC8DB@python.org> <47C9D802.2090205@v.loewis.de> <05D00F06-8B87-4B94-8878-42E1F3F50F90@python.org> <47C9DD86.4040300@v.loewis.de> <47D08643.50207@v.loewis.de> Message-ID: <47D0E372.1080105@v.loewis.de> > Ok, I have the 3MB log file. Should I upload it > somewhere or is there a pattern to find? Please compress it, and make a bug report on bugs.python.org. Regards, Martin From theller at ctypes.org Fri Mar 7 08:13:13 2008 From: theller at ctypes.org (Thomas Heller) Date: Fri, 07 Mar 2008 08:13:13 +0100 Subject: [Python-Dev] Auto-Assignment In-Reply-To: <47D051C8.1060402@v.loewis.de> References: <47D051C8.1060402@v.loewis.de> Message-ID: Martin v. L?wis schrieb: > I just implemented auto-assignment for the tracker, i.e. a component > can be linked to a developer so that issues mentioning the component > get assigned to that developer (unless an explicit assignment is > made). > You can autoassign ctypes issues to me, as you might have guessed. Thomas From martin at v.loewis.de Fri Mar 7 08:42:02 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 07 Mar 2008 08:42:02 +0100 Subject: [Python-Dev] Auto-Assignment In-Reply-To: References: <47D051C8.1060402@v.loewis.de> Message-ID: <47D0F1CA.4090100@v.loewis.de> Thomas Heller wrote: > Martin v. L?wis schrieb: >> I just implemented auto-assignment for the tracker, i.e. a component >> can be linked to a developer so that issues mentioning the component >> get assigned to that developer (unless an explicit assignment is >> made). >> > > You can autoassign ctypes issues to me, as you might have guessed. ctypes is currently not a "component" in the tracker. Should it be one? Regards, Martin From theller at ctypes.org Fri Mar 7 08:58:04 2008 From: theller at ctypes.org (Thomas Heller) Date: Fri, 07 Mar 2008 08:58:04 +0100 Subject: [Python-Dev] Auto-Assignment In-Reply-To: <47D0F1CA.4090100@v.loewis.de> References: <47D051C8.1060402@v.loewis.de> <47D0F1CA.4090100@v.loewis.de> Message-ID: Martin v. L?wis schrieb: > Thomas Heller wrote: >> Martin v. L?wis schrieb: >>> I just implemented auto-assignment for the tracker, i.e. a component >>> can be linked to a developer so that issues mentioning the component >>> get assigned to that developer (unless an explicit assignment is >>> made). >>> >> >> You can autoassign ctypes issues to me, as you might have guessed. > > ctypes is currently not a "component" in the tracker. Should it be > one? > Hm, I didn't see that. What is a components, and do you think it should be one? Thomas From ggpolo at gmail.com Fri Mar 7 11:04:24 2008 From: ggpolo at gmail.com (Guilherme Polo) Date: Fri, 7 Mar 2008 07:04:24 -0300 Subject: [Python-Dev] Auto-Assignment In-Reply-To: References: <47D051C8.1060402@v.loewis.de> <47D0F1CA.4090100@v.loewis.de> Message-ID: 2008/3/7, Thomas Heller : > Martin v. L?wis schrieb: > > > Thomas Heller wrote: > >> Martin v. L?wis schrieb: > >>> I just implemented auto-assignment for the tracker, i.e. a component > >>> can be linked to a developer so that issues mentioning the component > >>> get assigned to that developer (unless an explicit assignment is > >>> made). > >>> > >> > >> You can autoassign ctypes issues to me, as you might have guessed. > > > > ctypes is currently not a "component" in the tracker. Should it be > > one? > > > > Hm, I didn't see that. > > What is a components, and do you think it should be one? > > > Thomas > Hi Thomas, When you create a new issue, you have to select a component from a list, those are the available components. -- -- Guilherme H. Polo Goncalves From asmodai at in-nomine.org Fri Mar 7 14:35:32 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Fri, 7 Mar 2008 14:35:32 +0100 Subject: [Python-Dev] Auto-Assignment In-Reply-To: References: <47D051C8.1060402@v.loewis.de> <47D0F1CA.4090100@v.loewis.de> Message-ID: <20080307133531.GO60713@nexus.in-nomine.org> -On [20080307 11:05], Guilherme Polo (ggpolo at gmail.com) wrote: >When you create a new issue, you have to select a component from a >list, those are the available components. I think Thomas was wondering what the definition for a component was within Roundup/the Python project and if ctypes would need to be a component entry as such. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Absence makes the heart grow fonder... From theller at ctypes.org Fri Mar 7 15:59:45 2008 From: theller at ctypes.org (Thomas Heller) Date: Fri, 07 Mar 2008 15:59:45 +0100 Subject: [Python-Dev] Auto-Assignment In-Reply-To: <20080307133531.GO60713@nexus.in-nomine.org> References: <47D051C8.1060402@v.loewis.de> <47D0F1CA.4090100@v.loewis.de> <20080307133531.GO60713@nexus.in-nomine.org> Message-ID: Jeroen Ruigrok van der Werven schrieb: > -On [20080307 11:05], Guilherme Polo (ggpolo at gmail.com) wrote: >>When you create a new issue, you have to select a component from a >>list, those are the available components. > > I think Thomas was wondering what the definition for a component was within > Roundup/the Python project and if ctypes would need to be a component entry > as such. > Exactly. Thomas From guido at python.org Fri Mar 7 16:40:17 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 7 Mar 2008 07:40:17 -0800 Subject: [Python-Dev] Auto-Assignment In-Reply-To: References: <47D051C8.1060402@v.loewis.de> <47D0F1CA.4090100@v.loewis.de> <20080307133531.GO60713@nexus.in-nomine.org> Message-ID: ctypes is big and important enough IMO to deserve a component. On Fri, Mar 7, 2008 at 6:59 AM, Thomas Heller wrote: > Jeroen Ruigrok van der Werven schrieb: > > > -On [20080307 11:05], Guilherme Polo (ggpolo at gmail.com) wrote: > >>When you create a new issue, you have to select a component from a > >>list, those are the available components. > > > > I think Thomas was wondering what the definition for a component was within > > Roundup/the Python project and if ctypes would need to be a component entry > > as such. > > > > Exactly. > > Thomas > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From status at bugs.python.org Fri Mar 7 18:06:28 2008 From: status at bugs.python.org (Tracker) Date: Fri, 7 Mar 2008 18:06:28 +0100 (CET) Subject: [Python-Dev] Summary of Tracker Issues Message-ID: <20080307170628.BF02078173@psf.upfronthosting.co.za> ACTIVITY SUMMARY (02/29/08 - 03/07/08) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1730 open (+28) / 12348 closed (+13) / 14078 total (+41) Open issues with patches: 468 Average duration of open issues: 739 days. Median duration of open issues: 1136 days. Open Issues Breakdown open 1706 (+25) pending 24 ( +3) Issues Created Or Reopened (41) _______________________________ Cookie.Morsel interface needs update 02/29/08 http://bugs.python.org/issue2211 created astronouth7303 Cookie.BaseCookie has ambiguous unicode handling 02/29/08 http://bugs.python.org/issue2212 created astronouth7303 build_tkinter.py does not handle paths with spaces 03/01/08 http://bugs.python.org/issue2213 created JosephArmbruster make htmlhelp raises non-reproducable exception 03/01/08 CLOSED http://bugs.python.org/issue2214 created loewis test_sqlite fails in 2.6a1 on MacOS X 03/02/08 http://bugs.python.org/issue2215 created MrJean1 Problems using logging module with idle 03/02/08 http://bugs.python.org/issue2216 created ag6502 patch Problem with if clause in generator expression on class level 03/02/08 CLOSED http://bugs.python.org/issue2217 created cito Enhanced hotshot profiler with high-resolution timer 03/02/08 http://bugs.python.org/issue2218 created MrJean1 Py30a3: Possibly confusing message when module detection fails 03/03/08 http://bugs.python.org/issue2219 created mark bug in rlcompleter 03/03/08 CLOSED http://bugs.python.org/issue2220 created donlorenzo patch, easy Py30a3: calltip produces error output to stderr 03/03/08 http://bugs.python.org/issue2221 created mark Memory leak in os.rename? 03/03/08 http://bugs.python.org/issue2222 created ocean-city patch regrtest.py -R not working 03/03/08 http://bugs.python.org/issue2223 created ocean-city patch branches/trunk-math patch 03/03/08 http://bugs.python.org/issue2224 created tiran patch py_compile.main() does not return error code 03/03/08 CLOSED http://bugs.python.org/issue2225 created catlee patch, easy Small _abcoll Bugs / Oddities 03/03/08 http://bugs.python.org/issue2226 created aronacher time.strptime too strict? should it assume current year? 03/03/08 http://bugs.python.org/issue2227 created gregory.p.smith easy Imaplib speedup patch 03/03/08 http://bugs.python.org/issue2228 created aaronkaplan Search in 2.6 docs does not work 03/04/08 http://bugs.python.org/issue2229 created tebeka Document PyArg_Parse* behavior on failed conversion 03/04/08 CLOSED http://bugs.python.org/issue2230 created belopolsky patch Memory leak in itertools.chain() 03/04/08 CLOSED http://bugs.python.org/issue2231 created ocean-city patch Current os.tmpfile() implementation requires admin privs on Vist 03/04/08 CLOSED http://bugs.python.org/issue2232 created Trent.Nelson patch Makefile.pre.in contains extra slash before $(DESTDIR) which can 03/04/08 http://bugs.python.org/issue2233 created jlt63 patch, patch cygwinccompiler.py fails for latest MinGW releases. 03/04/08 http://bugs.python.org/issue2234 created kermode patch __eq__ / __hash__ check doesn't take inheritance into account 03/04/08 http://bugs.python.org/issue2235 created jek patch Distutils' mkpath implementation ignoring the "mode" parameter 03/04/08 http://bugs.python.org/issue2236 created henrique patch Inconsistent variable lookup 03/04/08 CLOSED http://bugs.python.org/issue2237 created swarecki TypeError instead of SyntaxError for syntactically invalid gen e 03/05/08 http://bugs.python.org/issue2238 created NathanCollins Tiny patch to cmdline docs 03/05/08 CLOSED http://bugs.python.org/issue2239 created tim.golden patch setitimer, getitimer wrapper 03/05/08 http://bugs.python.org/issue2240 created gpolo patch Additional Flag For Unit-Test Module: There Can Be Only One (Err 03/05/08 CLOSED http://bugs.python.org/issue2241 created bcwhite Decoding UTF-7 with "ignore warnings" crashes Python on Windows 03/06/08 http://bugs.python.org/issue2242 created cpalmer urllib2. strange behavior for getting chuncked transfer-ecnoded 03/06/08 http://bugs.python.org/issue2243 created begemoth urllib and urllib2 decode userinfo multiple times 03/06/08 http://bugs.python.org/issue2244 created carljm patch aifc cannot handle unrecognised chunk type "CHAN" 03/06/08 http://bugs.python.org/issue2245 created loki_dePlume itertools.groupby() leaks memory with circular reference 03/06/08 CLOSED http://bugs.python.org/issue2246 created asmodai patch Test auto-assignment 03/06/08 CLOSED http://bugs.python.org/issue2247 created loewis quit() method of SMTP instance (of smtplib) doesn't return it's 03/07/08 http://bugs.python.org/issue2248 created funagayama patch To document "assertTrue" in unittest 03/07/08 http://bugs.python.org/issue2249 created jcea rlcompleter raises Exception on bad input 03/07/08 http://bugs.python.org/issue2250 created donlorenzo patch tarfile using nonexistent function 03/07/08 CLOSED http://bugs.python.org/issue2251 created GotenXiao Issues Now Closed (24) ______________________ libffi needs an update to support mips64, arm and armeabi on lin 13 days http://bugs.python.org/issue1292 theller use unittest for test_logging 57 days http://bugs.python.org/issue1740 brett.cannon patch change the bool struct format code from 't' to '?' 46 days http://bugs.python.org/issue1872 theller patch with should be as fast as try/finally 12 days http://bugs.python.org/issue2179 amaury.forgeotdarc patch cPickle error with gtk GEnum classes 1 days http://bugs.python.org/issue2199 loewis Bug in Sphinx highlighting when pygments not available 0 days http://bugs.python.org/issue2207 georg.brandl patch Patch to doc/make.bat to allow non-standard HTML Help location 0 days http://bugs.python.org/issue2208 georg.brandl patch make htmlhelp raises non-reproducable exception 0 days http://bugs.python.org/issue2214 loewis Problem with if clause in generator expression on class level 1 days http://bugs.python.org/issue2217 georg.brandl bug in rlcompleter 3 days http://bugs.python.org/issue2220 georg.brandl patch, easy py_compile.main() does not return error code 3 days http://bugs.python.org/issue2225 georg.brandl patch, easy Document PyArg_Parse* behavior on failed conversion 0 days http://bugs.python.org/issue2230 georg.brandl patch Memory leak in itertools.chain() 0 days http://bugs.python.org/issue2231 rhettinger patch Current os.tmpfile() implementation requires admin privs on Vist 2 days http://bugs.python.org/issue2232 loewis patch Inconsistent variable lookup 0 days http://bugs.python.org/issue2237 amaury.forgeotdarc Tiny patch to cmdline docs 0 days http://bugs.python.org/issue2239 georg.brandl patch Additional Flag For Unit-Test Module: There Can Be Only One (Err 1 days http://bugs.python.org/issue2241 purcell itertools.groupby() leaks memory with circular reference 0 days http://bugs.python.org/issue2246 rhettinger patch Test auto-assignment 0 days http://bugs.python.org/issue2247 loewis tarfile using nonexistent function 0 days http://bugs.python.org/issue2251 lars.gustaebel Rational.py various bugfixes 1290 days http://bugs.python.org/issue1012468 jyasskin patch Memory leak in socket.py on Mac OS X 1164 days http://bugs.python.org/issue1092502 vila imaplib causes excessive fragmentation for large documents 805 days http://bugs.python.org/issue1389051 vila patch, easy Distutils default exclude doesn't match top level .svn 286 days http://bugs.python.org/issue1725737 schmir patch Top Issues Most Discussed (10) ______________________________ 13 regrtest.py -R not working 4 days open http://bugs.python.org/issue2223 9 setitimer, getitimer wrapper 2 days open http://bugs.python.org/issue2240 8 Current os.tmpfile() implementation requires admin privs on Vis 2 days closed http://bugs.python.org/issue2232 8 with should be as fast as try/finally 12 days closed http://bugs.python.org/issue2179 8 shutil.destinsrc returns wrong result when source path matches 28 days open http://bugs.python.org/issue2047 7 Tiny patch to cmdline docs 0 days closed http://bugs.python.org/issue2239 7 __eq__ / __hash__ check doesn't take inheritance into account 3 days open http://bugs.python.org/issue2235 7 test_sqlite fails in 2.6a1 on MacOS X 6 days open http://bugs.python.org/issue2215 6 itertools.groupby() leaks memory with circular reference 0 days closed http://bugs.python.org/issue2246 6 Memory leak in os.rename? 4 days open http://bugs.python.org/issue2222 From martin at v.loewis.de Fri Mar 7 19:01:11 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 07 Mar 2008 19:01:11 +0100 Subject: [Python-Dev] Auto-Assignment In-Reply-To: References: <47D051C8.1060402@v.loewis.de> <47D0F1CA.4090100@v.loewis.de> <20080307133531.GO60713@nexus.in-nomine.org> Message-ID: <47D182E7.2090801@v.loewis.de> > ctypes is big and important enough IMO to deserve a component. Ok, I just created the component and made Thomas the auto-assignee for it. http://bugs.python.org/component22 Regards, Martin From jek-gmane at kleckner.net Sat Mar 8 00:35:12 2008 From: jek-gmane at kleckner.net (Jim Kleckner) Date: Fri, 07 Mar 2008 15:35:12 -0800 Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6a1 and 3.0a3 In-Reply-To: <47D0E372.1080105@v.loewis.de> References: <6E72CEB8-D3BF-4440-A1EA-1A3D545CC8DB@python.org> <47C9D802.2090205@v.loewis.de> <05D00F06-8B87-4B94-8878-42E1F3F50F90@python.org> <47C9DD86.4040300@v.loewis.de> <47D08643.50207@v.loewis.de> <47D0E372.1080105@v.loewis.de> Message-ID: Martin v. L?wis wrote: > Please compress it, and make a bug report on > bugs.python.org. Submitted as: http://bugs.python.org/issue2256 From R.W.Thomas.02 at cantab.net Sat Mar 8 10:09:30 2008 From: R.W.Thomas.02 at cantab.net (Richard Thomas) Date: Sat, 8 Mar 2008 09:09:30 +0000 Subject: [Python-Dev] Readline completion hook silences exceptions. Message-ID: <1b54228d0803080109r4430dca1n8891a1d6857b4f38@mail.gmail.com> Hey, I've never posted to this list before but I think it is the right place for discussions such as this: The readline completion hook doesn't reraise exceptions. This is probably a good thing for actually using readline, less useful for debugging a completer. Possibly readline should have a 'debug' mode where it doesn't silence? Richard. From asmodai at in-nomine.org Sat Mar 8 10:24:34 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Sat, 8 Mar 2008 10:24:34 +0100 Subject: [Python-Dev] FreeBSD test suite failure -> curses Message-ID: <20080308092434.GZ60713@nexus.in-nomine.org> So I added my first buildbot yesterday (for FreeBSD and I hope to add a few more different BSD ones to the fray) and I see that it is failing in the test_curses part. I tracked it by hand and somewhere along the test it segfaults. The resulting coredump is not really helpful. (gdb) bt #0 0x281aa61f in pthread_testcancel () from /lib/libpthread.so.2 #1 0x281a2a52 in pthread_mutexattr_init () from /lib/libpthread.so.2 #2 0x28167450 in ?? () Anybody have any hints where I should poke around? -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Looking for the Sun that eclipsed behind black feathered wings... From martin at v.loewis.de Sat Mar 8 12:04:23 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 08 Mar 2008 12:04:23 +0100 Subject: [Python-Dev] Readline completion hook silences exceptions. In-Reply-To: <1b54228d0803080109r4430dca1n8891a1d6857b4f38@mail.gmail.com> References: <1b54228d0803080109r4430dca1n8891a1d6857b4f38@mail.gmail.com> Message-ID: <47D272B7.6070101@v.loewis.de> > The readline completion hook doesn't reraise exceptions. This is > probably a good thing for actually using readline, less useful for > debugging a completer. Possibly readline should have a 'debug' mode > where it doesn't silence? Errors should never pass silently. Unless explicitly silenced. Please submit a patch. Regards, Martin From martin at v.loewis.de Sat Mar 8 12:08:32 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 08 Mar 2008 12:08:32 +0100 Subject: [Python-Dev] FreeBSD test suite failure -> curses In-Reply-To: <20080308092434.GZ60713@nexus.in-nomine.org> References: <20080308092434.GZ60713@nexus.in-nomine.org> Message-ID: <47D273B0.4020809@v.loewis.de> > So I added my first buildbot yesterday (for FreeBSD and I hope to add a few > more different BSD ones to the fray) and I see that it is failing in the > test_curses part. > > I tracked it by hand and somewhere along the test it segfaults. The > resulting coredump is not really helpful. > > (gdb) bt > #0 0x281aa61f in pthread_testcancel () from /lib/libpthread.so.2 > #1 0x281a2a52 in pthread_mutexattr_init () from /lib/libpthread.so.2 > #2 0x28167450 in ?? () > > Anybody have any hints where I should poke around? If you want to ask for help, that's probably something for the FreeBSD lists. Unfortunately, the FreeBSD threads library is notorious for crashing, hanging, and doing other unpleasant things to Python. If you want to debug this, run Python in a debugger, and run the test suite from there. core files are often not too helpful. For the core file, make sure you select the right thread to inspect - this is probably not the one which caused the crash; use "info threads" as a starting point (hoping that the core file supports that). Regards, Martin From jimis at gmx.net Sun Mar 9 07:28:02 2008 From: jimis at gmx.net (Dimitrios Apostolou) Date: Sun, 9 Mar 2008 06:28:02 +0000 (GMT) Subject: [Python-Dev] Complexity documentation request Message-ID: Hello all, Is it possible to include algorithm complexity information for the various built-in types (strings, sets, lists, dictionaries...)? This would ease the decision for choosing the correct type. The information could simply be added as a new column in the tables found on pages as the following: http://docs.python.org/lib/types-set.html http://docs.python.org/lib/typesseq.html It took me a while to find some information for my purposes, however I'm not sure whether it's outdated or incomplete. The best sources I found are python-list archive and a PEP: http://www.python.org/dev/peps/pep-3128/ http://mail.python.org/pipermail/python-list/2007-June/446877.html Nevertheless, algorithm complexity is not always the answer. I remember some years ago I preferred using "x in list" rather than "x in set" as member checking in a tight loop, because the list was rather small (10-20 elements) and the overhead for the set was far greater than the better algorithm complexity advantage. So if this information is available, it would be nice to document constant time factors too. :-) Thanks in advance, Dimitris From ziade.tarek at gmail.com Sun Mar 9 12:25:43 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 9 Mar 2008 12:25:43 +0100 Subject: [Python-Dev] Distutils - changing the pypirc format Message-ID: <94bdd2610803090425q4fd82e42y6f3d3c1cc2434d8a@mail.gmail.com> Hello, I would like to suggest changing the .pypirc file format to a format that can deal with several servers. This has been discussed here : http://mail.python.org/pipermail/catalog-sig/2008-January/001607.html And a summary document is available here: http://wiki.python.org/moin/EnhancedPyPI (this document also contains a change ti PyPI server but this is another point) A patch was built and enhanced in the last bug day, and is available here: http://bugs.python.org/issue1858. Best Regards Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080309/806c7e84/attachment.htm From michal at yogi.htu.tuwien.ac.at Sun Mar 9 14:53:14 2008 From: michal at yogi.htu.tuwien.ac.at (Michal Revucky) Date: Sun, 9 Mar 2008 14:53:14 +0100 Subject: [Python-Dev] RQST: Master Thesis Message-ID: <20080309135314.GC6263@yogi> hi, I'm about to start my master thesis: "optimizing python interpreter" therefore i would require your help plz ;) first of all i need to find out, how the python interpreter is implemented, and which are the related sources, if this stuff is somewhere documented that would be very helpfull... thx in advance michal From aahz at pythoncraft.com Sun Mar 9 15:22:39 2008 From: aahz at pythoncraft.com (Aahz) Date: Sun, 9 Mar 2008 07:22:39 -0700 Subject: [Python-Dev] Complexity documentation request In-Reply-To: References: Message-ID: <20080309142239.GA18295@panix.com> On Sun, Mar 09, 2008, Dimitrios Apostolou wrote: > > Is it possible to include algorithm complexity information for the various > built-in types (strings, sets, lists, dictionaries...)? This would ease > the decision for choosing the correct type. This has been discussed before and rejected for two reasons: * Other Python implementations (Jython, PyPy, IronPython) may not be able to provide the same type implementations * Algorithmic information does sometimes change between versions, and keeping the docs updated is not trivial There probably would be some value in a wiki page on python.org that provides this information, particularly across versions. You may be able to find volunteers to help on comp.lang.python. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson From skip at pobox.com Sun Mar 9 15:42:30 2008 From: skip at pobox.com (skip at pobox.com) Date: Sun, 9 Mar 2008 09:42:30 -0500 Subject: [Python-Dev] RQST: Master Thesis In-Reply-To: <20080309135314.GC6263@yogi> References: <20080309135314.GC6263@yogi> Message-ID: <18387.63318.76728.195090@montanaro-dyndns-org.local> Michal> I'm about to start my master thesis: "optimizing python Michal> interpreter" therefore i would require your help plz ;) Michal> first of all i need to find out, how the python interpreter is Michal> implemented, and which are the related sources, if this stuff is Michal> somewhere documented that would be very helpfull... Michal, The best place to start is the implementation of the interpreter itself (Python/ceval.c in the source tree) and the byte code compiler (Python/compile.c). There have, at times, been modest attempts to document what's going on, but I believe the source code is far and away still your best bet. I recommend you get a Subversion checkout of either the trunk: svn co http://svn.python.org/projects/python/trunk or the py3k branch: svn co http://svn.python.org/projects/python/branches/py3k You will find the trunk more stable, but the py3k branch is the future. Also, study the archives of this list and the python-3000 at python.org list as well as checkin comments for the Python source, especially for the two files I referenced above. Finally, there may well be pending patches in the Python tracker which implement various optimizations: http://bugs.python.org/ Studying them (and trying them out and attaching comments or reviews of their efficacy) would be a good way to help understand the system. -- Skip Montanaro - skip at pobox.com - http://www.webfast.com/~skip/ From arigo at tunes.org Sun Mar 9 17:02:49 2008 From: arigo at tunes.org (Armin Rigo) Date: Sun, 9 Mar 2008 17:02:49 +0100 Subject: [Python-Dev] Equality on method objects Message-ID: <20080309160249.GA32007@code0.codespeak.net> Hi all, In Python 2.5, I made an attempt to make equality consistent for the various built-in and user-defined method types. I failed, though, as explained in http://bugs.python.org/issue1617161. The outcome of this discussion is that, first of all, we need to decide which behavior is "correct": >>> [].append == [].append True or False? (See the issue tracker for why the answer should probably be False.) The general question is: if x.foo and y.foo resolve to the same method, should "x.foo == y.foo" delegate to "x == y" or be based on "x is y"? The behavior about this has always been purely accidental, with three different results for user-defined methods versus built-in methods versus method wrappers (those who know what the latter are, raise your hand). (Yes, Python < 2.5 managed three different behaviors instead of just two: one of the types (don't ask me which) would base its equality on the identity of the 'self', but still compute its hash from the hash of 'self'...) A bientot, Armin From fdrake at acm.org Sun Mar 9 17:23:14 2008 From: fdrake at acm.org (Fred Drake) Date: Sun, 9 Mar 2008 12:23:14 -0400 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <20080309142239.GA18295@panix.com> References: <20080309142239.GA18295@panix.com> Message-ID: <76EFA410-DD8D-4F4B-A299-C298E92454FA@acm.org> On Mar 9, 2008, at 10:22 AM, Aahz wrote: > This has been discussed before and rejected for two reasons: > > * Other Python implementations (Jython, PyPy, IronPython) may not be > able to provide the same type implementations > > * Algorithmic information does sometimes change between versions, and > keeping the docs updated is not trivial Also, given the significance of the constant factors and the fact that these are significantly dependent, especially for containers, on the objects passed in (either to be contained or tested), it's not clear that it often makes sense to worry about at too detailed a level. Common sense, knowledge of the interpreter, and experience are often more valuable the easily-outdated documentation. -Fred -- Fred Drake From asmodai at in-nomine.org Sun Mar 9 17:29:00 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Sun, 9 Mar 2008 17:29:00 +0100 Subject: [Python-Dev] FreeBSD test suite failure -> curses In-Reply-To: <47D273B0.4020809@v.loewis.de> References: <20080308092434.GZ60713@nexus.in-nomine.org> <47D273B0.4020809@v.loewis.de> Message-ID: <20080309162900.GA60713@nexus.in-nomine.org> -On [20080308 12:08], "Martin v. L?wis" (martin at v.loewis.de) wrote: >If you want to ask for help, that's probably something for the FreeBSD >lists. Unfortunately, the FreeBSD threads library is notorious for >crashing, hanging, and doing other unpleasant things to Python. I think I can manage the FreeBSD side, I used to be a committer for it. ;) >If you want to debug this, run Python in a debugger, and run the >test suite from there. core files are often not too helpful. One result I get is this: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x8173000 (LWP 100121)] 0x28ac4102 in PyCurses_getsyx (self=0x0) at /usr/home/asmodai/projects/python/Modules/_cursesmodule.c:1770 1770 getsyx(y, x); Is a value of 0x0 valid for self? I'd figure it would be a memory address, but I do not know the internals well enough for that. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Delirious again, mesmerise my senses, our Souls entwine one more time... From rhamph at gmail.com Sun Mar 9 18:51:51 2008 From: rhamph at gmail.com (Adam Olsen) Date: Sun, 9 Mar 2008 10:51:51 -0700 Subject: [Python-Dev] Equality on method objects In-Reply-To: <20080309160249.GA32007@code0.codespeak.net> References: <20080309160249.GA32007@code0.codespeak.net> Message-ID: On Sun, Mar 9, 2008 at 9:02 AM, Armin Rigo wrote: > Hi all, > > In Python 2.5, I made an attempt to make equality consistent for the > various built-in and user-defined method types. I failed, though, as > explained in http://bugs.python.org/issue1617161. The outcome of this > discussion is that, first of all, we need to decide which behavior is > "correct": > > >>> [].append == [].append > True or False? > > (See the issue tracker for why the answer should probably be False.) > > The general question is: if x.foo and y.foo resolve to the same method, > should "x.foo == y.foo" delegate to "x == y" or be based on "x is y"? > > The behavior about this has always been purely accidental, with three > different results for user-defined methods versus built-in methods > versus method wrappers (those who know what the latter are, raise your > hand). > > (Yes, Python < 2.5 managed three different behaviors instead of just > two: one of the types (don't ask me which) would base its equality on > the identity of the 'self', but still compute its hash from the hash of > 'self'...) They should only compare equal if interchangeable. In the case of a mutable container (ie list), a value comparison of self is irrelevant garbage, so it should always be compared (and hashed) based on identity. IOW, "x = []; x.append == x.append" should be True, and everything else should be False. -- Adam Olsen, aka Rhamphoryncus From martin at v.loewis.de Sun Mar 9 20:23:18 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 09 Mar 2008 20:23:18 +0100 Subject: [Python-Dev] FreeBSD test suite failure -> curses In-Reply-To: <20080309162900.GA60713@nexus.in-nomine.org> References: <20080308092434.GZ60713@nexus.in-nomine.org> <47D273B0.4020809@v.loewis.de> <20080309162900.GA60713@nexus.in-nomine.org> Message-ID: <47D43926.4030902@v.loewis.de> > One result I get is this: > > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x8173000 (LWP 100121)] > 0x28ac4102 in PyCurses_getsyx (self=0x0) > at /usr/home/asmodai/projects/python/Modules/_cursesmodule.c:1770 > 1770 getsyx(y, x); > > Is a value of 0x0 valid for self? I'd figure it would be a memory address, > but I do not know the internals well enough for that. That's fine. It's a module-level function; self is typically NULL for these. Another issue is that there should be an additional (ignored) parameter args in PyCurses_getsyx; you might try adding one. However, the real culprit should be getsyx(). What does this entire function expand to when run through a preprocessor, and where does it crash when you run the expanded code in the debugger? For reference, on Linux (ncurses) it expands to do { if (((newscr)->_leaveok)) (y) = (x) = -1; else ((y) = ((newscr) ? (newscr)->_cury : (-1)), (x) = ((newscr) ? (newscr)->_curx : (-1))); } while(0); which should be equivalent to if (newscr->_leaveok) y = x = -1; else { y = newscr ? newscr->_cury : -1; x = newscr ? newscr->_curx : -1; } If it's similar on FreeBSD, the only reason why this could break is that newscr is NULL. Regards, Martin From greg.ewing at canterbury.ac.nz Sun Mar 9 20:50:02 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 10 Mar 2008 08:50:02 +1300 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <20080309142239.GA18295@panix.com> References: <20080309142239.GA18295@panix.com> Message-ID: <47D43F6A.8000808@canterbury.ac.nz> Aahz wrote: > * Other Python implementations (Jython, PyPy, IronPython) may not be > able to provide the same type implementations > > * Algorithmic information does sometimes change between versions, and > keeping the docs updated is not trivial Still, I think there are some general expectations one should be able to have of any implementation, e.g. that accessing a list item is roughly O(1), and not O(n) as it would be if they were implemented as linked lists. Dict access should probably be documented as no worse than O(log n) to allow for tree implementations. -- Greg From asmodai at in-nomine.org Sun Mar 9 22:02:10 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Sun, 9 Mar 2008 22:02:10 +0100 Subject: [Python-Dev] FreeBSD test suite failure -> curses In-Reply-To: <47D43926.4030902@v.loewis.de> References: <20080308092434.GZ60713@nexus.in-nomine.org> <47D273B0.4020809@v.loewis.de> <20080309162900.GA60713@nexus.in-nomine.org> <47D43926.4030902@v.loewis.de> Message-ID: <20080309210210.GB60713@nexus.in-nomine.org> -On [20080309 20:23], "Martin v. L?wis" (martin at v.loewis.de) wrote: >If it's similar on FreeBSD, the only reason why this >could break is that newscr is NULL. This is what I get: do { if(newscr->_leaveok) (y) = (x) = -1; else ((y) = ((newscr) ? (newscr)->_cury : (-1)), (x) = ((newscr) ? (newscr)->_curx : (-1))); } while(0); ncurses 5.6 In the same stack frame: (gdb) print newscr $1 = (WINDOW *) 0x0 So it seems that the newscr variable is indeed a NULL. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Let us eat and drink; for tomorrow we shall die... From guido at python.org Sun Mar 9 22:43:21 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 9 Mar 2008 13:43:21 -0800 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <47D43F6A.8000808@canterbury.ac.nz> References: <20080309142239.GA18295@panix.com> <47D43F6A.8000808@canterbury.ac.nz> Message-ID: On Sun, Mar 9, 2008 at 11:50 AM, Greg Ewing wrote: > Aahz wrote: > > * Other Python implementations (Jython, PyPy, IronPython) may not be > > able to provide the same type implementations > > > > * Algorithmic information does sometimes change between versions, and > > keeping the docs updated is not trivial > > Still, I think there are some general expectations one > should be able to have of any implementation, e.g. that > accessing a list item is roughly O(1), and not O(n) as > it would be if they were implemented as linked lists. > > Dict access should probably be documented as no worse > than O(log n) to allow for tree implementations. Well, there you have hit the nail on the head -- should we document the actual or the guaranteed O() expression? I think this is a can of worms better left unopened. At best we should include some hints to clarify that random access to list and tuple elements is constant time in CPython, and that dicts and sets are implemented using a hash table with open hashing -- readers can draw their own conclusions from that. What other implementations do should be up to them -- it becomes a Quality of Implementation issue. Regarding the OP's need for this kind of information (deciding between the various standard data types), I would recommend a different approach -- pick the data type that produces the shortest code. In all likelihood it's also the most efficient. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Mar 9 22:59:24 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 9 Mar 2008 13:59:24 -0800 Subject: [Python-Dev] Equality on method objects In-Reply-To: References: <20080309160249.GA32007@code0.codespeak.net> Message-ID: On Sun, Mar 9, 2008 at 9:51 AM, Adam Olsen wrote: > On Sun, Mar 9, 2008 at 9:02 AM, Armin Rigo wrote: > > Hi all, > > > > In Python 2.5, I made an attempt to make equality consistent for the > > various built-in and user-defined method types. I failed, though, as > > explained in http://bugs.python.org/issue1617161. The outcome of this > > discussion is that, first of all, we need to decide which behavior is > > "correct": > > > > >>> [].append == [].append > > True or False? > > > > (See the issue tracker for why the answer should probably be False.) > > > > The general question is: if x.foo and y.foo resolve to the same method, > > should "x.foo == y.foo" delegate to "x == y" or be based on "x is y"? > > > > The behavior about this has always been purely accidental, with three > > different results for user-defined methods versus built-in methods > > versus method wrappers (those who know what the latter are, raise your > > hand). > > > > (Yes, Python < 2.5 managed three different behaviors instead of just > > two: one of the types (don't ask me which) would base its equality on > > the identity of the 'self', but still compute its hash from the hash of > > 'self'...) > > They should only compare equal if interchangeable. In the case of a > mutable container (ie list), a value comparison of self is irrelevant > garbage, so it should always be compared (and hashed) based on > identity. > > IOW, "x = []; x.append == x.append" should be True, and everything > else should be False. Perhaps. I think we should approach this from the perspective of what is the most useful behavior that is straightforward to implement, not what is "right" (since there is more than one "right"). Do we have much of a use case for this? If not, I'd be okay with not providing these various forms of bound methods with equality at all, and just fall back on 'is'. While it may be surprising that x.append is not x.append, it should be no more surprising than x+1 is not x+1 under certain circumstances, and both actually provide a useful insight into the implementation. That said, if there's a use case, I agree that it would be okay with basing the equality of x.foo and y.foo on whether x and y are the same object, not on whether x==y (consider 0 .__add__ == 0.0 .__add__). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Sun Mar 9 23:42:05 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 10 Mar 2008 08:42:05 +1000 Subject: [Python-Dev] Equality on method objects In-Reply-To: References: <20080309160249.GA32007@code0.codespeak.net> Message-ID: <47D467BD.2000800@gmail.com> Guido van Rossum wrote: > That said, if there's a use case, I agree that it would be okay with > basing the equality of x.foo and y.foo on whether x and y are the same > object, not on whether x==y (consider 0 .__add__ == 0.0 .__add__). The use case in the issue tracker was maintaining a collection of unique callbacks, some of which could be bound methods - the current behaviour is actively harmful to that use case, since some of the later callbacks may fail to register properly (due to their self comparing equal to another instance of the same type that already had its callback method in the list). That same use case is what makes it useful to consider the same method on a single object to be equal - there is little point in having a bound method like "x.notify" in a callback list twice. So for myself, +1 on acknowledging issue 1617161 as a bug and fixing it as Armin suggests. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From daniel at stutzbachenterprises.com Sun Mar 9 23:17:53 2008 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sun, 9 Mar 2008 17:17:53 -0500 Subject: [Python-Dev] Complexity documentation request In-Reply-To: References: <20080309142239.GA18295@panix.com> <47D43F6A.8000808@canterbury.ac.nz> Message-ID: On Sun, Mar 9, 2008 at 4:43 PM, Guido van Rossum wrote: > Well, there you have hit the nail on the head -- should we document > the actual or the guaranteed O() expression? Either. Both. Put a note at the bottom saying that factors of O(log n) have been dropped and they're basically the same thing (this is sometimes called "Soft O notation"). Big O is technically an upper-bound anyway. When was the last time a new version caused a function to become slower by more than a factor of O(log n)? As is, some operations *are* documented, but in odd places. For example, the documentation for deque describes the complexity of some of the list type's operations. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From martin at v.loewis.de Sun Mar 9 23:58:58 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 09 Mar 2008 23:58:58 +0100 Subject: [Python-Dev] FreeBSD test suite failure -> curses In-Reply-To: <20080309210210.GB60713@nexus.in-nomine.org> References: <20080308092434.GZ60713@nexus.in-nomine.org> <47D273B0.4020809@v.loewis.de> <20080309162900.GA60713@nexus.in-nomine.org> <47D43926.4030902@v.loewis.de> <20080309210210.GB60713@nexus.in-nomine.org> Message-ID: <47D46BB2.1080202@v.loewis.de> > (gdb) print newscr > $1 = (WINDOW *) 0x0 > > So it seems that the newscr variable is indeed a NULL. So this now *is* a FreeBSD/ncurses expert's question. I don't think this is supposed to happen; newscr should become non-NULL when initscr is called, and should remain that way throughout. Regards, Martin From martin at v.loewis.de Mon Mar 10 00:03:05 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 10 Mar 2008 00:03:05 +0100 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <47D43F6A.8000808@canterbury.ac.nz> References: <20080309142239.GA18295@panix.com> <47D43F6A.8000808@canterbury.ac.nz> Message-ID: <47D46CA9.1090309@v.loewis.de> > Dict access should probably be documented as no worse > than O(log n) to allow for tree implementations. That should not be documented. The current dict implementation may use O(n) for lookup operations, where n is the number of keys in the dictionary, and counting comparison operations. Regards, Martin From pje at telecommunity.com Mon Mar 10 00:05:12 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 09 Mar 2008 19:05:12 -0400 Subject: [Python-Dev] Equality on method objects In-Reply-To: References: <20080309160249.GA32007@code0.codespeak.net> Message-ID: <20080309230515.15B0C3A40A5@sparrow.telecommunity.com> At 01:59 PM 3/9/2008 -0800, Guido van Rossum wrote: >Do we have much of a use case for this? I've often had APIs that take a callback that promise to only invoke the callback once, even if it's added more than once. And I've used dicts, lists, and sets for same. I did not, however, need the equality of bound methods to be based on object value equality, just value identity. ...at least until recently, anyway. I do have one library that wants to have equality-based comparison of im_self. What I ended up doing is writing code that tests what the current Python interpreter is doing, and if necessary implements a special method type, just for purposes of working around the absence of im_self equality testing. However, it's a pretty specialized case, and if I didn't have to support older Python versions I'd just use partial() -- assuming that partial() supports hashing and equality comparisons, that is, which I haven't checked. I imagine hashing a partial() might be at least as tricky as getting bound methods "right". :) >That said, if there's a use case, I agree that it would be okay with >basing the equality of x.foo and y.foo on whether x and y are the same >object, not on whether x==y (consider 0 .__add__ == 0.0 .__add__). +1 for making two bound methods m1 and m2 equal if and only if "m1.im_self is m2.im_self and m1.im_func==m2.im_func", and making the hash based on im_func and id(im_self). I don't think that the im_func comparison should be identity-based by default, however. (The im_func could be another bound method, for example.) From voights at gmail.com Mon Mar 10 00:21:31 2008 From: voights at gmail.com (Forrest Voight) Date: Sun, 9 Mar 2008 19:21:31 -0400 Subject: [Python-Dev] PEP Proposal: Revised slice objects & lists use slice objects as indexes Message-ID: This would simplify the handling of list slices. Slice objects that are produced in a list index area would be different, and optionally the syntax for slices in list indexes would be expanded to work everywhere. Instead of being containers for the start, end, and step numbers, they would be generators, similar to xranges. Lists would accept these slice objects as indexes, and would also accept any other list or generator. Last, slice objects would be able to be added for multiple index ranges of a list. The new slice object would keep a list of ranges. Optionally, the 1:2 syntax would create a slice object outside of list index areas. It would be shorthand for slice, as [] is for list. This would create some confusion in loops and conditionals due to the colon being used for the end of the structure. (see last example) This would be incompatible with classes that define __setitem__ and __getitem__, and would need changes in how slices are handled in CPython. Therefore, this is probably a proposal for Python 3000. Examples: >>> 1:5 1:5 >>> list(1:5) [1, 2, 3, 4] >>> list(1:5:2) [1, 3] >>> s = 1:3 >>> range(5)[s] [1, 2] >>> 1:5 + 15:17 1:5 + 15:17 >>> range(30)[1:5 + 15:17] [1, 2, 3, 4, 15, 16] >>> range(100)[[1,2,3]] [1, 2, 3] >>> range(100)[1,2,5] # maybe [1, 2, 5] >>> for x in :15:3: print x # maybe 0 3 6 9 12 --- Forrest Voight From alexandre at peadrop.com Mon Mar 10 01:35:09 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Sun, 9 Mar 2008 20:35:09 -0400 Subject: [Python-Dev] PEP Proposal: Revised slice objects & lists use slice objects as indexes In-Reply-To: References: Message-ID: On Sun, Mar 9, 2008 at 7:21 PM, Forrest Voight wrote: > This would simplify the handling of list slices. > > Slice objects that are produced in a list index area would be different, > and optionally the syntax for slices in list indexes would be expanded > to work everywhere. Instead of being containers for the start, end, > and step numbers, they would be generators, similar to xranges. I am not sure what you are trying to propose here. The slice object isn't special, it's just a regular built-in type. >>> slice(1,4) slice(1, 4, None) >>> [1,2,3,4,5,6][slice(1,4)] [2, 3, 4] I don't see how introducing new syntax would simplify indexing. > Lists would accept these slice objects as indexes, and would also > accept any other list or generator. > Why lists should accept a list or a generator as index? What is the use case you have in mind? > Optionally, the 1:2 syntax would create a slice object outside of list > index areas. Again, I don't see how this could be useful... > > >>> list(1:5) > [1, 2, 3, 4] > > >>> list(1:5:2) > [1, 3] > list(range(1,5,2))? > >>> range(30)[1:5 + 15:17] > [1, 2, 3, 4, 15, 16] > This is confusing, IMHO, and doesn't provide any advantage over: >>> s = list(range(30)) >>> s[1:5] + s[15:17] If you really needed it, you could define a custom class with a fancy __getitem__ class A: def __getitem__(self, x): return x >>> A()[1:3,2:5] (slice(1, 3, None), slice(2, 5, None)) P.S. You should consider using the python-ideas (http://mail.python.org/mailman/listinfo/python-ideas) mailing list, instead of python-dev for posting suggestions. Cheers, -- Alexandre From aleaxit at gmail.com Mon Mar 10 06:16:27 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Sun, 9 Mar 2008 22:16:27 -0700 Subject: [Python-Dev] RQST: Master Thesis In-Reply-To: <18387.63318.76728.195090@montanaro-dyndns-org.local> References: <20080309135314.GC6263@yogi> <18387.63318.76728.195090@montanaro-dyndns-org.local> Message-ID: Depending on which implementation[s] you want to target, Michal, you should also check out PyPy at http://codespeak.net/pypy/, IronPython at http://www.codeplex.com/IronPython, and Jython at http://www.jython.org/ -- Jython's currently a tad behind the other three, but Sun Microsystems has just announced new investments and high-profile hires to make Jython a top-quality Python implementation. Alex On Sun, Mar 9, 2008 at 7:42 AM, wrote: > > Michal> I'm about to start my master thesis: "optimizing python > Michal> interpreter" therefore i would require your help plz ;) > > Michal> first of all i need to find out, how the python interpreter is > Michal> implemented, and which are the related sources, if this stuff is > Michal> somewhere documented that would be very helpfull... > > Michal, > > The best place to start is the implementation of the interpreter itself > (Python/ceval.c in the source tree) and the byte code compiler > (Python/compile.c). There have, at times, been modest attempts to document > what's going on, but I believe the source code is far and away still your > best bet. > > I recommend you get a Subversion checkout of either the trunk: > > svn co http://svn.python.org/projects/python/trunk > > or the py3k branch: > > svn co http://svn.python.org/projects/python/branches/py3k > > You will find the trunk more stable, but the py3k branch is the future. > > Also, study the archives of this list and the python-3000 at python.org list as > well as checkin comments for the Python source, especially for the two files > I referenced above. Finally, there may well be pending patches in the > Python tracker which implement various optimizations: > > http://bugs.python.org/ > > Studying them (and trying them out and attaching comments or reviews of > their efficacy) would be a good way to help understand the system. > > -- > Skip Montanaro - skip at pobox.com - http://www.webfast.com/~skip/ > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com > From voights at gmail.com Mon Mar 10 11:57:30 2008 From: voights at gmail.com (Forrest Voight) Date: Mon, 10 Mar 2008 06:57:30 -0400 Subject: [Python-Dev] PEP Proposal: Revised slice objects & lists use slice objects as indexes In-Reply-To: References: Message-ID: > I am not sure what you are trying to propose here. The slice object > isn't special, it's just a regular built-in type. The idea is to have slice objects be generators. You could make a slice like 1:10:2 , and that would make a slice object which could be used as a list index. The list would return a list with the corresponding item for every index in the generator. Then, lists could transparently be used as list indexes, or you could supply your own generator instead of a slice object. > I don't see how introducing new syntax would simplify indexing. It would move the slice logic from the list to the slice object. Now the slice object is just a container, but with this it would be useful. > Why should lists accept a list or a generator as an index? What is the > use case you have in mind? For example, a multiple selection box's selection would be changed into its values by supplying the chosen indexes into a list of values. > > Optionally, the 1:2 syntax would create a slice object outside of list > > index areas. > > Again, I don't see how this could be useful... > > > > >>> list(1:5) > > [1, 2, 3, 4] > > > > >>> list(1:5:2) > > [1, 3] > > > > list(range(1,5,2))? It would only be useful as shorthand for xrange, but its addition capabilities would be useful. > > >>> range(30)[1:5 + 15:17] > > [1, 2, 3, 4, 15, 16] > > > > This is confusing, IMHO, and doesn't provide any advantage over: > > >>> s = list(range(30)) > >>> s[1:5] + s[15:17] > I don't think it's confusing. Also, an advantage would be if the slice object were being automatically generated, this would simplify the code. > If you really needed it, you could define a custom class with a fancy > __getitem__ > > class A: > def __getitem__(self, x): > return x > > >>> A()[1:3,2:5] > (slice(1, 3, None), slice(2, 5, None)) > > P.S. You should consider using the python-ideas > (http://mail.python.org/mailman/listinfo/python-ideas) mailing list, > instead of python-dev for posting suggestions. > > Cheers, > -- Alexandre > From arigo at tunes.org Mon Mar 10 12:26:45 2008 From: arigo at tunes.org (Armin Rigo) Date: Mon, 10 Mar 2008 12:26:45 +0100 Subject: [Python-Dev] Equality on method objects In-Reply-To: <20080309230515.15B0C3A40A5@sparrow.telecommunity.com> References: <20080309160249.GA32007@code0.codespeak.net> <20080309230515.15B0C3A40A5@sparrow.telecommunity.com> Message-ID: <20080310112645.GA3397@code0.codespeak.net> Hi Phillip, On Sun, Mar 09, 2008 at 07:05:12PM -0400, Phillip J. Eby wrote: > I did not, however, need the equality of bound methods to be based on > object value equality, just value identity. > > ...at least until recently, anyway. I do have one library that wants > to have equality-based comparison of im_self. What I ended up doing > is writing code that tests what the current Python interpreter is > doing, and if necessary implements a special method type, just for > purposes of working around the absence of im_self equality > testing. However, it's a pretty specialized case (...) I found myself in exactly the same case: a pretty specialized example where I wanted bound methods to use im_self equality rather than identity, solved by writing my own bound-method-like object. But that's not really hard to do, and the general tendency (which matches my own opinion too) seems to be that using im_self identity is less surprizing. In general, "x.append" is interchangeable with "x.append" even if "x.append is not x.append", so let's go for the least surprizing behavior: "m1.im_self is m2.im_self and m1.im_func==m2.im_func". Objection? A bientot, Armin From pje at telecommunity.com Mon Mar 10 15:55:50 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 10 Mar 2008 10:55:50 -0400 Subject: [Python-Dev] Equality on method objects In-Reply-To: <20080310112645.GA3397@code0.codespeak.net> References: <20080309160249.GA32007@code0.codespeak.net> <20080309230515.15B0C3A40A5@sparrow.telecommunity.com> <20080310112645.GA3397@code0.codespeak.net> Message-ID: <20080310145551.0FDB03A40AF@sparrow.telecommunity.com> At 12:26 PM 3/10/2008 +0100, Armin Rigo wrote: >Hi Phillip, > >On Sun, Mar 09, 2008 at 07:05:12PM -0400, Phillip J. Eby wrote: > > I did not, however, need the equality of bound methods to be based on > > object value equality, just value identity. > > > > ...at least until recently, anyway. I do have one library that wants > > to have equality-based comparison of im_self. What I ended up doing > > is writing code that tests what the current Python interpreter is > > doing, and if necessary implements a special method type, just for > > purposes of working around the absence of im_self equality > > testing. However, it's a pretty specialized case (...) > >I found myself in exactly the same case: a pretty specialized example >where I wanted bound methods to use im_self equality rather than >identity, solved by writing my own bound-method-like object. But that's >not really hard to do, and the general tendency (which matches my own >opinion too) seems to be that using im_self identity is less surprizing. > >In general, "x.append" is interchangeable with "x.append" even if >"x.append is not x.append", so let's go for the least surprizing >behavior: "m1.im_self is m2.im_self and m1.im_func==m2.im_func". >Objection? Nope; that's exactly what I proposed at the end of the email quoted above. From ncoghlan at gmail.com Mon Mar 10 16:54:11 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 11 Mar 2008 01:54:11 +1000 Subject: [Python-Dev] PEP Proposal: Revised slice objects & lists use slice objects as indexes In-Reply-To: References: Message-ID: <47D559A3.8030708@gmail.com> Forrest Voight wrote: >> I am not sure what you are trying to propose here. The slice object >> isn't special, it's just a regular built-in type. > > The idea is to have slice objects be generators. You could make a > slice like 1:10:2 , and that would make a slice object which could be > used as a list index. The list would return a list with the > corresponding item for every index in the generator. Then, lists could > transparently be used as list indexes, or you could supply your own > generator instead of a slice object. You can already pass whatever items you like to __getitem__ for your own sequences without even touching the builtin slice(). You can even write a decorator to convert slice objects to the relatively arbitrary index iterators you appear to favour (I wish you good luck in getting those to play well with numpy and the myriad of other C extensions that rely on the existing extended slicing semantics, or explaining how they work to a Python novice - you're going to need it). That said, and as Alexandre already pointed out, this thread is off-topic for python-dev - please take it to python-ideas to thrash out whether or not it has any practical applications, and whether those applications (assuming they exist) are even remotely close to compelling enough to justify the pain involved in making such a major change to the core language. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Mon Mar 10 16:56:27 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 11 Mar 2008 01:56:27 +1000 Subject: [Python-Dev] Equality on method objects In-Reply-To: <20080310145551.0FDB03A40AF@sparrow.telecommunity.com> References: <20080309160249.GA32007@code0.codespeak.net> <20080309230515.15B0C3A40A5@sparrow.telecommunity.com> <20080310112645.GA3397@code0.codespeak.net> <20080310145551.0FDB03A40AF@sparrow.telecommunity.com> Message-ID: <47D55A2B.6020700@gmail.com> Phillip J. Eby wrote: > At 12:26 PM 3/10/2008 +0100, Armin Rigo wrote: >> In general, "x.append" is interchangeable with "x.append" even if >> "x.append is not x.append", so let's go for the least surprizing >> behavior: "m1.im_self is m2.im_self and m1.im_func==m2.im_func". >> Objection? > > Nope; that's exactly what I proposed at the end of the email quoted above. +1 here - this is the behaviour I would expect if attempting to provide a called-once-only guarantee for a callback list. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From martin at v.loewis.de Mon Mar 10 17:12:30 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 10 Mar 2008 17:12:30 +0100 Subject: [Python-Dev] Get rid of strerror.c and memmove.c? In-Reply-To: References: Message-ID: <47D55DEE.7010706@v.loewis.de> > Since both strerror() and memmove() are both in C89 (they are listed > in K&R), can we ditch these files? I think they can safely be deleted. Regards, Martin From rhamph at gmail.com Mon Mar 10 17:20:14 2008 From: rhamph at gmail.com (Adam Olsen) Date: Mon, 10 Mar 2008 09:20:14 -0700 Subject: [Python-Dev] Equality on method objects In-Reply-To: <20080310112645.GA3397@code0.codespeak.net> References: <20080309160249.GA32007@code0.codespeak.net> <20080309230515.15B0C3A40A5@sparrow.telecommunity.com> <20080310112645.GA3397@code0.codespeak.net> Message-ID: On Mon, Mar 10, 2008 at 4:26 AM, Armin Rigo wrote: > Hi Phillip, > > > On Sun, Mar 09, 2008 at 07:05:12PM -0400, Phillip J. Eby wrote: > > I did not, however, need the equality of bound methods to be based on > > object value equality, just value identity. > > > > ...at least until recently, anyway. I do have one library that wants > > to have equality-based comparison of im_self. What I ended up doing > > is writing code that tests what the current Python interpreter is > > doing, and if necessary implements a special method type, just for > > purposes of working around the absence of im_self equality > > testing. However, it's a pretty specialized case (...) > > I found myself in exactly the same case: a pretty specialized example > where I wanted bound methods to use im_self equality rather than > identity, solved by writing my own bound-method-like object. But that's > not really hard to do, and the general tendency (which matches my own > opinion too) seems to be that using im_self identity is less surprizing. > > In general, "x.append" is interchangeable with "x.append" even if > "x.append is not x.append", so let's go for the least surprizing > behavior: "m1.im_self is m2.im_self and m1.im_func==m2.im_func". > Objection? +1 -- Adam Olsen, aka Rhamphoryncus From jimis at gmx.net Mon Mar 10 17:24:39 2008 From: jimis at gmx.net (Dimitrios Apostolou) Date: Mon, 10 Mar 2008 18:24:39 +0200 Subject: [Python-Dev] Complexity documentation request In-Reply-To: References: <20080309142239.GA18295@panix.com> <47D43F6A.8000808@canterbury.ac.nz> Message-ID: <47D560C7.3020908@gmx.net> Hello again, Guido van Rossum wrote: > Well, there you have hit the nail on the head -- should we document > the actual or the guaranteed O() expression? I think this is a can of > worms better left unopened. At best we should include some hints to I will open this can and say that average case complexity is the most important. For example sorting O(nlogn) and dict lookup O(1). But because worst case is important too, it would be nice (but not necessary) if there was an extra column on the table with that information, or blank if not applicable. > clarify that random access to list and tuple elements is constant time > in CPython, and that dicts and sets are implemented using a hash table > with open hashing -- readers can draw their own conclusions from that. Notes are nice and already exist at random places in the *huge* documentation archive. But I still believe that the best place for this are the already existing tables in the docs (I pointed them at my initial post). One should trivially be able to find this information. On the other hand notes could be added for various oddities according to experience. For example that for sets up to 10(?) elements, a list would probably be better because of the hashing overhead. > What other implementations do should be up to them -- it becomes a > Quality of Implementation issue. I agree that CPython docs should document CPython behaviour. This would be the most helpful for someone writing a program in CPython. People who use other implementations should consult the appropriate docs. Implementors with worst algorithms should try to reach CPython efficiency. > > Regarding the OP's need for this kind of information (deciding between > the various standard data types), I would recommend a different > approach -- pick the data type that produces the shortest code. In all > likelihood it's also the most efficient. Hmmm, the first thing that comes to mind is prepending an item in a list which is small and seems nice, but is O(n) I think! And besides that, for someone who *cares* about his code good looks is not enough... Thanks for your answers, Dimitris From jimis at gmx.net Mon Mar 10 17:37:51 2008 From: jimis at gmx.net (Dimitrios Apostolou) Date: Mon, 10 Mar 2008 18:37:51 +0200 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <76EFA410-DD8D-4F4B-A299-C298E92454FA@acm.org> References: <20080309142239.GA18295@panix.com> <76EFA410-DD8D-4F4B-A299-C298E92454FA@acm.org> Message-ID: <47D563DF.6070100@gmx.net> Fred Drake wrote: > On Mar 9, 2008, at 10:22 AM, Aahz wrote: >> This has been discussed before and rejected for two reasons: >> >> * Other Python implementations (Jython, PyPy, IronPython) may not be >> able to provide the same type implementations >> >> * Algorithmic information does sometimes change between versions, and >> keeping the docs updated is not trivial > > Also, given the significance of the constant factors and the fact that > these are significantly dependent, especially for containers, on the > objects passed in (either to be contained or tested), it's not clear > that it often makes sense to worry about at too detailed a level. > Common sense, knowledge of the interpreter, and experience are often > more valuable the easily-outdated documentation. Fair enough but the fact is that this documentation already exists, at random locations unfortunately. Who would expect to find such valuable info in a rejected PEP (3128)! I will agree that experience and interpreter inside knowledge is most valuable for choosing the right structure, but isn't this too much for occasional python programmers? Thanks, Dimitris > > > -Fred > From martin at v.loewis.de Mon Mar 10 17:57:30 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 10 Mar 2008 17:57:30 +0100 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <47D560C7.3020908@gmx.net> References: <20080309142239.GA18295@panix.com> <47D43F6A.8000808@canterbury.ac.nz> <47D560C7.3020908@gmx.net> Message-ID: <47D5687A.6010107@v.loewis.de> > I will open this can and say that average case complexity is the most > important. For example sorting O(nlogn) and dict lookup O(1). I would still debate the complexity of dict lookup. What are you averaging over? In absence of any distribution property of hash functions in the average case, you can't say anything about dictionary performance. I also disagree that the average complexity is "most important". I find the worst-case complexity most important. For average-case complexity, I just measure my application, and don't care about theoretical numbers. > Notes are nice and already exist at random places in the *huge* > documentation archive. But I still believe that the best place for this > are the already existing tables in the docs (I pointed them at my > initial post). One should trivially be able to find this information. Feel free to start a Wiki page then. With the right keywords on the page, it shouldn't take long for Google to pick up the page, and return it to people asking the right questions. > I agree that CPython docs should document CPython behaviour. Actually, no. The "CPython documentation" documents Python, the language, and its standard library. It is a specification that CPython conforms to (hopefully). There might be fragments in it that are both worthwhile-to-mention and implementation-specific, but they should be marked as the latter. > Hmmm, the first thing that comes to mind is prepending an item in a list > which is small and seems nice, but is O(n) I think! And besides that, > for someone who *cares* about his code good looks is not enough... Define "small". For a theoretically-reasonable definition of "small", prepending is O(1): namely, when a "small" list is one whose size is bounded by some upper bound (say, M). For such a list, prepending is O(1) (namely, not more than M copies). Of course, you can only prepend a finite number of times to such a list, unless you also delete in-between. Over a long series of insertions and deletions, prepending will be O(1) (if M is large, with a large factor). Regards, Martin From daniel at stutzbachenterprises.com Mon Mar 10 20:05:56 2008 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 10 Mar 2008 14:05:56 -0500 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <20080309142239.GA18295@panix.com> References: <20080309142239.GA18295@panix.com> Message-ID: On Sun, Mar 9, 2008 at 9:22 AM, Aahz wrote: > There probably would be some value in a wiki page on python.org that > provides this information, particularly across versions. You may be > able to find volunteers to help on comp.lang.python. I just created a very basic one at http://wiki.python.org/moin/TimeComplexity?action=show I'm not that familiar with the Wiki syntax, so the tables are kind of ugly at the moment. I wasn't sure about many of the set() operations, so I didn't include those. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From aahz at pythoncraft.com Mon Mar 10 20:18:55 2008 From: aahz at pythoncraft.com (Aahz) Date: Mon, 10 Mar 2008 12:18:55 -0700 Subject: [Python-Dev] Complexity documentation request In-Reply-To: References: <20080309142239.GA18295@panix.com> Message-ID: <20080310191855.GA1142@panix.com> On Mon, Mar 10, 2008, Daniel Stutzbach wrote: > On Sun, Mar 9, 2008 at 9:22 AM, Aahz wrote: >> >> There probably would be some value in a wiki page on python.org that >> provides this information, particularly across versions. You may be >> able to find volunteers to help on comp.lang.python. > > I just created a very basic one at > http://wiki.python.org/moin/TimeComplexity?action=show > > I'm not that familiar with the Wiki syntax, so the tables are kind of > ugly at the moment. Please specify which Python version you're using. Thanks! -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson From martin at v.loewis.de Mon Mar 10 23:06:44 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 10 Mar 2008 23:06:44 +0100 Subject: [Python-Dev] Complexity documentation request In-Reply-To: References: <20080309142239.GA18295@panix.com> Message-ID: <47D5B0F4.9050309@v.loewis.de> > I just created a very basic one at > http://wiki.python.org/moin/TimeComplexity?action=show I just knew that this could be a field of endless nitpicking. I think the dict.copy classification is incorrect. A dict.copy can take unbounded time, if the dict has seen many recent deletions (which didn't shrink it). Copying will have to iterate over all slots of the dictionary, and the ratio of slots to keys can grow unbounded if you have just deletions without insertions. IOW, if you do d = {} for i in xrange(N): d[i]=i for i in xrange(N-1): del d[i] then doing d.copy() takes O(N), not constant time (even though there is only one key in the dictionary). > I wasn't sure about many of the set() operations, so I didn't include those. set is just implemented like a dictionary, without keys. Regards, Martin From daniel at stutzbachenterprises.com Mon Mar 10 23:31:53 2008 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 10 Mar 2008 17:31:53 -0500 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <47D5B0F4.9050309@v.loewis.de> References: <20080309142239.GA18295@panix.com> <47D5B0F4.9050309@v.loewis.de> Message-ID: On Mon, Mar 10, 2008 at 5:06 PM, "Martin v. L?wis" wrote: > > I just created a very basic one at > > http://wiki.python.org/moin/TimeComplexity?action=show > > I just knew that this could be a field of endless nitpicking. Certainly. I am hoping that this thread will soon wind down and future nitpicking can come in the form of edits to the wiki page with footnotes or links to other pages for anything non-obvious. > I think the dict.copy classification is incorrect. A dict.copy > can take unbounded time, if the dict has seen many recent > deletions (which didn't shrink it). Copying will have to iterate > over all slots of the dictionary, and the ratio of slots to > keys can grow unbounded if you have just deletions without > insertions. I have updated the wiki page accordingly. I assume there is a reason that PyDict_DelItem never calls dictresize? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From martin at v.loewis.de Mon Mar 10 23:40:30 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 10 Mar 2008 23:40:30 +0100 Subject: [Python-Dev] Complexity documentation request In-Reply-To: References: <20080309142239.GA18295@panix.com> <47D5B0F4.9050309@v.loewis.de> Message-ID: <47D5B8DE.5050503@v.loewis.de> > I assume there is a reason that PyDict_DelItem never calls dictresize? Yes - the assumption is that more del calls will follow, so that the dictionary eventually ends up empty. Only when new inserts are made, that assumption is proven wrong, and the shrinking can be done in one sweep. Regards, Martin From guido at python.org Mon Mar 10 23:41:40 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 10 Mar 2008 15:41:40 -0700 Subject: [Python-Dev] Equality on method objects In-Reply-To: References: <20080309160249.GA32007@code0.codespeak.net> <20080309230515.15B0C3A40A5@sparrow.telecommunity.com> <20080310112645.GA3397@code0.codespeak.net> Message-ID: On Mon, Mar 10, 2008 at 9:20 AM, Adam Olsen wrote: > > On Mon, Mar 10, 2008 at 4:26 AM, Armin Rigo wrote: > > Hi Phillip, > > > > > > On Sun, Mar 09, 2008 at 07:05:12PM -0400, Phillip J. Eby wrote: > > > I did not, however, need the equality of bound methods to be based on > > > object value equality, just value identity. > > > > > > ...at least until recently, anyway. I do have one library that wants > > > to have equality-based comparison of im_self. What I ended up doing > > > is writing code that tests what the current Python interpreter is > > > doing, and if necessary implements a special method type, just for > > > purposes of working around the absence of im_self equality > > > testing. However, it's a pretty specialized case (...) > > > > I found myself in exactly the same case: a pretty specialized example > > where I wanted bound methods to use im_self equality rather than > > identity, solved by writing my own bound-method-like object. But that's > > not really hard to do, and the general tendency (which matches my own > > opinion too) seems to be that using im_self identity is less surprizing. > > > > In general, "x.append" is interchangeable with "x.append" even if > > "x.append is not x.append", so let's go for the least surprizing > > behavior: "m1.im_self is m2.im_self and m1.im_func==m2.im_func". > > Objection? > > +1 > > -- > Adam Olsen, aka Rhamphoryncus > +1 here too. For 2.6 as well as 3.0. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Tue Mar 11 00:51:05 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 11 Mar 2008 12:51:05 +1300 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <47D5B8DE.5050503@v.loewis.de> References: <20080309142239.GA18295@panix.com> <47D5B0F4.9050309@v.loewis.de> <47D5B8DE.5050503@v.loewis.de> Message-ID: <47D5C969.4040501@canterbury.ac.nz> Martin v. L?wis wrote: > Yes - the assumption is that more del calls will follow, so that the > dictionary eventually ends up empty. Only when new inserts are made, > that assumption is proven wrong, and the shrinking can be done in > one sweep. Hmmm. So the most efficient way to copy a dict that you've just deleted a lot of things from is to insert something, then delete it again, and then copy. :-) -- Greg From greg.ewing at canterbury.ac.nz Tue Mar 11 00:58:46 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 11 Mar 2008 12:58:46 +1300 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <47D5687A.6010107@v.loewis.de> References: <20080309142239.GA18295@panix.com> <47D43F6A.8000808@canterbury.ac.nz> <47D560C7.3020908@gmx.net> <47D5687A.6010107@v.loewis.de> Message-ID: <47D5CB36.2090305@canterbury.ac.nz> Martin v. L?wis wrote: > I would still debate the complexity of dict lookup. What are you > averaging over? All the Python programs I've ever run. :-) > I also disagree that the average complexity is "most important". > I find the worst-case complexity most important. While in theory the worst-case behaviour of a hash table is O(n), in practice the worst case is so vanishingly rare that nobody worries about it. Can you really say that you don't make any design decisions early on based on the assumption that dict lookup will almost certainly be a lot faster than searching a list? >>Hmmm, the first thing that comes to mind is prepending an item in a list >>which is small and seems nice, but is O(n) I think! > > Define "small". I think he was talking about the size of the code. In other words, just because the code is small doesn't necessarily mean the algorithm is efficient. > For a theoretically-reasonable definition of "small", > prepending is O(1): Big O-notation is all about the limit when things get big. So it doesn't make sense to talk about "O() when something is small". -- Greg From greg.ewing at canterbury.ac.nz Tue Mar 11 00:07:46 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 11 Mar 2008 12:07:46 +1300 Subject: [Python-Dev] PEP Proposal: Revised slice objects & lists use slice objects as indexes In-Reply-To: References: Message-ID: <47D5BF42.4080104@canterbury.ac.nz> Forrest Voight wrote: > Slice objects that are produced in a list index area would be different, > and optionally the syntax for slices in list indexes would be expanded > to work everywhere. Something like this was quite close to getting in a while back, but it was eventually dropped. Anyone advocating this should probably look back over the discussion and find out why. I think they were to be called "range expressions" or something like that. A point to consider is that iterating over a range of integers is actually quite a rare thing to do in idiomatic Python. It's much more common to iterate directly over a sequence of things you want to operate on. This is even more true now that we have enumerate(). So this proposal is addressing a fairly small set of use cases. -- Greg From greg.ewing at canterbury.ac.nz Tue Mar 11 00:22:44 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 11 Mar 2008 12:22:44 +1300 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <47D560C7.3020908@gmx.net> References: <20080309142239.GA18295@panix.com> <47D43F6A.8000808@canterbury.ac.nz> <47D560C7.3020908@gmx.net> Message-ID: <47D5C2C4.6090701@canterbury.ac.nz> Dimitrios Apostolou wrote: > On the other hand notes could be added for various oddities according to > experience. For example that for sets up to 10(?) elements, a list would > probably be better because of the hashing overhead. That's the sort of thing that tends to be *very* implementation dependent -- not just between CPython and others, but between different releases of CPython. > Hmmm, the first thing that comes to mind is prepending an item in a list > which is small and seems nice, but is O(n) I think! Yeah, there's no substitute for having at least a rough idea of how the object you're using is implemented, e.g. array vs. linked list. This kind of very basic information is something that I think ought to be documented, and some guarantees made in the language definition. For example, I think a Python implementation that implemented lists as linked lists would make many people unhappy, as their algorithms suddenly went from O(n**m) to O(n**(m+1)) without anyone telling them. -- Greg From aleaxit at gmail.com Tue Mar 11 02:21:53 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Mon, 10 Mar 2008 18:21:53 -0700 Subject: [Python-Dev] PEP Proposal: Revised slice objects & lists use slice objects as indexes In-Reply-To: References: Message-ID: On Mon, Mar 10, 2008 at 3:57 AM, Forrest Voight wrote: > > I am not sure what you are trying to propose here. The slice object > > isn't special, it's just a regular built-in type. > > The idea is to have slice objects be generators. You could make a > slice like 1:10:2 , and that would make a slice object which could be > used as a list index. The list would return a list with the > corresponding item for every index in the generator. Then, lists could And what indices would the slice 1:-1 return, for example? Your proposal can't play well with slices including negative indices. Also, your desired use case of alist[indices] is already pretty well covered by [alist[i] for i in indices]. Alex From tjreedy at udel.edu Tue Mar 11 04:11:16 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 10 Mar 2008 23:11:16 -0400 Subject: [Python-Dev] Complexity documentation request References: <20080309142239.GA18295@panix.com> <47D43F6A.8000808@canterbury.ac.nz><47D560C7.3020908@gmx.net> <47D5C2C4.6090701@canterbury.ac.nz> Message-ID: "Greg Ewing" wrote in message news:47D5C2C4.6090701 at canterbury.ac.nz... || Yeah, there's no substitute for having at least a | rough idea of how the object you're using is implemented, | e.g. array vs. linked list. As I understand it, the new 2.6 docs include a new one on CPython specifically. A page there might be appropriate. But someone has to write and submit a patch for review. | This kind of very basic information is something that | I think ought to be documented, and some guarantees | made in the language definition. For example, I think | a Python implementation that implemented lists as | linked lists would make many people unhappy, as their | algorithms suddenly went from O(n**m) to O(n**(m+1)) | without anyone telling them. Such an implementation should document such a design decision, but I don't see that an interpreter that runs the test suite should be prohibited from calling itself a 'Python interpreter' tjr From stefan_ml at behnel.de Tue Mar 11 14:55:04 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 11 Mar 2008 14:55:04 +0100 Subject: [Python-Dev] Python XML Validator In-Reply-To: <20080305102306.0a3db148@mbook-fbsd> References: <20080302230708.260fa4a9@bhuda.mired.org> <20080305102306.0a3db148@mbook-fbsd> Message-ID: (weird places these threads come up at, but now that it's here...) Mike Meyer wrote: > On Tue, 04 Mar 2008 15:44:32 -0800 Ned Deily wrote: >> In article <20080302230708.260fa4a9 at bhuda.mired.org>, >> Mike Meyer wrote: >>> On Thu, 28 Feb 2008 23:42:49 +0000 (UTC) Medhat Gayed >>> wrote: >>>> lxml is good but not written in python and difficult to install and didn't >>>> work on MacOS X. Due to a design problem in MacOS-X, not a problem in lxml. But it's not that hard to install either, as previous posts presented. >>> lxml is built on top of libxml2/libxslt, which are bundled with most >>> Unix-like OS's (including Mac OS X), or available in their package >>> systems. Trying to install it from the repository is a PITA, because >>> it uses both the easyinstall and Pyrex (later Cython) packages Using a release version of lxml does not require you to install any of the two. Just download the tar.gz, unpack it, and do the usual setup.py dance. That's how package installation in Python works. It does, however, require you to have libxml2 and libxslt installed with their dependencies, but that has nothing to do with Python. >>> aren't bundled with anything. On the other hand, if it's in the >>> package system (I no longer have macports installed anywhere, but >>> believe it was there at one time), that solves all those problems. I >>> believe they've excised the easyinstall source dependencies, though. There is no source dependency on easy_install. I assume that all they did is: build lxml against the libxml2/libxslt libraries that come with macports. Which is a sensible thing to do IMHO. > I think lxml is the best Python XML library that meets his > requirements, and it would make my life a lot easier if it were part > of the standard library. I don't object to that. I'm just not a major driver here as it would require a bit of work that I can't currently spare. > However, the authors tend to require recent > versions of libxml2 and libxslt, which means recent versions of lxml > won't build and/or work with the libraries bundled with many Unix and > Unix-like systems I wouldn't consider a dependency on an almost three year old library version "recent", libxml2 2.6.20 was released in July 2005. > - including OSX. That's different, because the system libraries here are a) horribly outdated for every new version of MacOS-X (i.e. usually more than two years old and very buggy), and b) difficult to replace by design. > Which means you wind up having to > build those yourself if you want a recent version of lxml, even if > you're using a system that includes lxml in it's package system. If you want a clean system, e.g. for production use, buildout has proven to be a good idea. And we also provide pretty good instructions on our web page on how to install lxml on MacOS-X and what to take care of. Stefan From stefan_ml at behnel.de Tue Mar 11 14:56:17 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 11 Mar 2008 14:56:17 +0100 Subject: [Python-Dev] Python XML Validator In-Reply-To: <20080305102553.5841f338@mbook-fbsd> References: <20080302230708.260fa4a9@bhuda.mired.org> <47CDE2CA.2080003@canterbury.ac.nz> <20080305102553.5841f338@mbook-fbsd> Message-ID: Mike Meyer wrote: > On Wed, 05 Mar 2008 13:01:14 +1300 Greg Ewing wrote: > >> Mike Meyer wrote: >>> Trying to install it from the repository is a PITA, because >>> it uses both the easyinstall and Pyrex >> It shouldn't depend on Pyrex as long as it's distributed >> with the generated C files. If it's not, that's an >> oversight on the part of the distributor. > > Sorry I wasn't clear. "from the repository" means building from > sourced checked out of the source repository, not from a > distribution. Ok, uhm, what do you think we do releases for? :) Stefan From stefan_ml at behnel.de Tue Mar 11 15:04:43 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 11 Mar 2008 15:04:43 +0100 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: <47CFA113.2080101@v.loewis.de> References: <47CDC837.8000104@rksystems.com> <47CDD183.2040201@cheimes.de> <47CDE509.8080801@holdenweb.com> <47CF0A89.2010804@canterbury.ac.nz> <47CFA113.2080101@v.loewis.de> Message-ID: <47D6917B.60502@behnel.de> Martin v. L?wis wrote: >> I think the best lesson here is Tcl. Because it uses stubs mechanism, >> you don't need to depend on tclXX.dll, you don't deal with really >> direct implementation details, you don't care about runtimes, >> everything is much easier. Maybe it's possible (and not too late) for >> Python to somehow embrace such mechanism? > > It would be possible, but it would be a fairly large project. You would > have to remove a lot of things from the Python header files, and that > would cause significant breakage in existing extension modules. Hmm, would it? I mean, you can currently build extensions with MinGW (at least from what I heard, I never managed to do so as it would require cross compiling for Windows), so why would you say you'd have to update the header files? Stefan From mwm at mired.org Tue Mar 11 17:10:06 2008 From: mwm at mired.org (Mike Meyer) Date: Tue, 11 Mar 2008 12:10:06 -0400 Subject: [Python-Dev] Python XML Validator In-Reply-To: References: <20080302230708.260fa4a9@bhuda.mired.org> <20080305102306.0a3db148@mbook-fbsd> Message-ID: <20080311121006.0da78de7@mbook-fbsd> On Tue, 11 Mar 2008 14:55:04 +0100 Stefan Behnel wrote: > (weird places these threads come up at, but now that it's here...) > Mike Meyer wrote: > > On Tue, 04 Mar 2008 15:44:32 -0800 Ned Deily wrote: > >> In article <20080302230708.260fa4a9 at bhuda.mired.org>, > >> Mike Meyer wrote: > >>> On Thu, 28 Feb 2008 23:42:49 +0000 (UTC) Medhat Gayed > >>> wrote: > >>>> lxml is good but not written in python and difficult to install and didn't > >>>> work on MacOS X. Please note that this original complaint is *not* mine. However... > Due to a design problem in MacOS-X, not a problem in lxml. I didn't find it noticeably harder to install lxml on MacOS-X than most other systems. > But it's not that hard to install either, as previous posts presented. Depends on how you define "hard". If I have to create a custom environment with updated version of system libraries just to use lxml, I'd call that "hard". That was pretty much the only route available the first time I wanted lxml on OS-X. And ubuntu. And RHEL. The second time for OS-X, I used an older version of lxml (1.3.6), and just did "setup.py install". Worked like a charm. That's not hard. The only system that installing a modern version of lxml on was easy was FreeBSD, probably because libxml2 and libxslt aren't part of the system software. > > However, the authors tend to require recent > > versions of libxml2 and libxslt, which means recent versions of lxml > > won't build and/or work with the libraries bundled with many Unix and > > Unix-like systems > I wouldn't consider a dependency on an almost three year old library version > "recent", libxml2 2.6.20 was released in July 2005. Well, if you're on a development box that you update regularly, you're right: three years old is pretty old. If you're talking about a production box that you don't touch unless you absolutely have to, you're wrong: three years old is still pretty recent. For example, the most recent release of RHEL is 4.6, which ships with libxml2 2.6.16. > > Which means you wind up having to > > build those yourself if you want a recent version of lxml, even if > > you're using a system that includes lxml in it's package system. > If you want a clean system, e.g. for production use, buildout has proven to be > a good idea. And we also provide pretty good instructions on our web page on > how to install lxml on MacOS-X and what to take care of. Yes, but the proposal was to include it in the Python standard library. Software that doesn't work on popular target platforms without updating a standard system library isn't really suitable for that. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. From stefan_ml at behnel.de Tue Mar 11 18:01:29 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 11 Mar 2008 18:01:29 +0100 Subject: [Python-Dev] Python XML Validator In-Reply-To: <20080311121006.0da78de7@mbook-fbsd> References: <20080302230708.260fa4a9@bhuda.mired.org> <20080305102306.0a3db148@mbook-fbsd> <20080311121006.0da78de7@mbook-fbsd> Message-ID: <47D6BAE9.4050105@behnel.de> Mike Meyer wrote: > On Tue, 11 Mar 2008 14:55:04 +0100 Stefan Behnel wrote: >> (weird places these threads come up at, but now that it's here...) >> Mike Meyer wrote: >>> On Tue, 04 Mar 2008 15:44:32 -0800 Ned Deily wrote: >>>> In article <20080302230708.260fa4a9 at bhuda.mired.org>, >>>> Mike Meyer wrote: >>>>> On Thu, 28 Feb 2008 23:42:49 +0000 (UTC) Medhat Gayed >>>>> wrote: >>>>>> lxml is good but not written in python and difficult to install and didn't >>>>>> work on MacOS X. > > Please note that this original complaint is *not* mine. However... > >> Due to a design problem in MacOS-X, not a problem in lxml. > > I didn't find it noticeably harder to install lxml on MacOS-X than > most other systems. It seems to be for a number of people, though, who turn up on the mailing list complaining about just that. >> But it's not that hard to install either, as previous posts presented. > > Depends on how you define "hard". If I have to create a custom > environment with updated version of system libraries just to use lxml, > I'd call that "hard". That was pretty much the only route available > the first time I wanted lxml on OS-X. And ubuntu. And RHEL. It got a lot better by now. The only problem is how to tell your operating system where to look for libraries that you installed yourself (which I still refuse to consider a problem of lxml). BTW, we had MacOS builds a while ago, so I wouldn't mind having someone volunteer to contribute builds on a regular basis (static builds preferred). > The second time for OS-X, I used an older version of lxml (1.3.6), and > just did "setup.py install". Worked like a charm. That's not hard. Interesting. 1.3.6 should also require libxml2 2.6.20 (although maybe less strictly than 2.0). >>> However, the authors tend to require recent >>> versions of libxml2 and libxslt, which means recent versions of lxml >>> won't build and/or work with the libraries bundled with many Unix and >>> Unix-like systems >> I wouldn't consider a dependency on an almost three year old library version >> "recent", libxml2 2.6.20 was released in July 2005. > > Well, if you're on a development box that you update regularly, you're > right: three years old is pretty old. If you're talking about a > production box that you don't touch unless you absolutely have to, > you're wrong: three years old is still pretty recent. For example, the > most recent release of RHEL is 4.6, which ships with libxml2 2.6.16. Ok, that's pretty old, although that was the last version we supported before requiring 2.6.20 (last summer, somewhere in the 1.3 series). Anyway, it's definitely less of a problem to upgrade system libraries on Linux (IIRC "rpm -bs" helps you on older RH*L versions) than under MacOS. >>> Which means you wind up having to >>> build those yourself if you want a recent version of lxml, even if >>> you're using a system that includes lxml in it's package system. >> If you want a clean system, e.g. for production use, buildout has proven to be >> a good idea. And we also provide pretty good instructions on our web page on >> how to install lxml on MacOS-X and what to take care of. > > Yes, but the proposal was to include it in the Python standard > library. Software that doesn't work on popular target platforms > without updating a standard system library isn't really suitable for > that. Hmm, coming somewhat back on-topic: how does Python currently handle its dependencies under MacOS-X? SQLite, for example? Does it use system libraries only, or are there libraries it ships with? (The MacOS distro is much bigger, but that might be due to the universal build - although that suggests that MacOS-X users do not care about disk space or download size anyway) It looks like it already ships with expat on all platforms, so why not add libxml2/libxslt to the distribution, at least on platforms where it's necessary? (happily ignoring the fact here that lxml isn't currently even close to being integrated) Admittedly, that would add some 1.3MB (uncompressed) to the distro... Regarding updated version requirements: those are always discussed on the mailing list. The only reason we had for continued support of 2.6.16 was MacOS-X 10.4, until we found that most users installed newer libraries anyway, because the old ones were just too old and crash-prone. Stefan From jimis at gmx.net Tue Mar 11 20:27:19 2008 From: jimis at gmx.net (Dimitrios Apostolou) Date: Tue, 11 Mar 2008 21:27:19 +0200 Subject: [Python-Dev] Complexity documentation request In-Reply-To: References: <20080309142239.GA18295@panix.com> Message-ID: <47D6DD17.3060905@gmx.net> Daniel Stutzbach wrote: > On Sun, Mar 9, 2008 at 9:22 AM, Aahz wrote: >> There probably would be some value in a wiki page on python.org that >> provides this information, particularly across versions. You may be >> able to find volunteers to help on comp.lang.python. > > I just created a very basic one at > http://wiki.python.org/moin/TimeComplexity?action=show > > I'm not that familiar with the Wiki syntax, so the tables are kind of > ugly at the moment. > > I wasn't sure about many of the set() operations, so I didn't include those. > Thanks! I'm interested too in some operations I don't know about, I think I'll just add them with blank fields so that eventually people who know fill them out. Dimitris P.S. This wiki is really bad in tables: I can't figure out how to draw a border for the tables and every table element contains a

tag, making the cell spanning 2-3 lines of height... :-@ From martin at v.loewis.de Tue Mar 11 21:17:33 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 11 Mar 2008 21:17:33 +0100 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: <47D6917B.60502@behnel.de> References: <47CDC837.8000104@rksystems.com> <47CDD183.2040201@cheimes.de> <47CDE509.8080801@holdenweb.com> <47CF0A89.2010804@canterbury.ac.nz> <47CFA113.2080101@v.loewis.de> <47D6917B.60502@behnel.de> Message-ID: <47D6E8DD.2020909@v.loewis.de> >>> I think the best lesson here is Tcl. Because it uses stubs mechanism, >>> you don't need to depend on tclXX.dll, you don't deal with really >>> direct implementation details, you don't care about runtimes, >>> everything is much easier. Maybe it's possible (and not too late) for >>> Python to somehow embrace such mechanism? >> It would be possible, but it would be a fairly large project. You would >> have to remove a lot of things from the Python header files, and that >> would cause significant breakage in existing extension modules. > > Hmm, would it? I mean, you can currently build extensions with MinGW (at least > from what I heard, I never managed to do so as it would require cross > compiling for Windows), so why would you say you'd have to update the header > files? I wasn't talking about building with other compilers, but about the subject of the Tcl stub mechanism being applied to Python. As others noted, that mechanism doesn't help at all with the issue of different CRTs. It does help with another issue, though, namely the constant ABI changes that we see. Every time the ABI changes, extension modules have to be recompiled. With a scheme similar to Tcl stubs, that need could go away. This would be particularly useful on Windows, where each feature release comes up with the new DLL name. With such a mechanism in place, we could do "python3.dll" (rather than python30.dll, python31.dll, and so on). To implement such a system, you need to get all ABI dependencies out of the header files; this includes the structure layouts in particular. You might want to keep struct _object, for sake of INCREF/DECREF. You then need to specify which of the remaining functions officially belongs to the ABI, and freeze them for the lifetime of Python 3. New functions can be added, but when changing the signature, the old functions must remain in place. In Tcl stubs, that is achieved through a function vector, IIUC. HTH, Martin From Martin.vonLoewis at hpi.uni-potsdam.de Tue Mar 11 21:22:31 2008 From: Martin.vonLoewis at hpi.uni-potsdam.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 11 Mar 2008 21:22:31 +0100 Subject: [Python-Dev] [ANN] Python 2.3.7 and 2.4.5 (final) Message-ID: <47D6EA07.5050409@hpi.uni-potsdam.de> On behalf of the Python development team and the Python community, I'm happy to announce the release of Python 2.4.5 and 2.3.7 (final). Both releases include only security fixes. Python 2.5 is the latest version of Python, we're making this release for people who are still running Python 2.3 or 2.4. See the release notes at the website (also available as Misc/NEWS in the source distribution) for details of bugs fixed; most of them prevent interpreter crashes (and now cause proper Python exceptions in cases where the interpreter may have crashed before). Since the release candidate, we received various reports that the this release may fail to build on current operating systems, in particular on OS X. We have made no attempt to fix these problems, as the release is targeted for systems that were current at the time Python 2.4 was originally released. For more recent systems, you might have to come up with work-arounds. For OS X in particular, try invoking:: ./configure MACOSX_DEPLOYMENT_TARGET=10.5 We have made no changes since the release candidate (except for the version numbers). For more information on Python 2.3.7 and 2.4.5, including download links for various platforms, release notes, and known issues, please see: http://www.python.org/2.3.7 http://www.python.org/2.4.5 Highlights of the previous major Python releases are available from the Python 2.4 page, at http://www.python.org/2.3/highlights.html http://www.python.org/2.4/highlights.html Enjoy this release, Martin Martin v. Loewis martin at v.loewis.de Python Release Manager (on behalf of the entire python-dev team) From martin at v.loewis.de Tue Mar 11 21:28:56 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 11 Mar 2008 21:28:56 +0100 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <47D5CB36.2090305@canterbury.ac.nz> References: <20080309142239.GA18295@panix.com> <47D43F6A.8000808@canterbury.ac.nz> <47D560C7.3020908@gmx.net> <47D5687A.6010107@v.loewis.de> <47D5CB36.2090305@canterbury.ac.nz> Message-ID: <47D6EB88.7060904@v.loewis.de> > Can you really say that you don't make any design > decisions early on based on the assumption that > dict lookup will almost certainly be a lot faster > than searching a list? I follow the advice Guido gave: I use the data structure that will make my code shortest and easiest to read, regardless of performance consequences initially. Premature optimization is the root of all evil. Regards, Martin From brett at python.org Wed Mar 12 00:34:57 2008 From: brett at python.org (Brett Cannon) Date: Tue, 11 Mar 2008 16:34:57 -0700 Subject: [Python-Dev] [OT] Python mentioned on OOPSLA 2008 postcard Message-ID: I just got my postcard announcing OOPSLA 2008. Interesting thing is that the postcard, which lists various things that will be at OOPSLA, includes Python. It's actually listed in the first line and the thing is not alphabetized. And even cooler, it is the first language listed (ruby, Objective-C, C#, and Java are also listed). Anyway, thought other might get a kick out of this like I did. -Brett From stephen at xemacs.org Wed Mar 12 00:56:28 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 12 Mar 2008 08:56:28 +0900 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <47D6EB88.7060904@v.loewis.de> References: <20080309142239.GA18295@panix.com> <47D43F6A.8000808@canterbury.ac.nz> <47D560C7.3020908@gmx.net> <47D5687A.6010107@v.loewis.de> <47D5CB36.2090305@canterbury.ac.nz> <47D6EB88.7060904@v.loewis.de> Message-ID: <87hcfczufn.fsf@uwakimon.sk.tsukuba.ac.jp> "Martin v. L?wis" writes: > Premature optimization is the root of all evil. Actually, Knuth's bon mot was "Premature optimization is the root of all error." Which is probably worse; it leaves the perpetrator the excuse "but I was only trying to help!" While we all know what to do to evildoers, it's hard to punish those who deliberately invite Murphy's attention as severely as they deserve. From guido at python.org Wed Mar 12 00:51:04 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 11 Mar 2008 15:51:04 -0800 Subject: [Python-Dev] [OT] Python mentioned on OOPSLA 2008 postcard In-Reply-To: References: Message-ID: Well maybe they'll invite me as a speaker now. ;-) On Tue, Mar 11, 2008 at 3:34 PM, Brett Cannon wrote: > I just got my postcard announcing OOPSLA 2008. Interesting thing is > that the postcard, which lists various things that will be at OOPSLA, > includes Python. It's actually listed in the first line and the thing > is not alphabetized. And even cooler, it is the first language listed > (ruby, Objective-C, C#, and Java are also listed). > > Anyway, thought other might get a kick out of this like I did. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Wed Mar 12 01:09:31 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 12 Mar 2008 13:09:31 +1300 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: <47D6E8DD.2020909@v.loewis.de> References: <47CDC837.8000104@rksystems.com> <47CDD183.2040201@cheimes.de> <47CDE509.8080801@holdenweb.com> <47CF0A89.2010804@canterbury.ac.nz> <47CFA113.2080101@v.loewis.de> <47D6917B.60502@behnel.de> <47D6E8DD.2020909@v.loewis.de> Message-ID: <47D71F3B.7080600@canterbury.ac.nz> Martin v. L?wis wrote: > To implement such a system, you need to get all ABI dependencies out > of the header files; this includes the structure layouts in particular. That could hurt the performance of some things. Macros like PyList_GET_ITEM etc. rely on knowing about struct layouts to get at things quickly. -- Greg From aahz at pythoncraft.com Wed Mar 12 02:49:53 2008 From: aahz at pythoncraft.com (Aahz) Date: Tue, 11 Mar 2008 18:49:53 -0700 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <87hcfczufn.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20080309142239.GA18295@panix.com> <47D43F6A.8000808@canterbury.ac.nz> <47D560C7.3020908@gmx.net> <47D5687A.6010107@v.loewis.de> <47D5CB36.2090305@canterbury.ac.nz> <47D6EB88.7060904@v.loewis.de> <87hcfczufn.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20080312014953.GB3379@panix.com> On Wed, Mar 12, 2008, Stephen J. Turnbull wrote: > "Martin v. L?wis" writes: >> >> Premature optimization is the root of all evil. > > Actually, Knuth's bon mot was "Premature optimization is the root of > all error." >From my .sig database: "Premature optimization is the root of all evil in programming." --C.A.R. Hoare (often misattributed to Knuth, who was himself quoting Hoare) "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil." --Knuth restates Hoare -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson From greg.ewing at canterbury.ac.nz Wed Mar 12 01:16:05 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 12 Mar 2008 13:16:05 +1300 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <47D6EB88.7060904@v.loewis.de> References: <20080309142239.GA18295@panix.com> <47D43F6A.8000808@canterbury.ac.nz> <47D560C7.3020908@gmx.net> <47D5687A.6010107@v.loewis.de> <47D5CB36.2090305@canterbury.ac.nz> <47D6EB88.7060904@v.loewis.de> Message-ID: <47D720C5.4060103@canterbury.ac.nz> Martin v. L?wis wrote: > I follow the advice Guido gave: I use the data > structure that will make my code shortest and easiest > to read, But given a choice such as list vs. dict, the code is almost identical either way. What do you do then? Toss a coin? Or make a few educated guesses based on what you know about each data type? > Premature optimization is the root of all evil. This phrase seems to be widely misused. Making your code bloated and convoluted without good evidence of the necessity is premature optimisation. Picking an appropriate data structure for the task based on experience is good design practice. -- Greg From martin at v.loewis.de Wed Mar 12 07:52:50 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 12 Mar 2008 07:52:50 +0100 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <47D720C5.4060103@canterbury.ac.nz> References: <20080309142239.GA18295@panix.com> <47D43F6A.8000808@canterbury.ac.nz> <47D560C7.3020908@gmx.net> <47D5687A.6010107@v.loewis.de> <47D5CB36.2090305@canterbury.ac.nz> <47D6EB88.7060904@v.loewis.de> <47D720C5.4060103@canterbury.ac.nz> Message-ID: <47D77DC2.2090508@v.loewis.de> >> I follow the advice Guido gave: I use the data >> structure that will make my code shortest and easiest >> to read, > > But given a choice such as list vs. dict, the code > is almost identical either way. What do you do then? Why do you say that? Lists and dictionaries are *completely* different things, not interchangeable at all. > Toss a coin? Or make a few educated guesses based on > what you know about each data type? I look at my problem, and the choice of container falls out of that naturally. Regards, Martin From martin at v.loewis.de Wed Mar 12 07:55:00 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 12 Mar 2008 07:55:00 +0100 Subject: [Python-Dev] Compiler used to build Python for Windows In-Reply-To: <47D71F3B.7080600@canterbury.ac.nz> References: <47CDC837.8000104@rksystems.com> <47CDD183.2040201@cheimes.de> <47CDE509.8080801@holdenweb.com> <47CF0A89.2010804@canterbury.ac.nz> <47CFA113.2080101@v.loewis.de> <47D6917B.60502@behnel.de> <47D6E8DD.2020909@v.loewis.de> <47D71F3B.7080600@canterbury.ac.nz> Message-ID: <47D77E44.4080701@v.loewis.de> >> To implement such a system, you need to get all ABI dependencies out >> of the header files; this includes the structure layouts in particular. > > That could hurt the performance of some things. Macros > like PyList_GET_ITEM etc. rely on knowing about struct > layouts to get at things quickly. OTOH, they are discouraged for use outside the core already. Use PyList_GetItem, and you won't notice the difference. Regards, Martin From asmodai at in-nomine.org Wed Mar 12 10:27:23 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Wed, 12 Mar 2008 10:27:23 +0100 Subject: [Python-Dev] FreeBSD test suite failure -> curses In-Reply-To: <47D46BB2.1080202@v.loewis.de> References: <20080308092434.GZ60713@nexus.in-nomine.org> <47D273B0.4020809@v.loewis.de> <20080309162900.GA60713@nexus.in-nomine.org> <47D43926.4030902@v.loewis.de> <20080309210210.GB60713@nexus.in-nomine.org> <47D46BB2.1080202@v.loewis.de> Message-ID: <20080312092723.GK60713@nexus.in-nomine.org> -On [20080309 23:59], "Martin v. L?wis" (martin at v.loewis.de) wrote: >So this now *is* a FreeBSD/ncurses expert's question. >I don't think this is supposed to happen; newscr should >become non-NULL when initscr is called, and should remain >that way throughout. Looking at the other FreeBSD build it seems to pass the tests. It's 6.2-RELEASE box. So right now I am getting my VMWare 6.2-R up and running and seeing if I also get the same results. At least that narrows my search to the point of my 6.3-STABLE from the 6.2-RELEASE date. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Alas! The world is full of enormous lights and mysteries, and man shuts them from himself with one small hand! From greg.ewing at canterbury.ac.nz Wed Mar 12 12:23:35 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 13 Mar 2008 00:23:35 +1300 Subject: [Python-Dev] Proxy form not getting through Message-ID: <47D7BD37.7050009@canterbury.ac.nz> I'm trying to send a proxy form, but all my mail to psf at python.org or goodger at python.org is getting bounced. Is there another address I can send it to that goes through a different mail server? -- Greg From andymac at bullseye.apana.org.au Wed Mar 12 13:15:42 2008 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Wed, 12 Mar 2008 22:15:42 +1000 Subject: [Python-Dev] FreeBSD test suite failure -> curses In-Reply-To: <20080312092723.GK60713@nexus.in-nomine.org> References: <20080308092434.GZ60713@nexus.in-nomine.org> <47D273B0.4020809@v.loewis.de> <20080309162900.GA60713@nexus.in-nomine.org> <47D43926.4030902@v.loewis.de> <20080309210210.GB60713@nexus.in-nomine.org> <47D46BB2.1080202@v.loewis.de> <20080312092723.GK60713@nexus.in-nomine.org> Message-ID: <47D7C96E.9090008@bullseye.andymac.org> Jeroen Ruigrok van der Werven wrote: > -On [20080309 23:59], "Martin v. L?wis" (martin at v.loewis.de) wrote: >> So this now *is* a FreeBSD/ncurses expert's question. >> I don't think this is supposed to happen; newscr should >> become non-NULL when initscr is called, and should remain >> that way throughout. > > Looking at the other FreeBSD build it seems to pass the tests. It's > 6.2-RELEASE box. So right now I am getting my VMWare 6.2-R up and running > and seeing if I also get the same results. At least that narrows my search > to the point of my 6.3-STABLE from the 6.2-RELEASE date. test_curses on its own passes on my FreeBSD 6.3 box. It segfaults when run in the context of a full regression test though. I've tried libthr instead of libpthread (via libmap.conf), and it also fails only in the full regression test. The backtrace from the coredump is different from the libpthread case though... libthr: ======= #0 0x2849513c in wecho_wchar () from /lib/libncursesw.so.6 #1 0x28496a89 in wecho_wchar () from /lib/libncursesw.so.6 #2 0x28497e62 in wecho_wchar () from /lib/libncursesw.so.6 #3 0x2849b134 in doupdate () from /lib/libncursesw.so.6 #4 0x287cf247 in PyCurses_doupdate (self=0x0) at /home/andymac/build/python-svn/trunk/Modules/_cursesmodule.c:182 #5 0x080c602e in PyEval_EvalFrameEx (f=0x93cbc0c, throwflag=0) at Python/ceval.c:3620 #6 0x080c6269 in PyEval_EvalFrameEx (f=0x93cba0c, throwflag=0) at Python/ceval.c:3722 #7 0x080c6269 in PyEval_EvalFrameEx (f=0x935b40c, throwflag=0) at Python/ceval.c:3722 #8 0x080c6eda in PyEval_EvalCodeEx (co=0x9b5bb60, globals=0x284c117c, locals=0x6e, args=0x817002c, argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2908 #9 0x080c702e in PyEval_EvalCode (co=0x9b5bb60, globals=0x93fb4f4, locals=0x93fb4f4) at Python/ceval.c:495 #10 0x080d938a in PyImport_ExecCodeModuleEx ( name=0xbfbfdc60 "test.test_curses", co=0x9b5bb60, pathname=0xbfbfd2e0 "/home/andymac/build/python-svn/trunk/Lib/test/test_curses.pyc") at Python/import.c:680 #11 0x080d97f3 in load_source_module (name=0xbfbfdc60 "test.test_curses", pathname=0xbfbfd2e0 "/home/andymac/build/python-svn/trunk/Lib/test/test_curses.pyc", fp=0x282801d8) at Python/import.c:968 ---Type to continue, or q to quit--- libpthread: =========== #0 0x281a655b in pthread_testcancel () from /lib/libpthread.so.2 #1 0x2819eeec in pthread_mutexattr_init () from /lib/libpthread.so.2 #2 0x28166450 in ?? () This is with a checkout updated to r61352. Andrew. -- ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac at bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac at pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From aahz at pythoncraft.com Wed Mar 12 16:27:34 2008 From: aahz at pythoncraft.com (Aahz) Date: Wed, 12 Mar 2008 08:27:34 -0700 Subject: [Python-Dev] Proxy form not getting through In-Reply-To: <47D7BD37.7050009@canterbury.ac.nz> References: <47D7BD37.7050009@canterbury.ac.nz> Message-ID: <20080312152733.GA10078@panix.com> On Thu, Mar 13, 2008, Greg Ewing wrote: > > I'm trying to send a proxy form, but all my mail to > psf at python.org or goodger at python.org is getting bounced. > > Is there another address I can send it to that goes > through a different mail server? What's the error message? I can relay it for you if you want, dunno about other options for mailservers. I think further discussion should go on psf-members. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson From rhamph at gmail.com Wed Mar 12 19:26:31 2008 From: rhamph at gmail.com (Adam Olsen) Date: Wed, 12 Mar 2008 11:26:31 -0700 Subject: [Python-Dev] Complexity documentation request In-Reply-To: References: <20080309142239.GA18295@panix.com> Message-ID: On Mon, Mar 10, 2008 at 12:05 PM, Daniel Stutzbach wrote: > On Sun, Mar 9, 2008 at 9:22 AM, Aahz wrote: > > There probably would be some value in a wiki page on python.org that > > provides this information, particularly across versions. You may be > > able to find volunteers to help on comp.lang.python. > > I just created a very basic one at > http://wiki.python.org/moin/TimeComplexity?action=show > > I'm not that familiar with the Wiki syntax, so the tables are kind of > ugly at the moment. > > I wasn't sure about many of the set() operations, so I didn't include those. For python's purposes, I think it's simpler to classify an operation as either "linear" or "near constant", then have an explanation that "near constant" is only the typical performance (it doesn't make guarantees about worst case behaviour), may include O(log n) implementations, etc. That suffices to distinguish use cases, and anything more specific may be dominated by constant factors anyway. Something like sort is a special case. I don't think the languages needs to guarantee any particular performance, yet it's worth documenting that CPython has a rather good implementation. -- Adam Olsen, aka Rhamphoryncus From jimis at gmx.net Wed Mar 12 20:52:46 2008 From: jimis at gmx.net (Dimitrios Apostolou) Date: Wed, 12 Mar 2008 21:52:46 +0200 Subject: [Python-Dev] Complexity documentation request In-Reply-To: References: <20080309142239.GA18295@panix.com> Message-ID: <47D8348E.8020507@gmx.net> Daniel Stutzbach wrote: > I just created a very basic one at > http://wiki.python.org/moin/TimeComplexity?action=show Hi, Just one quick note. What exactly do you mean by "Amortized worst case"? Shouldn't it just be "Worst case"? I think that the word "amortized" better describes the time complexity of specific operations. For example I think that the insertion in a dictionary should be noted as "O(1) amortized" for the average case meaning that when doing infinite random insertions, the time asymptotically tends to be constant. And worst case is simply O(n), not amortized. Am I missing something? Thanks, Dimitris From daniel at stutzbachenterprises.com Wed Mar 12 21:01:14 2008 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Wed, 12 Mar 2008 15:01:14 -0500 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <47D8348E.8020507@gmx.net> References: <20080309142239.GA18295@panix.com> <47D8348E.8020507@gmx.net> Message-ID: On Wed, Mar 12, 2008 at 2:52 PM, Dimitrios Apostolou wrote: > Just one quick note. What exactly do you mean by "Amortized worst case"? > Shouldn't it just be "Worst case"? I think that the word "amortized" > better describes the time complexity of specific operations. http://en.wikipedia.org/wiki/Amortized_analysis -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From ronaldoussoren at mac.com Wed Mar 12 21:04:11 2008 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 12 Mar 2008 21:04:11 +0100 Subject: [Python-Dev] Python XML Validator In-Reply-To: <47D6BAE9.4050105@behnel.de> References: <20080302230708.260fa4a9@bhuda.mired.org> <20080305102306.0a3db148@mbook-fbsd> <20080311121006.0da78de7@mbook-fbsd> <47D6BAE9.4050105@behnel.de> Message-ID: On 11 Mar, 2008, at 18:01, Stefan Behnel wrote: > Mike Meyer wrote: >> On Tue, 11 Mar 2008 14:55:04 +0100 Stefan Behnel >> wrote: >>> (weird places these threads come up at, but now that it's here...) >>> Mike Meyer wrote: >>>> On Tue, 04 Mar 2008 15:44:32 -0800 Ned Deily wrote: >>>>> In article <20080302230708.260fa4a9 at bhuda.mired.org>, >>>>> Mike Meyer wrote: >>>>>> On Thu, 28 Feb 2008 23:42:49 +0000 (UTC) Medhat Gayed >>>>>> wrote: >>>>>>> lxml is good but not written in python and difficult to >>>>>>> install and didn't >>>>>>> work on MacOS X. >> >> Please note that this original complaint is *not* mine. However... >> >>> Due to a design problem in MacOS-X, not a problem in lxml. >> >> I didn't find it noticeably harder to install lxml on MacOS-X than >> most other systems. > > It seems to be for a number of people, though, who turn up on the > mailing list > complaining about just that. What can make life a bit harder on OSX is universal binaries, although those aren't too hard either. BTW. Which design problem? BTW2. Discusion of problems with building lxml on OSX are better suited for the pythonmac-sig list (or the lxml one of course). >> >> Yes, but the proposal was to include it in the Python standard >> library. Software that doesn't work on popular target platforms >> without updating a standard system library isn't really suitable for >> that. > > Hmm, coming somewhat back on-topic: how does Python currently handle > its > dependencies under MacOS-X? SQLite, for example? Does it use system > libraries > only, or are there libraries it ships with? (The MacOS distro is > much bigger, > but that might be due to the universal build - although that > suggests that > MacOS-X users do not care about disk space or download size anyway) The .dmg on python.org includes it's own copies of sqlite, ncurses and berkeley db. That's mostly needed to be able to run on 10.3.9 or later. My guess is that the size difference with other binary distributions is mostly due to universal binaries, those double the size of executables. This might get worse in the future, I hope to find some time go make the python framework 4-way universal (32-bit and 64-bit code on PPC and Intel). Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20080312/e72d88a6/attachment.bin From asmodai at in-nomine.org Wed Mar 12 21:20:19 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Wed, 12 Mar 2008 21:20:19 +0100 Subject: [Python-Dev] FreeBSD test suite failure -> curses In-Reply-To: <47D7C96E.9090008@bullseye.andymac.org> References: <20080308092434.GZ60713@nexus.in-nomine.org> <47D273B0.4020809@v.loewis.de> <20080309162900.GA60713@nexus.in-nomine.org> <47D43926.4030902@v.loewis.de> <20080309210210.GB60713@nexus.in-nomine.org> <47D46BB2.1080202@v.loewis.de> <20080312092723.GK60713@nexus.in-nomine.org> <47D7C96E.9090008@bullseye.andymac.org> Message-ID: <20080312202019.GQ60713@nexus.in-nomine.org> -On [20080312 13:30], Andrew MacIntyre (andymac at bullseye.apana.org.au) wrote: >test_curses on its own passes on my FreeBSD 6.3 box. It segfaults when >run in the context of a full regression test though. So it does for my 6.3-STABLE. But 6.2-RELEASE goes through the entire regression though. I need to check, but I think in between 6.2-R and 6.3-R Rong-En Fan made the switch to a newer ncurses for the wide character support. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ My name is Legion: for we are many... From webmaster at h-labahn.de Thu Mar 13 01:16:12 2008 From: webmaster at h-labahn.de (Joost Behrends) Date: Thu, 13 Mar 2008 01:16:12 +0100 (CET) Subject: [Python-Dev] Why .index() is not a method of all sequence types ? Message-ID: <20080313010923.d1bc7ee9.webmaster@h-labahn.de> Hello, hopefully this mailing list is the right address for the following. Since there is a huge gap in performance between tuple(cur.execute(...)) and list(cur.execute(...)) - i saw a factor in the magnitude of 50 once - the first has always to be chosen when sufficient. Even if that difference would be smaller - you also document the resulting sequence as read-only in your code. I also use deque( ... some generator ... ) frequently (And i think, that this should have more room in tutorials for beginners - some of them have no idea, what tuples are for). With such a tuple tp i tried 'ix = tp.index(...)' recently and was astonished to learn, that this doesn't work. Since we have '... in tp' for me it seems, that it should make very little difference in the interpreter's code, if .index() would be a method of any sequence, mutable or not. Such a small difference, that this minor change wouldn't deserve a PEP. Do i overlook something here ? Kind regards, Joost Behrends From tnelson at onresolve.com Thu Mar 13 02:20:40 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Wed, 12 Mar 2008 18:20:40 -0700 Subject: [Python-Dev] Request for another build slave Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E16844C8A@EXMBX04.exchhosting.com> Can someone set me up with a build slave for an x86 FreeBSD box (6.2-STABLE, although we'll be migrating to 7.x in a week or so)? Thanks. [Suggestion: perhaps we could set up a python-buildbots at python.org list for discussing buildbot administrative minutiae, rather than polluting python-dev?] Trent. From tjreedy at udel.edu Thu Mar 13 02:26:31 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 12 Mar 2008 21:26:31 -0400 Subject: [Python-Dev] Why .index() is not a method of all sequence types ? References: <20080313010923.d1bc7ee9.webmaster@h-labahn.de> Message-ID: "Joost Behrends" wrote in message news:20080313010923.d1bc7ee9.webmaster at h-labahn.de... | With such a tuple tp i tried 'ix = tp.index(...)' recently and was | astonished to learn, that this doesn't work. Since we have '... in tp' | for me it seems, that it should make very little difference in | the interpreter's code, if .index() would be a method of any sequence, | mutable or not. Such a small difference, that this minor change wouldn't | deserve a PEP. I believe .index() is part of the 3.0 sequence protocol and hence has been added to tuples for 3.0. Don't know if has been or will be backported to 2.6. From Paul.James at amsa.gov.au Thu Mar 13 05:57:19 2008 From: Paul.James at amsa.gov.au (James, Paul) Date: Thu, 13 Mar 2008 15:57:19 +1100 Subject: [Python-Dev] Python 2.5.1 error when running cvs2svn [SEC=UNCLASSIFIED] Message-ID: Hello python-dev, We are trying to run the cvs2svn utility and get an execution problem coming from Python. The version of Python we have installed on a solaris sparc10 machine is 2.5.1. The error trace is below, the error is at end in blue- Traceback (most recent call last): File "/usr/local/bin/cvs2svn", line 27, in from cvs2svn_lib.main import main File "/usr/local/lib/python2.5/site-packages/cvs2svn_lib/main.py", line 31, in from cvs2svn_lib.run_options import RunOptions File "/usr/local/lib/python2.5/site-packages/cvs2svn_lib/run_options.py", line 41, in from cvs2svn_lib.svn_output_option import DumpfileOutputOption File "/usr/local/lib/python2.5/site-packages/cvs2svn_lib/svn_output_option.py ", line 42, in from cvs2svn_lib.svn_repository_mirror import SVNRepositoryMirror File "/usr/local/lib/python2.5/site-packages/cvs2svn_lib/svn_repository_mirro r.py", line 34, in from cvs2svn_lib.serializer import MarshalSerializer File "/usr/local/lib/python2.5/site-packages/cvs2svn_lib/serializer.py", line 24, in import zlib ImportError: ld.so.1: python: fatal: relocation error: file /usr/local/lib/python2.5/lib-dynload/zlib.so: symbol inflateCopy: referenced symbol not found -The zlib.so file exisits on our machine at the specified path. We are using Python 2.5.1. If possible, could you provide advice on where the problem is occuring? Thanks, Paul James Paul.James at amsa.gov.au ********************************************************************** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient, you are notified that any use or dissemination of this communication is prohibited. If you receive this transmission in error, please notify us immediately by telephone on +61 2 62795000 and delete all copies of this transmission together with any attachments. ********************************************************************** -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080313/35f08e63/attachment.htm From jimis at gmx.net Thu Mar 13 09:16:26 2008 From: jimis at gmx.net (Dimitrios Apostolou) Date: Thu, 13 Mar 2008 10:16:26 +0200 Subject: [Python-Dev] Complexity documentation request In-Reply-To: References: <20080309142239.GA18295@panix.com> <47D8348E.8020507@gmx.net> Message-ID: <47D8E2DA.4020709@gmx.net> Daniel Stutzbach wrote: > On Wed, Mar 12, 2008 at 2:52 PM, Dimitrios Apostolou wrote: >> Just one quick note. What exactly do you mean by "Amortized worst case"? >> Shouldn't it just be "Worst case"? I think that the word "amortized" >> better describes the time complexity of specific operations. > > http://en.wikipedia.org/wiki/Amortized_analysis > Thanks for this, I understand now what it means. However given that for the list and deque types both columns have exactly the same values, wouldn't it be more useful if we simply mentioned the worst case in the second, or another, column? On another note which sorting algorithm is python using? Perhaps we can add this as a footnote. I always thought it was quicksort, with a worst case of O(n^2). Thanks, Dimitris From lorgandon at gmail.com Thu Mar 13 09:20:48 2008 From: lorgandon at gmail.com (Imri Goldberg) Date: Thu, 13 Mar 2008 10:20:48 +0200 Subject: [Python-Dev] The Case Against Floating Point == Message-ID: <47D8E3E0.7010509@gmail.com> Heya I've been using Python for development for some years now, but up until now, I didn't participate much in work on Python itself. I've decided to 'take a shot at it', even if a small one at first. To the subject at hand: Most experienced programmers know that you shouldn?t compare floating point numbers with ==. If you want to check floating point equality, you usually decide on a precision, and check either the absolute error, or the relative error. Another option is to use the decimal module. Hence, floating point == isn?t used, maybe except for rare circumstances. I would even venture to say that when possible, static checkers should emit warnings on floating point ==. On the other hand there are the beginner programmers. They usually use == by mistake, and will be surprised by the strange results they sometimes get back. My suggestion is to do either of the following: 1. Change floating point == to behave like a valid floating point comparison. That means using precision and some error measure. 2. Change floating point == to raise an exception, with an error string suggesting using precision comparison, or the decimal module. Since this change is not backwards compatible, I suggest it be added only to Python 3. Personally, I prefer the second option (== to raise an exception). It is clearer, and less confusing. Arguments against this suggestion are: 1. This change breaks existing programs: I believe it most likely triggers hidden bugs. Since the suggestion is to change it only in Python 3, Those programs will most likely be broken by other changes as well, and will need to be changed anyway. 2. This change breaks compatibility with C-like languages: I agree. However, the already agreed on change of the / operator is even a stronger break. Most arguments for changing the / operator apply here as well. 3. Programmers will still need the regular ==: Maybe, and even then, only for very rare cases. For these, a special function\method might be used, which could be named floating_exact_eq. I tried looking up other material about this subject, and while there is plenty of material about how to do floating point code right, I didn't find any suggestion similar to this one. I also wrote this suggestion at my blog, and discussed it a little bit on #python. I'll be glad to read other peoples' thoughts on this. Cheers, Imri -- ------------------------- Imri Goldberg www.algorithm.co.il/blogs www.imri.co.il ------------------------- Insert Signature Here ------------------------- From duncan.booth at suttoncourtenay.org.uk Thu Mar 13 10:57:53 2008 From: duncan.booth at suttoncourtenay.org.uk (Duncan Booth) Date: Thu, 13 Mar 2008 09:57:53 +0000 (UTC) Subject: [Python-Dev] Complexity documentation request References: <20080309142239.GA18295@panix.com> <47D8348E.8020507@gmx.net> <47D8E2DA.4020709@gmx.net> Message-ID: Dimitrios Apostolou wrote: > On another note which sorting algorithm is python using? Perhaps we can > add this as a footnote. I always thought it was quicksort, with a worst > case of O(n^2). See http://svn.python.org/projects/python/trunk/Objects/listsort.txt From ncoghlan at gmail.com Thu Mar 13 13:37:19 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 13 Mar 2008 22:37:19 +1000 Subject: [Python-Dev] Why .index() is not a method of all sequence types ? In-Reply-To: References: <20080313010923.d1bc7ee9.webmaster@h-labahn.de> Message-ID: <47D91FFF.9000905@gmail.com> Terry Reedy wrote: > "Joost Behrends" wrote in message > news:20080313010923.d1bc7ee9.webmaster at h-labahn.de... > | With such a tuple tp i tried 'ix = tp.index(...)' recently and was > | astonished to learn, that this doesn't work. Since we have '... in tp' > | for me it seems, that it should make very little difference in > | the interpreter's code, if .index() would be a method of any sequence, > | mutable or not. Such a small difference, that this minor change wouldn't > | deserve a PEP. > > I believe .index() is part of the 3.0 sequence protocol and hence has been > added to tuples for 3.0. Don't know if has been or will be backported to > 2.6. Python 2.6a1+ (trunk:61289M, Mar 7 2008, 19:45:46) [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 'index' in dir(tuple) True Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From daniel at stutzbachenterprises.com Thu Mar 13 15:03:00 2008 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Thu, 13 Mar 2008 09:03:00 -0500 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <47D8E2DA.4020709@gmx.net> References: <20080309142239.GA18295@panix.com> <47D8348E.8020507@gmx.net> <47D8E2DA.4020709@gmx.net> Message-ID: On Thu, Mar 13, 2008 at 3:16 AM, Dimitrios Apostolou wrote: > On another note which sorting algorithm is python using? Perhaps we can > add this as a footnote. I always thought it was quicksort, with a worst > case of O(n^2). It's a highly optimized variant of mergesort, with some neat ideas to make the best-case O(n). I just made the word "Sort" into a hyperlink, pointing to the link that Duncan Booth pointed out in another response. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From aahz at pythoncraft.com Thu Mar 13 15:39:31 2008 From: aahz at pythoncraft.com (Aahz) Date: Thu, 13 Mar 2008 07:39:31 -0700 Subject: [Python-Dev] The Case Against Floating Point == In-Reply-To: <47D8E3E0.7010509@gmail.com> References: <47D8E3E0.7010509@gmail.com> Message-ID: <20080313143931.GA4803@panix.com> On Thu, Mar 13, 2008, Imri Goldberg wrote: > > My suggestion is to do either of the following: > 1. Change floating point == to behave like a valid floating point > comparison. That means using precision and some error measure. > 2. Change floating point == to raise an exception, with an error string > suggesting using precision comparison, or the decimal module. This is an interesting idea, but python-dev is the wrong place. You should probably start with python-ideas, then move to c.l.python, then finally bring it up on python-3000. But in any event, it probably doesn't matter: it's too late. Although we have not released a beta, we are far along the alpha cycle and this is too big a change. Remember that Python 3.0 is scheduled to have final release in August. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson From dickinsm at gmail.com Thu Mar 13 15:41:12 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Thu, 13 Mar 2008 10:41:12 -0400 Subject: [Python-Dev] The Case Against Floating Point == In-Reply-To: <47D8E3E0.7010509@gmail.com> References: <47D8E3E0.7010509@gmail.com> Message-ID: <5c6f2a5d0803130741k6adbb85cia0a19ea59ed830f4@mail.gmail.com> On Thu, Mar 13, 2008 at 4:20 AM, Imri Goldberg wrote: > My suggestion is to do either of the following: > 1. Change floating point == to behave like a valid floating point > comparison. That means using precision and some error measure. > 2. Change floating point == to raise an exception, with an error string > suggesting using precision comparison, or the decimal module. > I don't much like either of these; I think option 1 would cause a lot of confusion and difficulty---it changes a conceptually simple operation into something more complicated. As for option 2., I'd agree that there are situations where having a warning (not an exception) for floating-point equality (and inequality) tests might be helpful; but that warning should be off by default, or at least easily turned off. Some Fortran compilers have such a (compile-time) warning, I believe. But Fortran's users are much more likely to be writing the sort of code that cares about this. > Since this change is not backwards compatible, I suggest it be added > only to Python 3. > It's already too late for Python 3.0. > > 3. Programmers will still need the regular ==: > Maybe, and even then, only for very rare cases. For these, a special > function\method might be used, which could be named floating_exact_eq. > I disagree with the 'very rare' here. I've seen, and written, code like: if a == 0.0: # deal with exceptional case else: b = c/a ... or similarly, a test (a==b) before doing a division by a-b. That one's kind of dodgy, by the way: a != b doesn't always guarantee that a-b is nonzero, though you're okay if you're on an IEEE 754 platform and a and b are both finite numbers. Or what if you wanted to generate random numbers in the open interval (0.0, 1.0). random.random gives you numbers in [0.0, 1.0), so a careful programmer might well write: while True: x = random.random() if x != 0.0: break (A less fussy programmer might just say that the chance of getting 0.0 is about 1 in 2**53, so it's never going to happen...) Other thoughts: - what should x == x do? - what should 1.0 in set([0.0, 1.0, 2.0]) and 3.0 in set([0.0, 1.0, 2.0]) do? Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080313/a1306cd5/attachment.htm From brett at python.org Thu Mar 13 21:05:30 2008 From: brett at python.org (Brett Cannon) Date: Thu, 13 Mar 2008 15:05:30 -0500 Subject: [Python-Dev] How best to handle test_errno? Message-ID: I am going through my backlog of GHOP work and noticed that test_errno is essentially a no-op at the moment. I was wondering if anyone knew if there are any guaranteed values for the errno module? If not, the test current says that Linux has all the values; is that really true? -Brett From ggpolo at gmail.com Thu Mar 13 21:25:39 2008 From: ggpolo at gmail.com (Guilherme Polo) Date: Thu, 13 Mar 2008 17:25:39 -0300 Subject: [Python-Dev] How best to handle test_errno? In-Reply-To: References: Message-ID: 2008/3/13, Brett Cannon : > I am going through my backlog of GHOP work and noticed that test_errno > is essentially a no-op at the moment. I was wondering if anyone knew > if there are any guaranteed values for the errno module? If not, the > test current says that Linux has all the values; is that really true? > > -Brett Half-answering your email.. ENOTOBACCO is missing here, Linux. -- -- Guilherme H. Polo Goncalves From tjreedy at udel.edu Thu Mar 13 22:13:13 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 13 Mar 2008 17:13:13 -0400 Subject: [Python-Dev] The Case Against Floating Point == References: <47D8E3E0.7010509@gmail.com> Message-ID: "Imri Goldberg" wrote in message news:47D8E3E0.7010509 at gmail.com... | Most experienced programmers know that you shouldn't compare floating | point numbers with ==. As a general statement, this is wrong. Mark gave examples. Please drop the proposal. The reason for the int division change is the idea that expressions involving integral values should, insofar as possible, have the same value result regardless of the types used to carry the input (or output) values. Hence 1/2 should equal 1.0/2.0 == .5. The idea that 1 == 1 and 1.0 == 1.0 should give a different answer goes against this principle and the 2.x project of number unification. tjr From tnelson at onresolve.com Fri Mar 14 00:02:59 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Thu, 13 Mar 2008 16:02:59 -0700 Subject: [Python-Dev] Windows x64 & bsddb 4.4.20 woes Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E1684541D@EXMBX04.exchhosting.com> I've been trying to give the Windows x64 builds a bit of TLC the past few evenings. I managed to get a successful build with all external modules last night (Tcl/Tk required about a half a dozen code/configuration changes each in order to build in a Windows x64 environment with Visual Studio 9, I'll deal with that in a separate thread or roundup issue). Unfortunately, though, we're back to more bsddb issues. I got about 15 tests in without error before test_whichdb ran, which results in the following being called in dbhash.py: return bsddb.hashopen(file, flag, mode) I can trace that call to DBEnv_open() in _bsddb.c: static PyObject* DBEnv_open(DBEnvObject* self, PyObject* args) { int err, flags=0, mode=0660; char *db_home; if (!PyArg_ParseTuple(args, "z|ii:open", &db_home, &flags, &mode)) return NULL; CHECK_ENV_NOT_CLOSED(self); MYDB_BEGIN_ALLOW_THREADS; err = self->db_env->open(self->db_env, db_home, flags, mode); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Placing a breakpoint at the line above and stepping in results in Visual Studio reporting: " A buffer overrun has occurred in python_d.exe which has corrupted the program's internal state. Press Break to debug the program or Continue to terminate the program.". FWIW, the exception is being raised as part of the /GS buffer overflow checks (implemented in gs_result.c, which is provided in my VS9 installation). This has been annoyingly awkward to debug. I can't break down that call into multiple steps in order to try place breakpoints in the db_static module. The callstack isn't that useful either: _bsddb_d.pyd!__crt_debugger_hook() _bsddb_d.pyd!__report_gsfailure(unsigned __int64 StackCookie=2211040) _bsddb_d.pyd!__GSHandlerCheckCommon(void * EstablisherFrame=0x000000000021bce0, ...) _bsddb_d.pyd!__GSHandlerCheck(_EXCEPTION_RECORD * ExceptionRecord=0x000000000021bbc0, ...) ntdll.dll!00000000773ae13d() [Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll] ntdll.dll!00000000773aea57() ntdll.dll!00000000773b59f8() _bsddb_d.pyd!__os_strdup() + 0x18 bytes _bsddb_d.pyd!__os_tmpdir() + 0x281 bytes You'd think placing breakpoints in db 4.4.20's __os_strdup and __os_tmpdir methods would do something, but alas, the bufferoverflow exception is raised before any breakpoints are set. This makes me suspect there's something funky going on with the entire build and linking of db_static (VS should honour those breakpoints if the code is being entered, I even added db_static to pcbuild.sln and rebuilt but no dice). I've noticed that they're not using consistent compiler flags by default (we use /GS, they use /GS-, we allow function level linking, they don't -- note that I did change db_static's options to align with _bsddb's but the bufferoverflow exception is still being thrown). Greg, Jes?s, I'm CC'ing you guys as stfw'ing seems to bring back you two the most when it comes to bsddb issues. I've still got a list of things to try with regarding to debugging this x64 issue, but I wanted to reach out now to see if anyone else had encountered it before. Has bsddb ever been built successfully on Win64 and passed all tests or am I venturing into new ground? Martin, you've changed externals/bsddb-4.4.20 with regards to x64 builds recently -- have you been able to get things working in your x64 environments? Regards, Trent. From skip at pobox.com Thu Mar 13 14:41:08 2008 From: skip at pobox.com (skip at pobox.com) Date: Thu, 13 Mar 2008 08:41:08 -0500 Subject: [Python-Dev] Python 2.5.1 error when running cvs2svn [SEC=UNCLASSIFIED] In-Reply-To: References: Message-ID: <18393.12020.525648.214001@montanaro-dyndns-org.local> Paul> import zlib Paul> ImportError: ld.so.1: python: fatal: relocation error: file Paul> /usr/local/lib/python2.5/lib-dynload/zlib.so: symbol inflateCopy: Paul> referenced symbol not found Paul, Try running ldd /usr/local/lib/python2.5/lib-dynload/zlib.so I'll wager that libz.so is not found. Either rebuild Python using the -R flag to hard-code the location of various low-level libraries in Python's .so files or set LD_LIBRARY_PATH at runtime so the dynamic linker can find them. One other thing (well, two). python-dev at python.org is not the correct place for these sorts of questions. Try python-list at python.org instead. Also, note that many of us who might read this are at PyCon (http://us.pycon.org/) which starts this morning and continues through the middle of next week. Response rates from this list will be reduced. -- Skip Montanaro - skip at pobox.com - http://www.webfast.com/~skip/ From greg at krypto.org Fri Mar 14 03:00:44 2008 From: greg at krypto.org (Gregory P. Smith) Date: Thu, 13 Mar 2008 19:00:44 -0700 Subject: [Python-Dev] Windows x64 & bsddb 4.4.20 woes In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E1684541D@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E1684541D@EXMBX04.exchhosting.com> Message-ID: <52dc1c820803131900vd889255q4bd6fa0ff52d7c53@mail.gmail.com> I haven't built the bsddb stuff on windows myself in a few years and have never had access to a windows x64 system so I'm no silver bullet. Making the BerkeleyDB compile and link options match with those of python is the first place I'd start. Also you should be able to make a debug build of BerkeleyDB (though it sounds like you may have tried that already?). Next off in the debugging i'd take a look at the assembly to see what exactly it was failing to do. If you're at PyCon right now we should meet up and try to figure it out (I just arrived). On 3/13/08, Trent Nelson wrote: > > I've been trying to give the Windows x64 builds a bit of TLC the past few > evenings. I managed to get a successful build with all external modules > last night (Tcl/Tk required about a half a dozen code/configuration changes > each in order to build in a Windows x64 environment with Visual Studio 9, > I'll deal with that in a separate thread or roundup issue). > > Unfortunately, though, we're back to more bsddb issues. I got about 15 > tests in without error before test_whichdb ran, which results in the > following being called in dbhash.py: > > return bsddb.hashopen(file, flag, mode) > > I can trace that call to DBEnv_open() in _bsddb.c: > > static PyObject* > DBEnv_open(DBEnvObject* self, PyObject* args) > { > int err, flags=0, mode=0660; > char *db_home; > > if (!PyArg_ParseTuple(args, "z|ii:open", &db_home, &flags, &mode)) > return NULL; > > CHECK_ENV_NOT_CLOSED(self); > > MYDB_BEGIN_ALLOW_THREADS; > err = self->db_env->open(self->db_env, db_home, flags, mode); > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > Placing a breakpoint at the line above and stepping in results in Visual > Studio reporting: " A buffer overrun has occurred in python_d.exe which has > corrupted the program's internal state. Press Break to debug the program or > Continue to terminate the program.". FWIW, the exception is being raised as > part of the /GS buffer overflow checks (implemented in gs_result.c, which is > provided in my VS9 installation). > > This has been annoyingly awkward to debug. I can't break down that call > into multiple steps in order to try place breakpoints in the db_static > module. The callstack isn't that useful either: > > _bsddb_d.pyd!__crt_debugger_hook() > _bsddb_d.pyd!__report_gsfailure(unsigned __int64 StackCookie=2211040) > _bsddb_d.pyd!__GSHandlerCheckCommon(void * > EstablisherFrame=0x000000000021bce0, ...) > _bsddb_d.pyd!__GSHandlerCheck(_EXCEPTION_RECORD * > ExceptionRecord=0x000000000021bbc0, ...) > ntdll.dll!00000000773ae13d() > [Frames below may be incorrect and/or missing, no symbols loaded for > ntdll.dll] > ntdll.dll!00000000773aea57() > ntdll.dll!00000000773b59f8() > _bsddb_d.pyd!__os_strdup() + 0x18 bytes > _bsddb_d.pyd!__os_tmpdir() + 0x281 bytes > > You'd think placing breakpoints in db 4.4.20's __os_strdup and __os_tmpdir > methods would do something, but alas, the bufferoverflow exception is raised > before any breakpoints are set. This makes me suspect there's something > funky going on with the entire build and linking of db_static (VS should > honour those breakpoints if the code is being entered, I even added > db_static to pcbuild.sln and rebuilt but no dice). I've noticed that > they're not using consistent compiler flags by default (we use /GS, they use > /GS-, we allow function level linking, they don't -- note that I did change > db_static's options to align with _bsddb's but the bufferoverflow exception > is still being thrown). > > Greg, Jes?s, I'm CC'ing you guys as stfw'ing seems to bring back you two > the most when it comes to bsddb issues. I've still got a list of things to > try with regarding to debugging this x64 issue, but I wanted to reach out > now to see if anyone else had encountered it before. Has bsddb ever been > built successfully on Win64 and passed all tests or am I venturing into new > ground? > > Martin, you've changed externals/bsddb-4.4.20 with regards to x64 builds > recently -- have you been able to get things working in your x64 > environments? > > Regards, > > > Trent. > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080313/7fdde024/attachment.htm From tnelson at onresolve.com Fri Mar 14 04:02:46 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Thu, 13 Mar 2008 20:02:46 -0700 Subject: [Python-Dev] Windows x64 & bsddb 4.4.20 woes In-Reply-To: <52dc1c820803131900vd889255q4bd6fa0ff52d7c53@mail.gmail.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E1684541D@EXMBX04.exchhosting.com>, <52dc1c820803131900vd889255q4bd6fa0ff52d7c53@mail.gmail.com> Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE0D@EXMBX04.exchhosting.com> Hey Greg, I'm at PyCon indeed, staying through the sprints 'til next Thursday. I'll drop you a note offline re catching up. The other query I had was whether or not I should try a later version of BerkeleyDB -- are we committed to 4.4.20 (or 4.4.x?) for 2.6/3.0 or is it worth investigating newer versions? Martin/Jesus, any thoughts on this? Regarding the db_static build and conflicting compile/link options -- I'm going to bring the db_static source directly into the _bsddb project (for now) which should make this a lot easier to debug. Trent. From: Gregory P. Smith [greg at krypto.org] Sent: 13 March 2008 22:00 To: Trent Nelson Cc: python-dev at python.org; Jesus Cea Subject: Re: Windows x64 & bsddb 4.4.20 woes I haven't built the bsddb stuff on windows myself in a few years and have never had access to a windows x64 system so I'm no silver bullet. Making the BerkeleyDB compile and link options match with those of python is the first place I'd start. Also you should be able to make a debug build of BerkeleyDB (though it sounds like you may have tried that already?). Next off in the debugging i'd take a look at the assembly to see what exactly it was failing to do. If you're at PyCon right now we should meet up and try to figure it out (I just arrived). On 3/13/08, Trent Nelson wrote: I've been trying to give the Windows x64 builds a bit of TLC the past few evenings. I managed to get a successful build with all external modules last night (Tcl/Tk required about a half a dozen code/configuration changes each in order to build in a Windows x64 environment with Visual Studio 9, I'll deal with that in a separate thread or roundup issue). Unfortunately, though, we're back to more bsddb issues. I got about 15 tests in without error before test_whichdb ran, which results in the following being called in dbhash.py: return bsddb.hashopen(file, flag, mode) I can trace that call to DBEnv_open() in _bsddb.c: static PyObject* DBEnv_open(DBEnvObject* self, PyObject* args) { int err, flags=0, mode=0660; char *db_home; if (!PyArg_ParseTuple(args, "z|ii:open", &db_home, &flags, &mode)) return NULL; CHECK_ENV_NOT_CLOSED(self); MYDB_BEGIN_ALLOW_THREADS; err = self->db_env->open(self->db_env, db_home, flags, mode); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Placing a breakpoint at the line above and stepping in results in Visual Studio reporting: " A buffer overrun has occurred in python_d.exe which has corrupted the program's internal state. Press Break to debug the program or Continue to terminate the program.". FWIW, the exception is being raised as part of the /GS buffer overflow checks (implemented in gs_result.c, which is provided in my VS9 installation). This has been annoyingly awkward to debug. I can't break down that call into multiple steps in order to try place breakpoints in the db_static module. The callstack isn't that useful either: _bsddb_d.pyd!__crt_debugger_hook() _bsddb_d.pyd!__report_gsfailure(unsigned __int64 StackCookie=2211040) _bsddb_d.pyd!__GSHandlerCheckCommon(void * EstablisherFrame=0x000000000021bce0, ...) _bsddb_d.pyd!__GSHandlerCheck(_EXCEPTION_RECORD * ExceptionRecord=0x000000000021bbc0, ...) ntdll.dll!00000000773ae13d() [Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll] ntdll.dll!00000000773aea57() ntdll.dll!00000000773b59f8() _bsddb_d.pyd!__os_strdup() + 0x18 bytes _bsddb_d.pyd!__os_tmpdir() + 0x281 bytes You'd think placing breakpoints in db 4.4.20's __os_strdup and __os_tmpdir methods would do something, but alas, the bufferoverflow exception is raised before any breakpoints are set. This makes me suspect there's something funky going on with the entire build and linking of db_static (VS should honour those breakpoints if the code is being entered, I even added db_static to pcbuild.sln and rebuilt but no dice). I've noticed that they're not using consistent compiler flags by default (we use /GS, they use /GS-, we allow function level linking, they don't -- note that I did change db_static's options to align with _bsddb's but the bufferoverflow exception is still being thrown). Greg, Jes?s, I'm CC'ing you guys as stfw'ing seems to bring back you two the most when it comes to bsddb issues. I've still got a list of things to try with regarding to debugging this x64 issue, but I wanted to reach out now to see if anyone else had encountered it before. Has bsddb ever been built successfully on Win64 and passed all tests or am I venturing into new ground? Martin, you've changed externals/bsddb-4.4.20 with regards to x64 builds recently -- have you been able to get things working in your x64 environments? Regards, Trent. From tiagoaoa at cos.ufrj.br Fri Mar 14 03:50:14 2008 From: tiagoaoa at cos.ufrj.br (Tiago A.O.A.) Date: Thu, 13 Mar 2008 23:50:14 -0300 Subject: [Python-Dev] The Case Against Floating Point == In-Reply-To: References: <47D8E3E0.7010509@gmail.com> Message-ID: <47D9E7E6.9090401@cos.ufrj.br> I would suggest something like a ~= b, for "approximately equal to". How approximately? Well, there would be a default that could be changed somewhere. Don't know if it's all that useful, though. Tiago A.O.A. Terry Reedy escreveu: > "Imri Goldberg" wrote in message > news:47D8E3E0.7010509 at gmail.com... > | Most experienced programmers know that you shouldn't compare floating > | point numbers with ==. > > As a general statement, this is wrong. Mark gave examples. > Please drop the proposal. > > The reason for the int division change is the idea that expressions > involving integral values should, insofar as possible, have the same value > result regardless of the types used to carry the input (or output) values. > Hence 1/2 should equal 1.0/2.0 == .5. The idea that 1 == 1 and 1.0 == 1.0 > should give a different answer goes against this principle and the 2.x > project of number unification. > > tjr > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/tiagoaoa%40cos.ufrj.br > > From greg at krypto.org Fri Mar 14 05:23:54 2008 From: greg at krypto.org (Gregory P. Smith) Date: Thu, 13 Mar 2008 23:23:54 -0500 Subject: [Python-Dev] Windows x64 & bsddb 4.4.20 woes In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE0D@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E1684541D@EXMBX04.exchhosting.com> <52dc1c820803131900vd889255q4bd6fa0ff52d7c53@mail.gmail.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE0D@EXMBX04.exchhosting.com> Message-ID: <52dc1c820803132123q7fdc1cfdj136dc4f20698ce39@mail.gmail.com> On 3/13/08, Trent Nelson wrote: > > Hey Greg, > > I'm at PyCon indeed, staying through the sprints 'til next Thursday. I'll > drop you a note offline re catching up. > > The other query I had was whether or not I should try a later version of > BerkeleyDB -- are we committed to 4.4.20 (or 4.4.x?) for 2.6/3.0 or is it > worth investigating newer versions? Martin/Jesus, any thoughts on this? > Python 2.6/3.0 should be built on Windows using BerkeleyDB 4.5.x for now. 4.6.x is out but has some bugs on some platforms so i don't recommend shipping our release using it; 4.7.x is in beta and some bugs are being worked on; if its out and shows no signs of obvious issues before the 2.6/3.0 beta period is over I recommend we build our binary releases using it. Otherwise 4.5 it will be. There is no reason to use 4.4.x. Regarding the db_static build and conflicting compile/link options -- I'm > going to bring the db_static source directly into the _bsddb project (for > now) which should make this a lot easier to debug. > > Trent. > > > From: Gregory P. Smith [greg at krypto.org] > Sent: 13 March 2008 22:00 > To: Trent Nelson > Cc: python-dev at python.org; Jesus Cea > Subject: Re: Windows x64 & bsddb 4.4.20 woes > > > > I haven't built the bsddb stuff on windows myself in a few years and have > never had access to a windows x64 system so I'm no silver bullet. Making > the BerkeleyDB compile and link options match with those of python is the > first place I'd start. Also you should be able to make a debug build of > BerkeleyDB (though it sounds like you may have tried that already?). Next > off in the debugging i'd take a look at the assembly to see what exactly it > was failing to do. > > If you're at PyCon right now we should meet up and try to figure it out (I > just arrived). > > > On 3/13/08, Trent Nelson wrote: > I've been trying to give the Windows x64 builds a bit of TLC the past few > evenings. I managed to get a successful build with all external modules > last night (Tcl/Tk required about a half a dozen code/configuration changes > each in order to build in a Windows x64 environment with Visual Studio 9, > I'll deal with that in a separate thread or roundup issue). > > Unfortunately, though, we're back to more bsddb issues. I got about 15 > tests in without error before test_whichdb ran, which results in the > following being called in dbhash.py: > > return bsddb.hashopen(file, flag, mode) > > I can trace that call to DBEnv_open() in _bsddb.c: > > static PyObject* > DBEnv_open(DBEnvObject* self, PyObject* args) > { > int err, flags=0, mode=0660; > char *db_home; > > if (!PyArg_ParseTuple(args, "z|ii:open", &db_home, &flags, &mode)) > return NULL; > > CHECK_ENV_NOT_CLOSED(self); > > MYDB_BEGIN_ALLOW_THREADS; > err = self->db_env->open(self->db_env, db_home, flags, mode); > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > Placing a breakpoint at the line above and stepping in results in Visual > Studio reporting: " A buffer overrun has occurred in python_d.exe which has > corrupted the program's internal state. Press Break to debug the program or > Continue to terminate the program.". FWIW, the exception is being raised as > part of the /GS buffer overflow checks (implemented in gs_result.c, which is > provided in my VS9 installation). > > This has been annoyingly awkward to debug. I can't break down that call > into multiple steps in order to try place breakpoints in the db_static > module. The callstack isn't that useful either: > > _bsddb_d.pyd!__crt_debugger_hook() > _bsddb_d.pyd!__report_gsfailure(unsigned __int64 StackCookie=2211040) > _bsddb_d.pyd!__GSHandlerCheckCommon(void * > EstablisherFrame=0x000000000021bce0, ...) > _bsddb_d.pyd!__GSHandlerCheck(_EXCEPTION_RECORD * > ExceptionRecord=0x000000000021bbc0, ...) > ntdll.dll!00000000773ae13d() > [Frames below may be incorrect and/or missing, no symbols loaded for > ntdll.dll] > ntdll.dll!00000000773aea57() > ntdll.dll!00000000773b59f8() > _bsddb_d.pyd!__os_strdup() + 0x18 bytes > _bsddb_d.pyd!__os_tmpdir() + 0x281 bytes > > You'd think placing breakpoints in db 4.4.20's __os_strdup and __os_tmpdir > methods would do something, but alas, the bufferoverflow exception is raised > before any breakpoints are set. This makes me suspect there's something > funky going on with the entire build and linking of db_static (VS should > honour those breakpoints if the code is being entered, I even added > db_static to pcbuild.sln and rebuilt but no dice). I've noticed that > they're not using consistent compiler flags by default (we use /GS, they use > /GS-, we allow function level linking, they don't -- note that I did change > db_static's options to align with _bsddb's but the bufferoverflow exception > is still being thrown). > > Greg, Jes?s, I'm CC'ing you guys as stfw'ing seems to bring back you two > the most when it comes to bsddb issues. I've still got a list of things to > try with regarding to debugging this x64 issue, but I wanted to reach out > now to see if anyone else had encountered it before. Has bsddb ever been > built successfully on Win64 and passed all tests or am I venturing into new > ground? > > Martin, you've changed externals/bsddb-4.4.20 with regards to x64 builds > recently -- have you been able to get things working in your x64 > environments? > > Regards, > > > Trent. > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080313/6b6ff2b3/attachment-0001.htm From tnelson at onresolve.com Fri Mar 14 06:49:25 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Thu, 13 Mar 2008 22:49:25 -0700 Subject: [Python-Dev] Windows x64 & bsddb 4.4.20 woes In-Reply-To: <52dc1c820803132123q7fdc1cfdj136dc4f20698ce39@mail.gmail.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E1684541D@EXMBX04.exchhosting.com> <52dc1c820803131900vd889255q4bd6fa0ff52d7c53@mail.gmail.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE0D@EXMBX04.exchhosting.com>, <52dc1c820803132123q7fdc1cfdj136dc4f20698ce39@mail.gmail.com> Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE10@EXMBX04.exchhosting.com> Ah, and to think I just fixed 4.4.20 ;-) Removing the dependency on db_static.vcproj and merging the relevant source code files into _bsddb.vcproj did the trick -- all x64 bsddb-related tests now pass. The only issue with this approach is that it locks _bsddb.vcproj into 4.4.20. However, considering that this approach (i.e. bringing their source files into our build instead of linking against a static lib compiled with wildly incompatible flags) only took me about two minutes to implement and immediately fixed every bsddb problem I was encoutering, I'm convinced it's the right way to go. (I can separate the dependencies easily enough.) Woeful PyCon/hotel connectivity is preventing me from getting to bugs.python.org at the moment; I'll raise a ticket later to capture this stuff and we can move the discussion there once I've attached some patches. Trent. From: Gregory P. Smith [greg at krypto.org] Sent: 14 March 2008 00:23 To: Trent Nelson Cc: python-dev at python.org; Jesus Cea Subject: Re: Windows x64 & bsddb 4.4.20 woes On 3/13/08, Trent Nelson wrote: Hey Greg, I'm at PyCon indeed, staying through the sprints 'til next Thursday. I'll drop you a note offline re catching up. The other query I had was whether or not I should try a later version of BerkeleyDB -- are we committed to 4.4.20 (or 4.4.x?) for 2.6/3.0 or is it worth investigating newer versions? Martin/Jesus, any thoughts on this? Python 2.6/3.0 should be built on Windows using BerkeleyDB 4.5.x for now. 4.6.x is out but has some bugs on some platforms so i don't recommend shipping our release using it; 4.7.x is in beta and some bugs are being worked on; if its out and shows no signs of obvious issues before the 2.6/3.0 beta period is over I recommend we build our binary releases using it. Otherwise 4.5 it will be. There is no reason to use 4.4.x. Regarding the db_static build and conflicting compile/link options -- I'm going to bring the db_static source directly into the _bsddb project (for now) which should make this a lot easier to debug. Trent. From: Gregory P. Smith [greg at krypto.org] Sent: 13 March 2008 22:00 To: Trent Nelson Cc: python-dev at python.org; Jesus Cea Subject: Re: Windows x64 & bsddb 4.4.20 woes I haven't built the bsddb stuff on windows myself in a few years and have never had access to a windows x64 system so I'm no silver bullet. Making the BerkeleyDB compile and link options match with those of python is the first place I'd start. Also you should be able to make a debug build of BerkeleyDB (though it sounds like you may have tried that already?). Next off in the debugging i'd take a look at the assembly to see what exactly it was failing to do. If you're at PyCon right now we should meet up and try to figure it out (I just arrived). On 3/13/08, Trent Nelson wrote: I've been trying to give the Windows x64 builds a bit of TLC the past few evenings. I managed to get a successful build with all external modules last night (Tcl/Tk required about a half a dozen code/configuration changes each in order to build in a Windows x64 environment with Visual Studio 9, I'll deal with that in a separate thread or roundup issue). Unfortunately, though, we're back to more bsddb issues. I got about 15 tests in without error before test_whichdb ran, which results in the following being called in dbhash.py: return bsddb.hashopen(file, flag, mode) I can trace that call to DBEnv_open() in _bsddb.c: static PyObject* DBEnv_open(DBEnvObject* self, PyObject* args) { int err, flags=0, mode=0660; char *db_home; if (!PyArg_ParseTuple(args, "z|ii:open", &db_home, &flags, &mode)) return NULL; CHECK_ENV_NOT_CLOSED(self); MYDB_BEGIN_ALLOW_THREADS; err = self->db_env->open(self->db_env, db_home, flags, mode); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Placing a breakpoint at the line above and stepping in results in Visual Studio reporting: " A buffer overrun has occurred in python_d.exe which has corrupted the program's internal state. Press Break to debug the program or Continue to terminate the program.". FWIW, the exception is being raised as part of the /GS buffer overflow checks (implemented in gs_result.c, which is provided in my VS9 installation). This has been annoyingly awkward to debug. I can't break down that call into multiple steps in order to try place breakpoints in the db_static module. The callstack isn't that useful either: _bsddb_d.pyd!__crt_debugger_hook() _bsddb_d.pyd!__report_gsfailure(unsigned __int64 StackCookie=2211040) _bsddb_d.pyd!__GSHandlerCheckCommon(void * EstablisherFrame=0x000000000021bce0, ...) _bsddb_d.pyd!__GSHandlerCheck(_EXCEPTION_RECORD * ExceptionRecord=0x000000000021bbc0, ...) ntdll.dll!00000000773ae13d() [Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll] ntdll.dll!00000000773aea57() ntdll.dll!00000000773b59f8() _bsddb_d.pyd!__os_strdup() + 0x18 bytes _bsddb_d.pyd!__os_tmpdir() + 0x281 bytes You'd think placing breakpoints in db 4.4.20's __os_strdup and __os_tmpdir methods would do something, but alas, the bufferoverflow exception is raised before any breakpoints are set. This makes me suspect there's something funky going on with the entire build and linking of db_static (VS should honour those breakpoints if the code is being entered, I even added db_static to pcbuild.sln and rebuilt but no dice). I've noticed that they're not using consistent compiler flags by default (we use /GS, they use /GS-, we allow function level linking, they don't -- note that I did change db_static's options to align with _bsddb's but the bufferoverflow exception is still being thrown). Greg, Jes?s, I'm CC'ing you guys as stfw'ing seems to bring back you two the most when it comes to bsddb issues. I've still got a list of things to try with regarding to debugging this x64 issue, but I wanted to reach out now to see if anyone else had encountered it before. Has bsddb ever been built successfully on Win64 and passed all tests or am I venturing into new ground? Martin, you've changed externals/bsddb-4.4.20 with regards to x64 builds recently -- have you been able to get things working in your x64 environments? Regards, Trent. From martin at v.loewis.de Fri Mar 14 13:07:01 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMwpp3aXMi?=) Date: Fri, 14 Mar 2008 07:07:01 -0500 Subject: [Python-Dev] Windows x64 & bsddb 4.4.20 woes In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE10@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E1684541D@EXMBX04.exchhosting.com> <52dc1c820803131900vd889255q4bd6fa0ff52d7c53@mail.gmail.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE0D@EXMBX04.exchhosting.com>, <52dc1c820803132123q7fdc1cfdj136dc4f20698ce39@mail.gmail.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE10@EXMBX04.exchhosting.com> Message-ID: <47DA6A65.9020905@v.loewis.de> > Removing the dependency on db_static.vcproj and merging the relevant > source code files into _bsddb.vcproj did the trick -- all x64 > bsddb-related tests now pass. The only issue with this approach is > that it locks _bsddb.vcproj into 4.4.20. However, considering that > this approach (i.e. bringing their source files into our build > instead of linking against a static lib compiled with wildly > incompatible flags) only took me about two minutes to implement and > immediately fixed every bsddb problem I was encoutering, I'm > convinced it's the right way to go. (I can separate the dependencies > easily enough.) I'm convinced this is the wrong approach. Are you sure you copied all compiler settings over to the project correctly? What is the procedure to upgrade such a setup? What is the procedure for people who want to build with a different version of bsddb? Regards, Martin From tnelson at onresolve.com Fri Mar 14 13:54:31 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Fri, 14 Mar 2008 05:54:31 -0700 Subject: [Python-Dev] Windows x64 & bsddb 4.4.20 woes In-Reply-To: <47DA6A65.9020905@v.loewis.de> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E1684541D@EXMBX04.exchhosting.com> <52dc1c820803131900vd889255q4bd6fa0ff52d7c53@mail.gmail.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE0D@EXMBX04.exchhosting.com>, <52dc1c820803132123q7fdc1cfdj136dc4f20698ce39@mail.gmail.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE10@EXMBX04.exchhosting.com>, <47DA6A65.9020905@v.loewis.de> Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE12@EXMBX04.exchhosting.com> > > Removing the dependency on db_static.vcproj and merging the relevant > > source code files into _bsddb.vcproj did the trick -- all x64 > > bsddb-related tests now pass. The only issue with this approach is > > that it locks _bsddb.vcproj into 4.4.20. However, considering that > > this approach (i.e. bringing their source files into our build > > instead of linking against a static lib compiled with wildly > > incompatible flags) only took me about two minutes to implement and > > immediately fixed every bsddb problem I was encoutering, I'm > > convinced it's the right way to go. (I can separate the dependencies > > easily enough.) > > I'm convinced this is the wrong approach. Are you sure you copied > all compiler settings over to the project correctly? What is the > procedure to upgrade such a setup? What is the procedure for people > who want to build with a different version of bsddb? I reviewed all the compiler options used by db_static.vcproj -- the only thing I needed to bring over was -DDIAGNOSTIC for debug builds. Everything else either had no impact and could be safely dropped, or conflicted with compiler options used by the rest of the python build (function level linking, buffer overflow checks, etc). Regarding support for users who want to build with different versions of bsddb; if they want a functional build that passes tests they're going to have to do something similar to the work I've done anyway. As it stands now, the .lib generated by db_static.vcproj for x64 builds just straight out does not work. That can be fixed in two ways: coerce db_static.vcproj into matching our build, or mimicking db_static in a new .vcproj that's contained with our build, inheriting our property sheets. I chose the latter. Trent. From martin at v.loewis.de Fri Mar 14 16:59:21 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMwpp3aXMi?=) Date: Fri, 14 Mar 2008 10:59:21 -0500 Subject: [Python-Dev] Windows x64 & bsddb 4.4.20 woes In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE12@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E1684541D@EXMBX04.exchhosting.com> <52dc1c820803131900vd889255q4bd6fa0ff52d7c53@mail.gmail.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE0D@EXMBX04.exchhosting.com>, <52dc1c820803132123q7fdc1cfdj136dc4f20698ce39@mail.gmail.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE10@EXMBX04.exchhosting.com>, <47DA6A65.9020905@v.loewis.de> <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE12@EXMBX04.exchhosting.com> Message-ID: <47DAA0D9.7090409@v.loewis.de> > I reviewed all the compiler options used by db_static.vcproj -- the > only thing I needed to bring over was -DDIAGNOSTIC for debug builds. > Everything else either had no impact and could be safely dropped, or > conflicted with compiler options used by the rest of the python build > (function level linking, buffer overflow checks, etc). If you take out those options from the db_static project, it *should* work just the same as what you've got now, right? Can you use that approach to determine the culprit for making the static library approach fail? > As it stands now, the .lib generated by db_static.vcproj for x64 > builds just straight out does not work. That can be fixed in two > ways: coerce db_static.vcproj into matching our build, or mimicking > db_static in a new .vcproj that's contained with our build, > inheriting our property sheets. I chose the latter. I would *really* prefer the former. Regards, Martin From status at bugs.python.org Fri Mar 14 18:06:18 2008 From: status at bugs.python.org (Tracker) Date: Fri, 14 Mar 2008 18:06:18 +0100 (CET) Subject: [Python-Dev] Summary of Tracker Issues Message-ID: <20080314170618.9BD8C780F7@psf.upfronthosting.co.za> ACTIVITY SUMMARY (03/07/08 - 03/14/08) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1735 open (+23) / 12380 closed (+14) / 14115 total (+37) Open issues with patches: 474 Average duration of open issues: 739 days. Median duration of open issues: 1152 days. Open Issues Breakdown open 1711 (+23) pending 24 ( +0) Issues Created Or Reopened (37) _______________________________ "continue" documentation 03/07/08 CLOSED http://bugs.python.org/issue2252 created blp "continue" documentation internally inconsistent 03/07/08 CLOSED http://bugs.python.org/issue2253 created blp Python CGIHTTPServer information disclosure 03/07/08 http://bugs.python.org/issue2254 created m.sucajtys patch Change Mandrake by Mandriva for platform.dist() 03/07/08 CLOSED http://bugs.python.org/issue2255 created neoclust patch Install failure of 2.6a1 on Windows XP without VS8 installed 03/07/08 http://bugs.python.org/issue2256 created jkleckner typo in tutorial section 4.4 - final break statement is missing 03/08/08 CLOSED http://bugs.python.org/issue2257 created Kyte999 Update command line docs for issue 1739468 03/09/08 http://bugs.python.org/issue2258 created ncoghlan easy Poor support other than 44.1khz, 16bit audio files? 03/09/08 http://bugs.python.org/issue2259 created loki_dePlume patch conditional jump to a POP_TOP optimization 03/09/08 http://bugs.python.org/issue2260 created _doublep patch Warning: could not send message for past 4 hours 03/09/08 CLOSED http://bugs.python.org/issue2261 created barnabas79 Helping the compiler avoid memory references in PyEval_EvalFrame 03/09/08 http://bugs.python.org/issue2262 created jyasskin patch, patch struct.pack() + numpy int raises SystemError 03/10/08 http://bugs.python.org/issue2263 created jvr empty specifier for float.__format__ does not always print at le 03/10/08 http://bugs.python.org/issue2264 created eric.smith A line in the second example of "7.3.5 Examples" in "Python Libr 03/10/08 CLOSED http://bugs.python.org/issue2265 created furutaka Missing documentation about old/new-style classes 03/10/08 http://bugs.python.org/issue2266 created Yinon datetime.datetime operator methods are not subclass-friendly 03/10/08 CLOSED http://bugs.python.org/issue2267 created stingray patch Fold slice constants 03/10/08 http://bugs.python.org/issue2268 created belopolsky patch Problem reporting non-keyword arg after keyword arg syntax error 03/10/08 CLOSED http://bugs.python.org/issue2269 created ijmorlan Typo on 8.6.2.5 Document Objects page 03/10/08 CLOSED http://bugs.python.org/issue2270 created throw6617 msi installs to the incorrect location (C drive) 03/10/08 http://bugs.python.org/issue2271 created rossmclendon Segment Violation when using smtp with tls 03/11/08 CLOSED http://bugs.python.org/issue2272 created hareldvd test_decimal: possible thread lockup in test case 03/11/08 http://bugs.python.org/issue2273 created jaredgrubb patch heapq API 03/11/08 CLOSED http://bugs.python.org/issue2274 created rhettinger urllib2 header capitalization 03/11/08 http://bugs.python.org/issue2275 created frispete patch distutils out-of-date for runtime_library_dirs flag on OS X 03/12/08 http://bugs.python.org/issue2276 created janssen easy MozillaCookieJar does not support Firefox3 cookie files 03/12/08 http://bugs.python.org/issue2277 created thekorn patch [Py30a3] xml.parsers.expat recognizes encoding="utf-8" but not e 03/12/08 http://bugs.python.org/issue2278 created mark distutils sdist add_defaults does not add data_files 03/12/08 http://bugs.python.org/issue2279 created dripton parser module chokes on unusual characters 03/12/08 http://bugs.python.org/issue2280 created dbinger patch Enhanced cPython profiler with high-resolution timer 03/12/08 http://bugs.python.org/issue2281 created MrJean1 TextIOWrapper.seekable() always returns False 03/13/08 http://bugs.python.org/issue2282 created netzhen lambda *a, **k: a, k # does not work 03/13/08 CLOSED http://bugs.python.org/issue2283 created mark [PATCH] -x64 option added to pcbuild/rt.bat to facilitate testin 03/14/08 CLOSED http://bugs.python.org/issue2284 created Trent.Nelson patch list.sort.__doc__ says "cmp" is a keyword, but it isn't. 03/14/08 CLOSED http://bugs.python.org/issue2285 created dbinger Stack overflow exception caused by test_marshal on Windows x64 03/14/08 http://bugs.python.org/issue2286 created Trent.Nelson Problems using logging module with logging.basicConfig(level=log 03/14/08 http://bugs.python.org/issue2287 created panhudie Confusing documentation for urllib.urlopen 03/14/08 http://bugs.python.org/issue2288 created marketdickinson Issues Now Closed (35) ______________________ Patch to make py3k/Lib/test/test_thread.py use unittest 182 days http://bugs.python.org/issue1163 brett.cannon patch str.format() produces different output on different platforms (P 89 days http://bugs.python.org/issue1600 eric.smith locale.strxfrm can't handle non-ascii strings 86 days http://bugs.python.org/issue1618 loewis test_select.py converted to unittest 46 days http://bugs.python.org/issue1952 brett.cannon patch test_gdbm.py converted to unittest 45 days http://bugs.python.org/issue1960 brett.cannon patch localeconv() does not encode returned strings 36 days http://bugs.python.org/issue1995 loewis test_fcntl.py converted to unittest 33 days http://bugs.python.org/issue2055 brett.cannon patch Empty test 23 days http://bugs.python.org/issue2119 loewis with should be as fast as try/finally 14 days http://bugs.python.org/issue2179 jyasskin patch map and filter shouldn't support None as first argument (in Py3k 17 days http://bugs.python.org/issue2186 belopolsky patch, easy map and filter objects shouldn't call themselves itertools.imap 17 days http://bugs.python.org/issue2187 rhettinger easy Additional Flag For Unit-Test Module: There Can Be Only One (Err 3 days http://bugs.python.org/issue2241 bcwhite To document "assertTrue" in unittest 2 days http://bugs.python.org/issue2249 georg.brandl patch "continue" documentation 0 days http://bugs.python.org/issue2252 georg.brandl "continue" documentation internally inconsistent 1 days http://bugs.python.org/issue2253 georg.brandl Change Mandrake by Mandriva for platform.dist() 0 days http://bugs.python.org/issue2255 lemburg patch typo in tutorial section 4.4 - final break statement is missing 0 days http://bugs.python.org/issue2257 georg.brandl Warning: could not send message for past 4 hours 0 days http://bugs.python.org/issue2261 loewis A line in the second example of "7.3.5 Examples" in "Python Libr 3 days http://bugs.python.org/issue2265 georg.brandl datetime.datetime operator methods are not subclass-friendly 0 days http://bugs.python.org/issue2267 belopolsky patch Problem reporting non-keyword arg after keyword arg syntax error 0 days http://bugs.python.org/issue2269 facundobatista Typo on 8.6.2.5 Document Objects page 2 days http://bugs.python.org/issue2270 georg.brandl Segment Violation when using smtp with tls 1 days http://bugs.python.org/issue2272 facundobatista heapq API 2 days http://bugs.python.org/issue2274 rhettinger lambda *a, **k: a, k # does not work 0 days http://bugs.python.org/issue2283 draghuram [PATCH] -x64 option added to pcbuild/rt.bat to facilitate testin 0 days http://bugs.python.org/issue2284 loewis patch list.sort.__doc__ says "cmp" is a keyword, but it isn't. 0 days http://bugs.python.org/issue2285 georg.brandl struct.pack of floats in non-native endian order 1823 days http://bugs.python.org/issue705836 marketdickinson patch pyconfig.h duplicates common defines 1697 days http://bugs.python.org/issue771479 loewis exclude CVS conflict files in sdist command 1159 days http://bugs.python.org/issue1095784 georg.brandl patch slightly easier way to debug from the exception handler 1143 days http://bugs.python.org/issue1106316 facundobatista long -> Py_ssize_t (C/API 1.2.1) 588 days http://bugs.python.org/issue1533486 georg.brandl thread + import => crashes? 18 days http://bugs.python.org/issue1720705 georg.brandl slice type is unhashable 275 days http://bugs.python.org/issue1733184 rhettinger string formatter %x problem with indirectly given long 260 days http://bugs.python.org/issue1741218 facundobatista Top Issues Most Discussed (10) ______________________________ 8 To document "assertTrue" in unittest 2 days closed http://bugs.python.org/issue2249 7 slice type is unhashable 275 days closed http://bugs.python.org/issue1733184 6 Helping the compiler avoid memory references in PyEval_EvalFram 5 days open http://bugs.python.org/issue2262 6 Instance methods compare equal when their self's are equal 454 days open http://bugs.python.org/issue1617161 5 Poor support other than 44.1khz, 16bit audio files? 5 days open http://bugs.python.org/issue2259 5 setitimer, getitimer wrapper 9 days open http://bugs.python.org/issue2240 4 datetime.datetime operator methods are not subclass-friendly 0 days closed http://bugs.python.org/issue2267 4 Change Mandrake by Mandriva for platform.dist() 0 days closed http://bugs.python.org/issue2255 4 Enhanced hotshot profiler with high-resolution timer 12 days open http://bugs.python.org/issue2218 4 map and filter shouldn't support None as first argument (in Py3 17 days closed http://bugs.python.org/issue2186 From g.brandl at gmx.net Sat Mar 15 01:24:21 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 15 Mar 2008 01:24:21 +0100 Subject: [Python-Dev] Link in license broken Message-ID: While fixing the broken links in the docs, I saw that the link to http://www.pythonlabs.com/logos.html in the "BEOPEN PYTHON OPEN SOURCE LICENSE AGREEMENT VERSION 1" is broken. What to do about that? Georg From eric+python-dev at trueblade.com Sat Mar 15 02:14:30 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Fri, 14 Mar 2008 20:14:30 -0500 Subject: [Python-Dev] range in future_builtins? Message-ID: <47DB22F6.4080707@trueblade.com> In the keynote, Guido mentioned switching from range to xrange in 2.6 code, as a migration strategy. Another option would be to add range to future_builtins, and have it call xrange. Would that be desirable? From guido at python.org Sat Mar 15 05:07:22 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 14 Mar 2008 23:07:22 -0500 Subject: [Python-Dev] Link in license broken In-Reply-To: References: Message-ID: On Fri, Mar 14, 2008 at 7:24 PM, Georg Brandl wrote: > While fixing the broken links in the docs, I saw that the link to > http://www.pythonlabs.com/logos.html in the "BEOPEN PYTHON OPEN SOURCE > LICENSE AGREEMENT VERSION 1" is broken. > > What to do about that? Too bad. BeOpen doesn't exist any more. We can't very well retroactively change their license; they were stupid to put a web link in a license (and if only that was their only stupidity :-). I think having it be a 404 forever is probably the right solution. I have no idea which logos were once there, ane neither does the wayback machine at archive.org... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Mar 15 05:10:01 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 14 Mar 2008 23:10:01 -0500 Subject: [Python-Dev] range in future_builtins? In-Reply-To: <47DB22F6.4080707@trueblade.com> References: <47DB22F6.4080707@trueblade.com> Message-ID: Sure. The 3.0 range() isn't exactly the same as the 2.6 xrange(), so it would have to be a proper backport (sorry, I don't recall the exact difference, but I remember it's been redone, perhaps to support long integers). It seems pretty minor though. The advantage of using xrange() is that you remain backwards compatible all the way to 2.0 and probably even 1.5.2... On Fri, Mar 14, 2008 at 8:14 PM, Eric Smith wrote: > In the keynote, Guido mentioned switching from range to xrange in 2.6 > code, as a migration strategy. Another option would be to add range to > future_builtins, and have it call xrange. Would that be desirable? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ebenze at hotmail.com Sat Mar 15 05:57:51 2008 From: ebenze at hotmail.com (Eric B.) Date: Sat, 15 Mar 2008 00:57:51 -0400 Subject: [Python-Dev] Problems building python2.4 SRPM on RHEL4 x64 Message-ID: Hi, I appologize if this is not the right place to post this, but searching through the old archives, I ran across the same issue from 3 years ago, but I cannot find the resolution to it. Currently, I am trying to build the python2.4 SRPM from Python.org on a CentOS4.6_x64 platform, but the build is failing with a very non-descript error message. .... + cd /var/tmp/python2.4-2.4-root/usr/bin + mv -f pydoc pydoc2.4 + cd /var/tmp/python2.4-2.4-root/usr/bin + mv -f idle idle2.4 + echo '#!/bin/bash' + echo 'exec /usr/bin/python2.4 /usr/lib64/python2.4/idlelib/idle.py' + chmod 755 /var/tmp/python2.4-2.4-root/usr/bin/idle2.4 + cp -a Tools /var/tmp/python2.4-2.4-root/usr/lib64/python2.4 + rm -f mainpkg.files + find /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/lib-dynload -type f + sed 's|^/var/tmp/python2.4-2.4-root|/|' + grep -v -e '_tkinter.so$' error: Bad exit status from /var/tmp/rpm-tmp.55639 (%install) RPM build errors: user jafo does not exist - using root group jafo does not exist - using root user jafo does not exist - using root group jafo does not exist - using root Bad exit status from /var/tmp/rpm-tmp.55639 (%install) The old post I found about this can be found here: http://mail.python.org/pipermail/python-bugs-list/2005-October/030670.html I checked the var/tmp/python2.4-2.4-root/usr/lib64 directories and see plenty of files in there, but not sure what I should be looking for. Can anyone point me in the right direction? Am not sure what to be looking for in order to get this to work. Thanks! Eric From martin at v.loewis.de Sat Mar 15 10:49:21 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMwpp3aXMi?=) Date: Sat, 15 Mar 2008 04:49:21 -0500 Subject: [Python-Dev] Problems building python2.4 SRPM on RHEL4 x64 In-Reply-To: References: Message-ID: <47DB9BA1.5010702@v.loewis.de> Eric B. schrieb: > Hi, > > I appologize if this is not the right place to post this, but searching > through the old archives, I ran across the same issue from 3 years ago, but > I cannot find the resolution to it. > > Currently, I am trying to build the python2.4 SRPM from Python.org on a > CentOS4.6_x64 platform, but the build is failing with a very non-descript > error message. > > .... > + cd /var/tmp/python2.4-2.4-root/usr/bin > + mv -f pydoc pydoc2.4 > + cd /var/tmp/python2.4-2.4-root/usr/bin > + mv -f idle idle2.4 > + echo '#!/bin/bash' > + echo 'exec /usr/bin/python2.4 /usr/lib64/python2.4/idlelib/idle.py' > + chmod 755 /var/tmp/python2.4-2.4-root/usr/bin/idle2.4 > + cp -a Tools /var/tmp/python2.4-2.4-root/usr/lib64/python2.4 > + rm -f mainpkg.files > + find /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/lib-dynload -type f > + sed 's|^/var/tmp/python2.4-2.4-root|/|' > + grep -v -e '_tkinter.so$' > error: Bad exit status from /var/tmp/rpm-tmp.55639 (%install) The last command executed imediately before the error output seems to be find "$RPM_BUILD_ROOT""%{__prefix}"/%{libdirname}/python%{libvers}/lib-dynload -type f | sed "s|^${RPM_BUILD_ROOT}|/|" | grep -v -e '_tkinter.so$' >mainpkg.files That is not the last command in the %install script (atleast not according to the spec file). So it is not at all clear why the shell should stop executing at that point, and spit out that error message. The only theory I can come up with is that the shell *crashed*. Can you get hold of the rpm-tmp file (e.g. by asking RPM not to delete it)? Then run it independently, perhaps under strace. If it's indeed the case that the shell crashes, something is seriously wrong with your operating system. Regards, Martin From jimis at gmx.net Sat Mar 15 11:50:52 2008 From: jimis at gmx.net (Dimitrios Apostolou) Date: Sat, 15 Mar 2008 12:50:52 +0200 Subject: [Python-Dev] Complexity documentation request In-Reply-To: References: <20080309142239.GA18295@panix.com> <47D8348E.8020507@gmx.net> <47D8E2DA.4020709@gmx.net> Message-ID: <47DBAA0C.7050108@gmx.net> Hi, I just dug into the source code looking for complexity of set operations. In the wiki page I documented an interesting finding, that it is different to do s-t and s.difference(t). It is also interesting that you can do the first only for sets, but the second for every iterable in t. Are these portable characteristics of the python language or just implementation specific details? In addition, can someone explain me the usefulness of the loop starting with 'if (PyDict_CheckExact(other))' in set_difference()? As I understand it set_difference() is always called with two sets as arguments (set_sub() does the actual call). I'm just trying to figure out the complexity of the other set operations, but things get more complicated. I'd appreciate your help. Thanks, Dimitris From jimis at gmx.net Sat Mar 15 12:25:53 2008 From: jimis at gmx.net (Dimitrios Apostolou) Date: Sat, 15 Mar 2008 13:25:53 +0200 Subject: [Python-Dev] Complexity documentation request In-Reply-To: <47DBAA0C.7050108@gmx.net> References: <20080309142239.GA18295@panix.com> <47D8348E.8020507@gmx.net> <47D8E2DA.4020709@gmx.net> <47DBAA0C.7050108@gmx.net> Message-ID: <47DBB241.6070107@gmx.net> Correcting myself: Dimitrios Apostolou wrote: > Hi, > > I just dug into the source code looking for complexity of set > operations. In the wiki page I documented an interesting finding, that > it is different to do s-t and s.difference(t). It is also interesting it is different to do s-t than s.difference_update(t), as fas as complexity is involved. The first one is O(len(s)) while the second is O(len(t)) (I *think so, I may have missed lots of things in the source code). > that you can do the first only for sets, but the second for every > iterable in t. > > Are these portable characteristics of the python language or just > implementation specific details? In addition, can someone explain me the I just found it documented in the library reference, that s.method() can accept any iterable while s-t can't. So I guess it is a language characteristic. > usefulness of the loop starting with 'if (PyDict_CheckExact(other))' in > set_difference()? As I understand it set_difference() is always called > with two sets as arguments (set_sub() does the actual call). > > I'm just trying to figure out the complexity of the other set > operations, but things get more complicated. I'd appreciate your help. > > > Thanks, > Dimitris P.S. Who is the wiki admin? I'm desperately trying to improve the looks of tables (Add border, remove the

element from every cell) but I can't. I think that the page stylesheet needs to be modified, for starters... From ebenze at hotmail.com Sat Mar 15 13:59:07 2008 From: ebenze at hotmail.com (Eric B.) Date: Sat, 15 Mar 2008 08:59:07 -0400 Subject: [Python-Dev] Problems building python2.4 SRPM on RHEL4 x64 References: <47DB9BA1.5010702@v.loewis.de> Message-ID: ""Martin v. L?wis"" wrote in message news:47DB9BA1.5010702 at v.loewis.de... > Eric B. schrieb: >> Hi, >> >> I appologize if this is not the right place to post this, but searching >> through the old archives, I ran across the same issue from 3 years ago, >> but >> I cannot find the resolution to it. >> >> Currently, I am trying to build the python2.4 SRPM from Python.org on a >> CentOS4.6_x64 platform, but the build is failing with a very non-descript >> error message. >> >> .... >> + cd /var/tmp/python2.4-2.4-root/usr/bin >> + mv -f pydoc pydoc2.4 >> + cd /var/tmp/python2.4-2.4-root/usr/bin >> + mv -f idle idle2.4 >> + echo '#!/bin/bash' >> + echo 'exec /usr/bin/python2.4 /usr/lib64/python2.4/idlelib/idle.py' >> + chmod 755 /var/tmp/python2.4-2.4-root/usr/bin/idle2.4 >> + cp -a Tools /var/tmp/python2.4-2.4-root/usr/lib64/python2.4 >> + rm -f mainpkg.files >> + find /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/lib-dynload -type >> f >> + sed 's|^/var/tmp/python2.4-2.4-root|/|' >> + grep -v -e '_tkinter.so$' >> error: Bad exit status from /var/tmp/rpm-tmp.55639 (%install) > > The last command executed imediately before the error output > seems to be > > find > "$RPM_BUILD_ROOT""%{__prefix}"/%{libdirname}/python%{libvers}/lib-dynload > -type f | > sed "s|^${RPM_BUILD_ROOT}|/|" | > grep -v -e '_tkinter.so$' >mainpkg.files > > That is not the last command in the %install script (atleast not > according to the spec file). So it is not at all clear why the > shell should stop executing at that point, and spit out > that error message. > > The only theory I can come up with is that the shell *crashed*. > Can you get hold of the rpm-tmp file (e.g. by asking RPM not > to delete it)? Then run it independently, perhaps under > strace. Forgive the "newbie-ness" to this question, but I'm not quite sure what you mean by the rpm-tmp file; I'm assuming you mean the rpm-temp.45231 shell script that is left in /var/tmp. I tried running that myself from the cmd line using # bash -x rpm-tmp.45231 and it runs properly to completion. If I try running just: # rpmbuild -bi --short-circuit /usr/src/redhat/SPECS/python-2.4.spec it (not surprisingly) exists with the same error message. If I try to run # strace rpmbuild -bi --short-circuit /usr/src/redhat/SPECS/python-2.4.spec I end up with a whole bunch of output I don't understand: ..... + find /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/lib-dynload -type f + sed 's|^/var/tmp/python2.4-2.4-root|/|' + grep -v -e '_tkinter.so$' [{WIFEXITED(s) && WEXITSTATUS(s) == 1}], 0, NULL) = 14779 --- SIGCHLD (Child exited) @ 0 (0) --- write(2, "error: ", 7error: ) = 7 write(2, "Bad exit status from /var/tmp/rp"..., 53Bad exit status from /var/tmp/rpm-tmp.156 (%install) ) = 53 write(1, "\n\nRPM build errors:\n", 20 RPM build errors: ) = 20 write(2, " Bad exit status from /var/tm"..., 57 Bad exit status from /var/tmp/rpm-tmp.156 (%install) ) = 57 open("/usr/lib/rpm/rpmrc", O_RDONLY) = 3 fcntl(3, F_SETFD, FD_CLOEXEC) = 0 fstat(3, {st_mode=S_IFREG|0644, st_size=11452, ...}) = 0 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2a983ac000 poll([{fd=3, events=POLLIN, revents=POLLIN}], 1, 1000) = 1 read(3, "#/*! \\page config_rpmrc Default "..., 8192) = 8192 poll([{fd=3, events=POLLIN, revents=POLLIN}], 1, 1000) = 1 read(3, "t: armv4l: armv3l\narch_compat: a"..., 8192) = 3260 poll([{fd=3, events=POLLIN, revents=POLLIN}], 1, 1000) = 1 ..... > If it's indeed the case that the shell crashes, something > is seriously wrong with your operating system. I would be surprised if it was something wrong with the OS at is a brand spanking new install. In fact, I installed it specifically in order to build python2.4 on RHEL4 x64 so I can then install the rpm pkg on my production x64 server (I only managed to find precompiled i386 binaries for RHEL4). Is it possible I'm missing some libraries somewhere? I don't know if there is any way I can complete the rpm build manually? I've looked at the SPEC file, but don't see anything particularly special in there, nor am I sure how I can modify this rpm-tmp script that it runs to skip that line and see if it can continue without it. (Am not very well versed with building srpms). Any ideas what I can do/try next? Thanks, Eric From facundobatista at gmail.com Sat Mar 15 16:30:26 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Sat, 15 Mar 2008 10:30:26 -0500 Subject: [Python-Dev] No __bases__ in dir() Message-ID: Hi! My head crashed into this: >>> class C(object): ...: pass ...: >>> >>> dir(C) ['__class__', ...] >>> C.__bases__ (,) Why __bases__ does not appear in dir()? Is there a good reason for this or should I file a bug? Thanks! -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From lists at cheimes.de Sat Mar 15 17:09:32 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 15 Mar 2008 17:09:32 +0100 Subject: [Python-Dev] No __bases__ in dir() In-Reply-To: References: Message-ID: > Why __bases__ does not appear in dir()? > > Is there a good reason for this or should I file a bug? __bases__ and several other methods like mro and __subclasses__ are defined on the meta class. dir() doesn't list the attributes of the meta class of a class. >>> class C(object): ... pass ... >>> dir(type(C)) ['__base__', '__bases__', '__basicsize__', '__call__', '__class__', '__cmp__', '__delattr__', '__dict__', '__dictoffset__', '__doc__', '__flags__', '__getattribute__', '__hash__', '__init__', '__itemsize__', '__module__', '__mro__', '__name__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', '__subclasses__', '__weakrefoffset__', 'mro'] Christian From guido at python.org Sat Mar 15 17:20:59 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 15 Mar 2008 11:20:59 -0500 Subject: [Python-Dev] No __bases__ in dir() In-Reply-To: References: Message-ID: This is because dir() special-cases classes, isn't it? On Sat, Mar 15, 2008 at 11:09 AM, Christian Heimes wrote: > > Why __bases__ does not appear in dir()? > > > > Is there a good reason for this or should I file a bug? > > __bases__ and several other methods like mro and __subclasses__ are > defined on the meta class. dir() doesn't list the attributes of the meta > class of a class. > > > >>> class C(object): > ... pass > ... > >>> dir(type(C)) > ['__base__', '__bases__', '__basicsize__', '__call__', '__class__', > '__cmp__', '__delattr__', '__dict__', '__dictoffset__', '__doc__', > '__flags__', '__getattribute__', '__hash__', '__init__', '__itemsize__', > '__module__', '__mro__', '__name__', '__new__', '__reduce__', > '__reduce_ex__', '__repr__', '__setattr__', '__str__', '__subclasses__', > '__weakrefoffset__', 'mro'] > > Christian > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ebenze at hotmail.com Sat Mar 15 17:23:07 2008 From: ebenze at hotmail.com (Eric B.) Date: Sat, 15 Mar 2008 12:23:07 -0400 Subject: [Python-Dev] Problems building python2.4 SRPM on RHEL4 x64 References: <47DB9BA1.5010702@v.loewis.de> Message-ID: >>> I appologize if this is not the right place to post this, but searching >>> through the old archives, I ran across the same issue from 3 years ago, >>> but >>> I cannot find the resolution to it. >>> >>> Currently, I am trying to build the python2.4 SRPM from Python.org on a >>> CentOS4.6_x64 platform, but the build is failing with a very >>> non-descript >>> error message. >>> >>> .... >>> + cd /var/tmp/python2.4-2.4-root/usr/bin >>> + mv -f pydoc pydoc2.4 >>> + cd /var/tmp/python2.4-2.4-root/usr/bin >>> + mv -f idle idle2.4 >>> + echo '#!/bin/bash' >>> + echo 'exec /usr/bin/python2.4 /usr/lib64/python2.4/idlelib/idle.py' >>> + chmod 755 /var/tmp/python2.4-2.4-root/usr/bin/idle2.4 >>> + cp -a Tools /var/tmp/python2.4-2.4-root/usr/lib64/python2.4 >>> + rm -f mainpkg.files >>> + find /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/lib-dynload -type >>> f >>> + sed 's|^/var/tmp/python2.4-2.4-root|/|' >>> + grep -v -e '_tkinter.so$' >>> error: Bad exit status from /var/tmp/rpm-tmp.55639 (%install) >> >> The last command executed imediately before the error output >> seems to be >> >> find >> "$RPM_BUILD_ROOT""%{__prefix}"/%{libdirname}/python%{libvers}/lib-dynload >> -type f | >> sed "s|^${RPM_BUILD_ROOT}|/|" | >> grep -v -e '_tkinter.so$' >mainpkg.files >> >> That is not the last command in the %install script (atleast not >> according to the spec file). So it is not at all clear why the >> shell should stop executing at that point, and spit out >> that error message. I've done a little more debugging to try to determine where the problem lies and ran across some interesting things. 1) I first edited the SPECS file and commented out the find "$RPM_BUILD_ROOT""%{__prefix}"/%{libdirname}/python%{libvers}/lib-dynload -type f | sed "s|^${RPM_BUILD_ROOT}|/|" | grep -v -e '_tkinter.so$' >mainpkg.files line. If I remove this line, the rpmbuild no longer crashes, but does still fail, citing missing files in the lib64 directories. + rm -f /tmp/python-rpm-files.4859 + /usr/lib/rpm/brp-compress + /usr/lib/rpm/brp-strip + /usr/lib/rpm/brp-strip-static-archive + /usr/lib/rpm/brp-strip-comment-note Processing files: python2.4-2.4-1pydotorg error: File not found by glob: /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/*.txt error: File not found by glob: /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/*.py* error: File not found: /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/pdb.doc error: File not found: /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/profile.doc error: File not found: /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/curses error: File not found: /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/distutils error: File not found: /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/encodings error: File not found: /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/plat-linux2 error: File not found: /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/site-packages error: File not found: /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/test error: File not found: /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/xml error: File not found: /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/email error: File not found: /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/compiler error: File not found: /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/bsddb error: File not found: /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/hotshot error: File not found: /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/logging error: File not found: /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/lib-old Executing(%doc): /bin/sh -e /var/tmp/rpm-tmp.95118 + umask 022 + cd /usr/src/redhat/BUILD + cd Python-2.4 + DOCDIR=/var/tmp/python2.4-2.4-root/usr/share/doc/python2.4-2.4 + export DOCDIR + rm -rf /var/tmp/python2.4-2.4-root/usr/share/doc/python2.4-2.4 + /bin/mkdir -p /var/tmp/python2.4-2.4-root/usr/share/doc/python2.4-2.4 + cp -pr Misc/README Misc/cheatsheet Misc/Porting /var/tmp/python2.4-2.4-root/usr/share/doc/python2.4-2.4 + cp -pr LICENSE Misc/ACKS Misc/HISTORY Misc/NEWS /var/tmp/python2.4-2.4-root/usr/share/doc/python2.4-2.4 + exit 0 Processing files: python2.4-devel-2.4-1pydotorg error: File not found: /var/tmp/python2.4-2.4-root/usr/lib64/python2.4/config Processing files: python2.4-tools-2.4-1pydotorg Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(VersionedDependencies) <= 3.0.3-1 Requires: /bin/bash /bin/sh /usr/bin/env 2) If I run the same find command in the /var/tmp/python2.4-2.4-root/usr/lib/python2.4/lib-dynload (instead of the lib64 directory), then I find a whole bunch of files, whereas the lib64/ directory returns no files. It seems as though the make script is either installing files in the wrong location, or the SPECS is expecting files in the lib64/ directory when instead they are in the lib/ directory. Anyone have any ideas? Thanks, Eric From ncoghlan at gmail.com Sat Mar 15 19:11:34 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 16 Mar 2008 04:11:34 +1000 Subject: [Python-Dev] No __bases__ in dir() In-Reply-To: References: Message-ID: <47DC1156.5040407@gmail.com> Guido van Rossum wrote: > This is because dir() special-cases classes, isn't it? Avoiding infinite recursion in dir(type) might be fun if that special case was removed without due care and attention... Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sat Mar 15 19:37:59 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 16 Mar 2008 04:37:59 +1000 Subject: [Python-Dev] [Python-checkins] r61403 - python/trunk/Misc/NEWS In-Reply-To: <20080315160711.CE8F31E4007@bag.python.org> References: <20080315160711.CE8F31E4007@bag.python.org> Message-ID: <47DC1787.9020707@gmail.com> skip.montanaro wrote: > Author: skip.montanaro > Date: Sat Mar 15 17:07:11 2008 > New Revision: 61403 > > Modified: > python/trunk/Misc/NEWS > Log: > . > > > Modified: python/trunk/Misc/NEWS > ============================================================================== > --- python/trunk/Misc/NEWS (original) > +++ python/trunk/Misc/NEWS Sat Mar 15 17:07:11 2008 > @@ -21,6 +21,10 @@ > Library > ------- > > +- Issue #1158: add %f format (fractions of a second represented as > + microseconds) to datetime objects. Understood by both strptime and > + strftime. %f makes me think femtoseconds :) Any particular reason we can't use '%u' to align with the convention of abbreviating microseconds as 'us' when a character encoding doesn't provide convenient access to the Greek letter mu? (e.g. ASCII) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From skip at pobox.com Sat Mar 15 19:49:09 2008 From: skip at pobox.com (skip at pobox.com) Date: Sat, 15 Mar 2008 13:49:09 -0500 Subject: [Python-Dev] [Python-checkins] r61403 - python/trunk/Misc/NEWS In-Reply-To: <47DC1787.9020707@gmail.com> References: <20080315160711.CE8F31E4007@bag.python.org> <47DC1787.9020707@gmail.com> Message-ID: <18396.6693.791396.284548@montanaro-dyndns-org.local> Nick> %f makes me think femtoseconds :) Not fraction? Nick> Any particular reason we can't use '%u' to align with the Nick> convention of abbreviating microseconds as 'us' when a character Nick> encoding doesn't provide convenient access to the Greek letter mu? Nick> (e.g. ASCII) Well, %u is already in use by at least some implementations of strftime. >From the Solaris 10 man page: %u Weekday as a decimal number [1,7], with 1 representing Monday. See NOTES below. I see the same on my Mac. I think it's better to use the same format code for both parsing and formatting if possible. Skip From guido at python.org Sat Mar 15 20:10:59 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 15 Mar 2008 14:10:59 -0500 Subject: [Python-Dev] No __bases__ in dir() In-Reply-To: <47DC1156.5040407@gmail.com> References: <47DC1156.5040407@gmail.com> Message-ID: On Sat, Mar 15, 2008 at 1:11 PM, Nick Coghlan wrote: > Guido van Rossum wrote: > > This is because dir() special-cases classes, isn't it? > > Avoiding infinite recursion in dir(type) might be fun if that special > case was removed without due care and attention... I wasn't suggeting removing the special-casing -- rather I was explaining the observed behavior. In Py3k, dir() will allow any class to makes its instances special cases by defining __dir__(). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Sat Mar 15 23:13:18 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 15 Mar 2008 18:13:18 -0400 Subject: [Python-Dev] No __bases__ in dir() References: <47DC1156.5040407@gmail.com> Message-ID: "Guido van Rossum" wrote in message news:ca471dc20803151210l30209673wb44556a3650c4a5 at mail.gmail.com... | In Py3k, dir() will allow any class to makes its instances special | cases by defining __dir__(). Nice. Then the current special case will become explainable as type.__dir__ excluding type's numerous attibutes, to avoid clutter, and we should have fewer discussions about what dir() 'ought' to return ;-) From nnorwitz at gmail.com Sat Mar 15 23:54:54 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sat, 15 Mar 2008 17:54:54 -0500 Subject: [Python-Dev] difference between diff string implementations Message-ID: This inconsistency goes back to 2.3 at least and probably to the initial unicode implementation. >>> set(dir(u'')) - set(dir('')) ['isnumeric', 'isdecimal'] UserString contains these two methods even though 8-bit strings do not. I'm not sure what we should do for 2.6 or 3.0. My preference would be to remove these methods on unicode/UserString if they aren't useful to a large audience. However, removing for 2.6 without a deprecation seems bad. Suggestions? n From greg at krypto.org Sun Mar 16 02:07:53 2008 From: greg at krypto.org (Gregory P. Smith) Date: Sat, 15 Mar 2008 20:07:53 -0500 Subject: [Python-Dev] difference between diff string implementations In-Reply-To: References: Message-ID: <52dc1c820803151807p60319325g849a911372a1700@mail.gmail.com> Shouldn't isnumeric and isdecimal apply to 8-bit strings as well? Are there localization issues with them that I'm blissfully unaware of? why not just add the methods there for consistency instead? -gps On 3/15/08, Neal Norwitz wrote: > > This inconsistency goes back to 2.3 at least and probably to the > initial unicode implementation. > > >>> set(dir(u'')) - set(dir('')) > ['isnumeric', 'isdecimal'] > > UserString contains these two methods even though 8-bit strings do > not. I'm not sure what we should do for 2.6 or 3.0. My preference > would be to remove these methods on unicode/UserString if they aren't > useful to a large audience. However, removing for 2.6 without a > deprecation seems bad. > > Suggestions? > > n > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/greg%40krypto.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080315/be610518/attachment.htm From guido at python.org Sun Mar 16 05:29:10 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 15 Mar 2008 23:29:10 -0500 Subject: [Python-Dev] difference between diff string implementations In-Reply-To: References: Message-ID: On Sat, Mar 15, 2008 at 5:54 PM, Neal Norwitz wrote: > This inconsistency goes back to 2.3 at least and probably to the > initial unicode implementation. > > >>> set(dir(u'')) - set(dir('')) > ['isnumeric', 'isdecimal'] > > UserString contains these two methods even though 8-bit strings do > not. I'm not sure what we should do for 2.6 or 3.0. My preference > would be to remove these methods on unicode/UserString if they aren't > useful to a large audience. However, removing for 2.6 without a > deprecation seems bad. > > Suggestions? It looks like they all denote different character classes though. I'd be inclined to keep the status quo in 2.6; the inconsistency will disappear in 3.0 (I don't think we need to add them to bytes). They should be documented though. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Sun Mar 16 09:25:05 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 16 Mar 2008 18:25:05 +1000 Subject: [Python-Dev] [Python-checkins] r61403 - python/trunk/Misc/NEWS In-Reply-To: <18396.6693.791396.284548@montanaro-dyndns-org.local> References: <20080315160711.CE8F31E4007@bag.python.org> <47DC1787.9020707@gmail.com> <18396.6693.791396.284548@montanaro-dyndns-org.local> Message-ID: <47DCD961.4050806@gmail.com> skip at pobox.com wrote: > Well, %u is already in use by at least some implementations of strftime. >>From the Solaris 10 man page: > > %u Weekday as a decimal number [1,7], with 1 > representing Monday. See NOTES below. > > I see the same on my Mac. > > I think it's better to use the same format code for both parsing and > formatting if possible. Yep - %u already being used for something else in some contexts is the kind of reason I was looking for. Given that, then %f for fractions of a second sounds fine. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Sun Mar 16 14:51:20 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 16 Mar 2008 08:51:20 -0500 Subject: [Python-Dev] 2.6 and 3.0 project management Message-ID: Python 3.0 and 2.6 are coming along really nice. I am optimistic that we can make the projected August date for the final releases of 2.6 and 3.0. As you may remember, Barry (the new release manager for both) suggested that we synchronize releases of 2.6 and 3.0. Not only could this potentially save the release manager and his assistants some time, doing the final releases together sends a clear signal to the community that both versions will receive equal support. However, looking at the calendar, I think we need to do a little more planning and management than we've typically done for Python releases. A final release in August means that we should plan to release a first beta in June and a second one in July. That means we have time for only two more alpha releases (April and May). I'm thinking of 1-2 release candidates 1-2 weeks ahead of the final release. Barry can make up a more detailed timetable. I'm fine with some slippage (especially if planned ahead) of individual releases due to availability of release personnel; but I don't want to be held up by features or bugs unless they are of absolutely dramatic show-stopping nature. In order to make such a tight release schedule we should try to come up with a list of tasks that need to be done, and prioritize them. This should include documentation, and supporting tools like 2to3. It should include features, backports of features, cleanup, bugs, and whatever else needs to be done (e.g. bugbot maintenance). In the past we've used shared spreadsheets in Google for this purpose, but seeing that these haven't been updated in ages, I'm skeptical that they are a sufficient tool. In my day job at Google we've started to do all task management for our project in the bug tracker (but that tracker has some features that make it particularly easy). Does anyone have a suggestion for an online open shared task management system that we cold adopt? Or should we bite the bullet and put everything in the bug tracker? Other suggestions? Anything's better than just email... PS. I realize that I've been rather absent from the day to day activities in the Python 3000 world lately. This is a temporary condition due to an important impending launch in my day job; I expect to have much more time for Python again starting in April. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From facundobatista at gmail.com Sun Mar 16 15:01:27 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Sun, 16 Mar 2008 09:01:27 -0500 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: References: Message-ID: 2008/3/16, Guido van Rossum : > they are a sufficient tool. In my day job at Google we've started to > do all task management for our project in the bug tracker (but that > tracker has some features that make it particularly easy). Does anyone Like which? Something that could be added to our tracker in a short time? -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From guido at python.org Sun Mar 16 15:18:27 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 16 Mar 2008 09:18:27 -0500 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: References: Message-ID: On Sun, Mar 16, 2008 at 9:01 AM, Facundo Batista wrote: > 2008/3/16, Guido van Rossum : > > they are a sufficient tool. In my day job at Google we've started to > > do all task management for our project in the bug tracker (but that > > tracker has some features that make it particularly easy). Does anyone > > Like which? Something that could be added to our tracker in a short time? It has a much more detailed set of categories, organized as a tree. Our project alone probably has 20-30 different bug categories. New bugs in those categories are automatically CC'ed to our group's mailing list (which isn't the same as auto-assignment). There are also more "bug states" you can use to track progress of a bug through the system: unassigned, assigned, accepted (meaning the assignee is actually working on it). (There are also a whole bunch that I don't find so useful, and severam that roundup already supports.) But perhaps the best feature is "hot lists" -- arbitrary, ordered, groupings of selected bugs. Each bug can be assigned to as many hot lists as you want. Seeing the list of all bugs in a particular hot list is one click away. We use this for overlaying project management categories and priorities, such as "code", "documentation", "configuration" as well as "next internal release", "must have", "post launch" etc. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at pobox.com Sun Mar 16 15:31:25 2008 From: skip at pobox.com (skip at pobox.com) Date: Sun, 16 Mar 2008 09:31:25 -0500 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: References: Message-ID: <18397.12093.765076.737023@montanaro-dyndns-org.local> Guido> It has a much more detailed set of categories, organized as a Guido> tree. Our project alone probably has 20-30 different bug Guido> categories. New bugs in those categories are automatically CC'ed Guido> to our group's mailing list (which isn't the same as Guido> auto-assignment). Adding categories should be easy. Organized in trees? Not so sure. Guido> There are also more "bug states" you can use to track progress of Guido> a bug through the system: unassigned, assigned, accepted (meaning Guido> the assignee is actually working on it). (There are also a whole Guido> bunch that I don't find so useful, and severam that roundup Guido> already supports.) Again, I think this should be easy. Guido> But perhaps the best feature is "hot lists" -- arbitrary, Guido> ordered, groupings of selected bugs. Each bug can be assigned to Guido> as many hot lists as you want. Seeing the list of all bugs in a Guido> particular hot list is one click away. We use this for overlaying Guido> project management categories and priorities, such as "code", Guido> "documentation", "configuration" as well as "next internal Guido> release", "must have", "post launch" etc. A hot list sounds like a saved search, which Roundup already supports. It also supports making these saved searches public. I suspect you could define one or more saved public searches which correspond to desired hot lists. Aside: Today's my last day here. I'd like to say hi sometime today. Free for lunch? Maybe this would be a good lunchtime discussion. Skip From lists at cheimes.de Sun Mar 16 15:42:21 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 16 Mar 2008 15:42:21 +0100 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: References: Message-ID: <47DD31CD.9030008@cheimes.de> Guido van Rossum wrote: > In order to make such a tight release schedule we should try to come > up with a list of tasks that need to be done, and prioritize them. > This should include documentation, and supporting tools like 2to3. It > should include features, backports of features, cleanup, bugs, and > whatever else needs to be done (e.g. bugbot maintenance). I did a quick brainstorming with me, myself and I. I came up with a list of (IMHO) important tasks. * Stabilize the C API of Python 3.0. I like to rename several prefixes to reduce the confusing: PyBytes -> PyByteArray, PyString -> PyBytes ... * Backport the new buffer protocol to 2.6. I spoke to Travis yesterday and he said he is trying to get it done during the PyCon sprint. Maybe somebody can assist him? When the new buffer protocol is available in 2.6 we can start back porting bytesarray and the new IO framework. * Replace Windows API calls with wide versions to support unicode for file names, environment etc. * Get the stdlib reorg done and add the fixers to 2to3 * Speed up 2to3. I suggested a GSoC task to improve and speed up 2to3 but it may be too late when we plan to ship out 3.0 in August. Christian From gregor.lingl at aon.at Sun Mar 16 16:13:03 2008 From: gregor.lingl at aon.at (Gregor Lingl) Date: Sun, 16 Mar 2008 16:13:03 +0100 Subject: [Python-Dev] [Python-3000] 2.6 and 3.0 project management In-Reply-To: References: Message-ID: <47DD38FF.9010004@aon.at> Hi everyone, with this posting I refer to a paragraph in PEP 361, which says: """Each non-trivial feature listed here that is not a PEP must be discussed on python-dev. Other enhancements include: - ... - turtle.py replacement or enhancements """ Some time ago I had offered my xturtle.py as a replacement of or supplement to turtle.py. The discussion that followed is here: http://www.python.org/dev/summary/2006-06-16_2006-06-30/ At Europython 2006 I gave a talk on xturtle and there and since then I had quite positive feedback including encouragement to offer it again to include it in the python standard distribution by quite a few people including Guido van Rossum. During the last few weeks I did some enhancements to xturtle and put the current version on the xturtle website for download in order to get same feedback about the API as well as possible bug reports. This version still needs some code polishing. http://www.rg16.at/~python/xturtle/download.html So I'm interested to know if this is still an issue for you. If so there should be initiated some procedure to decide that. If this decision were negative, things were done (- and I'd continue to develop xturtle elsewhere.) If the decision were positive, I'd be able to prepare two equivalent versions for Python2.6 and Python3000 within two or three weeks. (The port to Python3000 is nearly ready.) These could include say 85% of the documentation, albeit still not in the correct format. I think these had to be examined my some reviewer(s) and also a discussion about features to include or not include would be useful. I'd like to intensivly take part in this discussion and development. After a possible decision to include xturtle into Python, which certainly should take place before the first beta release, there would be enough time to polish the documentation and to fix bugs. For their discovery it would certainly be an advantage to put it in some prerelease as early as possible. Of course I know that xturtle is only a side issue in the current development efforts. Unfortunately I'm not familiar with the procedures needed to get a new module into Python, so I kindly ask you for your advice how to proceed, at the same time offering my cooperation. With best regards Gregor Lingl Guido van Rossum schrieb: > Python 3.0 and 2.6 are coming along really nice. I am optimistic that > we can make the projected August date for the final releases of 2.6 > and 3.0. As you may remember, Barry (the new release manager for both) > suggested that we synchronize releases of 2.6 and 3.0. Not only could > this potentially save the release manager and his assistants some > time, doing the final releases together sends a clear signal to the > community that both versions will receive equal support. > > However, looking at the calendar, I think we need to do a little more > planning and management than we've typically done for Python releases. > A final release in August means that we should plan to release a first > beta in June and a second one in July. That means we have time for > only two more alpha releases (April and May). I'm thinking of 1-2 > release candidates 1-2 weeks ahead of the final release. Barry can > make up a more detailed timetable. I'm fine with some slippage > (especially if planned ahead) of individual releases due to > availability of release personnel; but I don't want to be held up by > features or bugs unless they are of absolutely dramatic show-stopping > nature. > > In order to make such a tight release schedule we should try to come > up with a list of tasks that need to be done, and prioritize them. > This should include documentation, and supporting tools like 2to3. It > should include features, backports of features, cleanup, bugs, and > whatever else needs to be done (e.g. bugbot maintenance). > > In the past we've used shared spreadsheets in Google for this purpose, > but seeing that these haven't been updated in ages, I'm skeptical that > they are a sufficient tool. In my day job at Google we've started to > do all task management for our project in the bug tracker (but that > tracker has some features that make it particularly easy). Does anyone > have a suggestion for an online open shared task management system > that we cold adopt? Or should we bite the bullet and put everything in > the bug tracker? Other suggestions? Anything's better than just > email... > > PS. I realize that I've been rather absent from the day to day > activities in the Python 3000 world lately. This is a temporary > condition due to an important impending launch in my day job; I expect > to have much more time for Python again starting in April. > > From guido at python.org Sun Mar 16 16:23:56 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 16 Mar 2008 10:23:56 -0500 Subject: [Python-Dev] 2.6 and 3.0 tasks Message-ID: Moving this to a new subject to keep the discussion of tasks and the discussion of task tracking tools separate. On Sun, Mar 16, 2008 at 9:42 AM, Christian Heimes wrote: > I did a quick brainstorming with me, myself and I. I came up with a list > of (IMHO) important tasks. > > * Stabilize the C API of Python 3.0. I like to rename several prefixes > to reduce the confusing: PyBytes -> PyByteArray, +1 (also +1 to backporting this to 2.6) > PyString -> PyBytes ... -1. This will make merging code from 2.6 harder, and causes more work for porting C extensions. > * Backport the new buffer protocol to 2.6. I spoke to Travis yesterday > and he said he is trying to get it done during the PyCon sprint. Maybe > somebody can assist him? Does he need assistance? > When the new buffer protocol is available in 2.6 we can start back > porting bytesarray and the new IO framework. Are these really so closely tied that they have to wait before they can be backported? > * Replace Windows API calls with wide versions to support unicode for > file names, environment etc. +1. This should be broken into separate tasks for each API. > * Get the stdlib reorg done +1. But again, I think this is many small tasks. > and add the fixers to 2to3 +1. I think quite a few changes have not had a fixer added. Again, I think we should maintain a specific list of needed fixers; fixers can easily be developed independently. > * Speed up 2to3. I suggested a GSoC task to improve and speed up 2to3 > but it may be too late when we plan to ship out 3.0 in August. While I know that some people are expecting to use a development model that invokes 2to3 very frequently, I think this is at best a nice-to-have. (I also don't see how it could be done, but maybe I'm blind for the obvious, as the original author.) I have some other tasks to add: - getargs.c (Georg posted about this; his post actually inspired my post.) - Tweak structseq to be more like namedtuple. Maybe they could have a common ABC. I also think we should get rid of the concept of "hidden" fields (that are accessible by name but not through the tuple API). - Devise a way to handle str instances pickled in 2.6 and unpickled in 3.0, and also bytes instances pickled in 3.0 and unpickled in 2.6. - Someone should go over https://spreadsheets.google.com/ccc?key=pCKY4oaXnT81FrGo3ShGHGg and extract more tasks. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tnelson at onresolve.com Sun Mar 16 16:27:05 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Sun, 16 Mar 2008 08:27:05 -0700 Subject: [Python-Dev] 3.0 buildbots all red Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD173@EXMBX04.exchhosting.com> http://www.python.org/dev/buildbot/3.0/ New sprint idea: getting all (inc. trunk) the buildbots green by Thursday. Anyone interested? Trent. From guido at python.org Sun Mar 16 16:32:17 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 16 Mar 2008 10:32:17 -0500 Subject: [Python-Dev] xturtle and 3.0 Message-ID: I'm changing the subject to keep this separate from the project management tools discussion. On Sun, Mar 16, 2008 at 10:13 AM, Gregor Lingl wrote: > > Hi everyone, > > with this posting I refer to a paragraph in PEP 361, which says: > > """Each non-trivial feature listed here that is not a PEP must be discussed on python-dev. Other enhancements include: > > - ... > - turtle.py replacement or enhancements > """ > > Some time ago I had offered my xturtle.py as a replacement of or > supplement to turtle.py. The discussion that followed is here: > > http://www.python.org/dev/summary/2006-06-16_2006-06-30/ > > At Europython 2006 I gave a talk on xturtle and there and since then I > had quite positive feedback including encouragement to offer it again to > include it in the python standard distribution by quite a few people > including Guido van Rossum. > > During the last few weeks I did some enhancements to xturtle and put the > current version on the xturtle website for download in order to get same > feedback about the API as well as possible bug reports. This version > still needs some code polishing. > > http://www.rg16.at/~python/xturtle/download.html > > So I'm interested to know if this is still an issue for you. If so there > should be initiated some procedure to decide that. I think that for 3.0, replacing turtle with xturtle is great, *IF* xturtle doesn't add any additional dependencies *AND* it works well on Mac OSX, Linux and Windows. > If this decision were negative, things were done (- and I'd continue to > develop xturtle elsewhere.) > > If the decision were positive, I'd be able to prepare two equivalent > versions for Python2.6 and Python3000 within two or three weeks. (The > port to Python3000 is nearly ready.) These could include say 85% of the > documentation, albeit still not in the correct format. That sounds cool. In 2.6 I'm reluctant to replace the existing turtle module; xturtle can be added as xturtle. > I think these had to be examined my some reviewer(s) and also a > discussion about features to include or not include would be useful. I'd > like to intensivly take part in this discussion and development. > > After a possible decision to include xturtle into Python, which > certainly should take place before the first beta release, there would > be enough time to polish the documentation and to fix bugs. For their > discovery it would certainly be an advantage to put it in some > prerelease as early as possible. > > Of course I know that xturtle is only a side issue in the current > development efforts. Unfortunately I'm not familiar with the procedures > needed to get a new module into Python, so I kindly ask you for your > advice how to proceed, at the same time offering my cooperation. I think that for a library module like this, an email like you've sent is just fine. Maybe Brett has a suggestion on whether it would remain a toplevel module or could be placed in some umbrella package (is Tkinter being moved around?). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tnelson at onresolve.com Sun Mar 16 16:33:49 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Sun, 16 Mar 2008 08:33:49 -0700 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: References: Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD176@EXMBX04.exchhosting.com> > > * Replace Windows API calls with wide versions to support unicode > > for file names, environment etc. > > +1. This should be broken into separate tasks for each API. What are we referring to here? Calling the W versions explicitly and using wchar_t for everything, or using the TCHAR/TEXT() approach and keeping the API calls the same, letting the #define UNICODE do the work behind the scenes? Trent. From martin at v.loewis.de Sun Mar 16 16:48:47 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMwpp3aXMi?=) Date: Sun, 16 Mar 2008 10:48:47 -0500 Subject: [Python-Dev] 3.0 buildbots all red In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD173@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD173@EXMBX04.exchhosting.com> Message-ID: <47DD415F.7090109@v.loewis.de> > New sprint idea: getting all (inc. trunk) the buildbots green by Thursday. Anyone interested? I think the chance to achieve that is close to zero. Regards, Martin From tnelson at onresolve.com Sun Mar 16 16:55:35 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Sun, 16 Mar 2008 08:55:35 -0700 Subject: [Python-Dev] 3.0 buildbots all red In-Reply-To: <47DD415F.7090109@v.loewis.de> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD173@EXMBX04.exchhosting.com> <47DD415F.7090109@v.loewis.de> Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD17D@EXMBX04.exchhosting.com> > > New sprint idea: getting all (inc. trunk) the buildbots green by > Thursday. Anyone interested? > > I think the chance to achieve that is close to zero. Sounds like a challenge if ever I've heard one -- care to wager a beer on it? (Only applies to buildbots that are connected/online.) (FWIW, I've got the x64 Windows build green on my dev server, tcl/tk and bsddb required patching, so did some tests, and so did some C code -- I'm in the process of filtering the efforts back into the tracker.) Trent. From g.brandl at gmx.net Sun Mar 16 17:19:40 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 16 Mar 2008 17:19:40 +0100 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: References: Message-ID: Guido van Rossum schrieb: > But perhaps the best feature is "hot lists" -- arbitrary, ordered, > groupings of selected bugs. Each bug can be assigned to as many hot > lists as you want. Seeing the list of all bugs in a particular hot > list is one click away. We use this for overlaying project management > categories and priorities, such as "code", "documentation", > "configuration" as well as "next internal release", "must have", "post > launch" etc. Doesn't this match Roundup's keywords? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From musiccomposition at gmail.com Sun Mar 16 17:19:37 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sun, 16 Mar 2008 11:19:37 -0500 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: References: Message-ID: <1afaf6160803160919qbf49800n584ea4f965ebc6b2@mail.gmail.com> On Sun, Mar 16, 2008 at 8:51 AM, Guido van Rossum wrote: > Python 3.0 and 2.6 are coming along really nice. I am optimistic that > we can make the projected August date for the final releases of 2.6 > and 3.0. As you may remember, Barry (the new release manager for both) > suggested that we synchronize releases of 2.6 and 3.0. Not only could > this potentially save the release manager and his assistants some > time, doing the final releases together sends a clear signal to the > community that both versions will receive equal support. > > However, looking at the calendar, I think we need to do a little more > planning and management than we've typically done for Python releases. > A final release in August means that we should plan to release a first > beta in June and a second one in July. That means we have time for > only two more alpha releases (April and May). I'm thinking of 1-2 > release candidates 1-2 weeks ahead of the final release. Barry can > make up a more detailed timetable. I'm fine with some slippage > (especially if planned ahead) of individual releases due to > availability of release personnel; but I don't want to be held up by > features or bugs unless they are of absolutely dramatic show-stopping > nature. > > In order to make such a tight release schedule we should try to come > up with a list of tasks that need to be done, and prioritize them. > This should include documentation, and supporting tools like 2to3. It > should include features, backports of features, cleanup, bugs, and > whatever else needs to be done (e.g. bugbot maintenance). > > In the past we've used shared spreadsheets in Google for this purpose, > but seeing that these haven't been updated in ages, I'm skeptical that > they are a sufficient tool. In my day job at Google we've started to > do all task management for our project in the bug tracker (but that > tracker has some features that make it particularly easy). Does anyone > have a suggestion for an online open shared task management system > that we cold adopt? Or should we bite the bullet and put everything in > the bug tracker? Other suggestions? Anything's better than just > email... I don't like the idea of task like items in the main bug tracker. I do, however, like the bug tracker interface for this sort of thing. Could we start a new instance of the the tracker just for internal development tasks? We could change the statuses around to "Work in progress", "Completed", "Incomplete", and such. It'd be easy to search for tasks that have to be accomplished for a given release. > > > PS. I realize that I've been rather absent from the day to day > activities in the Python 3000 world lately. This is a temporary > condition due to an important impending launch in my day job; I expect > to have much more time for Python again starting in April. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/ > ) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080316/ce742145/attachment.htm From musiccomposition at gmail.com Sun Mar 16 17:26:58 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sun, 16 Mar 2008 11:26:58 -0500 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: References: Message-ID: <1afaf6160803160926i37d3f2b9l9bd23953e77f7101@mail.gmail.com> On Sun, Mar 16, 2008 at 10:23 AM, Guido van Rossum wrote: > Moving this to a new subject to keep the discussion of tasks and the > discussion of task tracking tools separate. > > On Sun, Mar 16, 2008 at 9:42 AM, Christian Heimes > wrote: > > I did a quick brainstorming with me, myself and I. I came up with a > list > > of (IMHO) important tasks. > > > > * Stabilize the C API of Python 3.0. I like to rename several prefixes > > to reduce the confusing: PyBytes -> PyByteArray, > > +1 (also +1 to backporting this to 2.6) > > > PyString -> PyBytes ... > > -1. This will make merging code from 2.6 harder, and causes more work > for porting C extensions. There was a thread about this a few weeks ago: http://mail.python.org/pipermail/python-dev/2008-March/077339.html We can still do the renaming, but alias PyString to PyBytes. > > > > * Backport the new buffer protocol to 2.6. I spoke to Travis yesterday > > and he said he is trying to get it done during the PyCon sprint. Maybe > > somebody can assist him? > > Does he need assistance? > > > When the new buffer protocol is available in 2.6 we can start back > > porting bytesarray and the new IO framework. > > Are these really so closely tied that they have to wait before they > can be backported? > > > * Replace Windows API calls with wide versions to support unicode for > > file names, environment etc. > > +1. This should be broken into separate tasks for each API. > > > * Get the stdlib reorg done > > +1. But again, I think this is many small tasks. > > > and add the fixers to 2to3 > > +1. I think quite a few changes have not had a fixer added. Again, I > think we should maintain a specific list of needed fixers; fixers can > easily be developed independently. > > > * Speed up 2to3. I suggested a GSoC task to improve and speed up 2to3 > > but it may be too late when we plan to ship out 3.0 in August. > > While I know that some people are expecting to use a development model > that invokes 2to3 very frequently, I think this is at best a > nice-to-have. (I also don't see how it could be done, but maybe I'm > blind for the obvious, as the original author.) > > I have some other tasks to add: > > - getargs.c (Georg posted about this; his post actually inspired my post.) > > - Tweak structseq to be more like namedtuple. Maybe they could have a > common ABC. I also think we should get rid of the concept of "hidden" > fields (that are accessible by name but not through the tuple API). > > - Devise a way to handle str instances pickled in 2.6 and unpickled in > 3.0, and also bytes instances pickled in 3.0 and unpickled in 2.6. > > - Someone should go over > https://spreadsheets.google.com/ccc?key=pCKY4oaXnT81FrGo3ShGHGg > and extract more tasks. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/ > ) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080316/79f70334/attachment-0001.htm From guido at python.org Sun Mar 16 17:37:45 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 16 Mar 2008 11:37:45 -0500 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: <1afaf6160803160919qbf49800n584ea4f965ebc6b2@mail.gmail.com> References: <1afaf6160803160919qbf49800n584ea4f965ebc6b2@mail.gmail.com> Message-ID: On Sun, Mar 16, 2008 at 11:19 AM, Benjamin Peterson wrote: > I don't like the idea of task like items in the main bug tracker. Why not? Bugs are also tasks, and need to be managed and triaged in the same way. It might be convenient to have everything in one tracker. What's your objection? It should be easy enough to search for tasks or bugs etc. > I do, however, like the bug tracker interface for this sort of thing. Could we > start a new instance of the the tracker just for internal development tasks? I'm not sure how easy it would be to start a new instance, but I expect setting up the database, configuration etc. would require a fairly significant amount of work. I'd much rather use existing infrastructure. > We could change the statuses around to "Work in progress", "Completed", > "Incomplete", and such. It'd be easy to search for tasks that have to be > accomplished for a given release. That would be good for bugs too, actually. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Mar 16 17:40:32 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 16 Mar 2008 11:40:32 -0500 Subject: [Python-Dev] Fwd: 2.6 and 3.0 project management In-Reply-To: References: <18397.12093.765076.737023@montanaro-dyndns-org.local> Message-ID: Sorry, forgot to CC this to the list. On Sun, Mar 16, 2008 at 9:31 AM, wrote: > > Guido> It has a much more detailed set of categories, organized as a > Guido> tree. Our project alone probably has 20-30 different bug > Guido> categories. New bugs in those categories are automatically CC'ed > Guido> to our group's mailing list (which isn't the same as > Guido> auto-assignment). > > Adding categories should be easy. Organized in trees? Not so sure. The tree is really useful because it means that end users can assign bugs to the top-level node for a project and the project members can move it to the correct subnode. This can even happen in two triage stages for large projects. > Guido> There are also more "bug states" you can use to track progress of > Guido> a bug through the system: unassigned, assigned, accepted (meaning > Guido> the assignee is actually working on it). (There are also a whole > Guido> bunch that I don't find so useful, and severam that roundup > Guido> already supports.) > > Again, I think this should be easy. It's also the least important one on my list. > Guido> But perhaps the best feature is "hot lists" -- arbitrary, > Guido> ordered, groupings of selected bugs. Each bug can be assigned to > Guido> as many hot lists as you want. Seeing the list of all bugs in a > Guido> particular hot list is one click away. We use this for overlaying > Guido> project management categories and priorities, such as "code", > Guido> "documentation", "configuration" as well as "next internal > Guido> release", "must have", "post launch" etc. > > A hot list sounds like a saved search, which Roundup already supports. It > also supports making these saved searches public. I suspect you could > define one or more saved public searches which correspond to desired hot > lists. Not quite. Items don't automatically end up on a hot list; they must explicitly be put on one. I'm not sure how you'd simulate this via saved searches. Maybe a combination of a custom keyword *and* a saved search would help. However this doesn't scale so well, because keywords show up in everybody's UI. Hot lists are only visible to users who care to subscribe to them. [Georg, in a later post] > Doesn't this match Roundup's keywords? See above answer. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Mar 16 17:44:43 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 16 Mar 2008 11:44:43 -0500 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: <1afaf6160803160926i37d3f2b9l9bd23953e77f7101@mail.gmail.com> References: <1afaf6160803160926i37d3f2b9l9bd23953e77f7101@mail.gmail.com> Message-ID: On Sun, Mar 16, 2008 at 11:26 AM, Benjamin Peterson wrote: > On Sun, Mar 16, 2008 at 10:23 AM, Guido van Rossum wrote: > > > PyString -> PyBytes ... > > > > -1. This will make merging code from 2.6 harder, and causes more work > > for porting C extensions. > There was a thread about this a few weeks ago: > http://mail.python.org/pipermail/python-dev/2008-March/077339.html > We can still do the renaming, but alias PyString to PyBytes. That's a rather long thread. Was any conclusion reached? I'm not sure how introducing a set of aliases will help merging 2.6 code to 3.0. Can you or Christian describe the proposed approach in more detail? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Sun Mar 16 17:44:06 2008 From: barry at python.org (Barry Warsaw) Date: Sun, 16 Mar 2008 11:44:06 -0500 Subject: [Python-Dev] 3.0 buildbots all red In-Reply-To: <47DD415F.7090109@v.loewis.de> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD173@EXMBX04.exchhosting.com> <47DD415F.7090109@v.loewis.de> Message-ID: <08FEA482-95A1-4EA8-92D3-5A6AFEFEDF90@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 16, 2008, at 10:48 AM, Martin v. L?wis wrote: >> New sprint idea: getting all (inc. trunk) the buildbots green by >> Thursday. Anyone interested? > > I think the chance to achieve that is close to zero. How broken do you want the next monthly alphas to be? :) - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR91OWHEjvBPtnXfVAQIL/gP/T5TqGLt6Z7LD3/vEKt/0qjZErkOQmmFH BciuGOs6CCud//N4fsSev2CBmeAQ/RspHhjLSSKBd6H4rimZWv5ePo0gMC2N7nGY 1wpUvixaZpYol4zywjBQRO+bjlnZtdt4WG09DdMrn0MsYHNpVFaAPTWx4X3BFHm+ k0QLzsJ7T2o= =7czp -----END PGP SIGNATURE----- From collinw at gmail.com Sun Mar 16 17:50:16 2008 From: collinw at gmail.com (Collin Winter) Date: Sun, 16 Mar 2008 09:50:16 -0700 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: References: Message-ID: <43aa6ff70803160950l736a20ceg1acb4c17251d9a0d@mail.gmail.com> On Sun, Mar 16, 2008 at 8:23 AM, Guido van Rossum wrote: > On Sun, Mar 16, 2008 at 9:42 AM, Christian Heimes wrote: ...... > > and add the fixers to 2to3 > > +1. I think quite a few changes have not had a fixer added. Again, I > think we should maintain a specific list of needed fixers; fixers can > easily be developed independently. Neal and I are coming up with a list to feed tasks to interested PyCon sprinters. > > * Speed up 2to3. I suggested a GSoC task to improve and speed up 2to3 > > but it may be too late when we plan to ship out 3.0 in August. > > While I know that some people are expecting to use a development model > that invokes 2to3 very frequently, I think this is at best a > nice-to-have. (I also don't see how it could be done, but maybe I'm > blind for the obvious, as the original author.) The biggest win in terms of performance would be to reimplement the pattern matching engine used by the fixers.: it by far dominates the running time, taking 99+% of the runtime when I ran 2to3 over Twisted, for example. The current design is a heavily-recursive system, and as such bombs out when it encounters, e.g., files with a thousand assignment statements in a row. I'd also like something more expressive: the current DSL can't express recursive patterns. Collin Winter From skip at pobox.com Sun Mar 16 17:49:45 2008 From: skip at pobox.com (skip at pobox.com) Date: Sun, 16 Mar 2008 11:49:45 -0500 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: References: <1afaf6160803160919qbf49800n584ea4f965ebc6b2@mail.gmail.com> Message-ID: <18397.20393.103687.882317@montanaro-dyndns-org.local> >> I don't like the idea of task like items in the main bug tracker. Guido> Why not? Bugs are also tasks, and need to be managed and triaged Guido> in the same way. Agreed. Both bugs and tasks would be "issues" in Roundup parlance, along with patches. A further reason to keep this in Roundup if possible is that people like Martin have already committed to help maintain the tracker. We already have a separate Trac instance for the website, which I would like to see folded into the Roundup tracker, freeing up limited (human) resources used to maintain that tracker so they can either spend more time with their families or work on other things that need doing. Skip From musiccomposition at gmail.com Sun Mar 16 17:51:19 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sun, 16 Mar 2008 11:51:19 -0500 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: References: <1afaf6160803160919qbf49800n584ea4f965ebc6b2@mail.gmail.com> Message-ID: <1afaf6160803160951l7a580ee8i796bcfe77d424cf8@mail.gmail.com> On Sun, Mar 16, 2008 at 11:37 AM, Guido van Rossum wrote: > On Sun, Mar 16, 2008 at 11:19 AM, Benjamin Peterson > wrote: > > I don't like the idea of task like items in the main bug tracker. > > Why not? Bugs are also tasks, and need to be managed and triaged in > the same way. It might be convenient to have everything in one > tracker. What's your objection? It should be easy enough to search for > tasks or bugs etc. It's just depends on how you see the tracker. It's not just to "bug" tracker anymore, is it? On other projects I've worked with, we had separate areas for bugs, features, and tasks. (yes, it's SourceForge.) I found it easier to keep organized. However, if this is Python's way, I'm not going to stand in it. > > > > I do, however, like the bug tracker interface for this sort of thing. > Could we > > start a new instance of the the tracker just for internal development > tasks? > > I'm not sure how easy it would be to start a new instance, but I > expect setting up the database, configuration etc. would require a > fairly significant amount of work. I'd much rather use existing > infrastructure. I'm now fine with that. > > > > We could change the statuses around to "Work in progress", "Completed", > > "Incomplete", and such. It'd be easy to search for tasks that have to be > > accomplished for a given release. > > That would be good for bugs too, actually. Well, I'm glad to help somehow! :P > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/ > ) > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080316/cf09856b/attachment.htm From martin at v.loewis.de Sun Mar 16 17:54:26 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMwpp3aXMi?=) Date: Sun, 16 Mar 2008 11:54:26 -0500 Subject: [Python-Dev] [Python-3000] Fwd: 2.6 and 3.0 project management In-Reply-To: References: <18397.12093.765076.737023@montanaro-dyndns-org.local> Message-ID: <47DD50C2.1070701@v.loewis.de> > Not quite. Items don't automatically end up on a hot list; they must > explicitly be put on one. I'm not sure how you'd simulate this via > saved searches. Maybe a combination of a custom keyword *and* a saved > search would help. However this doesn't scale so well, because > keywords show up in everybody's UI. Hot lists are only visible to > users who care to subscribe to them. It would be relatively easy to implement this directly in roundup. IIUC, there should be a hotlist object, and either a) an issue has a multilink to multiple hotlists, or b) a hotlist has a multilink to multiple issues. I cannot envision the "add to hotlist" user interface, though. It should be possible to add an issue to a hotlist from the issue's page, right? So would a comma-separated list be reasonable? (with a popup menu to checkmark hotlists) Regards, Martin From musiccomposition at gmail.com Sun Mar 16 17:57:32 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sun, 16 Mar 2008 11:57:32 -0500 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: References: <1afaf6160803160926i37d3f2b9l9bd23953e77f7101@mail.gmail.com> Message-ID: <1afaf6160803160957v1e15a8c9s1e709a0d47bb52f@mail.gmail.com> On Sun, Mar 16, 2008 at 11:44 AM, Guido van Rossum wrote: > On Sun, Mar 16, 2008 at 11:26 AM, Benjamin Peterson > wrote: > > On Sun, Mar 16, 2008 at 10:23 AM, Guido van Rossum > wrote: > > > > PyString -> PyBytes ... > > > > > > -1. This will make merging code from 2.6 harder, and causes more work > > > for porting C extensions. > > > There was a thread about this a few weeks ago: > > http://mail.python.org/pipermail/python-dev/2008-March/077339.html > > We can still do the renaming, but alias PyString to PyBytes. > > That's a rather long thread. Was any conclusion reached? I'm not sure > how introducing a set of aliases will help merging 2.6 code to 3.0. > Can you or Christian describe the proposed approach in more detail? As far as I can see, no objections were raised in that thread. Christian explained the probable approach: http://mail.python.org/pipermail/python-dev/2008-March/077362.html > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/ > ) > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080316/d18e1550/attachment-0001.htm From martin at v.loewis.de Sun Mar 16 17:58:00 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMwpp3aXMi?=) Date: Sun, 16 Mar 2008 11:58:00 -0500 Subject: [Python-Dev] [Python-3000] 2.6 and 3.0 project management In-Reply-To: <1afaf6160803160951l7a580ee8i796bcfe77d424cf8@mail.gmail.com> References: <1afaf6160803160919qbf49800n584ea4f965ebc6b2@mail.gmail.com> <1afaf6160803160951l7a580ee8i796bcfe77d424cf8@mail.gmail.com> Message-ID: <47DD5198.9090001@v.loewis.de> > It's just depends on how you see the tracker. It's not just to "bug" > tracker anymore, is it? On other projects I've worked with, we had > separate areas for bugs, features, and tasks. (yes, it's SourceForge.) I > found it easier to keep organized. However, if this is Python's way, I'm > not going to stand in it. Actually, one of the main complaints about the SF tracker is that it splits into several ones. Something starts out as a bug, but then becomes a patch as soon as somebody attaches a patch. So on SF, people had to open a *separate* issue to provide a patch, and leave a message in the original bug report pointing to the patch. They hated it, and insisted that the new tracker should have a single list of issues. Regards, Martin From brett at python.org Sun Mar 16 18:00:01 2008 From: brett at python.org (Brett Cannon) Date: Sun, 16 Mar 2008 12:00:01 -0500 Subject: [Python-Dev] [Python-3000] xturtle and 3.0 In-Reply-To: References: Message-ID: On Sun, Mar 16, 2008 at 10:32 AM, Guido van Rossum wrote: > I'm changing the subject to keep this separate from the project > management tools discussion. > > > > Of course I know that xturtle is only a side issue in the current > > development efforts. Unfortunately I'm not familiar with the procedures > > needed to get a new module into Python, so I kindly ask you for your > > advice how to proceed, at the same time offering my cooperation. > > I think that for a library module like this, an email like you've sent > is just fine. Maybe Brett has a suggestion on whether it would remain > a toplevel module or could be placed in some umbrella package (is > Tkinter being moved around?). The current plan is to introduce a tk package and turtle was to become tk.turtle. xturtle, if picked up, can just take the place of the current turtle at that location. -Brett From martin at v.loewis.de Sun Mar 16 18:04:29 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMwpp3aXMi?=) Date: Sun, 16 Mar 2008 12:04:29 -0500 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD176@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD176@EXMBX04.exchhosting.com> Message-ID: <47DD531D.5030801@v.loewis.de> >>> * Replace Windows API calls with wide versions to support unicode >>> for file names, environment etc. >> +1. This should be broken into separate tasks for each API. > > What are we referring to here? Calling the W versions explicitly and > using wchar_t for everything, or using the TCHAR/TEXT() approach and > keeping the API calls the same, letting the #define UNICODE do the > work behind the scenes? Not sure whose being attributed here, but I think "breaking down" should be done by "information source": command line, environment, registry, file names, sys.path, module names, etc. I have a patch on SF to resolve the command line issue. I don't think we should use Microsoft's Unicode programming model around TCHAR/TEXT. It would allow for two different builds, and given that we don't need to support W9X anymore, an "ANSI" build is pointless, IMO. Regards, Martin From martin at v.loewis.de Sun Mar 16 18:07:21 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 16 Mar 2008 12:07:21 -0500 Subject: [Python-Dev] 3.0 buildbots all red In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD17D@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD173@EXMBX04.exchhosting.com> <47DD415F.7090109@v.loewis.de> <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD17D@EXMBX04.exchhosting.com> Message-ID: <47DD53C9.8000704@v.loewis.de> >>> New sprint idea: getting all (inc. trunk) the buildbots green by >> Thursday. Anyone interested? >> >> I think the chance to achieve that is close to zero. > > Sounds like a challenge if ever I've heard one -- care to wager a > beer on it? (Only applies to buildbots that are connected/online.) Even with that restriction: I'll happily buy you a beer if you manage to get all the online ones pass all tests consistently. Regards, Martin From lists at cheimes.de Sun Mar 16 18:09:49 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 16 Mar 2008 18:09:49 +0100 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: References: Message-ID: <47DD545D.6070300@cheimes.de> Guido van Rossum wrote: > -1. This will make merging code from 2.6 harder, and causes more work > for porting C extensions. I'm happy to pay the price for the sake of a clean and easy-to-recall C API. The for the C extension problem I already proposed a solution in the thread Benjamin mentioned before. #include "Python.h" #if PY_VERSION_HEX > 0x03000000 #include "python2_compat.h" #endif Where python2_compat provides aliases for PyInt and PyString: #define PyInt_Spam PyLong_Spam ... #define PyString_Egg PyBytes_Egg >> * Backport the new buffer protocol to 2.6. I spoke to Travis yesterday >> and he said he is trying to get it done during the PyCon sprint. Maybe >> somebody can assist him? > > Does he need assistance? I don't know. > >> When the new buffer protocol is available in 2.6 we can start back >> porting bytesarray and the new IO framework. > > Are these really so closely tied that they have to wait before they > can be backported? Bytesarray depends on the buffer protocol and the new io framework is much easier to back port when both the buffer protocol and bytesarray is available. > While I know that some people are expecting to use a development model > that invokes 2to3 very frequently, I think this is at best a > nice-to-have. (I also don't see how it could be done, but maybe I'm > blind for the obvious, as the original author.) I'm not sure if and how 2to3 can be speed up. Maybe some profiling and some C code can increase the speed. It's worth a shot. Christian From guido at python.org Sun Mar 16 18:13:42 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 16 Mar 2008 12:13:42 -0500 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: <1afaf6160803160951l7a580ee8i796bcfe77d424cf8@mail.gmail.com> References: <1afaf6160803160919qbf49800n584ea4f965ebc6b2@mail.gmail.com> <1afaf6160803160951l7a580ee8i796bcfe77d424cf8@mail.gmail.com> Message-ID: On Sun, Mar 16, 2008 at 11:51 AM, Benjamin Peterson wrote: > It's just depends on how you see the tracker. It's not just to "bug" tracker > anymore, is it? On other projects I've worked with, we had separate areas > for bugs, features, and tasks. (yes, it's SourceForge.) I found it easier to > keep organized. However, if this is Python's way, I'm not going to stand in > it. Ah, sourceforge. I am so glad we're not using that any more. The random separation between patches and bugs was more a distraction rather than a feature; often bugs turn into patches or patches turn out to be useless except for the fact that they highlight a bug... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Sun Mar 16 18:17:00 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 16 Mar 2008 18:17:00 +0100 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: <43aa6ff70803160950l736a20ceg1acb4c17251d9a0d@mail.gmail.com> References: <43aa6ff70803160950l736a20ceg1acb4c17251d9a0d@mail.gmail.com> Message-ID: <47DD560C.8090202@cheimes.de> Collin Winter wrote: > The biggest win in terms of performance would be to reimplement the > pattern matching engine used by the fixers.: it by far dominates the > running time, taking 99+% of the runtime when I ran 2to3 over Twisted, > for example. The current design is a heavily-recursive system, and as > such bombs out when it encounters, e.g., files with a thousand > assignment statements in a row. I'd also like something more > expressive: the current DSL can't express recursive patterns. Do you have time to mentor a GSoC project? Or can you mentor a mentor ...? :) Christian From barry at python.org Sun Mar 16 18:24:19 2008 From: barry at python.org (Barry Warsaw) Date: Sun, 16 Mar 2008 12:24:19 -0500 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: References: Message-ID: <2C2D7EC7-51F2-4F4E-8127-87ACCC6F8413@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 16, 2008, at 8:51 AM, Guido van Rossum wrote: > Python 3.0 and 2.6 are coming along really nice. I am optimistic that > we can make the projected August date for the final releases of 2.6 > and 3.0. As you may remember, Barry (the new release manager for both) > suggested that we synchronize releases of 2.6 and 3.0. Not only could > this potentially save the release manager and his assistants some > time, doing the final releases together sends a clear signal to the > community that both versions will receive equal support. I mentioned this to Guido and got a positive response, so let me state my preference for your feedback. I plan on holding up the final releases until both versions are ready to go. I think this will help motivate us to give Python 2.6 the love it needs if it's lagging behind 3.0, and I completely agree with Guido that this let's our community know that both versions are equally important to us. The other thing is that I'd really like is a "show stoppers" Roundup search. The idea is that if our core buildbots look good and the "show stoppers" search turns up no items, then I know I can cut a release (at least for alphas, betas, and rcs). If there are "show stoppers" then I have something that I can triage (and maybe re-assign severity) or start publicly harassing people into fixing. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR91XxHEjvBPtnXfVAQIINwQAsaJ8uWb0FXqD9wZV8AuE7CxG2RLEZl42 vz2EUbAs7n/txV/lWs3N4syZvfS7g0Q3rgd65LvpUCi4+r6M3MpKcl9VFxGfheUD mHf5LajS/wEEXyFpxG9+dGfhPpD7JFx21PKyOpHKnq3LP0fCt3yenOi/nEF5zfHr Ogc6JOapRiM= =0f9G -----END PGP SIGNATURE----- From barry at python.org Sun Mar 16 18:25:27 2008 From: barry at python.org (Barry Warsaw) Date: Sun, 16 Mar 2008 12:25:27 -0500 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: References: Message-ID: <65366C3B-A839-454E-A748-9CF24C39A980@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 16, 2008, at 8:51 AM, Guido van Rossum wrote: > However, looking at the calendar, I think we need to do a little more > planning and management than we've typically done for Python releases. > A final release in August means that we should plan to release a first > beta in June and a second one in July. That means we have time for > only two more alpha releases (April and May). I'm thinking of 1-2 > release candidates 1-2 weeks ahead of the final release. Barry can > make up a more detailed timetable. I'm fine with some slippage > (especially if planned ahead) of individual releases due to > availability of release personnel; but I don't want to be held up by > features or bugs unless they are of absolutely dramatic show-stopping > nature. Oh yeah, I'd like to hash a more detailed timeline out at the sprint. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR91YCXEjvBPtnXfVAQJsxQP9FgyAn2JOl/+SShBNgTqlxWBwGAbrjSlS ySbm/55PKs6bnjx1lkyvptzHFdsfh1LlBVrC5rxQzQIjSX00x8fAPAoseQZ/hDm0 cnVSvPhJhBLsZCmsSRfvDYPdZXnanDd5ff69uV3jd+NLasuJBNrQ5d+eQGiQOTZm BTgsjt+YKpQ= =L9yF -----END PGP SIGNATURE----- From nnorwitz at gmail.com Sun Mar 16 18:32:33 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 16 Mar 2008 12:32:33 -0500 Subject: [Python-Dev] 3.0 buildbots all red In-Reply-To: <47DD53C9.8000704@v.loewis.de> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD173@EXMBX04.exchhosting.com> <47DD415F.7090109@v.loewis.de> <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD17D@EXMBX04.exchhosting.com> <47DD53C9.8000704@v.loewis.de> Message-ID: On Sun, Mar 16, 2008 at 12:07 PM, "Martin v. L?wis" wrote: > >>> New sprint idea: getting all (inc. trunk) the buildbots green by > >> Thursday. Anyone interested? > >> > >> I think the chance to achieve that is close to zero. > > > > Sounds like a challenge if ever I've heard one -- care to wager a > > beer on it? (Only applies to buildbots that are connected/online.) > > Even with that restriction: I'll happily buy you a beer if you > manage to get all the online ones pass all tests consistently. I think this is possible, though considerable work. Probably the biggest win will be creating a mock for socket and using mock sockets in the tests for asyn{core,chat}, smtplib, xmlrpc, etc. That will fix about 75% of the problems on 2.6. The remaining problems are: * test_asyn{chat,core} might not be meaningful with mock sockets and are flaky * the alpha fails test_signal/socket for weird alarm conditions. this might be hard to debug/fix (I have access to this box though) * test_sqlite is broken on x86 with an old sqlite (I have access to this box) * test_bsddb may be flaky, I'm not sure * probably a few platform specific problems 3.0 will get most of the improvements as the fixes are ported. 3.0 has a different problem on the x86 box dealing with signals. There are probably some other 3.0 specific problems. Although, using a mock socket could address this too. I can help on fixing these issues during the sprints. It will be happy to work with Trent to try to fix the problems. I'm sure we can greatly improve the situation. n From guido at python.org Sun Mar 16 18:32:57 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 16 Mar 2008 12:32:57 -0500 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: <47DD545D.6070300@cheimes.de> References: <47DD545D.6070300@cheimes.de> Message-ID: On Sun, Mar 16, 2008 at 11:57 AM, Benjamin Peterson wrote: [Guido] > > That's a rather long thread. Was any conclusion reached? I'm not sure > > how introducing a set of aliases will help merging 2.6 code to 3.0. > > Can you or Christian describe the proposed approach in more detail? > As far as I can see, no objections were raised in that thread. Hm. I had wanted to register a complaint, but I guess I was too busy. > Christian explained the probable approach: > http://mail.python.org/pipermail/python-dev/2008-March/077362.html That's not much of a plan. It doesn't discuss any of the effects of the proposed change on merging code from the 2.6 trunk to the py3k branch. On Sun, Mar 16, 2008 at 12:09 PM, Christian Heimes wrote: > Guido van Rossum wrote: > > -1. This will make merging code from 2.6 harder, and causes more work > > for porting C extensions. > > I'm happy to pay the price for the sake of a clean and easy-to-recall C > API. > > The for the C extension problem I already proposed a solution in the > thread Benjamin mentioned before. > > #include "Python.h" > #if PY_VERSION_HEX > 0x03000000 > #include "python2_compat.h" > #endif > > Where python2_compat provides aliases for PyInt and PyString: > > #define PyInt_Spam PyLong_Spam > ... > #define PyString_Egg PyBytes_Egg So this doesn't address merges at all. Suppose we have some C code that's shared between 2.6 and 3.0 and manipulates binary data (e.g. the gzip codec). It currently uses PyString on both branches, so any changes to the trunk merge smoothly into the py3k branch. But if you change PyString -> PyBytes in the py3k branch, every change you make in the 2.6 code will cause a merge conflict. Is that really what you want? I worry that it will effectively separate the trunk and the py3k branch, losing the advantage of doing development that effects both at once. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From musiccomposition at gmail.com Sun Mar 16 19:01:29 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sun, 16 Mar 2008 13:01:29 -0500 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: References: <47DD545D.6070300@cheimes.de> Message-ID: <1afaf6160803161101jed4b63bl3ae599622bb56bfe@mail.gmail.com> On Sun, Mar 16, 2008 at 12:32 PM, Guido van Rossum wrote: > On Sun, Mar 16, 2008 at 11:57 AM, Benjamin Peterson > wrote: > [Guido] > > > That's a rather long thread. Was any conclusion reached? I'm not sure > > > how introducing a set of aliases will help merging 2.6 code to 3.0. > > > Can you or Christian describe the proposed approach in more detail? > > > As far as I can see, no objections were raised in that thread. > > Hm. I had wanted to register a complaint, but I guess I was too busy. > > > Christian explained the probable approach: > > http://mail.python.org/pipermail/python-dev/2008-March/077362.html > > That's not much of a plan. It doesn't discuss any of the effects of > the proposed change on merging code from the 2.6 trunk to the py3k > branch. > > On Sun, Mar 16, 2008 at 12:09 PM, Christian Heimes > wrote: > > Guido van Rossum wrote: > > > -1. This will make merging code from 2.6 harder, and causes more work > > > for porting C extensions. > > > > I'm happy to pay the price for the sake of a clean and easy-to-recall C > > API. > > > > The for the C extension problem I already proposed a solution in the > > thread Benjamin mentioned before. > > > > #include "Python.h" > > #if PY_VERSION_HEX > 0x03000000 > > #include "python2_compat.h" > > #endif > > > > Where python2_compat provides aliases for PyInt and PyString: > > > > #define PyInt_Spam PyLong_Spam > > ... > > #define PyString_Egg PyBytes_Egg > > So this doesn't address merges at all. Suppose we have some C code > that's shared between 2.6 and 3.0 and manipulates binary data (e.g. > the gzip codec). It currently uses PyString on both branches, so any > changes to the trunk merge smoothly into the py3k branch. But if you > change PyString -> PyBytes in the py3k branch, every change you make > in the 2.6 code will cause a merge conflict. Is that really what you > want? I worry that it will effectively separate the trunk and the py3k > branch, losing the advantage of doing development that effects both at > once. We could backport the python2_compact header. > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/ > ) > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080316/99e23fbc/attachment.htm From oliphant.travis at ieee.org Sun Mar 16 19:03:20 2008 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Sun, 16 Mar 2008 13:03:20 -0500 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: References: Message-ID: Guido van Rossum wrote: > Moving this to a new subject to keep the discussion of tasks and the > discussion of task tracking tools separate. > > On Sun, Mar 16, 2008 at 9:42 AM, Christian Heimes wrote: >> I did a quick brainstorming with me, myself and I. I came up with a list >> of (IMHO) important tasks. >> >> * Stabilize the C API of Python 3.0. I like to rename several prefixes >> to reduce the confusing: PyBytes -> PyByteArray, > > +1 (also +1 to backporting this to 2.6) > >> PyString -> PyBytes ... > > -1. This will make merging code from 2.6 harder, and causes more work > for porting C extensions. > >> * Backport the new buffer protocol to 2.6. I spoke to Travis yesterday >> and he said he is trying to get it done during the PyCon sprint. Maybe >> somebody can assist him? > > Does he need assistance? I don't really need help with back-porting the protocol. However, I do need help with the struct module changes. This is a standard-library that I'm hoping to get help with. -Travis From lists at cheimes.de Sun Mar 16 19:04:46 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 16 Mar 2008 19:04:46 +0100 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: References: <47DD545D.6070300@cheimes.de> Message-ID: <47DD613E.6090007@cheimes.de> Guido van Rossum wrote: > So this doesn't address merges at all. Suppose we have some C code > that's shared between 2.6 and 3.0 and manipulates binary data (e.g. > the gzip codec). It currently uses PyString on both branches, so any > changes to the trunk merge smoothly into the py3k branch. But if you > change PyString -> PyBytes in the py3k branch, every change you make > in the 2.6 code will cause a merge conflict. Is that really what you > want? I worry that it will effectively separate the trunk and the py3k > branch, losing the advantage of doing development that effects both at > once. I'm fully aware of the extra burden. The removal of the PyInt_* functions is already causing merge conflicts and compile errors. Nearly every C code merge contains at least one place that requires manual intervention. The PyInt merge conflicts are trivial to fix - so would errors caused PyString -> PyBytes rename. I'm not worried about the extra work since it's usually trivial and fast to fix. I'm more worried about the API names. Do you *really* want to drag down dead bodies along the road for the next ten years? The dead bodies are going to rot and stink sooner than later. By Python 3.2 everybody surely regrets the confusing names. ;) Christian From musiccomposition at gmail.com Sun Mar 16 19:06:18 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sun, 16 Mar 2008 13:06:18 -0500 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: <1afaf6160803161101jed4b63bl3ae599622bb56bfe@mail.gmail.com> References: <47DD545D.6070300@cheimes.de> <1afaf6160803161101jed4b63bl3ae599622bb56bfe@mail.gmail.com> Message-ID: <1afaf6160803161106o5c06ce6dx7839e0d5decd2cc0@mail.gmail.com> On Sun, Mar 16, 2008 at 1:01 PM, Benjamin Peterson < musiccomposition at gmail.com> wrote: > > > On Sun, Mar 16, 2008 at 12:32 PM, Guido van Rossum > wrote: > > > On Sun, Mar 16, 2008 at 11:57 AM, Benjamin Peterson > > wrote: > > [Guido] > > > > That's a rather long thread. Was any conclusion reached? I'm not > > sure > > > > how introducing a set of aliases will help merging 2.6 code to 3.0. > > > > Can you or Christian describe the proposed approach in more detail? > > > > > As far as I can see, no objections were raised in that thread. > > > > Hm. I had wanted to register a complaint, but I guess I was too busy. > > > > > Christian explained the probable approach: > > > http://mail.python.org/pipermail/python-dev/2008-March/077362.html > > > > That's not much of a plan. It doesn't discuss any of the effects of > > the proposed change on merging code from the 2.6 trunk to the py3k > > branch. > > > > On Sun, Mar 16, 2008 at 12:09 PM, Christian Heimes > > wrote: > > > Guido van Rossum wrote: > > > > -1. This will make merging code from 2.6 harder, and causes more > > work > > > > for porting C extensions. > > > > > > I'm happy to pay the price for the sake of a clean and easy-to-recall > > C > > > API. > > > > > > The for the C extension problem I already proposed a solution in the > > > thread Benjamin mentioned before. > > > > > > #include "Python.h" > > > #if PY_VERSION_HEX > 0x03000000 > > > #include "python2_compat.h" > > > #endif > > > > > > Where python2_compat provides aliases for PyInt and PyString: > > > > > > #define PyInt_Spam PyLong_Spam > > > ... > > > #define PyString_Egg PyBytes_Egg > > > > So this doesn't address merges at all. Suppose we have some C code > > that's shared between 2.6 and 3.0 and manipulates binary data (e.g. > > the gzip codec). It currently uses PyString on both branches, so any > > changes to the trunk merge smoothly into the py3k branch. But if you > > change PyString -> PyBytes in the py3k branch, every change you make > > in the 2.6 code will cause a merge conflict. Is that really what you > > want? I worry that it will effectively separate the trunk and the py3k > > branch, losing the advantage of doing development that effects both at > > once. > > We could backport the python2_compact header. > Sorry, that needs some explanation. We could do the opposite that we do for Py3K: Add a header file aliasing PyBytes to PyString. > > > > > > -- > > --Guido van Rossum (home page: http://www.python.org/~guido/ > > ) > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080316/5cd56eed/attachment.htm From lists at cheimes.de Sun Mar 16 19:23:09 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 16 Mar 2008 19:23:09 +0100 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: <1afaf6160803160951l7a580ee8i796bcfe77d424cf8@mail.gmail.com> References: <1afaf6160803160919qbf49800n584ea4f965ebc6b2@mail.gmail.com> <1afaf6160803160951l7a580ee8i796bcfe77d424cf8@mail.gmail.com> Message-ID: Benjamin Peterson wrote: > It's just depends on how you see the tracker. It's not just to "bug" tracker > anymore, is it? On other projects I've worked with, we had separate areas > for bugs, features, and tasks. (yes, it's SourceForge.) I found it easier to > keep organized. However, if this is Python's way, I'm not going to stand in > it. Despite the url bugs.python.org it's an issue tracker and not a bug tracker. We track patches, feature requests, ideas and bugs in the same tracker. Christian From guido at python.org Sun Mar 16 20:46:26 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 16 Mar 2008 14:46:26 -0500 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: <47DD613E.6090007@cheimes.de> References: <47DD545D.6070300@cheimes.de> <47DD613E.6090007@cheimes.de> Message-ID: On Sun, Mar 16, 2008 at 1:04 PM, Christian Heimes wrote: > Guido van Rossum wrote: > > So this doesn't address merges at all. Suppose we have some C code > > that's shared between 2.6 and 3.0 and manipulates binary data (e.g. > > the gzip codec). It currently uses PyString on both branches, so any > > changes to the trunk merge smoothly into the py3k branch. But if you > > change PyString -> PyBytes in the py3k branch, every change you make > > in the 2.6 code will cause a merge conflict. Is that really what you > > want? I worry that it will effectively separate the trunk and the py3k > > branch, losing the advantage of doing development that effects both at > > once. > > I'm fully aware of the extra burden. The removal of the PyInt_* > functions is already causing merge conflicts and compile errors. Even though the more popular PyInt_ APIs are still available (even if only as macros). > Nearly every C code merge contains at least one place that requires manual > intervention. The PyInt merge conflicts are trivial to fix - so would > errors caused PyString -> PyBytes rename. I disagree. Bad merges are already a frequent cause of instability in 3.0. This could easily double the problems. And while I want to reduce the instability (I really wish you would no commit until all unittests pass), I also don't want the merges to cost more of your time! > I'm not worried about the extra work since it's usually trivial and fast > to fix. I'm more worried about the API names. Do you *really* want to > drag down dead bodies along the road for the next ten years? The dead > bodies are going to rot and stink sooner than later. By Python 3.2 > everybody surely regrets the confusing names. ;) It doesn't have to be so black and white. Using the macro approach you propose we can fix the situation at any time later. We could also make the changes in 2.6, so that the 2.6 code looks the same as 3.0. (However for binary compatibility I think it would be better if in 2.6 the linker sees PyString where in 3.0 it sees PyBytes.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From collinw at gmail.com Sun Mar 16 21:08:02 2008 From: collinw at gmail.com (Collin Winter) Date: Sun, 16 Mar 2008 13:08:02 -0700 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: <47DD560C.8090202@cheimes.de> References: <43aa6ff70803160950l736a20ceg1acb4c17251d9a0d@mail.gmail.com> <47DD560C.8090202@cheimes.de> Message-ID: <43aa6ff70803161308u10f97081vf3244927f165ab75@mail.gmail.com> On Sun, Mar 16, 2008 at 10:17 AM, Christian Heimes wrote: > Collin Winter wrote: > > The biggest win in terms of performance would be to reimplement the > > pattern matching engine used by the fixers.: it by far dominates the > > running time, taking 99+% of the runtime when I ran 2to3 over Twisted, > > for example. The current design is a heavily-recursive system, and as > > such bombs out when it encounters, e.g., files with a thousand > > assignment statements in a row. I'd also like something more > > expressive: the current DSL can't express recursive patterns. > > Do you have time to mentor a GSoC project? Or can you mentor a mentor > ...? :) I think I'd have time to mentor a GSoC project. Let's talk off-list about that. From greg at krypto.org Sun Mar 16 21:22:08 2008 From: greg at krypto.org (Gregory P. Smith) Date: Sun, 16 Mar 2008 15:22:08 -0500 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: References: Message-ID: <52dc1c820803161322r4ed68f30lfc59519b35de47a8@mail.gmail.com> On 3/16/08, Travis Oliphant wrote: > > Guido van Rossum wrote: > > Moving this to a new subject to keep the discussion of tasks and the > > discussion of task tracking tools separate. > > > > On Sun, Mar 16, 2008 at 9:42 AM, Christian Heimes > wrote: > >> I did a quick brainstorming with me, myself and I. I came up with a > list > >> of (IMHO) important tasks. > >> > >> * Stabilize the C API of Python 3.0. I like to rename several prefixes > >> to reduce the confusing: PyBytes -> PyByteArray, > > > > +1 (also +1 to backporting this to 2.6) > > > >> PyString -> PyBytes ... > > > > -1. This will make merging code from 2.6 harder, and causes more work > > for porting C extensions. > > > >> * Backport the new buffer protocol to 2.6. I spoke to Travis yesterday > >> and he said he is trying to get it done during the PyCon sprint. Maybe > >> somebody can assist him? > > > > Does he need assistance? > > > I don't really need help with back-porting the protocol. However, I do > need help with the struct module changes. This is a standard-library > that I'm hoping to get help with. > > -Travis I'm happy to help with this stuff during the sprint. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080316/8e2bb33d/attachment.htm From guido at python.org Sun Mar 16 21:54:51 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 16 Mar 2008 15:54:51 -0500 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: References: <1afaf6160803160919qbf49800n584ea4f965ebc6b2@mail.gmail.com> <1afaf6160803160951l7a580ee8i796bcfe77d424cf8@mail.gmail.com> Message-ID: I don't see a lot of objections left against using the bug tracker. I just talked to Neal and he's going to transfer all tasks from the 2.6 spreadsheet to the bug tracker. I'll also be adding various other tasks., as I think of them. We'll have to think about which keywords to use. We'll probably pick the initial set of keywords at the sprint tomorrow morning. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Mar 16 21:56:52 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 16 Mar 2008 15:56:52 -0500 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: <2C2D7EC7-51F2-4F4E-8127-87ACCC6F8413@python.org> References: <2C2D7EC7-51F2-4F4E-8127-87ACCC6F8413@python.org> Message-ID: On Sun, Mar 16, 2008 at 12:24 PM, Barry Warsaw wrote: > I mentioned this to Guido and got a positive response, so let me state > my preference for your feedback. I plan on holding up the final > releases until both versions are ready to go. I think this will help > motivate us to give Python 2.6 the love it needs if it's lagging > behind 3.0, and I completely agree with Guido that this let's our > community know that both versions are equally important to us. It's a deal. > The other thing is that I'd really like is a "show stoppers" Roundup > search. The idea is that if our core buildbots look good and the > "show stoppers" search turns up no items, then I know I can cut a > release (at least for alphas, betas, and rcs). If there are "show > stoppers" then I have something that I can triage (and maybe re-assign > severity) or start publicly harassing people into fixing. How about using the "critical" Severity for show stoppers? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg at krypto.org Sun Mar 16 22:38:48 2008 From: greg at krypto.org (Gregory P. Smith) Date: Sun, 16 Mar 2008 16:38:48 -0500 Subject: [Python-Dev] [Python-3000] 2.6 and 3.0 project management In-Reply-To: References: <1afaf6160803160919qbf49800n584ea4f965ebc6b2@mail.gmail.com> <1afaf6160803160951l7a580ee8i796bcfe77d424cf8@mail.gmail.com> Message-ID: <52dc1c820803161438wcb14acfw7e5fb2e6b482233d@mail.gmail.com> On 3/16/08, Guido van Rossum wrote: > > I don't see a lot of objections left against using the bug tracker. I > just talked to Neal and he's going to transfer all tasks from the 2.6 > spreadsheet to the bug tracker. > > I'll also be adding various other tasks., as I think of them. yay, thanks Neal! We'll have to think about which keywords to use. We'll probably pick > the initial set of keywords at the sprint tomorrow morning. > > -- > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/greg%40krypto.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080316/c176eeeb/attachment.htm From dickinsm at gmail.com Sun Mar 16 23:13:36 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Sun, 16 Mar 2008 18:13:36 -0400 Subject: [Python-Dev] 3.0 buildbots all red In-Reply-To: References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD173@EXMBX04.exchhosting.com> <47DD415F.7090109@v.loewis.de> <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD17D@EXMBX04.exchhosting.com> <47DD53C9.8000704@v.loewis.de> Message-ID: <5c6f2a5d0803161513r44050141m4888a3107ee6fc77@mail.gmail.com> On Sun, Mar 16, 2008 at 1:32 PM, Neal Norwitz wrote: > > I think this is possible, though considerable work. Probably the > biggest win will be creating a mock for socket and using mock sockets > in the tests for asyn{core,chat}, smtplib, xmlrpc, etc. That will fix > about 75% of the problems on 2.6. The remaining problems are: > > * test_asyn{chat,core} might not be meaningful with mock sockets and are flaky > * the alpha fails test_signal/socket for weird alarm conditions. > this might be hard to debug/fix (I have access to this box though) > * test_sqlite is broken on x86 with an old sqlite (I have access to this box) > * test_bsddb may be flaky, I'm not sure > * probably a few platform specific problems > test_tokenize is also currently (sometimes) failing on many of the bots. I've been looking into it, but I'm struggling to find the problem. The traceback e.g. for the amd64 gentoo buildbot ends with File "/home/buildbot/slave/py-build/3.0.norwitz-amd64/build/Lib/io.py", line 1081, in decode output = self.decoder.decode(input, final=final) File "/home/buildbot/slave/py-build/3.0.norwitz-amd64/build/Lib/codecs.py", line 291, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf8' codec can't decode bytes in position 12-15: invalid data On my own machine (SuSE 9.3/i686) I'm seeing this test pass about 80% of the time and fail the other 20% with something like the above, the position of the reported invalid data changing from run to run. It looks like data are getting corrupted somewhere along the line. Anyone have any ideas? Mark From stephen at xemacs.org Sun Mar 16 23:52:56 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 17 Mar 2008 07:52:56 +0900 Subject: [Python-Dev] [Python-3000] 2.6 and 3.0 project management In-Reply-To: <1afaf6160803160951l7a580ee8i796bcfe77d424cf8@mail.gmail.com> References: <1afaf6160803160919qbf49800n584ea4f965ebc6b2@mail.gmail.com> <1afaf6160803160951l7a580ee8i796bcfe77d424cf8@mail.gmail.com> Message-ID: <87myoyi8mv.fsf@uwakimon.sk.tsukuba.ac.jp> Benjamin Peterson writes: > It's just depends on how you see the tracker. It's not just to "bug" tracker > anymore, is it? On other projects I've worked with, we had separate areas > for bugs, features, and tasks. (yes, it's SourceForge.) I found it easier to > keep organized. However, if this is Python's way, I'm not going to stand in > it. You (as an ordinary user) can have it both ways in the same instance. If Martin adds a "task" issue type (which is easy to do in Roundup), then you personally can create and save queries for "task", "bug", "feature", etc. Your view of the database will then be more like sourceforge. On the other hand, cross-referencing and creating dependencies across issue types becomes a lot easier if they're in the same database. That's important for some issues. > > > We could change the statuses around to "Work in progress", "Completed", > > > "Incomplete", and such. It'd be easy to search for tasks that have to be > > > accomplished for a given release. I've done this for XEmacs's tracker. It's definitely feasible. I'm subscribed to tracker-discuss, so I'll not go into detail here. From nnorwitz at gmail.com Sun Mar 16 23:59:46 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 16 Mar 2008 17:59:46 -0500 Subject: [Python-Dev] 3.0 buildbots all red In-Reply-To: <5c6f2a5d0803161513r44050141m4888a3107ee6fc77@mail.gmail.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD173@EXMBX04.exchhosting.com> <47DD415F.7090109@v.loewis.de> <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD17D@EXMBX04.exchhosting.com> <47DD53C9.8000704@v.loewis.de> <5c6f2a5d0803161513r44050141m4888a3107ee6fc77@mail.gmail.com> Message-ID: On Sun, Mar 16, 2008 at 5:13 PM, Mark Dickinson wrote: > On Sun, Mar 16, 2008 at 1:32 PM, Neal Norwitz wrote: > > > > > I think this is possible, though considerable work. Probably the > > biggest win will be creating a mock for socket and using mock sockets > > in the tests for asyn{core,chat}, smtplib, xmlrpc, etc. That will fix > > about 75% of the problems on 2.6. The remaining problems are: > > > > * test_asyn{chat,core} might not be meaningful with mock sockets and are flaky > > * the alpha fails test_signal/socket for weird alarm conditions. > > this might be hard to debug/fix (I have access to this box though) > > * test_sqlite is broken on x86 with an old sqlite (I have access to this box) > > * test_bsddb may be flaky, I'm not sure > > * probably a few platform specific problems > > > > > test_tokenize is also currently (sometimes) failing on many of the bots. > I've been looking into it, but I'm struggling to find the problem. The > traceback e.g. for the amd64 gentoo buildbot ends with > > File "/home/buildbot/slave/py-build/3.0.norwitz-amd64/build/Lib/io.py", > line 1081, in decode > output = self.decoder.decode(input, final=final) > File "/home/buildbot/slave/py-build/3.0.norwitz-amd64/build/Lib/codecs.py", > line 291, in decode > (result, consumed) = self._buffer_decode(data, self.errors, final) > UnicodeDecodeError: 'utf8' codec can't decode bytes in position > 12-15: invalid data > > On my own machine (SuSE 9.3/i686) I'm seeing this test pass about 80% > of the time and fail the other 20% with something like the above, > the position of the reported invalid data changing from run to run. It > looks like data are getting corrupted somewhere along the line. > > Anyone have any ideas? Yeah, sounds like a memory issue. Did you try running with valgrind or purify? I haven't done so for a long time, perhaps never on 3k branch. It would be a good thing to run a tool soon. n From guido at python.org Mon Mar 17 00:13:00 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 16 Mar 2008 18:13:00 -0500 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) Message-ID: Phillip asked me to give an opinion on his pkg_resources PEP. While the PEP is short and sweet, the pkg_resources module itself is huge (1800 non-blank lines; 16 classes plus 5 exceptions; it exports 67 names in total according to __all__). And pkg_resources.txt is another 1700 lines of documentation. I find that hard to swallow. Is there anyone besides Phillip who can claim he understands this module? If its inclusion is really meant just as a bootstrap to simplify installing other package management solutions, as the PEP claims, I would prefer to see something with a much smaller footprint. Surely there is no need for example to have support for C extensions inside zip files *as part of the bootstrap module*? Unless I find someone besides Phillip who is interested in having this included and is willing to help maintain it, I don't really think it would be wise to accept this into the standard library. Phillip, in the PEP you mention that there are several other package management tools that also like to use pkg_resources. Maybe you can get some folks from those tools to speak up and explain what pkg_resources means to them, and maybe even volunteer to co-own it once it's in the standard library? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tnelson at onresolve.com Mon Mar 17 00:19:48 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Sun, 16 Mar 2008 16:19:48 -0700 Subject: [Python-Dev] 3.0 buildbots all red In-Reply-To: <5c6f2a5d0803161513r44050141m4888a3107ee6fc77@mail.gmail.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD173@EXMBX04.exchhosting.com> <47DD415F.7090109@v.loewis.de> <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD17D@EXMBX04.exchhosting.com> <47DD53C9.8000704@v.loewis.de> , <5c6f2a5d0803161513r44050141m4888a3107ee6fc77@mail.gmail.com> Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE16@EXMBX04.exchhosting.com> Yeah test_tokenize is weird, I've been looking into it as well. Here's a sample failure from a Windows buildbot: File "S:\buildbots\python\3.0.nelson-windows\build\lib\test\test_tokenize.py", line ?, in test.test_tokenize.__test__.doctests Failed example: for testfile in testfiles: if not roundtrip(open(testfile)): break else: True Exception raised: Traceback (most recent call last): File "S:\buildbots\python\3.0.nelson-windows\build\lib\doctest.py", line 1227, in __run compileflags, 1), test.globs) File "", line 2, in if not roundtrip(open(testfile)): break File "", line 3, in roundtrip token_list = list(generate_tokens(f.readline)) File "S:\buildbots\python\3.0.nelson-windows\build\lib\tokenize.py", line 264, in generate_tokens line = readline() File "S:\buildbots\python\3.0.nelson-windows\build\lib\io.py", line 1467, in readline readahead, pending = self._read_chunk() File "S:\buildbots\python\3.0.nelson-windows\build\lib\io.py", line 1278, in _read_chunk pending = self._decoder.decode(readahead, not readahead) File "S:\buildbots\python\3.0.nelson-windows\build\lib\io.py", line 1081, in decode output = self.decoder.decode(input, final=final) File "S:\buildbots\python\3.0.nelson-windows\build\lib\encodings\cp1252.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0] UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 17: character maps to The following is at the end of the doctests in test_tokenize: >>> tempdir = os.path.dirname(f) or os.curdir >>> testfiles = glob.glob(os.path.join(tempdir, "test*.py")) >>> if not test_support.is_resource_enabled("compiler"): ... testfiles = random.sample(testfiles, 10) ... >>> for testfile in testfiles: ... if not roundtrip(open(testfile)): break ... else: True True On that first line, 'f' is lib/test/tokenize_tests.txt, so basically, it's grabbing ten random test*.py files in lib/test and running untokenize(generate_tokens(f.readline)) on each one. In order to figure out which file it's dying on, I added the following to test_tokenize.py: def test_tokenize_all(): import glob import os tempdir = os.path.dirname(__file__) or os.curdir testfiles = glob.glob(os.path.join(tempdir, "test*.py")) for file in testfiles: print("processing file: " + file) print("roundtrip(open(file)): " + roundtrip(open(file))) This results in different results: Python 3.0a3+ (py3k, Mar 16 2008, 10:41:45) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> from test import test_tokenize [50808 refs] >>> test_tokenize.test_tokenize_all() processing file: s:\src\svn\svn.python.org\projects\python\branches\py3k\lib\test\testcodec.py Traceback (most recent call last): File "", line 1, in File "s:\src\svn\svn.python.org\projects\python\branches\py3k\lib\test\test_tokenize.py", line 565, in test_tokenize_all print("roundtrip(open(file)): " + roundtrip(open(file))) File "s:\src\svn\svn.python.org\projects\python\branches\py3k\lib\test\test_tokenize.py", line 514, in roundtrip source = untokenize(generate_tokens(f.readline)) File "s:\src\svn\svn.python.org\projects\python\branches\py3k\lib\tokenize.py", line 238, in untokenize return ut.untokenize(iterable) File "s:\src\svn\svn.python.org\projects\python\branches\py3k\lib\tokenize.py", line 183, in untokenize self.add_whitespace(start) File "s:\src\svn\svn.python.org\projects\python\branches\py3k\lib\tokenize.py", line 172, in add_whitespace assert row <= self.prev_row AssertionError [52668 refs] Yay. And to make this even more interesting: s:\src\svn\svn.python.org\projects\python\branches\py3k\PCbuild>python_d ..\Lib\test\test_tokenize.py doctest (test.test_tokenize) ... 62 tests with zero failures [61919 refs] Oh, and while we're here: s:\src\svn\svn.python.org\projects\python\branches\py3k\PCbuild>python_d ..\lib\test\regrtest.py -q -uall -rw test_tokenize ********************************************************************** File "s:\src\svn\svn.python.org\projects\python\branches\py3k\lib\test\test_tokenize.py", line ?, in test.test_tokenize.__test__.doc tests Failed example: for testfile in testfiles: if not roundtrip(open(testfile)): break else: True Exception raised: Traceback (most recent call last): File "s:\src\svn\svn.python.org\projects\python\branches\py3k\lib\doctest.py", line 1227, in __run compileflags, 1), test.globs) File "", line 2, in if not roundtrip(open(testfile)): break File "", line 3, in roundtrip token_list = list(generate_tokens(f.readline)) File "s:\src\svn\svn.python.org\projects\python\branches\py3k\lib\tokenize.py", line 264, in generate_tokens line = readline() File "s:\src\svn\svn.python.org\projects\python\branches\py3k\lib\io.py", line 1467, in readline readahead, pending = self._read_chunk() File "s:\src\svn\svn.python.org\projects\python\branches\py3k\lib\io.py", line 1278, in _read_chunk pending = self._decoder.decode(readahead, not readahead) File "s:\src\svn\svn.python.org\projects\python\branches\py3k\lib\io.py", line 1081, in decode output = self.decoder.decode(input, final=final) File "s:\src\svn\svn.python.org\projects\python\branches\py3k\lib\encodings\cp1252.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0] UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 17: character maps to ********************************************************************** 1 items had failures: 1 of 57 in test.test_tokenize.__test__.doctests ***Test Failed*** 1 failures. test test_tokenize failed -- 1 of 62 doctests failed 1 test failed: test_tokenize Trent. ________________________________________ From: python-dev-bounces+tnelson=onresolve.com at python.org [python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of Mark Dickinson [dickinsm at gmail.com] Sent: 16 March 2008 18:13 To: Python Dev Subject: Re: [Python-Dev] 3.0 buildbots all red On Sun, Mar 16, 2008 at 1:32 PM, Neal Norwitz wrote: > > I think this is possible, though considerable work. Probably the > biggest win will be creating a mock for socket and using mock sockets > in the tests for asyn{core,chat}, smtplib, xmlrpc, etc. That will fix > about 75% of the problems on 2.6. The remaining problems are: > > * test_asyn{chat,core} might not be meaningful with mock sockets and are flaky > * the alpha fails test_signal/socket for weird alarm conditions. > this might be hard to debug/fix (I have access to this box though) > * test_sqlite is broken on x86 with an old sqlite (I have access to this box) > * test_bsddb may be flaky, I'm not sure > * probably a few platform specific problems > test_tokenize is also currently (sometimes) failing on many of the bots. I've been looking into it, but I'm struggling to find the problem. The traceback e.g. for the amd64 gentoo buildbot ends with File "/home/buildbot/slave/py-build/3.0.norwitz-amd64/build/Lib/io.py", line 1081, in decode output = self.decoder.decode(input, final=final) File "/home/buildbot/slave/py-build/3.0.norwitz-amd64/build/Lib/codecs.py", line 291, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf8' codec can't decode bytes in position 12-15: invalid data On my own machine (SuSE 9.3/i686) I'm seeing this test pass about 80% of the time and fail the other 20% with something like the above, the position of the reported invalid data changing from run to run. It looks like data are getting corrupted somewhere along the line. Anyone have any ideas? Mark _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/tnelson%40onresolve.com From barry at python.org Mon Mar 17 00:46:50 2008 From: barry at python.org (Barry Warsaw) Date: Sun, 16 Mar 2008 18:46:50 -0500 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: References: <2C2D7EC7-51F2-4F4E-8127-87ACCC6F8413@python.org> Message-ID: <5E86F9D7-4421-4C57-8294-B0EC6A4EE03C@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 16, 2008, at 3:56 PM, Guido van Rossum wrote: > On Sun, Mar 16, 2008 at 12:24 PM, Barry Warsaw > wrote: >> I mentioned this to Guido and got a positive response, so let me >> state >> my preference for your feedback. I plan on holding up the final >> releases until both versions are ready to go. I think this will help >> motivate us to give Python 2.6 the love it needs if it's lagging >> behind 3.0, and I completely agree with Guido that this let's our >> community know that both versions are equally important to us. > > It's a deal. Excellent. >> The other thing is that I'd really like is a "show stoppers" Roundup >> search. The idea is that if our core buildbots look good and the >> "show stoppers" search turns up no items, then I know I can cut a >> release (at least for alphas, betas, and rcs). If there are "show >> stoppers" then I have something that I can triage (and maybe re- >> assign >> severity) or start publicly harassing people into fixing. > > How about using the "critical" Severity for show stoppers? 'critical' is fine (or 'immediate'). My problem before was that I couldn't do one query that gave me all the critical issues for both 2.6 and 3.0. That certainly could have been pebkac though. Neal mentioned that that kind of query should be possible, if it's not already there. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR92xb3EjvBPtnXfVAQLZGQP+O+FQwlXDXAT5lz+DKPer7+5n9ivy/YmD 94RYUnHEsVLA5aWpZB0O23/wVavS5FdUfJxuCvMOwHhZ6i58GHF4i6gfrtWDefX7 BWSfm82rIOAw/UX10JiUpkPp7vRlqfPdPOteqzFq0yo0vM49HOqFOL5fIU02MbRj unVdo8uYJ5c= =SB9I -----END PGP SIGNATURE----- From dickinsm at gmail.com Mon Mar 17 00:53:40 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Sun, 16 Mar 2008 19:53:40 -0400 Subject: [Python-Dev] 3.0 buildbots all red In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE16@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD173@EXMBX04.exchhosting.com> <47DD415F.7090109@v.loewis.de> <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD17D@EXMBX04.exchhosting.com> <47DD53C9.8000704@v.loewis.de> <5c6f2a5d0803161513r44050141m4888a3107ee6fc77@mail.gmail.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE16@EXMBX04.exchhosting.com> Message-ID: <5c6f2a5d0803161653l7d791476sf153a728309f6639@mail.gmail.com> On Sun, Mar 16, 2008 at 7:19 PM, Trent Nelson wrote: > Yeah test_tokenize is weird, I've been looking into it as well. Here's a sample failure from a Windows buildbot: > [failure log snipped...] > On that first line, 'f' is lib/test/tokenize_tests.txt, so basically, it's grabbing ten random test*.py files in lib/test and running untokenize(generate_tokens(f.readline)) on each one. In order to figure out which file it's dying on, I added the following to test_tokenize.py: Aha. This is very helpful! It explains the randomness of the failures, at least. So it's probably not a C-level data corruption bug after all. test_shlex seems to be one of the problem files. Investigations are continuing. Trent, thanks for your help! I'll take further discussions off line and spare python-dev the gory details... Mark From pje at telecommunity.com Mon Mar 17 01:06:24 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 16 Mar 2008 20:06:24 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: Message-ID: <20080317000630.735823A40B0@sparrow.telecommunity.com> Quick summary of the below: I'm definitely fine with doing a simpler, pure-bootstrap module, if there's some consensus on what should go in it. I just wish we could've had this discussion last year, when OSAF was still able to fund the work... ;-) At 06:13 PM 3/16/2008 -0500, Guido van Rossum wrote: >Phillip asked me to give an opinion on his pkg_resources PEP. While >the PEP is short and sweet, the pkg_resources module itself is huge >(1800 non-blank lines; 16 classes plus 5 exceptions; it exports 67 >names in total according to __all__). And pkg_resources.txt is another >1700 lines of documentation. I find that hard to swallow. Is there >anyone besides Phillip who can claim he understands this module? Bob Ippolito actually wrote the very first version of pkg_resources. Others, such as Philip Jenvey of the Jython project, have provided patches. From previous discussions on the distutils-sig, I know that Jim Fulton has in-depth knowledge of both pkg_resources and easy_install. Of course, that's not the same as any of these guys volunteering to be maintainers. :) >If its inclusion is really meant just as a bootstrap to simplify >installing other package management solutions, as the PEP claims, I >would prefer to see something with a much smaller footprint. Actually, the PEP says: "pkg_resources is a module used to find and manage Python package/version dependencies and access bundled files and resources, including those inside of zipped .egg files. Currently, pkg_resources is only available through installing the entire setuptools distribution, but it does not depend on any other part of setuptools; in effect, it comprises the entire runtime support library for Python Eggs, and is independently useful." This kind of glosses over the part where this is also for runtime support of projects that use eggs. Which, these days, is, well, almost any large Python project, from Chandler to Enthought to Zope. > Surely >there is no need for example to have support for C extensions inside >zip files *as part of the bootstrap module*? It's a runtime; the PEP actually merely proposes that a further addition to be made to support bootstrapping, *also*. Otherwise, the PEP would be even shorter. :) The reason I proposed it this way was for simplicity -- and politics. Currently, people using setuptools in their setup.py have to include a similar bootstrap module to download setuptools if it's not available, and pkg_resources already has version checking logic and everything needed to find dependencies and download them. (Plus, I figured it'd be easier to just use what was already there and stable, rather than creating something different.) That was the simplicity part. The politics part was that: 1. I thought it would be less controversial to include the "runtime for eggs" than to include something that's just a bootstrapper for setuptools. However, MvL surprised me by actually being in *favor* of including a setuptools bootstrapper. 2. I thought that it would have broader acceptance if it was oriented towards bootstrapping *any* package, not just setuptools. So, if the consensus is that it would be better to have a module that only does bootstrap installs of pure-Python eggs from PyPI, I'm totally fine with that. >Unless I find someone besides Phillip who is interested in having this >included and is willing to help maintain it, I don't really think it >would be wise to accept this into the standard library. > >Phillip, in the PEP you mention that there are several other package >management tools that also like to use pkg_resources. Maybe you can >get some folks from those tools to speak up and explain what >pkg_resources means to them, and maybe even volunteer to co-own it >once it's in the standard library? The distutils-sig is the de facto place for discussions regarding those tools, so I've cc'd this there. Hopefully, one or more volunteers will step up if they want this. From eikeon at eikeon.com Mon Mar 17 02:16:22 2008 From: eikeon at eikeon.com (Daniel Krech) Date: Sun, 16 Mar 2008 21:16:22 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <20080317000630.735823A40B0@sparrow.telecommunity.com> References: <20080317000630.735823A40B0@sparrow.telecommunity.com> Message-ID: On Mar 16, 2008, at 8:06 PM, Phillip J. Eby wrote: > Quick summary of the below: I'm definitely fine with doing a simpler, > pure-bootstrap module, if there's some consensus on what should go in > it. I just wish we could've had this discussion last year, when OSAF > was still able to fund the work... ;-) > > > At 06:13 PM 3/16/2008 -0500, Guido van Rossum wrote: >> Phillip asked me to give an opinion on his pkg_resources PEP. While >> the PEP is short and sweet, the pkg_resources module itself is huge >> (1800 non-blank lines; 16 classes plus 5 exceptions; it exports 67 >> names in total according to __all__). And pkg_resources.txt is >> another >> 1700 lines of documentation. I find that hard to swallow. Is there >> anyone besides Phillip who can claim he understands this module? > > Bob Ippolito actually wrote the very first version of > pkg_resources. Others, such as Philip Jenvey of the Jython project, > have provided patches. From previous discussions on the > distutils-sig, I know that Jim Fulton has in-depth knowledge of both > pkg_resources and easy_install. > > Of course, that's not the same as any of these guys volunteering to > be maintainers. :) > > >> If its inclusion is really meant just as a bootstrap to simplify >> installing other package management solutions, as the PEP claims, I >> would prefer to see something with a much smaller footprint. > > Actually, the PEP says: > > "pkg_resources is a module used to find and manage Python > package/version dependencies and access bundled files and resources, > including those inside of zipped .egg files. Currently, pkg_resources > is only available through installing the entire setuptools > distribution, but it does not depend on any other part of setuptools; > in effect, it comprises the entire runtime support library for Python > Eggs, and is independently useful." > > This kind of glosses over the part where this is also for runtime > support of projects that use eggs. Which, these days, is, well, > almost any large Python project, from Chandler to Enthought to Zope. > > >> Surely >> there is no need for example to have support for C extensions inside >> zip files *as part of the bootstrap module*? > > It's a runtime; the PEP actually merely proposes that a further > addition to be made to support bootstrapping, *also*. Otherwise, the > PEP would be even shorter. :) > > The reason I proposed it this way was for simplicity -- and politics. > > Currently, people using setuptools in their setup.py have to include > a similar bootstrap module to download setuptools if it's not > available, and pkg_resources already has version checking logic and > everything needed to find dependencies and download them. (Plus, I > figured it'd be easier to just use what was already there and stable, > rather than creating something different.) > > That was the simplicity part. The politics part was that: > > 1. I thought it would be less controversial to include the "runtime > for eggs" than to include something that's just a bootstrapper for > setuptools. However, MvL surprised me by actually being in *favor* > of including a setuptools bootstrapper. > > 2. I thought that it would have broader acceptance if it was oriented > towards bootstrapping *any* package, not just setuptools. > > So, if the consensus is that it would be better to have a module that > only does bootstrap installs of pure-Python eggs from PyPI, I'm > totally fine with that. > > >> Unless I find someone besides Phillip who is interested in having >> this >> included and is willing to help maintain it, I don't really think it >> would be wise to accept this into the standard library. >> >> Phillip, in the PEP you mention that there are several other package >> management tools that also like to use pkg_resources. Maybe you can >> get some folks from those tools to speak up and explain what >> pkg_resources means to them, and maybe even volunteer to co-own it >> once it's in the standard library? > > The distutils-sig is the de facto place for discussions regarding > those tools, so I've cc'd this there. Hopefully, one or more > volunteers will step up if they want this. I'd like to see it in and am willing to help maintain it. Especially if it only does the bootstrap installs of pure-Python egg bit. I've dug into pkg_resource some, but can't claim to understand it all. From martin at v.loewis.de Mon Mar 17 05:27:04 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 16 Mar 2008 23:27:04 -0500 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: <5E86F9D7-4421-4C57-8294-B0EC6A4EE03C@python.org> References: <2C2D7EC7-51F2-4F4E-8127-87ACCC6F8413@python.org> <5E86F9D7-4421-4C57-8294-B0EC6A4EE03C@python.org> Message-ID: <47DDF318.4080303@v.loewis.de> > 'critical' is fine (or 'immediate'). My problem before was that I > couldn't do one query that gave me all the critical issues for both > 2.6 and 3.0. That certainly could have been pebkac though. Neal > mentioned that that kind of query should be possible, if it's not > already there. I just created a "Showstoppers" public query (go to Your Queries/*edit*, and set it to "leave in"). This *just* searches for critical open issues, all versions. Given that there are currently really no critical issues, that semantically is the same as further restricting it to 2.6 and 3.0. Creating a query that searches for multiple versions is possible through URL editing, but not the form (currently). I'm not sure whether that would search for issues which are marked both 2.6 and 3.0, or for issues that are either marked 2.6 *or* 3.0. Regards, Martin From ocean at m2.ccsnet.ne.jp Mon Mar 17 06:34:31 2008 From: ocean at m2.ccsnet.ne.jp (ocean) Date: Mon, 17 Mar 2008 14:34:31 +0900 Subject: [Python-Dev] 3.0 buildbots all red References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD173@EXMBX04.exchhosting.com><47DD415F.7090109@v.loewis.de><87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD17D@EXMBX04.exchhosting.com><47DD53C9.8000704@v.loewis.de><5c6f2a5d0803161513r44050141m4888a3107ee6fc77@mail.gmail.com> Message-ID: <001101c887f0$93d63910$0300a8c0@whiterabc2znlh> > Yeah, sounds like a memory issue. Did you try running with valgrind > or purify? I haven't done so for a long time, perhaps never on 3k > branch. It would be a good thing to run a tool soon. Maybe is this related? [Potential overflows due to incorrect usage of PyUnicode_AsString] http://bugs.python.org/issue1950 Thank you. From tnelson at onresolve.com Mon Mar 17 07:14:05 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Sun, 16 Mar 2008 23:14:05 -0700 Subject: [Python-Dev] 3.0 buildbots all red In-Reply-To: <001101c887f0$93d63910$0300a8c0@whiterabc2znlh> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD173@EXMBX04.exchhosting.com><47DD415F.7090109@v.loewis.de><87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD17D@EXMBX04.exchhosting.com><47DD53C9.8000704@v.loewis.de><5c6f2a5d0803161513r44050141m4888a3107ee6fc77@mail.gmail.com> , <001101c887f0$93d63910$0300a8c0@whiterabc2znlh> Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE18@EXMBX04.exchhosting.com> As it turns out, it's not memory related, but has to do with tokenize not supporting coding cookies in files. Mark picked up on this and linked it to an issue already in roundup that was raised way back in 2003: http://bugs.python.org/issue71988. I've just finished patching test_tokenizer.py to better represent this test case -- the current implementation doesn't lend itself very well to being debugged when things go wrong (I think Mark and I both felt like we were on a bit of a wild goose chase). I've fixed that and have a bunch of text files with various utf-8/bom sig permutations that are now used to test tokenizer's compliance with PEP 0263. I'll upload that now then move on to actually patching tokenizer.py. Trent "wishes-there-was-somewhere-to-get-some-food-after-11pm-at-pycon" Nelson. ________________________________________ From: python-dev-bounces+tnelson=onresolve.com at python.org [python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of ocean [ocean at m2.ccsnet.ne.jp] Sent: 17 March 2008 01:34 To: Neal Norwitz; Mark Dickinson Cc: Python Dev Subject: Re: [Python-Dev] 3.0 buildbots all red > Yeah, sounds like a memory issue. Did you try running with valgrind > or purify? I haven't done so for a long time, perhaps never on 3k > branch. It would be a good thing to run a tool soon. Maybe is this related? [Potential overflows due to incorrect usage of PyUnicode_AsString] http://bugs.python.org/issue1950 Thank you. _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/tnelson%40onresolve.com From tnelson at onresolve.com Mon Mar 17 07:51:23 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Sun, 16 Mar 2008 23:51:23 -0700 Subject: [Python-Dev] 3.0 buildbots all red In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE18@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD173@EXMBX04.exchhosting.com><47DD415F.7090109@v.loewis.de><87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD17D@EXMBX04.exchhosting.com><47DD53C9.8000704@v.loewis.de><5c6f2a5d0803161513r44050141m4888a3107ee6fc77@mail.gmail.com> , <001101c887f0$93d63910$0300a8c0@whiterabc2znlh>, <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE18@EXMBX04.exchhosting.com> Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE19@EXMBX04.exchhosting.com> > As it turns out, it's not memory related, but has to do with > tokenize not supporting coding cookies in files. > Mark picked up on this and linked it to an issue already > in roundup that was raised way back in 2003: > http://bugs.python.org/issue71988. Oops, left off an 8. That's meant to read http://bugs.python.org/issue719888. From guido at python.org Mon Mar 17 14:48:51 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 17 Mar 2008 08:48:51 -0500 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <20080317000630.735823A40B0@sparrow.telecommunity.com> References: <20080317000630.735823A40B0@sparrow.telecommunity.com> Message-ID: On Sun, Mar 16, 2008 at 7:06 PM, Phillip J. Eby wrote: > So, if the consensus is that it would be better to have a module that > only does bootstrap installs of pure-Python eggs from PyPI, I'm > totally fine with that. Let's just do this; it will avoid a protracted discussion of the merits of eggs, pkg_resources, and setuptools. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Mon Mar 17 15:19:11 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 17 Mar 2008 10:19:11 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317000630.735823A40B0@sparrow.telecommunity.com> Message-ID: <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> At 08:48 AM 3/17/2008 -0500, Guido van Rossum wrote: >On Sun, Mar 16, 2008 at 7:06 PM, Phillip J. Eby wrote: > > So, if the consensus is that it would be better to have a module that > > only does bootstrap installs of pure-Python eggs from PyPI, I'm > > totally fine with that. > >Let's just do this; it will avoid a protracted discussion of the >merits of eggs, pkg_resources, and setuptools. Well, it might be replaced by a protracted discussion of how the module should work and what its API should be, but perhaps that would be a better one to have. :) So, the original proposal (from the previous thread about this) was that the module be named easy_install, and that it simply downloads setuptools and delegates to the "real" easy_install. That way, people can simply use "python -m easy_install ...", without worrying about whether setuptools has been installed yet. IIRC, other package management tools such as zc.buildout and workingenv can then be installed using easy_install. Any objections? Should I revise the PEP? From barry at python.org Mon Mar 17 15:27:23 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 17 Mar 2008 09:27:23 -0500 Subject: [Python-Dev] [Python-3000] 2.6 and 3.0 project management In-Reply-To: <5E86F9D7-4421-4C57-8294-B0EC6A4EE03C@python.org> References: <2C2D7EC7-51F2-4F4E-8127-87ACCC6F8413@python.org> <5E86F9D7-4421-4C57-8294-B0EC6A4EE03C@python.org> Message-ID: <81BCED86-19FC-4BDC-9A2D-136FE484B3A6@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 16, 2008, at 6:46 PM, Barry Warsaw wrote: > > 'critical' is fine (or 'immediate'). My problem before was that I > couldn't do one query that gave me all the critical issues for both > 2.6 and 3.0. That certainly could have been pebkac though. Neal > mentioned that that kind of query should be possible, if it's not > already there. So, I just looked again and that wasn't quite my problem. When you search, it seems like you have a choice of version "don't care" or you can pick a single version. But it looks like once you're on the search results you can further refine the version filter via the list box. It's a little clunky, but it'll work. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR95/y3EjvBPtnXfVAQKlmgQAg+1azln0Ljb32iyoreALUwepHwrN0XLp Fg6OVPp70iYIhctFhT3QYAN+59wzqy8x2PKsuZV/k48ebsJWwLsbU1yHH0WImHoo 56Dso3sfbsj2GzK7i3cF903RiIVE4geQRbnttDnrQVmI9U3jrs9iyWMjw/5Znohz DtLTEEf2fQs= =mQXt -----END PGP SIGNATURE----- From martin at v.loewis.de Mon Mar 17 15:45:25 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 17 Mar 2008 09:45:25 -0500 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> References: <20080317000630.735823A40B0@sparrow.telecommunity.com> <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> Message-ID: <47DE8405.2030201@v.loewis.de> > Well, it might be replaced by a protracted discussion of how the > module should work and what its API should be, but perhaps that would > be a better one to have. :) Indeed, that's likely to happen :-) > So, the original proposal (from the previous thread about this) was > that the module be named easy_install, and that it simply downloads > setuptools and delegates to the "real" easy_install. That way, > people can simply use "python -m easy_install ...", without worrying > about whether setuptools has been installed yet. I thought the original proposal was to install a *binary* easy_install that takes that function. > IIRC, other package management tools such as zc.buildout and > workingenv can then be installed using easy_install. > > Any objections? Should I revise the PEP? I'm fine with the module, but would really like to see a command line utility in addition. This, of course, would raise the issue who "owns" the easy_install script name; ideally, the script would not have to be overwritten when setuptools gets installed. Regards, Martin From pje at telecommunity.com Mon Mar 17 16:10:36 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 17 Mar 2008 11:10:36 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <47DE8405.2030201@v.loewis.de> References: <20080317000630.735823A40B0@sparrow.telecommunity.com> <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> Message-ID: <20080317151637.532D23A409D@sparrow.telecommunity.com> At 09:45 AM 3/17/2008 -0500, Martin v. L?wis wrote: > > Well, it might be replaced by a protracted discussion of how the > > module should work and what its API should be, but perhaps that would > > be a better one to have. :) > >Indeed, that's likely to happen :-) > > > So, the original proposal (from the previous thread about this) was > > that the module be named easy_install, and that it simply downloads > > setuptools and delegates to the "real" easy_install. That way, > > people can simply use "python -m easy_install ...", without worrying > > about whether setuptools has been installed yet. > >I thought the original proposal was to install a *binary* easy_install >that takes that function. What do you mean by "binary"? I thought we were talking about a module. Do you mean a script to be installed alongside Python itself in e.g. /usr/bin? In the original discussion, it was a module to be added alongside pkg_resources, which would use pkg_resources to find and/or install setuptools. I also personally like the use of -m instead of a script because it makes it quite clear that this is a Python-specific installation tool, and *which* version of Python, as well, without having to have easy_install-2.5, easy_install-2.6, etc. > > IIRC, other package management tools such as zc.buildout and > > workingenv can then be installed using easy_install. > > > > Any objections? Should I revise the PEP? > >I'm fine with the module, but would really like to see a command >line utility in addition. > >This, of course, would raise the issue who "owns" the easy_install >script name; ideally, the script would not have to be overwritten >when setuptools gets installed. It won't have to. The module will attempt to import the setuptools-supplied version of easy_install, and delegate to it if possible, before trying to do any download. From martin at v.loewis.de Mon Mar 17 16:51:28 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 17 Mar 2008 10:51:28 -0500 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <20080317151637.532D23A409D@sparrow.telecommunity.com> References: <20080317000630.735823A40B0@sparrow.telecommunity.com> <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> Message-ID: <47DE9380.5000809@v.loewis.de> >> I thought the original proposal was to install a *binary* easy_install >> that takes that function. > > What do you mean by "binary"? I thought we were talking about a > module. Do you mean a script to be installed alongside Python itself in > e.g. /usr/bin? Exactly so. > In the original discussion, it was a module to be added alongside > pkg_resources, which would use pkg_resources to find and/or install > setuptools. I also personally like the use of -m instead of a script > because it makes it quite clear that this is a Python-specific > installation tool, and *which* version of Python, as well, without > having to have easy_install-2.5, easy_install-2.6, etc. If that becomes the official interface to easy_install, that's fine with me. I'm worried about web instructions that tell people that there is an "easy_install" utility, so that people never find out the module actually exists. Regards, Martin From guido at python.org Mon Mar 17 16:53:21 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 17 Mar 2008 10:53:21 -0500 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <20080317151637.532D23A409D@sparrow.telecommunity.com> References: <20080317000630.735823A40B0@sparrow.telecommunity.com> <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> Message-ID: I don't think this should play games with scripts being overridden or whatever. If a bootstrap script is to be installed it should have a separate name. I'm not sure what the advantage is of a bootstrap script over "python -m bootstrap_module ..." though. The PEP suggests that other package managers also benefit. How do they benefit if the bootstrap script installs setuptools? That sounds like the bootstrap script would make setuptools the de-facto standard, which I'd like to avoid given the politics around this topic. I'd also like to avoid the specific name "easy_install" for any of this. That's a "brand name" (and a misleading one if you ask me, but that's politics again :-). --Guido On Mon, Mar 17, 2008 at 10:10 AM, Phillip J. Eby wrote: > At 09:45 AM 3/17/2008 -0500, Martin v. L?wis wrote: > > > Well, it might be replaced by a protracted discussion of how the > > > module should work and what its API should be, but perhaps that would > > > be a better one to have. :) > > > >Indeed, that's likely to happen :-) > > > > > So, the original proposal (from the previous thread about this) was > > > that the module be named easy_install, and that it simply downloads > > > setuptools and delegates to the "real" easy_install. That way, > > > people can simply use "python -m easy_install ...", without worrying > > > about whether setuptools has been installed yet. > > > >I thought the original proposal was to install a *binary* easy_install > >that takes that function. > > What do you mean by "binary"? I thought we were talking about a > module. Do you mean a script to be installed alongside Python itself > in e.g. /usr/bin? > > In the original discussion, it was a module to be added alongside > pkg_resources, which would use pkg_resources to find and/or install > setuptools. I also personally like the use of -m instead of a script > because it makes it quite clear that this is a Python-specific > installation tool, and *which* version of Python, as well, without > having to have easy_install-2.5, easy_install-2.6, etc. > > > > > > IIRC, other package management tools such as zc.buildout and > > > workingenv can then be installed using easy_install. > > > > > > Any objections? Should I revise the PEP? > > > >I'm fine with the module, but would really like to see a command > >line utility in addition. > > > >This, of course, would raise the issue who "owns" the easy_install > >script name; ideally, the script would not have to be overwritten > >when setuptools gets installed. > > It won't have to. The module will attempt to import the > setuptools-supplied version of easy_install, and delegate to it if > possible, before trying to do any download. > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Mon Mar 17 17:12:45 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 17 Mar 2008 12:12:45 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317000630.735823A40B0@sparrow.telecommunity.com> <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> Message-ID: <20080317161305.22B183A409D@sparrow.telecommunity.com> At 10:53 AM 3/17/2008 -0500, Guido van Rossum wrote: >I don't think this should play games with scripts being overridden or >whatever. If a bootstrap script is to be installed it should have a >separate name. I'm not sure what the advantage is of a bootstrap >script over "python -m bootstrap_module ..." though. And -m also makes explicit: 1. that it's a Python-specific tool 2. which Python version it will apply to >The PEP suggests that other package managers also benefit. How do they >benefit if the bootstrap script installs setuptools? Because those other package managers depend, in fact, on setuptools, or at least pkg_resources... which was why the original proposal was to just include pkg_resources in the first place. :) >I'd also like to avoid the specific name "easy_install" for any of >this. That's a "brand name" (and a misleading one if you ask me, but >that's politics again :-). Ok, so if someone will propose a name and API for the thing, I'll implement it. (Assuming the proposed API is sane and reasonably implementable, of course.) From guido at python.org Mon Mar 17 17:31:59 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 17 Mar 2008 11:31:59 -0500 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <20080317161305.22B183A409D@sparrow.telecommunity.com> References: <20080317000630.735823A40B0@sparrow.telecommunity.com> <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> Message-ID: On Mon, Mar 17, 2008 at 11:12 AM, Phillip J. Eby wrote: > At 10:53 AM 3/17/2008 -0500, Guido van Rossum wrote: > >I don't think this should play games with scripts being overridden or > >whatever. If a bootstrap script is to be installed it should have a > >separate name. I'm not sure what the advantage is of a bootstrap > >script over "python -m bootstrap_module ..." though. > > And -m also makes explicit: > > 1. that it's a Python-specific tool > 2. which Python version it will apply to Right! > >The PEP suggests that other package managers also benefit. How do they > >benefit if the bootstrap script installs setuptools? > > Because those other package managers depend, in fact, on setuptools, > or at least pkg_resources... which was why the original proposal was > to just include pkg_resources in the first place. :) On how much of pkg_resources do they depend? Or is that an unanswerable question? > >I'd also like to avoid the specific name "easy_install" for any of > >this. That's a "brand name" (and a misleading one if you ask me, but > >that's politics again :-). > > Ok, so if someone will propose a name and API for the thing, I'll > implement it. (Assuming the proposed API is sane and reasonably > implementable, of course.) Perhaps it can be called package_bootstrap? I don't know enough about the required functionality to propose an API, but here goes anyway. Would be reasonable for it to have a command line that allows you to specify a package name, optionally a version string, and (very) optionally a server to contact (specified as an URL?). It should default to the highest non-experimental version that's applicable to the current Python version (assuming there's an easy way to determine this; I don't want it to try and download different versions until one works). It should not handle any kind of dependency management or package management administration. It should be able to download a Python-only module or package and install it into site-packages (or perhaps elsewhere if so directed via another optional command line flag). It should support zip, tar and tar.gz/tgz files (and perhaps tar.bz2). It should simply unpack the zip/tar file using the zip or tar modules, and extract the package/module into site-packages in such a way that it can be imported directly without messing with sys.path. It should not mess with .pth files, setup.py scripts, or eggs. If the contents of the tar/zip file has a toplevel directory with version numbers in its name (e.g. Django-0.96.1) it should skip that and just use the subdirectory whose name is the "pure" package name (e.g. django). Does this make sense? I'm happy to take push-back, this is just something I cooked up off the top of my head based on my experience with manually installing packages. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From p.f.moore at gmail.com Mon Mar 17 17:35:24 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 17 Mar 2008 16:35:24 +0000 Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module) In-Reply-To: <20080317161305.22B183A409D@sparrow.telecommunity.com> References: <20080317000630.735823A40B0@sparrow.telecommunity.com> <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> Message-ID: <79990c6b0803170935v59129ac8k3b0ff4ece23fe591@mail.gmail.com> On 17/03/2008, Phillip J. Eby wrote: > >The PEP suggests that other package managers also benefit. How do they > >benefit if the bootstrap script installs setuptools? > > Because those other package managers depend, in fact, on setuptools, > or at least pkg_resources... which was why the original proposal was > to just include pkg_resources in the first place. :) I'm puzzled. We seem to be talking about adding a module to the stdlib whose basic function is to download and install setuptools? How is this better than just including setuptools in the stdlib? Personally, I have no problem per se with including setuptools in the stdlib. Maybe that means I miss the subtle benefit of this approach... I'm -1 on having a module which just installs setuptools. I'm +0 on including pkg_resources (as described in PEP 365) in the stdlib. I'm +lots on someone giving a clear explanation of the meaning and interrelationship of the various terms involved in this discussion (setuptools, easy_install, pkg_resources, eggs, "package managers" as distinct from setuptools, etc etc) so that the discussion gets some much-needed clarity :-( I'm -1 on adding anything until PEP 365 is updated to match what is being proposed, and then that amended PEP is submitted for discussion. Paul. From guido at python.org Mon Mar 17 17:40:21 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 17 Mar 2008 11:40:21 -0500 Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module) In-Reply-To: <79990c6b0803170935v59129ac8k3b0ff4ece23fe591@mail.gmail.com> References: <20080317000630.735823A40B0@sparrow.telecommunity.com> <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <79990c6b0803170935v59129ac8k3b0ff4ece23fe591@mail.gmail.com> Message-ID: On Mon, Mar 17, 2008 at 11:35 AM, Paul Moore wrote: > On 17/03/2008, Phillip J. Eby wrote: > > >The PEP suggests that other package managers also benefit. How do they > > >benefit if the bootstrap script installs setuptools? > > > > Because those other package managers depend, in fact, on setuptools, > > or at least pkg_resources... which was why the original proposal was > > to just include pkg_resources in the first place. :) > > I'm puzzled. We seem to be talking about adding a module to the stdlib > whose basic function is to download and install setuptools? How is > this better than just including setuptools in the stdlib? I'm not in favor of this either. > Personally, I have no problem per se with including setuptools in the > stdlib. Maybe that means I miss the subtle benefit of this approach... > > I'm -1 on having a module which just installs setuptools. > I'm +0 on including pkg_resources (as described in PEP 365) in the stdlib. > > I'm +lots on someone giving a clear explanation of the meaning and > interrelationship of the various terms involved in this discussion > (setuptools, easy_install, pkg_resources, eggs, "package managers" as > distinct from setuptools, etc etc) so that the discussion gets some > much-needed clarity :-( Right. But finding someone who can explain all this is apparently hard. All the owners of package managers seem busy... > I'm -1 on adding anything until PEP 365 is updated to match what is > being proposed, and then that amended PEP is submitted for discussion. Well, if we can reach consensus here first on what to put into PEP 365 first I'd be fine with updating the PEP as an afterthought, especially of the consensus also comes with working code (hopefully less than 2500 lines :-). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Mon Mar 17 17:48:09 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 17 Mar 2008 11:48:09 -0500 Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module) In-Reply-To: <79990c6b0803170935v59129ac8k3b0ff4ece23fe591@mail.gmail.com> References: <20080317000630.735823A40B0@sparrow.telecommunity.com> <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <79990c6b0803170935v59129ac8k3b0ff4ece23fe591@mail.gmail.com> Message-ID: <47DEA0C9.1020300@v.loewis.de> > I'm puzzled. We seem to be talking about adding a module to the stdlib > whose basic function is to download and install setuptools? How is > this better than just including setuptools in the stdlib? I can do a review of such a module in an hour. To review setuptools (which I would have to do if it were to be included), I would likely need a week or so, full time. > Personally, I have no problem per se with including setuptools in the > stdlib. Maybe that means I miss the subtle benefit of this approach... Did you review setuptools and can vouch that it is written cleanly, follows the coding style, has correct documentation, and so on? Regards, Martin From stefan_ml at behnel.de Mon Mar 17 17:55:17 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 17 Mar 2008 17:55:17 +0100 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317000630.735823A40B0@sparrow.telecommunity.com> <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> Message-ID: <47DEA275.4080306@behnel.de> Guido van Rossum wrote: > It should be able to download a Python-only module or package and > install it into site-packages (or perhaps elsewhere if so directed via > another optional command line flag). It should support zip, tar and > tar.gz/tgz files (and perhaps tar.bz2). It should simply unpack the > zip/tar file using the zip or tar modules, and extract the > package/module into site-packages in such a way that it can be > imported directly without messing with sys.path. It should not mess > with .pth files, setup.py scripts, or eggs. Do you mean "existing eggs" or does that include the (potential .egg) package that is being installed? If I understood correctly, this bootstrap module currently supports installing eggs (although I'm not sure how they are supposed to work without the current way of keeping a .pth file). Is it *wanted* that eggs are being supported by core Python? Stefan From p.f.moore at gmail.com Mon Mar 17 18:12:40 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 17 Mar 2008 17:12:40 +0000 Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module) In-Reply-To: <47DEA0C9.1020300@v.loewis.de> References: <20080317000630.735823A40B0@sparrow.telecommunity.com> <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <79990c6b0803170935v59129ac8k3b0ff4ece23fe591@mail.gmail.com> <47DEA0C9.1020300@v.loewis.de> Message-ID: <79990c6b0803171012uaae8e9cmf6e0f4f189fa7ade@mail.gmail.com> On 17/03/2008, "Martin v. L?wis" wrote: > > Personally, I have no problem per se with including setuptools in the > > stdlib. Maybe that means I miss the subtle benefit of this approach... > > Did you review setuptools and can vouch that it is written cleanly, > follows the coding style, has correct documentation, and so on? No, I concede that when I say "I have no problem" with including setuptools, I'm assuming that someone does that review - and there's no reason to assume that anyone will do that (that's why I say "I have no problem with" rather than "I want"). But I don't see any practical difference with including setuptools and including a module that installs setuptools. Would you be happy with the standard library including a module whose sole function was to install a package that you weren't happy to include directly in the standard library? I wouldn't, personally. Paul From guido at python.org Mon Mar 17 18:17:49 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 17 Mar 2008 12:17:49 -0500 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <47DEA275.4080306@behnel.de> References: <20080317000630.735823A40B0@sparrow.telecommunity.com> <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <47DEA275.4080306@behnel.de> Message-ID: On Mon, Mar 17, 2008 at 11:55 AM, Stefan Behnel wrote: > Guido van Rossum wrote: > > It should be able to download a Python-only module or package and > > install it into site-packages (or perhaps elsewhere if so directed via > > another optional command line flag). It should support zip, tar and > > tar.gz/tgz files (and perhaps tar.bz2). It should simply unpack the > > zip/tar file using the zip or tar modules, and extract the > > package/module into site-packages in such a way that it can be > > imported directly without messing with sys.path. It should not mess > > with .pth files, setup.py scripts, or eggs. > > Do you mean "existing eggs" or does that include the (potential .egg) package > that is being installed? The latter. > If I understood correctly, this bootstrap module > currently supports installing eggs (although I'm not sure how they are > supposed to work without the current way of keeping a .pth file). My proposal is that the boostrap module only be able to install non-egg archives. > Is it *wanted* that eggs are being supported by core Python? No. There will be no egg support in the standard library. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Mar 17 18:18:41 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 17 Mar 2008 12:18:41 -0500 Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module) In-Reply-To: <79990c6b0803171012uaae8e9cmf6e0f4f189fa7ade@mail.gmail.com> References: <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <79990c6b0803170935v59129ac8k3b0ff4ece23fe591@mail.gmail.com> <47DEA0C9.1020300@v.loewis.de> <79990c6b0803171012uaae8e9cmf6e0f4f189fa7ade@mail.gmail.com> Message-ID: On Mon, Mar 17, 2008 at 12:12 PM, Paul Moore wrote: > But I don't see any practical difference with including setuptools and > including a module that installs setuptools. Would you be happy with > the standard library including a module whose sole function was to > install a package that you weren't happy to include directly in the > standard library? I wouldn't, personally. Actually it's quite different -- the user or administrator is in control and they can choose not to install setuptools. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Mon Mar 17 18:28:28 2008 From: brett at python.org (Brett Cannon) Date: Mon, 17 Mar 2008 12:28:28 -0500 Subject: [Python-Dev] Currently adding known 2.6 showstoppers to the tracker Message-ID: Right now at the sprint I am going through a list of issues Neal and I compiled of what needs to happen to get 2.6/3.0 out the door (although the list is pretty much 2.6-specific). They are all being flagged as "immediate" priority. Hopefully it won't take too long to add all of them (although the list ain't short =). If you want to see the list, Martin added a pre-defined Showstoppers search to make it easy to list everything that is set to "immediate". -Brett From guido at python.org Mon Mar 17 18:35:43 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 17 Mar 2008 12:35:43 -0500 Subject: [Python-Dev] Currently adding known 2.6 showstoppers to the tracker In-Reply-To: References: Message-ID: Great! I wonder though if these should really all be given "showstopper" priority. IMO things don't reach showstopper status until they are blocking the next release (alpha, beta or final). On Mon, Mar 17, 2008 at 12:28 PM, Brett Cannon wrote: > Right now at the sprint I am going through a list of issues Neal and I > compiled of what needs to happen to get 2.6/3.0 out the door (although > the list is pretty much 2.6-specific). They are all being flagged as > "immediate" priority. Hopefully it won't take too long to add all of > them (although the list ain't short =). > > If you want to see the list, Martin added a pre-defined Showstoppers > search to make it easy to list everything that is set to "immediate". -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Mon Mar 17 18:45:23 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 17 Mar 2008 13:45:23 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317000630.735823A40B0@sparrow.telecommunity.com> <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <47DEA275.4080306@behnel.de> Message-ID: <20080317174542.89A463A409D@sparrow.telecommunity.com> At 12:17 PM 3/17/2008 -0500, Guido van Rossum wrote: >There will be no egg support in the standard library. Are there any qualifications on that statement, or is this in the same category as "from __future__ import braces"? From p.f.moore at gmail.com Mon Mar 17 18:48:45 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 17 Mar 2008 17:48:45 +0000 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <47DEA275.4080306@behnel.de> Message-ID: <79990c6b0803171048t140a1dbfvcb1fa4dc005e5cb5@mail.gmail.com> On 17/03/2008, Guido van Rossum wrote: > On Mon, Mar 17, 2008 at 11:55 AM, Stefan Behnel wrote: > > Is it *wanted* that eggs are being supported by core Python? > > No. There will be no egg support in the standard library. This bothers me somewhat. At a certain level, all that egg files are is zip files with a different extension - and the zipimport module and PEP 302 certainly do support them. There is a lot more conceptual baggage associated with the egg "brand name", but unless that extra is clearly specified, a statement like "there will be no egg support" doesn't mean much. My view on PEP 365 is that it offers a standard way of doing from pkg_resources import resource_string foo_config = resource_string(__name__, 'foo.conf') which is basically a version of foo_config = open(os.path.join(os.path.dirname(__file__),'foo.conf').read() which also supports PEP 302 importers such as zipimport. This has nothing to do with eggs, and everything to do with solidifying the support for cleanly handling PEP 302 importers. For me, that would be far more useful that a package download&installer (as I'm on Windows, I tend to use bdist_wininst installers, and a bootstrap module which gave no uninstall capability would suck for me). The sticking point here, is deciding what parts of pkg_resources are OK, and which constitute "egg support". It may not be possible to come to a clear understanding of this. But I remain -1 on any module that just does download and install, with no uninstall capability. I don't like *eggs* for precisely this reason! Paul. From martin at v.loewis.de Mon Mar 17 18:49:25 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 17 Mar 2008 12:49:25 -0500 Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module) In-Reply-To: <79990c6b0803171012uaae8e9cmf6e0f4f189fa7ade@mail.gmail.com> References: <20080317000630.735823A40B0@sparrow.telecommunity.com> <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <79990c6b0803170935v59129ac8k3b0ff4ece23fe591@mail.gmail.com> <47DEA0C9.1020300@v.loewis.de> <79990c6b0803171012uaae8e9cmf6e0f4f189fa7ade@mail.gmail.com> Message-ID: <47DEAF25.20101@v.loewis.de> > But I don't see any practical difference with including setuptools and > including a module that installs setuptools. Would you be happy with > the standard library including a module whose sole function was to > install a package that you weren't happy to include directly in the > standard library? I wouldn't, personally. If the actual code is not in Python, I'm not responsible for it. I don't have to deal with bug reports, and I don't have to help users with that feature. As users want to get the functionality anyway, and out-of-the-box, I consider it a useful compromise. Regards, Martin From guido at python.org Mon Mar 17 18:59:46 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 17 Mar 2008 12:59:46 -0500 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <20080317174542.89A463A409D@sparrow.telecommunity.com> References: <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <47DEA275.4080306@behnel.de> <20080317174542.89A463A409D@sparrow.telecommunity.com> Message-ID: On Mon, Mar 17, 2008 at 12:45 PM, Phillip J. Eby wrote: > At 12:17 PM 3/17/2008 -0500, Guido van Rossum wrote: > >There will be no egg support in the standard library. > > Are there any qualifications on that statement, or is this in the > same category as "from __future__ import braces"? IIUC eggs are a method of package management that includes support for dependencies, multiple versions, and C extensions in zip files, as well as conventions for naming these and for encoding metadata (e.g. how to find out the version or the dependencies). This whole set of conventions is IMO too much to include into the stdlib ATM -- if only because it has proved controversial in the past. Maybe a few years from now it will be no longer controversial and then my objections will disappear. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Mon Mar 17 19:10:28 2008 From: brett at python.org (Brett Cannon) Date: Mon, 17 Mar 2008 13:10:28 -0500 Subject: [Python-Dev] Currently adding known 2.6 showstoppers to the tracker In-Reply-To: References: Message-ID: On Mon, Mar 17, 2008 at 12:35 PM, Guido van Rossum wrote: > Great! > > I wonder though if these should really all be given "showstopper" > priority. IMO things don't reach showstopper status until they are > blocking the next release (alpha, beta or final). > Fine by me. If Barry has no issue with that I will drop all of their priority to urgent. But until then I will just keep doing immediate and change them en-mass. -Brett > > > On Mon, Mar 17, 2008 at 12:28 PM, Brett Cannon wrote: > > Right now at the sprint I am going through a list of issues Neal and I > > compiled of what needs to happen to get 2.6/3.0 out the door (although > > the list is pretty much 2.6-specific). They are all being flagged as > > "immediate" priority. Hopefully it won't take too long to add all of > > them (although the list ain't short =). > > > > If you want to see the list, Martin added a pre-defined Showstoppers > > search to make it easy to list everything that is set to "immediate". > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > From lists at cheimes.de Mon Mar 17 18:58:44 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 17 Mar 2008 18:58:44 +0100 Subject: [Python-Dev] Currently adding known 2.6 showstoppers to the tracker In-Reply-To: References: Message-ID: <47DEB154.8020909@cheimes.de> Brett Cannon schrieb: > Right now at the sprint I am going through a list of issues Neal and I > compiled of what needs to happen to get 2.6/3.0 out the door (although > the list is pretty much 2.6-specific). They are all being flagged as > "immediate" priority. Hopefully it won't take too long to add all of > them (although the list ain't short =). > > If you want to see the list, Martin added a pre-defined Showstoppers > search to make it easy to list everything that is set to "immediate". In the past I've used "high" for beta/final show stoppers, "urgent" for show stoppers for the next release and "immediate" for critical bugs that either affects security or the current development process. I like to keep one priority reserved for issue that must be resolevd ASAP and within hours and not until the next release. Christian From pje at telecommunity.com Mon Mar 17 19:45:40 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 17 Mar 2008 14:45:40 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <47DEA275.4080306@behnel.de> <20080317174542.89A463A409D@sparrow.telecommunity.com> Message-ID: <20080317184553.913413A409D@sparrow.telecommunity.com> At 12:59 PM 3/17/2008 -0500, Guido van Rossum wrote: >On Mon, Mar 17, 2008 at 12:45 PM, Phillip J. Eby > wrote: > > At 12:17 PM 3/17/2008 -0500, Guido van Rossum wrote: > > >There will be no egg support in the standard library. > > > > Are there any qualifications on that statement, or is this in the > > same category as "from __future__ import braces"? > >IIUC eggs are a method of package management that includes support for >dependencies, multiple versions, and C extensions in zip files, as >well as conventions for naming these and for encoding metadata (e.g. >how to find out the version or the dependencies). This whole set of >conventions is IMO too much to include into the stdlib ATM -- if only >because it has proved controversial in the past. Maybe a few years >from now it will be no longer controversial and then my objections >will disappear. So, does this mean that the bootstrap tool must not use eggs? That seems a little bit odd, in that setuptools will at least need its .egg-info directory to get installed, and all of the people who'll be using this initially will be using it precisely in order to have support for eggs... So, it might be simpler all around to just clear up the "controversy". To the best of my recollection, only MAL and MvL have ever objected on Python-Dev to the idea of supporting eggs. Note: I'm specifically segregating "egg support" from the topic of including setuptools or easy_install in the stdlib directly. There are many legitimate reservations and open questions about the latter, including availability of volunteer support, choice of defaults, whether to replace distutils with setuptools, etc. etc. I recognize and respect the validity of those issues, which is precisely why I withdrew setuptools from inclusion in Python 2.5. However, regarding support for eggs, my understanding is that there were only two objections to eggs -- even at the time of the 2.5 setuptools discussions. And even though MvL objects to the idea of eggs in *principle*, I didn't read his recent posts as objecting to having the bootstrap tool download and install eggs in *practice*. (Although I hope he will clarify that stance one way or the other.) That leaves MAL, whose objections to PEP 365 centered on the API (he said he was "+1 on the concepts being added to the stdlib, -1 on adding the module in its current state"). Among other concerns, he wanted pkg_resources to be split into pkgutil and a new "egglib" module. I don't have a problem with this in principle, if there were a pkg_resources module that reconstituted the merged API. (But there are some practical problems with that approach, such as trying to split namespace package support between two theoretically-unrelated modules.) I would guess, however, that MAL's issues with the pkg_resources API would not apply to a bootstrap module whose sole purpose was to download eggs and put them on sys.path. Or, perhaps he would object *more*, I don't know. We could certainly ask him, though. :) So, was there anyone else you were counting towards "controversy"? The only other person I recall objecting to setuptools in any way on Python-Dev was effbot, and IIUC his objections were practical/administrative re: supporting easy_install and setuptools, not to the idea of .egg support in general. In summary, I think the controversy on Python-Dev regarding .egg support has actually been over for some time now. From guido at python.org Mon Mar 17 19:59:38 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 17 Mar 2008 13:59:38 -0500 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <20080317184553.913413A409D@sparrow.telecommunity.com> References: <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <47DEA275.4080306@behnel.de> <20080317174542.89A463A409D@sparrow.telecommunity.com> <20080317184553.913413A409D@sparrow.telecommunity.com> Message-ID: On Mon, Mar 17, 2008 at 1:45 PM, Phillip J. Eby wrote: > At 12:59 PM 3/17/2008 -0500, Guido van Rossum wrote: > >On Mon, Mar 17, 2008 at 12:45 PM, Phillip J. Eby > > wrote: > > > At 12:17 PM 3/17/2008 -0500, Guido van Rossum wrote: > > > >There will be no egg support in the standard library. > > > > > > Are there any qualifications on that statement, or is this in the > > > same category as "from __future__ import braces"? > > > >IIUC eggs are a method of package management that includes support for > >dependencies, multiple versions, and C extensions in zip files, as > >well as conventions for naming these and for encoding metadata (e.g. > >how to find out the version or the dependencies). This whole set of > >conventions is IMO too much to include into the stdlib ATM -- if only > >because it has proved controversial in the past. Maybe a few years > >from now it will be no longer controversial and then my objections > >will disappear. > > So, does this mean that the bootstrap tool must not use eggs? Correct. > That > seems a little bit odd, in that setuptools will at least need its > .egg-info directory to get installed, and all of the people who'll be > using this initially will be using it precisely in order to have > support for eggs... I'm okay if setuptools, once it's been installed, runs some setup code that creates the .egg-info directory and whatever else. This means I'm also okay with the bootstrap module finding and invoking that setup code. But I'm *not* okay with building any kind of egg management into the bootstrap module. The bootstrap module must be be neutral w.r.t. the package management style. > So, it might be simpler all around to just clear up the > "controversy". To the best of my recollection, only MAL and MvL have > ever objected on Python-Dev to the idea of supporting eggs. You can add my name to the list. I've heard plenty of people speak highly of eggs, but I've *also* heard from plenty of people (besides MAL and MvL) who have serious difficulties with the concept of eggs. I have certainly personally encountered plenty of situations where I wasn't able to complete an egg-based install because some dependency was broken (e.g. not available for the Python version I was using). > Note: I'm specifically segregating "egg support" from the topic of > including setuptools or easy_install in the stdlib directly. There > are many legitimate reservations and open questions about the latter, > including availability of volunteer support, choice of defaults, > whether to replace distutils with setuptools, etc. etc. I recognize > and respect the validity of those issues, which is precisely why I > withdrew setuptools from inclusion in Python 2.5. > > However, regarding support for eggs, my understanding is that there > were only two objections to eggs -- even at the time of the 2.5 > setuptools discussions. And even though MvL objects to the idea of > eggs in *principle*, I didn't read his recent posts as objecting to > having the bootstrap tool download and install eggs in > *practice*. (Although I hope he will clarify that stance one way or > the other.) You can do it in two stages. The bootstrap can download and install egg support, even though the bootstrap itself knows nothing about eggs. Then another bootstrap can download and install eggs. > That leaves MAL, whose objections to PEP 365 centered on the API (he > said he was "+1 on the concepts being added to the stdlib, -1 on > adding the module in its current state"). Among other concerns, he > wanted pkg_resources to be split into pkgutil and a new "egglib" > module. I don't have a problem with this in principle, if there were > a pkg_resources module that reconstituted the merged API. (But there > are some practical problems with that approach, such as trying to > split namespace package support between two theoretically-unrelated modules.) Well, you've heard my position now. > I would guess, however, that MAL's issues with the pkg_resources API > would not apply to a bootstrap module whose sole purpose was to > download eggs and put them on sys.path. Or, perhaps he would object > *more*, I don't know. We could certainly ask him, though. :) No need. I object to this myself. > So, was there anyone else you were counting towards > "controversy"? The only other person I recall objecting to > setuptools in any way on Python-Dev was effbot, and IIUC his > objections were practical/administrative re: supporting easy_install > and setuptools, not to the idea of .egg support in general. > > In summary, I think the controversy on Python-Dev regarding .egg > support has actually been over for some time now. Not really. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tseaver at palladion.com Mon Mar 17 20:05:15 2008 From: tseaver at palladion.com (Tres Seaver) Date: Mon, 17 Mar 2008 15:05:15 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317000630.735823A40B0@sparrow.telecommunity.com> <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> Message-ID: <47DEC0EB.3000404@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Guido van Rossum wrote: > On Mon, Mar 17, 2008 at 11:12 AM, Phillip J. Eby wrote: >> At 10:53 AM 3/17/2008 -0500, Guido van Rossum wrote: >> >I don't think this should play games with scripts being overridden or >> >whatever. If a bootstrap script is to be installed it should have a >> >separate name. I'm not sure what the advantage is of a bootstrap >> >script over "python -m bootstrap_module ..." though. >> >> And -m also makes explicit: >> >> 1. that it's a Python-specific tool >> 2. which Python version it will apply to > > Right! > >> >The PEP suggests that other package managers also benefit. How do they >> >benefit if the bootstrap script installs setuptools? >> >> Because those other package managers depend, in fact, on setuptools, >> or at least pkg_resources... which was why the original proposal was >> to just include pkg_resources in the first place. :) > > On how much of pkg_resources do they depend? Or is that an > unanswerable question? > >> >I'd also like to avoid the specific name "easy_install" for any of >> >this. That's a "brand name" (and a misleading one if you ask me, but >> >that's politics again :-). >> >> Ok, so if someone will propose a name and API for the thing, I'll >> implement it. (Assuming the proposed API is sane and reasonably >> implementable, of course.) > > Perhaps it can be called package_bootstrap? I don't know enough about > the required functionality to propose an API, but here goes anyway. > > Would be reasonable for it to have a command line that allows you to > specify a package name, optionally a version string, and (very) > optionally a server to contact (specified as an URL?). It should > default to the highest non-experimental version that's applicable to > the current Python version (assuming there's an easy way to determine > this; I don't want it to try and download different versions until one > works). It should not handle any kind of dependency management or > package management administration. > > It should be able to download a Python-only module or package and > install it into site-packages (or perhaps elsewhere if so directed via > another optional command line flag). It should support zip, tar and > tar.gz/tgz files (and perhaps tar.bz2). It should simply unpack the > zip/tar file using the zip or tar modules, and extract the > package/module into site-packages in such a way that it can be > imported directly without messing with sys.path. It should not mess > with .pth files, setup.py scripts, or eggs. If the contents of the > tar/zip file has a toplevel directory with version numbers in its name > (e.g. Django-0.96.1) it should skip that and just use the subdirectory > whose name is the "pure" package name (e.g. django). > > Does this make sense? I'm happy to take push-back, this is just > something I cooked up off the top of my head based on my experience > with manually installing packages. > I would prefer to see it just: - Find a source distribution (the variants you specified) on the given server which matches the supplied version string, using the same semantics as the current 'pkg_resources' constraints. - Unpack the source distribution to a temp directory. - cd into that directory and use sys.executable to invoke 'setup.py install'. Any extra command-line arguments beyond those used to find the source distribution would be passed on to the 'install' command, which would allow alternate locations, etc. That makes the installation as much like a manual one as possible, and means that existing pacakges will be installable whether or not they know about setuptools, etc. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFH3sDr+gerLs4ltQ4RAhjWAKCbFP87mJN4UnVt0pzDs81JovVpoACdGI7A tohpRJnXah0/QnyQeYiqoYY= =9Cif -----END PGP SIGNATURE----- From pje at telecommunity.com Mon Mar 17 20:36:49 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 17 Mar 2008 15:36:49 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <47DEA275.4080306@behnel.de> <20080317174542.89A463A409D@sparrow.telecommunity.com> <20080317184553.913413A409D@sparrow.telecommunity.com> Message-ID: <20080317194030.D6BAF3A409D@sparrow.telecommunity.com> At 01:59 PM 3/17/2008 -0500, Guido van Rossum wrote: >I have certainly personally encountered plenty of situations where I >wasn't able to complete an egg-based install because some dependency >was broken (e.g. not available for the Python version I was using). That's odd -- setuptools-based installs should be able to find and install packages from source. I have noticed a recent phenomenon where new developers upload *only* an egg to PyPI, without the source, but that's usually short-lived until someone points it out to them. Do you happen to know what packages you had this problem with? >I'm okay if setuptools, once it's been installed, runs some setup code >that creates the .egg-info directory and whatever else. This means I'm >also okay with the bootstrap module finding and invoking that setup >code. But I'm *not* okay with building any kind of egg management into >the bootstrap module. The bootstrap module must be be neutral w.r.t. >the package management style. Ok, well then we'll have to invent a new kind of binary package, whose name isn't 'egg'. Supporting distutils source packages is almost certainly a non-starter, if you want to avoid bringing the rest of setuptools into play. The only way to correctly determine what a source package contains is to run its setup script... and running unboxed setup scripts isn't safe because there are people who hardcode paths (or more precisely, use bad ways of computing them) in their setup scripts. I'm not saying the tool needs to guard against *malicious* scripts, just badly-written ones. (Setuptools does this with its "sandboxing" module, when running source packages' setup scripts.) So, if source is out, then some binary format is needed, which means defining the conventions for said format... i.e. "eggs lite" or "egg substitutes". :) > > So, it might be simpler all around to just clear up the > > "controversy". To the best of my recollection, only MAL and MvL have > > ever objected on Python-Dev to the idea of supporting eggs. > >You can add my name to the list. I've heard plenty of people speak >highly of eggs, but I've *also* heard from plenty of people (besides >MAL and MvL) who have serious difficulties with the concept of eggs. I did say "on Python-Dev", and you implied that it was not controversial with you, except for the maintenance-related concerns. I'm not fighting about this, but I would rather you were straight-up with your objections rather than deferring it to a controversy that "might go away in a few years". That way, I could at least attempt to do something about the concerns. OTOH, if your objections were non-specific and likely to stay that way, then I could have at least not wasted your time with any of this. From barry at python.org Mon Mar 17 20:46:01 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 17 Mar 2008 14:46:01 -0500 Subject: [Python-Dev] Currently adding known 2.6 showstoppers to the tracker In-Reply-To: References: Message-ID: <063477B9-0FB1-4CB4-A263-2A6A8C72E477@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 17, 2008, at 12:28 PM, Brett Cannon wrote: > Right now at the sprint I am going through a list of issues Neal and I > compiled of what needs to happen to get 2.6/3.0 out the door (although > the list is pretty much 2.6-specific). They are all being flagged as > "immediate" priority. Hopefully it won't take too long to add all of > them (although the list ain't short =). > > If you want to see the list, Martin added a pre-defined Showstoppers > search to make it easy to list everything that is set to "immediate". Brett, I'm going to use "immediate" to mark issues that must be fixed before the next release, which includes alpha releases. So I think "urgent" is a better priority to use for things that have to get fixed in 2.6/3.0. We can bounce these issues up higher if we want to fix them before the next alpha of course. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR97KeXEjvBPtnXfVAQKcCAP/YxZ7LQJepi9pg5IONjg1NePJSWwqggG7 rdFexmeHVockllBr5GMMmYgnr+jytnNDtRMTdy/uBvn3Ocl8+JRUHb5g/sDz2KQr usHcHnqIgkoQIgBnHGjnOg6nWmrn1CsAxJBMFuw3vcOksoYsJDdPiy+67GtJ0XIh ClZGJ8zbQQg= =Li3K -----END PGP SIGNATURE----- From jeff at taupro.com Mon Mar 17 20:44:07 2008 From: jeff at taupro.com (Jeff Rush) Date: Mon, 17 Mar 2008 14:44:07 -0500 Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317000630.735823A40B0@sparrow.telecommunity.com> <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <79990c6b0803170935v59129ac8k3b0ff4ece23fe591@mail.gmail.com> Message-ID: <47DECA07.6000600@taupro.com> Guido van Rossum wrote: > On Mon, Mar 17, 2008 at 11:35 AM, Paul Moore wrote: >> >> I'm +lots on someone giving a clear explanation of the meaning and >> interrelationship of the various terms involved in this discussion >> (setuptools, easy_install, pkg_resources, eggs, "package managers" as >> distinct from setuptools, etc etc) so that the discussion gets some >> much-needed clarity :-( > > Right. But finding someone who can explain all this is apparently > hard. All the owners of package managers seem busy... In preparing for my PyCon 2008 tutorial on eggs and buildout, I spent three full-time weeks carefully going over sources for distutils, setuptools and buildout to discover those aspects not documented. I can explain how they work, although I'm not sure this is the correct forum. I'd like to first offer my slides from my tutorial, 150 of them with detailed handout notes on many of them. http://wiki.python.org/moin/buildout/pycon2008_tutorial I'm happy to answer questions after that. I'm in the Hanada B room for OLPC at PyCon and on IRC #pycon. -Jeff From pje at telecommunity.com Mon Mar 17 21:09:23 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 17 Mar 2008 16:09:23 -0400 Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module) In-Reply-To: <47DECA07.6000600@taupro.com> References: <20080317000630.735823A40B0@sparrow.telecommunity.com> <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <79990c6b0803170935v59129ac8k3b0ff4ece23fe591@mail.gmail.com> <47DECA07.6000600@taupro.com> Message-ID: <20080317200941.372403A409D@sparrow.telecommunity.com> At 02:44 PM 3/17/2008 -0500, Jeff Rush wrote: >Guido van Rossum wrote: > > On Mon, Mar 17, 2008 at 11:35 AM, Paul Moore wrote: > >> > >> I'm +lots on someone giving a clear explanation of the meaning and > >> interrelationship of the various terms involved in this discussion > >> (setuptools, easy_install, pkg_resources, eggs, "package managers" as > >> distinct from setuptools, etc etc) so that the discussion gets some > >> much-needed clarity :-( > > > > Right. But finding someone who can explain all this is apparently > > hard. All the owners of package managers seem busy... > >In preparing for my PyCon 2008 tutorial on eggs and buildout, I spent three >full-time weeks carefully going over sources for distutils, setuptools and >buildout to discover those aspects not documented. I can explain how they >work, although I'm not sure this is the correct forum. I'd like to first >offer my slides from my tutorial, 150 of them with detailed handout notes on >many of them. > > http://wiki.python.org/moin/buildout/pycon2008_tutorial Wow. I am skimming over the 44-page one on setuptools, and that is definitely the most comprehensive doc anyone has produced on it, aside from the official docs. Thank you! From p.f.moore at gmail.com Mon Mar 17 21:19:11 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 17 Mar 2008 20:19:11 +0000 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <20080317184553.913413A409D@sparrow.telecommunity.com> References: <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <47DEA275.4080306@behnel.de> <20080317174542.89A463A409D@sparrow.telecommunity.com> <20080317184553.913413A409D@sparrow.telecommunity.com> Message-ID: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> On 17/03/2008, Phillip J. Eby wrote: > That leaves MAL, whose objections to PEP 365 centered on the API (he > said he was "+1 on the concepts being added to the stdlib, -1 on > adding the module in its current state"). Among other concerns, he > wanted pkg_resources to be split into pkgutil and a new "egglib" > module. I don't have a problem with this in principle, if there were > a pkg_resources module that reconstituted the merged API. (But there > are some practical problems with that approach, such as trying to > split namespace package support between two theoretically-unrelated modules.) I would personally like to see such a separation. As one of the authors of PEP 302 (well, the documentation - Just did all of the implementation) I have an interest in strengthening the standard library's support for transparent use of PEP 302 importers. To that end, I would like to see such functions as pkg_resources.resource_string() standardised. That covers the pkgutil aspect of pkg_resources. The "egglib" side of things is where the controversy lies, IMHO. I have a (somewhat unfocussed) dislike of eggs, largely because of the lack of a package management tool to handle them. I can live with them or without them, and it doesn't bother me if others use them, but the thing that really, really bothers me is that the controversy (and yes, there is such) over eggs is hampering the adoption of the pkgutil side of pkg_resources. So from me, there's a strong +1 for the split of pkg_resources into additions to the existing pkgutil module, and an independent egglib. You say there are practical problems to doing this. OK, but could we not have a discussion on how to resolve those issues as they come up? Could the split not be initially into 3 parts - pkgutil (eg, resource_string), egglib, and "difficult"? Then there would be something concrete to discuss and resolve. Paul. From gregor.lingl at aon.at Mon Mar 17 21:35:55 2008 From: gregor.lingl at aon.at (Gregor Lingl) Date: Mon, 17 Mar 2008 21:35:55 +0100 Subject: [Python-Dev] [Python-3000] xturtle and 3.0 In-Reply-To: References: Message-ID: <47DED62B.4040300@aon.at> Brett Cannon schrieb: > ... > The current plan is to introduce a tk package and turtle was to become > tk.turtle. xturtle, if picked up, can just take the place of the > current turtle at that location. > > -Brett > Hi Brett, as you probably can imagine, I'd like to try out xturtle.py with Python 2.6 Alas, I didn't succeed installing Python 2.6 correctly on my Windows machine using the Windows msi installer. Whereas I could start the python interpreter successfully it was impossible to use it to execute either idle.py nor turtle.py In the first case I got an import error: import _socket Import Error: DLL load failed in the second one likewise import _tkinter Import Error: DLL load failed A look on sys.path showed the DLLs directory to be present there. Do you have an explanation for this behaviour? What can I do to avoid it? Do I have to take some special action when installing the alpha release (I did it "for this user only")? With best regards, Gregor From brett at python.org Mon Mar 17 21:43:39 2008 From: brett at python.org (Brett Cannon) Date: Mon, 17 Mar 2008 15:43:39 -0500 Subject: [Python-Dev] [Python-3000] xturtle and 3.0 In-Reply-To: <47DED62B.4040300@aon.at> References: <47DED62B.4040300@aon.at> Message-ID: On Mon, Mar 17, 2008 at 3:35 PM, Gregor Lingl wrote: > > > Brett Cannon schrieb: > > ... > > > The current plan is to introduce a tk package and turtle was to become > > tk.turtle. xturtle, if picked up, can just take the place of the > > current turtle at that location. > > > > -Brett > > > Hi Brett, > > as you probably can imagine, I'd like to try out xturtle.py with Python 2.6 > Alas, I didn't succeed installing Python 2.6 correctly on my Windows > machine using > the Windows msi installer. > > Whereas I could start the python interpreter successfully it was > impossible to use it > to execute either idle.py nor turtle.py > > In the first case I got an import error: > > import _socket > Import Error: DLL load failed > > in the second one likewise > > import _tkinter > Import Error: DLL load failed > > A look on sys.path showed the DLLs directory to be present there. > Do you have an explanation for this behaviour? What can I do to > avoid it? Do I have to take some special action when installing the > alpha release (I did it "for this user only")? I don't use Windows so I can't help with that. But you might want to try a checkout and build from source. You can find instructions in the PCbuild directory. -Brett From p.f.moore at gmail.com Mon Mar 17 22:03:58 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 17 Mar 2008 21:03:58 +0000 Subject: [Python-Dev] [Python-3000] xturtle and 3.0 In-Reply-To: <47DED62B.4040300@aon.at> References: <47DED62B.4040300@aon.at> Message-ID: <79990c6b0803171403j4b7167f6x473ff36b9afe5c0e@mail.gmail.com> On 17/03/2008, Gregor Lingl wrote: > as you probably can imagine, I'd like to try out xturtle.py with Python 2.6 > Alas, I didn't succeed installing Python 2.6 correctly on my Windows > machine using the Windows msi installer. > > Whereas I could start the python interpreter successfully it was > impossible to use it to execute either idle.py nor turtle.py It worked for me. I have Python 2.5 installed, so I did an install of Python 2.6a1 "for all users", but I deselected the "register extensions" option (which makes this the default Python). I then ran idle using C:\Apps\Python26\python C:\Apps\Python26\Lib\idlelib\idle.py This worked fine for me. > A look on sys.path showed the DLLs directory to be present there. > Do you have an explanation for this behaviour? What can I do to > avoid it? Do I have to take some special action when installing the > alpha release (I did it "for this user only")? This is my sys.path: >C:\Apps\Python26\python Python 2.6a1 (r26a1:61155, Mar 1 2008, 12:11:56) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path ['', 'C:\\Apps\\Python26\\python26.zip', 'C:\\Apps\\Python26\\DLLs', 'C:\\Apps\\Python26\\lib', 'C:\\Apps\\Python26\\lib\\plat-win', 'C:\\Apps\\Python26\\lib\\lib-tk', 'C:\\Apps\\Python26', 'C:\\Apps\\Python26\\lib\\site-packages'] I don't see why "for this user only" should work any differently. No, I just tried it and it's OK as well. I can't see any reason why "register extensions" sould cause you a problem either. Can I suggest you uninstall and reinstall and see if that helps? Keep a log of what you didn, and if it's still a problem let me know and I'll see what I can do. Paul. PS What version of Windows are you on? I'm using XP Home (32 bit). If you're using 64-bit, I can't help, I'm afraid... From jeff at taupro.com Mon Mar 17 23:10:51 2008 From: jeff at taupro.com (Jeff Rush) Date: Mon, 17 Mar 2008 17:10:51 -0500 Subject: [Python-Dev] Capsule Summary of Some Packaging/Deployment Technology Concerns Message-ID: <47DEEC6B.5040602@taupro.com> I was in a Packaging BoF yesterday and, although not very relevant to the packager bootstrap thread, Guido has asked me to post some of the concerns. The BoF drew about 15 people, many of whom were packagers for Red Hat, Ubuntu and such. Everyone had strong expressions of frustration with the status quo and most had tried to resolve their issues but had their patches rejected. I am not taking either side and whether those rejections were justified I cannot say, but the general feeling of their concerns intentionally not being addressed isn't healthy. Several had abandoned setuptools, deeming it a failed solution and others called for a fork. To start, I am not a leader of the group nor do I claim I accurately captured and expressed all their concerns. I apologize to those in the BoF for any misrepresentations. 1. Many felt the existing dependency resolver was not correct. They wanted a full tree traversal resulting in an intersection of all restrictions, instead of a first-acceptable-solution approach taking now, which can result in top-level dependencies not being enforced upon lower levels. The latter is faster however. One solution would be to make the resolver pluggable. 2. People want a solution for the handling of documentation. The distutils module has had commented out sections related to this for several years. 3. A more flexible internal handing of the different types of files is needed. Currently the code, data, lib, etc. files are aggregated at build time and people would like them to be kept separate until install/packaging time. They also want greater flexibility in the kinds of files identified for packaging. There is currently a single plugin entrypoint for file_finding, so people have resorted to abusing the setuptools function find_packages() again and again with different include/exclude args. A solution is to expand the set of entrypoints into finer grained categories. They also want a way to expand the set of categories rather than a fixed set, which can be easily done with entrypoint groups and names. People also want a greater variety of file_finders to be included with setuptools. Instead of just CVS and SVN, they want it to comprehend Mercurial, Bazaar, Git and so forth. 4. They want an uninstall setuptools command. Adding one to remove a specific egg isn't difficult but correctly removing those dependencies that came in with that egg, without breaking later installs can be tricky. This is complicated because there isn't a single global package namespace to manage, when you factor in virtualenv and buildout sandboxes and per-user package areas. This differs from how RPMs and .debs are viewed. 5. There was concern over the .pth mechanism used by setuptools re activation. First, there is a (perceived) performance issue with increasingly adding every ZIP file explicitly onto sys.path. This may or may not be a red herring. The other is the use of a single .pth file to control the list of activated packages. Those who produce distributions would prefer a magic directory into which links to distributions could be dropped, similar to the current best practices for Linux, with /etc/conf.d/, /etc/profile.d/, /etc/xinetd.d/ and so forth. 6. There is a need for more extensibility hooks. People want places to plug in special handling. For example: a) setuptools has a --record option to capture the list of files installed for use by subsequent packaging tools. Some want that list to be available to a setuptools plugin. b) some want hooks for post-build/post-install actions, instead of the current approach of writing a custom build class that handles it all. 7. Many wanted to ability to install files anywhere in the install tree and not just under the Python package. Under distutils this was possible but it was removed in setuptools for security reasons. Custom code can still be written to do this explicitly but this is not popular. Neither setuptools nor distutils has the ability to rename files at install time. A fair question is whether it is the job of setuptools (or any Python packaging solution) to cover all these bases. The risk of not doing the job is that some of those in attendance were rolling their own solutions which do not play well with packages installed using other means, not seeing them. Distutils has intentionally tried to -not- be a general replacement packaging solution, with its support of the "bdist" command for various platform-specific distribution formats. We should continue not trying to replace platform-specific packaging technologies but perhaps improve our control of their creation. As mentioned, some of these concerns can be resolved by adding customization-pressure-release entrypoints to setuptools, and some can be handled with much better documentation of use cases and what to do. And some of it is confusion over packaging libraries versus applications, where setuptools focuses on the former and zc.buildout focuses on the latter. But buildout is very young, maintains isolation from the system Python and was not known to many of the packaging BoF attendees. Some of this may seem down on eggs, but I think they are really cool and would like to see them adopted as the standard for packaging Python software. There are rough spots on setuptools and buildout that would benefit from opening up the process and bringing in more developers, and communicating what they are and more importantly, what they are not. I believe the lack of a coherent packaging and deployment story for Python is hurting its uptake in many sectors and would like to work with others to strengthen this area of Python. -Jeff From gregor.lingl at aon.at Mon Mar 17 23:21:10 2008 From: gregor.lingl at aon.at (Gregor Lingl) Date: Mon, 17 Mar 2008 23:21:10 +0100 Subject: [Python-Dev] Installing Python 2.6 alpha1 on Windows XP Message-ID: <47DEEED6.7070904@aon.at> Hi Paul, thanks for you efforts, but up to now it still didn't work. I'm using Windows XP Professional (32 bit). I tried an install on two different machines with the same negative result. I proceeded like you suggested. - I installed for all users, - I disabled the register extensions When doing the same call to execute idle as you, I got the following: Traceback (most recent call last): File "c:\Python26\Lib\idlelib\idle.py", line 6, in import PyShell File "c:\Python26\Lib\idlelib\PyShell.py", line 9, in import socket File "c:\Python26\Lib\socket.py", line 46, in import _socket ImportError: DLL load failed: I get a similar error message, when I do from turtle import * with ... import _tkinter Import Error. DLL load failed .... sys.path is exactly like yours. (So the DLLs directory is contained in sys.path) _tkinter.pyd and _socket.pyd are present in DLLs. I've installed Python 2.5 on both machines. On the first one I moreover deleted all entries concerning Python (2.5) from the PATH variable, with no positive effect. On the other machine I have also installed Python 3000 successfully, which even still doesn't have IDLE in it's Menu. Nevertheless there C:\Python30> python Lib\idlelib\idle.py brings up IDLE 3.0a1 (and I even can import and use a port of xturtle.py to Python 3000 there). I never experienced a similar Problem before when installing Python. Any ideas? Regards, Gregor From alexander.belopolsky at gmail.com Mon Mar 17 23:35:46 2008 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 17 Mar 2008 18:35:46 -0400 Subject: [Python-Dev] New/Old class exception pitfall Message-ID: While discussing issue2291, I presented the following argument: """ Consider the following code: class x: pass class y(x): pass try: raise y except y: print "a" except: print "b" It prints 'b'. Now, suppose in preparation for 3.0 transition someone adds "__metaclass__ = type" to the module with that code. The result: it prints 'a'. Since the difference in behavior is in error handling code, which in my experience is often not thoroughly tested, the bug introduced by a seemingly innocuous move from old to new style classes is likely to trigger in the worst possible moment. (For example a wrong roll-back logic is applied after a failed database commit.) """ http://bugs.python.org/msg63584 This issue is only partially alleviated by the -3 warning because the warning is not issued unless the error condition raising a new style class not deriving from BaseException is actually tested for. It is my understanding that subclass check is skipped for new style classes not derived from BaseException in order to enable the identity check when a string exception is caught. With the deprecation of string exceptions, this logic is hard to justify. In any case, I believe this issue is either code or documentation bug: """ Exceptions are identified by class instances. The except clause is selected depending on the class of the instance: it must reference the class of the instance or a base class thereof. """ http://docs.python.org/dev/reference/executionmodel.html#id2 I don't see anything in the documentation that would suggest that old and new class instances should behave differently. From guido at python.org Mon Mar 17 23:49:14 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 17 Mar 2008 17:49:14 -0500 Subject: [Python-Dev] New/Old class exception pitfall In-Reply-To: References: Message-ID: On Mon, Mar 17, 2008 at 5:35 PM, Alexander Belopolsky wrote: > While discussing issue2291, I presented the following argument: > > """ > Consider the following code: > > class x: > pass > class y(x): > pass > try: > raise y > except y: > print "a" > except: > print "b" > > It prints 'b'. Really? Under which version exactly? On which platform? I cannot reproduce this with either 2.4, 2.5 or 2.6 on OS X. > Now, suppose in preparation for 3.0 transition someone > adds "__metaclass__ = type" to the module with that code. The result: > it prints 'a'. Which one would expect regardless of the metaclass, right? > Since the difference in behavior is in error handling > code, which in my experience is often not thoroughly tested, the bug > introduced by a seemingly innocuous move from old to new style classes > is likely to trigger in the worst possible moment. (For example a wrong > roll-back logic is applied after a failed database commit.) > """ http://bugs.python.org/msg63584 > > This issue is only partially alleviated by the -3 warning because the warning > is not issued unless the error condition raising a new style class not deriving > from BaseException is actually tested for. > > It is my understanding that subclass check is skipped for new style classes > not derived from BaseException in order to enable the identity check when a > string exception is caught. I have no idea what you are talking about. Can you quote a file, revision and line number where this is done? > With the deprecation of string exceptions, this logic is hard to justify. > > In any case, I believe this issue is either code or documentation bug: > > """ > Exceptions are identified by class instances. The except clause is > selected depending on the class of the instance: it must reference the class of > the instance or a base class thereof. > """ http://docs.python.org/dev/reference/executionmodel.html#id2 > > I don't see anything in the documentation that would suggest that old and new > class instances should behave differently. Me neither. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From phd at phd.pp.ru Tue Mar 18 00:13:14 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Tue, 18 Mar 2008 02:13:14 +0300 Subject: [Python-Dev] New/Old class exception pitfall In-Reply-To: References: Message-ID: <20080317231314.GA29474@phd.pp.ru> On Mon, Mar 17, 2008 at 06:35:46PM -0400, Alexander Belopolsky wrote: > class x: > pass > class y(x): > pass > try: > raise y > except y: > print "a" > except: > print "b" > > It prints 'b'. Python 2.2, 2.3, 2.4 and 2.5 on Linux: prints 'a'. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From alexander.belopolsky at gmail.com Tue Mar 18 00:18:25 2008 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 17 Mar 2008 19:18:25 -0400 Subject: [Python-Dev] New/Old class exception pitfall In-Reply-To: References: Message-ID: On Mon, Mar 17, 2008 at 6:49 PM, Guido van Rossum wrote: .. > Really? Under which version exactly? On which platform? I cannot > reproduce this with either 2.4, 2.5 or 2.6 on OS X. Just retested in Python 2.6a1+ (trunk:61449M, Mar 17 2008, 17:29:21) [GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2 and Python 2.5 (r25:51908, Nov 24 2006, 11:03:50) [GCC 3.4.4 20050721 (Red Hat 3.4.4-2)] on linux2 I don't have my PowerBook here, but I am sure I've seen in on Mac OS too. Only new-style class behavior is problematic. The following code prints 'b' for me: __metaclass__ = type class x: pass class y(x): pass try: raise y except y: print "a" except: print "b" > Which one would expect regardless of the metaclass, right? Yes. > I have no idea what you are talking about. Can you quote a file, > revision and line number where this is done? > Sorry for the lack of details. The code in question is at trunk/Python/errors.c:108 as of r61467: """ if (PyExceptionClass_Check(err) && PyExceptionClass_Check(exc)) { /* problems here!? not sure PyObject_IsSubclass expects to be called with an exception pending... */ return PyObject_IsSubclass(err, exc); } return err == exc; """ (flushed left hoping it won't get garbled) As you can see, subclass check is only performed if PyExceptionClass_Check(err) passes, which includes a check for err being derived from BaseException (see Include/pyerrors.h). This logic allows returning err == exc when err is a string. From phd at phd.pp.ru Tue Mar 18 00:26:02 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Tue, 18 Mar 2008 02:26:02 +0300 Subject: [Python-Dev] New/Old class exception pitfall In-Reply-To: References: Message-ID: <20080317232602.GA29708@phd.pp.ru> On Mon, Mar 17, 2008 at 07:18:25PM -0400, Alexander Belopolsky wrote: > I don't have my PowerBook here, but I am sure I've seen in on Mac OS > too. Only new-style class behavior is problematic. The following > code prints 'b' for me: > > __metaclass__ = type Ah, yes - with this addition it prints 'b'. > class x: > pass > class y(x): > pass > try: > raise y > except y: > print "a" > except: > print "b" Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From p.f.moore at gmail.com Tue Mar 18 00:25:30 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 17 Mar 2008 23:25:30 +0000 Subject: [Python-Dev] Installing Python 2.6 alpha1 on Windows XP In-Reply-To: <47DEEED6.7070904@aon.at> References: <47DEEED6.7070904@aon.at> Message-ID: <79990c6b0803171625u5159380at46e4464b2a72dcdd@mail.gmail.com> On 17/03/2008, Gregor Lingl wrote: > When doing the same call to execute idle as you, I got the following: > > Traceback (most recent call last): > File "c:\Python26\Lib\idlelib\idle.py", line 6, in > import PyShell > File "c:\Python26\Lib\idlelib\PyShell.py", line 9, in > import socket > File "c:\Python26\Lib\socket.py", line 46, in > import _socket > ImportError: DLL load failed: Can you try running C:\Python26\python.exe, and then at the interpreter prompt, execute: import sys print sys.path import socket and post the results? I expect you will get the same error about _socket not being loadable, but I'd like to check. Also can you try just "import _socket"? What is the size of _socket.pyd - mine is 44,032 bytes. Another thought - do you have any copies of msvcr90.dll on your PATH? I don't think it'll make a difference, but if you do can you try renaming them? > I never experienced a similar Problem before when installing Python. > > Any ideas? Not many :-( One final thought, what is the value of your PATH variable? Mine has no Python entries in it at all - that's normal, the Python installer doesn't set PATH. Sorry I can't be of more help, Paul. From alexander.belopolsky at gmail.com Tue Mar 18 00:29:54 2008 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 17 Mar 2008 23:29:54 +0000 (UTC) Subject: [Python-Dev] New/Old class exception pitfall References: <20080317231314.GA29474@phd.pp.ru> Message-ID: Oleg Broytmann phd.pp.ru> writes: > > On Mon, Mar 17, 2008 at 06:35:46PM -0400, Alexander Belopolsky wrote: > > class x: > > pass > > class y(x): > > pass > > try: > > raise y > > except y: > > print "a" > > except: > > print "b" > > > > It prints 'b'. > > Python 2.2, 2.3, 2.4 and 2.5 on Linux: prints 'a'. > Sorry, my fault. It prints 'a' without __metaclass__ = type, but prints 'b' with the metaclass. The output should be the same in both cases. The problematic case is: __metaclass__ = type class x: pass class y(x): pass try: raise y except y: print "a" except: print "b" the above code prints 'b' in 2.x From gregor.lingl at aon.at Tue Mar 18 00:53:20 2008 From: gregor.lingl at aon.at (Gregor Lingl) Date: Tue, 18 Mar 2008 00:53:20 +0100 Subject: [Python-Dev] Installing Python 2.6 alpha1 on Windows XP In-Reply-To: <79990c6b0803171625u5159380at46e4464b2a72dcdd@mail.gmail.com> References: <47DEEED6.7070904@aon.at> <79990c6b0803171625u5159380at46e4464b2a72dcdd@mail.gmail.com> Message-ID: <47DF0470.5060108@aon.at> Paul Moore schrieb: > On 17/03/2008, Gregor Lingl wrote: >> When doing the same call to execute idle as you, I got the following: >> >> Traceback (most recent call last): >> File "c:\Python26\Lib\idlelib\idle.py", line 6, in >> import PyShell >> File "c:\Python26\Lib\idlelib\PyShell.py", line 9, in >> import socket >> File "c:\Python26\Lib\socket.py", line 46, in >> import _socket >> ImportError: DLL load failed: > > Can you try running C:\Python26\python.exe, and then at the > interpreter prompt, execute: > > import sys > print sys.path > import socket > > and post the results? > >>> import sys >>> print sys.path ['', 'C:\\Python26\\python26.zip', 'C:\\Python26\\DLLs', 'C:\\Python26\\lib', ] 'C:\\Python26\\lib\\plat-win', 'C:\\Python26\\lib\\lib-tk', 'C:\\Python26', 'C:\\Python26\\lib\\site-packages'] >>> import socket Traceback (most recent call last): File "", line 1, in File "C:\Python26\lib\socket.py", line 46, in import _socket ImportError: DLL load failed: Das System kann die angegebene Datei nicht finden. ;-) :-( >>> > I expect you will get the same error about _socket not being loadable, > but I'd like to check. Also can you try just "import _socket"? > >>> import _socket Traceback (most recent call last): File "", line 1, in ImportError: DLL load failed: Das System kann die angegebene Datei nicht finden. > What is the size of _socket.pyd - mine is 44,032 bytes. > The same > Another thought - do you have any copies of msvcr90.dll on your PATH? > I don't think it'll make a difference, but if you do can you try > renaming them? > > No I don't! Only in c:\Python26, in c:\Python30 and on c:\Python30\DLLs. Strange that there are two copies of msvcr90.dll in Python30. So I'll copy this beast also to C:\Python26\DLLs, and ... it works! I can import socket and I even can start IDLE from the Python2.6 Menu Thanks for your advice. Do you have an idea if this is a 'bug' in the installer? Why the differences between 2.6 and 3000. Why two copies of that .dll in Python 30.0? I'm rather happy now :-) Have a nice evening. (Here in Vienna it's already 0:51 am.) All the best Gregor >> I never experienced a similar Problem before when installing Python. >> >> Any ideas? >> > > Not many :-( > > One final thought, what is the value of your PATH variable? Mine has > no Python entries in it at all - that's normal, the Python installer > doesn't set PATH. > > Sorry I can't be of more help, > Paul. > > > From guido at python.org Tue Mar 18 01:30:20 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 17 Mar 2008 19:30:20 -0500 Subject: [Python-Dev] 3rd party developers: don't change your APIs when porting to Py3k! Message-ID: For those who don't read blogs, I just blogged the slides for my keynote, and added an important admonishment to 3rd party developers. Here's the full text of the blog: The slides of my `keynote`_ are now up on python.org. There's both a `PowerPoint`_ and a `PDF`_ file. .. _keynote: http://www.python.org/doc/essays/ppt/ .. _PowerPoint: http://www.python.org/doc/essays/ppt/pycon2008/Py3kAndYou.ppt .. _PDF: http://www.python.org/doc/essays/ppt/pycon2008/Py3kAndYou.pdf I'd like to take this opportunity to remind you of a really important issue that I neglected to mention in the talk: **Don't change your APIs incompatibly when porting to Py3k.** Yes, you heard that right: even though Python 3.0 is changing incompatibly, I implore you (especially if you're maintaining a library that's used by others) *not* to make incompatible changes to your API. If you *have* make API changes, do them *before* you port to 3.0 -- release a version with the new API for Python 2.5, or 2.6 if you must. (Or do it later, *after* you've released a port to 3.0 without adding new features.) Why? Think of your users. Suppose Ima Lumberjack has implemented a web 2.0 app for managing his sawmill. Ima is a happy user of your most excellent web 2.0 framework. Now Ima wants to upgrade his app to Py3k. He waits until you have ported your framework to Py3k. He does everything by the books, runs his source code through the 2to3 tool, and starts testing. Imagine his despair when the tests fail: how is he going to tell whether the breakage is due to your API changes or due to his own code not being Py3k-ready? On the other hand, if port your web 2.0 framework to Py3k *without* making API changes, Ima's task is much more focused: the bugs he is left with after running 2to3 are definitely in his own code, which (presumably :-) he knows how to debug and fix. The same recommendation applies even more strongly if your library is a dependency for other libraries -- due to the fan-out the pain caused to others multiplies. If one of those packages gives up (even temporarily) its Py3k porting effort, this may prevent many other libraries and apps from being ported at all! So, once more for emphasis: **Don't change your APIs at the same time as porting to Py3k!** *PS.* The 3.0final release is now scheduled for September 3, 2008. See `PEP 361`_. .. _PEP 361: http://python.org/dev/peps/pep-0361/ -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Tue Mar 18 01:37:30 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 17 Mar 2008 20:37:30 -0400 Subject: [Python-Dev] Capsule Summary of Some Packaging/Deployment Technology Concerns In-Reply-To: <47DEEC6B.5040602@taupro.com> References: <47DEEC6B.5040602@taupro.com> Message-ID: <20080318004718.56E173A40FF@sparrow.telecommunity.com> At 05:10 PM 3/17/2008 -0500, Jeff Rush wrote: >I was in a Packaging BoF yesterday and, although not very relevant to the >packager bootstrap thread, Guido has asked me to post some of the concerns. > >The BoF drew about 15 people, many of whom were packagers for Red Hat, Ubuntu >and such. Everyone had strong expressions of frustration with the status quo >and most had tried to resolve their issues but had their patches rejected. I >am not taking either side and whether those rejections were >justified I cannot >say, but the general feeling of their concerns intentionally not being >addressed isn't healthy. Several had abandoned setuptools, deeming it a >failed solution and others called for a fork. > >To start, I am not a leader of the group nor do I claim I accurately captured >and expressed all their concerns. I apologize to those in the BoF for any >misrepresentations. I'm actually happy to hear that there's this much energy available -- hopefully some of it can be harnessed towards positive solutions. When I began developing setuptools, I often asked for the input of packagers, developers, etc., through the distutils-sig... and was met with overwhelming silence. So the fact that there is now a group of people who are ready to work for some solutions seems like a positive change, to me. It's hard to make design decisions regarding itches you don't personally have, and which other people won't help scratch. Unfortunately, a lot of the proposals from packaging system people have been of the form of, "fix this for us by breaking things for other people". Not all of them, though. Many have been very helpful, contributing troubleshooting help and good patches. That some of those good patches took nearly a year to get into setuptools (some from Fedora just got into 0.6c8 that were sent to me almost a year ago) is because I'm the only person reviewing setuptools patches, and I've spent only a few days in the last year doing focused development work on setuptools (as opposed to answering questions about it on the SIG). It's never a good thing when people's patches sit around, regardless of where they come from. But that's not the same thing as *rejecting* the patches. >1. Many felt the existing dependency resolver was not correct. They wanted a > full tree traversal resulting in an intersection of all restrictions, > instead of a first-acceptable-solution approach taking now, which can > result in top-level dependencies not being enforced upon lower > levels. The > latter is faster however. One solution would be to make the resolver > pluggable. Patches welcome, on both counts. Personally, Bob and I originally wanted a full-tree intersection, too, but it turned out to be hairier to implement than it seems at first. My guess is that none of the people who want it, have actually tried to implement it without a factorial or exponential O(). But that doesn't mean I'll be unhappy if somebody succeeds. :) Intuitively, it seems easy, just gather the requirements and intersect. In practice, different versions of a package may have different dependencies, so the intersection is not nearly as simple as that. We ended up just going for a depth-first version of the current algorithm (switched to breadth-first later, after field tests showed some problems with that), being greedy by testing latest-version-first, on the assumption that more recent versions would be likely to have the most-restrictive version requirements. In other words, we attempt to achieve heuristically what's being proposed to do algorithmically. And my guess is that whatever cases the heuristic is failing at, would probably not be helped by an algorithmic approach either. But I would welcome some actual data, either way. Again, though, patches are welcome. :) (Specifically, for the trunk; I don't see a resolver overhaul as being suitable for the 0.6 stable branch.) >2. People want a solution for the handling of documentation. The distutils > module has had commented out sections related to this for several years. As with so many other things, this gets tossed around the distutils-sig every now and then. A couple of times I've thrown out some options for how this might be done, but then the conversation peters out around the time anybody would have to actually do some work on it. (Me included, since I don't have an itch that needs scratching in this area.) In particular, if somebody wants to come up with a metadata standard for including documentation in eggs, we've got a boatload of hooks by which it could be done. Nothing's stopping anybody from proposing a standard and building a tool, here. (e.g. using the setuptools command hook, .egg-info writer hook, etc.) >3. A more flexible internal handing of the different types of files is needed. > Currently the code, data, lib, etc. files are aggregated at > build time and > people would like them to be kept separate until install/packaging time. I don't know what this means, exactly. > They also want greater flexibility in the kinds of files identified for > packaging. There is currently a single plugin entrypoint for > file_finding, > so people have resorted to abusing the setuptools function > find_packages() > again and again with different include/exclude args. A solution is to > expand the set of entrypoints into finer grained categories. They also > want a way to expand the set of categories rather than a fixed set, which > can be easily done with entrypoint groups and names. > > People also want a greater variety of file_finders to be included with > setuptools. Instead of just CVS and SVN, they want it to comprehend > Mercurial, Bazaar, Git and so forth. Did you point them to the Cheeseshop? There are plugins already available for all the systems you mentioned, plus Darcs and Monotone. If you mean "included" as in "bundled", this doesn't make a whole lot of sense to me. I'd think that if you're using setuptools as a developer (the only reason you need the file finders, since source distributions include a prebuilt manifest), you'd not have a problem saying "easy_install setuptools-git" or adding a "setup_requires='setuptools-git'" line to your setup.py. (Although the latter would only be needed for *development*, not deployment.) If you mean support for *installing* from these tools, I really wanted to add a pluggable download/retrieval mechanism for easy_install in 0.7, and would still love to see it happen. >4. They want an uninstall setuptools command. Adding one to remove a specific > egg isn't difficult but correctly removing those dependencies > that came in > with that egg, without breaking later installs can be tricky. Patches, once again, are welcome. :) (Btw, "setup.py develop" supports uninstallation, although it doesn't do a blessed thing about dependencies.) By the way, there are also third-party tools on the Cheeseshop that show egg dependency graphs (e.g. tl.eggdeps) or dump out information about installed eggs (e.g. "yolk"). > This is complicated because there isn't a single global package namespace > to manage, when you factor in virtualenv and buildout sandboxes and > per-user package areas. This differs from how RPMs and .debs are viewed. Yep. I really wanted, for 0.7, to study virtualenv and buildout and try to design a more general mechanism for managing things with the vaporware I've been calling "nest". Unfortunately, I've lost both the will and the budget for working on that any time soon. >5. There was concern over the .pth mechanism used by setuptools re activation. > First, there is a (perceived) performance issue with increasingly adding > every ZIP file explicitly onto sys.path. This may or may not be a red > herring. It is. My tests a few years back, when MAL first brought this up on the distutils-sig, showed the startup cost to be positively miniscule, if actual zipfiles are used. At the same time, I myself have come to the conclusion that if I had it to do over, I would use something more like the .egg-info egg style for general package installation, and added an installation manifest to it. If I ever write "nest", it will use such, with the ability to also support .egg files and directories. .egg files were created for extensible application platforms like Chandler and Zope and Trac and so on. Plugins usually need libraries, though, so the rest got added on because it was useful, and then the whole thing escaped its niche like a foreign organism added to an ecosystem with no natural predators. :) > The other is the use of a single .pth file to control the list > of activated > packages. Those who produce distributions would prefer a magic directory > into which links to distributions could be dropped, similar to > the current > best practices for Linux, with /etc/conf.d/, /etc/profile.d/, > /etc/xinetd.d/ and so forth. site-packages is that directory, and has been since long before setuptools. Just drop uniquely-named .pth files there, and you're good to go. >6. There is a need for more extensibility hooks. People want places to plug > in special handling. For example: > > a) setuptools has a --record option to capture the list of > files installed > for use by subsequent packaging tools. Some want that list to be > available to a setuptools plugin. > > b) some want hooks for post-build/post-install actions, instead of the > current approach of writing a custom build class that handles it all. Patches welcome! >7. Many wanted to ability to install files anywhere in the install tree and > not just under the Python package. Under distutils this was possible but > it was removed in setuptools for security reasons. It wasn't security, it was manageability. Egg-based installation means containment, (analagous to GNU stow) and therefore portability and disposability of plugins. (Which again is what eggs were really developed for in the first place.) > Custom code can still > be written to do this explicitly but this is not popular. No kidding. :) Current best practice is to include a script or module in the package that can install other files to a designated location. Personally, though, I tend to view applications and libraries that target specific install locations to be overreaching their bounds, and stepping into sysadmin territory. Give me the tools to install the data, don't just dump it somewhere on my system where *you* think it should go, in other words. On the other hand, I've been puzzling over how to handle legitimate post-install features. On Windows, both wx and pywin32 have a real need to do some actuall "install" operations. Some is just copying files, but pywin32 also has to do some registry stuff. I don't know how to allow just what's sensible, without opening up a huge can of worms, though. Proposals welcome. From gregor.lingl at aon.at Tue Mar 18 01:47:45 2008 From: gregor.lingl at aon.at (Gregor Lingl) Date: Tue, 18 Mar 2008 01:47:45 +0100 Subject: [Python-Dev] xturtle and 2.6 In-Reply-To: References: Message-ID: <47DF1131.8070702@aon.at> Hi everyone, I happily like to report, that xturtle is running under Python 2.6 seemingly without any problems. Thanks to Paul Moore's advice I could get Python 2.6 running on my windows machine. I tested xturtle running those 30+ sample scripts, which are contained in the xturtle package with the included demoViewer and not a single (new) problem did occur. Quite a few of these samplescripts contain runtime measurements, which consistently showed that they ran 5 to 15% faster than under Python2.5 Regards, Gregor From barry at python.org Tue Mar 18 01:49:43 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 17 Mar 2008 19:49:43 -0500 Subject: [Python-Dev] PEP 361: Python 2.6/3.0 release schedule Message-ID: Greetings from Pycon 2008! Neal Norwitz and I have worked out the schedule for Python 2.6 and 3.0, which will be released in lockstep. We will be following a monthly release schedule, with releases to occur on the first Wednesday of the month. We'll move to a 2 week schedule for the release candidates. Executive summary: Python 2.6 and 3.0 finals are planned for September 3, 2008. PEP 361 contains all the gory details; from the PEP: Feb 29 2008: Python 2.6a1 and 3.0a3 are released Apr 02 2008: Python 2.6a2 and 3.0a4 planned May 07 2008: Python 2.6a3 and 3.0a5 planned Jun 04 2008: Python 2.6b1 and 3.0b1 planned Jul 02 2008: Python 2.6b2 and 3.0b2 planned Aug 06 2008: Python 2.6rc1 and 3.0rc1 planned Aug 20 2008: Python 2.6rc2 and 3.0rc2 planned Sep 03 2008: Python 2.6 and 3.0 final http://www.python.org/dev/peps/pep-0361/ This schedule gives everybody plenty of advanced notice, so you should be able to get your code in on time. Please be very careful about not breaking the branches. Neal and I will be cracking the whip of public shame when your commit turns the buildbots red. Embarrassing Pycon pictures of you will be posted if such broken revisions cause us to slip a release, and remember, we know how to use GIMP. On behalf of everyone, here's to an awesome release! -Barry Python 2.6/3.0 release manager -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: pep-0361.txt Url: http://mail.python.org/pipermail/python-dev/attachments/20080317/c34e843c/attachment.txt -------------- next part -------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 304 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20080317/c34e843c/attachment.pgp From zooko at zooko.com Tue Mar 18 02:40:25 2008 From: zooko at zooko.com (zooko) Date: Mon, 17 Mar 2008 19:40:25 -0600 Subject: [Python-Dev] 3rd party developers: don't change your APIs when porting to Py3k! (but consider using ctypes) In-Reply-To: References: Message-ID: I'm the maintainer of a few Python packages which wrap native C or C+ + code. At Pycon, I learned that PyPy and Jython support ctypes or have plans to do so in the near future. I don't know about IronPython. However, having CPython, PyPy, and Jython all supporting ctypes makes it the obvious choice for interfacing to native code in the future. Regards, Zooko O'Whielacronx From janssen at parc.com Tue Mar 18 02:41:22 2008 From: janssen at parc.com (Bill Janssen) Date: Mon, 17 Mar 2008 18:41:22 PDT Subject: [Python-Dev] 3rd party developers: don't change your APIs when porting to Py3k! In-Reply-To: References: Message-ID: <08Mar17.184131pdt."58696"@synergy1.parc.xerox.com> Now I apparently need an email reader that understands reStructuredText :-). Bill From brett at python.org Tue Mar 18 03:10:28 2008 From: brett at python.org (Brett Cannon) Date: Mon, 17 Mar 2008 21:10:28 -0500 Subject: [Python-Dev] Change in priority fields Message-ID: Barry, Neal, and myself had a conversation and changed the priority fields in the tracker. You can click on 'priority' to see an explanation, but the new fields are: - release blocker - critical - high - normal - low So "release blocker" blocks a release. "Critical" could very easily block a release, but not the current one. "High" issues should be addressed, but won't block anything. "Normal" is normal. And "low" is for spelling errors and such. -Brett From nnorwitz at gmail.com Tue Mar 18 05:31:12 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 17 Mar 2008 23:31:12 -0500 Subject: [Python-Dev] Change in priority fields In-Reply-To: References: Message-ID: On Mon, Mar 17, 2008 at 9:10 PM, Brett Cannon wrote: > Barry, Neal, and myself had a conversation and changed the priority > fields in the tracker. You can click on 'priority' to see an > explanation, but the new fields are: > > - release blocker > - critical > - high > - normal > - low > > So "release blocker" blocks a release. "Critical" could very easily > block a release, but not the current one. "High" issues should be > addressed, but won't block anything. "Normal" is normal. And "low" is > for spelling errors and such. Primarily everyone should use normal for issues that are, uh, normal. "Critical" should be used for bugs that are things like: crashing the interpreter, serious memory/reference leaks. "High" could be used for large problems with resource usage (too much memory) or something that is otherwise, important. n From guido at python.org Tue Mar 18 05:35:48 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 17 Mar 2008 23:35:48 -0500 Subject: [Python-Dev] Change in priority fields In-Reply-To: References: Message-ID: What do I do for something that should absolutely go into the 2.6final release (say) but is otherwise pretty minor? I've been using critical to make sure it doesn't get put off until past the release. On Mon, Mar 17, 2008 at 11:31 PM, Neal Norwitz wrote: > On Mon, Mar 17, 2008 at 9:10 PM, Brett Cannon wrote: > > Barry, Neal, and myself had a conversation and changed the priority > > fields in the tracker. You can click on 'priority' to see an > > explanation, but the new fields are: > > > > - release blocker > > - critical > > - high > > - normal > > - low > > > > So "release blocker" blocks a release. "Critical" could very easily > > block a release, but not the current one. "High" issues should be > > addressed, but won't block anything. "Normal" is normal. And "low" is > > for spelling errors and such. > > Primarily everyone should use normal for issues that are, uh, normal. > "Critical" should be used for bugs that are things like: crashing the > interpreter, serious memory/reference leaks. "High" could be used for > large problems with resource usage (too much memory) or something that > is otherwise, important. > > > > n > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Tue Mar 18 05:49:01 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 17 Mar 2008 23:49:01 -0500 Subject: [Python-Dev] Change in priority fields In-Reply-To: References: Message-ID: <6527F3DE-7122-4570-81A2-B53E5AF8D0C4@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 17, 2008, at 11:35 PM, Guido van Rossum wrote: > What do I do for something that should absolutely go into the 2.6final > release (say) but is otherwise pretty minor? I've been using critical > to make sure it doesn't get put off until past the release. Critical is the right one to use. Neal and I will basically be moving issues between 'release blocker' and 'critical' with the former meaning this issue blocks the upcoming release. The latter means it will (probably) block an upcoming release. I think when we make a major milestone, e.g. the first beta, the first release candidate, etc., we'll triage those critical and move some up to release blockers. We should not release the finals until there are no release blockers or criticals. Either the critical gets moved to high and ignored, or it gets moved to release blocker and gets fixed. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR99JvnEjvBPtnXfVAQJwuwQAh5mhXdwg7t9FAsNXJ69OoPM6qj37OjP4 +3SjZMn9A1qObFB7biUV6P47KwydzDskMaswifMv9eV94+ccb0mIlDC/SgdjB9h8 JtuJq6mZ1nIUqQSLtX4W6op4G/Zpk/cerrbBzTWt06VU8yi7XhoBCVjDDVn37Nwv Kh260/8ewnw= =dMJV -----END PGP SIGNATURE----- From guido at python.org Tue Mar 18 06:11:18 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 18 Mar 2008 00:11:18 -0500 Subject: [Python-Dev] Change in priority fields In-Reply-To: <6527F3DE-7122-4570-81A2-B53E5AF8D0C4@python.org> References: <6527F3DE-7122-4570-81A2-B53E5AF8D0C4@python.org> Message-ID: On Mon, Mar 17, 2008 at 11:49 PM, Barry Warsaw wrote: > On Mar 17, 2008, at 11:35 PM, Guido van Rossum wrote: > > > What do I do for something that should absolutely go into the 2.6final > > release (say) but is otherwise pretty minor? I've been using critical > > to make sure it doesn't get put off until past the release. > > Critical is the right one to use. Neal and I will basically be moving > issues between 'release blocker' and 'critical' with the former > meaning this issue blocks the upcoming release. The latter means it > will (probably) block an upcoming release. I think when we make a > major milestone, e.g. the first beta, the first release candidate, > etc., we'll triage those critical and move some up to release blockers. > > We should not release the finals until there are no release blockers > or criticals. Either the critical gets moved to high and ignored, or > it gets moved to release blocker and gets fixed. Hm... This feels a bit like inflation of priorities to me; there would be lots of critical bugs and quite a few release blockers... The highest priority just pertains to the upcoming release which could be weeks in the future. I'd be more comfortable if there were 1-2 priorities above that, e.g. one higher for stuff that makes a buildbot go red (i.e. breaks a test) and two higher for stuff that affects most developers (e.g. stuff checked in that doesn't even build). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Mar 18 06:31:55 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 18 Mar 2008 00:31:55 -0500 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> References: <20080317161305.22B183A409D@sparrow.telecommunity.com> <47DEA275.4080306@behnel.de> <20080317174542.89A463A409D@sparrow.telecommunity.com> <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> Message-ID: After reading all this, I really don't believe that adding egg support to the stdlib at this time is the right thing to do. I am therefore rejecting the PEP. I am hoping that someone will create a simpler bootstrap module that is able to download a file of pure Python code and install it, perhaps by running its setup.py, assuming that it only depends on distutils (or other things previously installed). I will welcome such a module into the stdlib. I'm not sure a PEP is even needed, though interested parties are certainly welcome to write a PEP specifying the behavior first. With 2.6 and 3.0 slated for release in September, there should be enough time to get this done before then. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From oliphant.travis at ieee.org Tue Mar 18 08:04:41 2008 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 18 Mar 2008 02:04:41 -0500 Subject: [Python-Dev] PEP 361: Python 2.6/3.0 release schedule In-Reply-To: References: Message-ID: <47DF6989.1010808@ieee.org> Barry Warsaw wrote: > Greetings from Pycon 2008! > > Neal Norwitz and I have worked out the schedule for Python 2.6 and 3.0, > which will be released in lockstep. We will be following a monthly > release schedule, with releases to occur on the first Wednesday of the > month. We'll move to a 2 week schedule for the release candidates. > Hey Barry, Thanks for putting this PEP together. This is really helpful. I didn't see discussion of PEP 3118 and it's features back-ported to Python 2.6. I've already back-ported the new buffer API as an addition to the old buffer protocol. In addition, I've planned to back-port the improvements to the struct module and the addition of the memoryview object (both in PEP 3118). If you have questions, we can talk tomorrow. Best regards, -Travis Oliphant From asmodai at in-nomine.org Tue Mar 18 09:16:57 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Tue, 18 Mar 2008 09:16:57 +0100 Subject: [Python-Dev] PEP 361: Python 2.6/3.0 release schedule In-Reply-To: References: Message-ID: <20080318081657.GQ60713@nexus.in-nomine.org> -On [20080318 01:52], Barry Warsaw (barry at python.org) wrote: >This schedule gives everybody plenty of advanced notice, so you should be >able to get your code in on time. In particular the memory related fixes over the last weeks, please. :) -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ To love someone deeply gives you strength. Being loved by someone deeply gives you courage... From p.f.moore at gmail.com Tue Mar 18 10:39:40 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 18 Mar 2008 09:39:40 +0000 Subject: [Python-Dev] Installing Python 2.6 alpha1 on Windows XP In-Reply-To: <47DF0470.5060108@aon.at> References: <47DEEED6.7070904@aon.at> <79990c6b0803171625u5159380at46e4464b2a72dcdd@mail.gmail.com> <47DF0470.5060108@aon.at> Message-ID: <79990c6b0803180239r5167d5b8x8c9d2d74641a323@mail.gmail.com> On 17/03/2008, Gregor Lingl wrote: > > Another thought - do you have any copies of msvcr90.dll on your PATH? > > I don't think it'll make a difference, but if you do can you try > > renaming them? > > > > > No I don't! Only in c:\Python26, in c:\Python30 and on c:\Python30\DLLs. > Strange that there are two copies of msvcr90.dll in Python30. > > So I'll copy this beast also to C:\Python26\DLLs, > and ... it works! > I can import socket and I even can start IDLE from the Python2.6 Menu That's odd. In theory, having msvcr90.dll in C:\Python26 should be sufficient, as once python.exe is loaded, its directory is added to the DLL search path. Maybe it's something to do with the "side by side manifest installation" stuff (or whatever it's called). Martin, can you comment? It looks like the 3.0 installer uses 2 copies of msvcr90.dll, where the 2.6 one doesn't. I would have thought that only one is necessary, but Gregor's experiments seem to demonstrate otherwise. > Thanks for your advice. I'm not sure I did much more than get you to the point where you solved the problem for yourself, but I'm glad you've got things working :-) > Do you have an idea if this is a 'bug' in the installer? I suspect it is - I've copied Martin for comment, but could you raise a bug in the tracker? > Why the differences between 2.6 and 3000. I don't know, but that's what makes me think it's a bug (although I'm not entirely convinced that having 2 copies of the DLL, like 3.0 does, is the correct solution). > Why two copies of that .dll in Python 30.0? I suspect it's a result of the msvcr90 "side by side" stuff. But beyond that, I don't know. > I'm rather happy now :-) So am I. Glad things are working :-) Paul. From ncoghlan at gmail.com Tue Mar 18 10:47:59 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 18 Mar 2008 19:47:59 +1000 Subject: [Python-Dev] Change in priority fields In-Reply-To: References: <6527F3DE-7122-4570-81A2-B53E5AF8D0C4@python.org> Message-ID: <47DF8FCF.50501@gmail.com> Guido van Rossum wrote: > On Mon, Mar 17, 2008 at 11:49 PM, Barry Warsaw wrote: >> We should not release the finals until there are no release blockers >> or criticals. Either the critical gets moved to high and ignored, or >> it gets moved to release blocker and gets fixed. > > Hm... This feels a bit like inflation of priorities to me; there would > be lots of critical bugs and quite a few release blockers... The > highest priority just pertains to the upcoming release which could be > weeks in the future. I'd be more comfortable if there were 1-2 > priorities above that, e.g. one higher for stuff that makes a buildbot > go red (i.e. breaks a test) and two higher for stuff that affects most > developers (e.g. stuff checked in that doesn't even build). It would be good if someone at the sprints could take the PEP 3 redraft I posted a few weeks back, update it with whatever you all come up with in relation to tracker management and check it in. (Attaching that draft here so people don't have to go trawling through email archives for it) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: pep-0003.txt Url: http://mail.python.org/pipermail/python-dev/attachments/20080318/be8ea51b/attachment.txt From lists at cheimes.de Tue Mar 18 11:38:28 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 18 Mar 2008 11:38:28 +0100 Subject: [Python-Dev] Installing Python 2.6 alpha1 on Windows XP In-Reply-To: <79990c6b0803180239r5167d5b8x8c9d2d74641a323@mail.gmail.com> References: <47DEEED6.7070904@aon.at> <79990c6b0803171625u5159380at46e4464b2a72dcdd@mail.gmail.com> <47DF0470.5060108@aon.at> <79990c6b0803180239r5167d5b8x8c9d2d74641a323@mail.gmail.com> Message-ID: <47DF9BA4.6010305@cheimes.de> Paul Moore schrieb: > That's odd. In theory, having msvcr90.dll in C:\Python26 should be > sufficient, as once python.exe is loaded, its directory is added to > the DLL search path. Maybe it's something to do with the "side by side > manifest installation" stuff (or whatever it's called). > > Martin, can you comment? It looks like the 3.0 installer uses 2 copies > of msvcr90.dll, where the 2.6 one doesn't. I would have thought that > only one is necessary, but Gregor's experiments seem to demonstrate > otherwise. In practice it's not enough on XP and higher. Starting with XP, Windows supports side by side assemblies (SxS). SxS assemblies must be installed following a special convention. The MSDN web site contains some examples. It suc.. err it's fun :/ We are still having problems to integrate the MS Merge module into our MSI. Martin is working on the problem. In the mean time you can work around the problem by installing the MSVCRT 9.0 Redist. Christian From hsoft at hardcoded.net Tue Mar 18 14:18:00 2008 From: hsoft at hardcoded.net (Virgil Dupras) Date: Tue, 18 Mar 2008 14:18:00 +0100 Subject: [Python-Dev] test_support.have_unicode Message-ID: <2a70578d0803180618p2dbd7241s85db8de30d82472a@mail.gmail.com> The test_support unit has this have_unicode. Do we need the Python's test unit to be *that* backward compatible? Is there still an implementation of Python that doesn't support unicode? If there is, should the test suite care? As a side question. Considering that I'm not sure whether have_unicode is relevant or not, is it more appropriate to create a ticket for it or to ask python-dev? Virgil Dupras From musiccomposition at gmail.com Tue Mar 18 14:26:02 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 18 Mar 2008 08:26:02 -0500 Subject: [Python-Dev] test_support.have_unicode In-Reply-To: <2a70578d0803180618p2dbd7241s85db8de30d82472a@mail.gmail.com> References: <2a70578d0803180618p2dbd7241s85db8de30d82472a@mail.gmail.com> Message-ID: <1afaf6160803180626m5445f88cja7a760d83e06108c@mail.gmail.com> On Tue, Mar 18, 2008 at 8:18 AM, Virgil Dupras wrote: > The test_support unit has this have_unicode. Do we need the Python's > test unit to be *that* backward compatible? Is there still an > implementation of Python that doesn't support unicode? If there is, > should the test suite care? Python 2.x can be compiled without Unicode. > > > As a side question. Considering that I'm not sure whether have_unicode > is relevant or not, is it more appropriate to create a ticket for it > or to ask python-dev? > > Virgil Dupras > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080318/22144ed9/attachment.htm From martin at v.loewis.de Tue Mar 18 14:37:45 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 18 Mar 2008 08:37:45 -0500 Subject: [Python-Dev] Installing Python 2.6 alpha1 on Windows XP In-Reply-To: <79990c6b0803180239r5167d5b8x8c9d2d74641a323@mail.gmail.com> References: <47DEEED6.7070904@aon.at> <79990c6b0803171625u5159380at46e4464b2a72dcdd@mail.gmail.com> <47DF0470.5060108@aon.at> <79990c6b0803180239r5167d5b8x8c9d2d74641a323@mail.gmail.com> Message-ID: <47DFC5A9.90401@v.loewis.de> > That's odd. In theory, having msvcr90.dll in C:\Python26 should be > sufficient, as once python.exe is loaded, its directory is added to > the DLL search path. Maybe it's something to do with the "side by side > manifest installation" stuff (or whatever it's called). Yes, with VS 2008, the DLL search path becomes irrelevant (or so it seems). > Martin, can you comment? It looks like the 3.0 installer uses 2 copies > of msvcr90.dll, where the 2.6 one doesn't. I would have thought that > only one is necessary, but Gregor's experiments seem to demonstrate > otherwise. I haven't figured it out myself; it's a complete mess, and Microsoft is heavily wasting our time. It seems that you absolutely *must* have the manifest file in each directory that has a DLL which links with the CRT. Whether or not separate copies of the DLL are then also necessary, and whether or not that causes two copies to be loaded into the address space, I don't know. HELP!!!! To reproduce the problem, you probably have to test on a machine which doesn't have the CRT redistributable installed centrally (neither through VS 2008 installation, nor by running the standalone CRT installer, nor by having installed any other software that provides an SxS copy of the CRT). Regards, Martin From barry at python.org Tue Mar 18 14:39:46 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 18 Mar 2008 08:39:46 -0500 Subject: [Python-Dev] Change in priority fields In-Reply-To: References: <6527F3DE-7122-4570-81A2-B53E5AF8D0C4@python.org> Message-ID: <53B76010-C290-41C8-AC87-076A053ACCFC@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 18, 2008, at 12:11 AM, Guido van Rossum wrote: > Hm... This feels a bit like inflation of priorities to me; there would > be lots of critical bugs and quite a few release blockers... The > highest priority just pertains to the upcoming release which could be > weeks in the future. I want a very simple roundup query that I can consult during the release cycle to know whether everything's fine, or whether I have to start pestering people and pull the big red "STOP RELEASE" button. Critical gives us a priority for things we really need to fix, but maybe not right this second. > I'd be more comfortable if there were 1-2 > priorities above that, e.g. one higher for stuff that makes a buildbot > go red (i.e. breaks a test) and two higher for stuff that affects most > developers (e.g. stuff checked in that doesn't even build). I think neither of these use cases should get that far. Neal and I talked it over and we're in agreement that if a commit makes the buildbots go red or breaks the build, we're going to just revert it. Tough luck Joe Dev, please test your changes more carefully next time. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR9/GJHEjvBPtnXfVAQJEugQAnyPR4WkSW7R2QN4F6v1zgQakD/8yVxCn TMESNJaG1XHhZJlZ6gSl5SBmy5PFS0w4GeUayXjbxFbH/hdpNWfAeWwgY+5+W/6S A3JO96nz89JUXqiRvOab7QaDl8N1KSd0Om7rJo50XKZZqJBNO6/ypL9mr1nAvUp/ ppZ614lz15I= =HCH0 -----END PGP SIGNATURE----- From martin at v.loewis.de Tue Mar 18 14:44:34 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 18 Mar 2008 08:44:34 -0500 Subject: [Python-Dev] Installing Python 2.6 alpha1 on Windows XP In-Reply-To: <47DF9BA4.6010305@cheimes.de> References: <47DEEED6.7070904@aon.at> <79990c6b0803171625u5159380at46e4464b2a72dcdd@mail.gmail.com> <47DF0470.5060108@aon.at> <79990c6b0803180239r5167d5b8x8c9d2d74641a323@mail.gmail.com> <47DF9BA4.6010305@cheimes.de> Message-ID: <47DFC742.9090501@v.loewis.de> > We are still having problems to integrate the MS Merge module into our > MSI. Martin is working on the problem. In the mean time you can work > around the problem by installing the MSVCRT 9.0 Redist. While this is a work-around for the people trying to run the alpha releases, it effectively prevents them from doing useful tests in that area for the later releases. IIUC, there is NO way to uninstall the CRT redist (perhaps except for deleting the files from disk, and cleaning an unknown number of registry keys), so once you have run that on a machine, the machine becomes useless for determining whether the installer would work without it. As a consequence, regular testers won't report the problem anymore, as has been demonstrated with 2.6a1, it seems (which apparently doesn't work). Non-regular testers will have no clue what happened (which also was just demonstrated). Regards, Martin From martin at v.loewis.de Tue Mar 18 14:47:21 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 18 Mar 2008 08:47:21 -0500 Subject: [Python-Dev] test_support.have_unicode In-Reply-To: <2a70578d0803180618p2dbd7241s85db8de30d82472a@mail.gmail.com> References: <2a70578d0803180618p2dbd7241s85db8de30d82472a@mail.gmail.com> Message-ID: <47DFC7E9.2060006@v.loewis.de> > The test_support unit has this have_unicode. Do we need the Python's > test unit to be *that* backward compatible? Is there still an > implementation of Python that doesn't support unicode? If there is, > should the test suite care? It's still intended that you can build Python 2.6 without Unicode support, and that the test suite "mostly" works. If it doesn't, it's up to users who care about that feature to provide fixes, but you should not actively break it. Regards, Martin From lists at cheimes.de Tue Mar 18 15:04:03 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 18 Mar 2008 15:04:03 +0100 Subject: [Python-Dev] test_support.have_unicode In-Reply-To: <47DFC7E9.2060006@v.loewis.de> References: <2a70578d0803180618p2dbd7241s85db8de30d82472a@mail.gmail.com> <47DFC7E9.2060006@v.loewis.de> Message-ID: <47DFCBD3.9040507@cheimes.de> Martin v. L?wis schrieb: > It's still intended that you can build Python 2.6 without Unicode > support, and that the test suite "mostly" works. > > If it doesn't, it's up to users who care about that feature to provide > fixes, but you should not actively break it. About two months ago I fixed the most critical bugs but the unicode free build is treated like a poor cousin at best. It's neither actively developed nor tested in regular intervals. IMO it's a deprecation candiate. Christian From p.f.moore at gmail.com Tue Mar 18 15:07:17 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 18 Mar 2008 14:07:17 +0000 Subject: [Python-Dev] Installing Python 2.6 alpha1 on Windows XP In-Reply-To: <47DFC5A9.90401@v.loewis.de> References: <47DEEED6.7070904@aon.at> <79990c6b0803171625u5159380at46e4464b2a72dcdd@mail.gmail.com> <47DF0470.5060108@aon.at> <79990c6b0803180239r5167d5b8x8c9d2d74641a323@mail.gmail.com> <47DFC5A9.90401@v.loewis.de> Message-ID: <79990c6b0803180707o69ca3210n227e0fca1dc2cfb8@mail.gmail.com> On 18/03/2008, "Martin v. L?wis" wrote: > > Martin, can you comment? It looks like the 3.0 installer uses 2 copies > > of msvcr90.dll, where the 2.6 one doesn't. I would have thought that > > only one is necessary, but Gregor's experiments seem to demonstrate > > otherwise. > > I haven't figured it out myself; it's a complete mess, and Microsoft > is heavily wasting our time. > > It seems that you absolutely *must* have the manifest file in each > directory that has a DLL which links with the CRT. Whether or not > separate copies of the DLL are then also necessary, and whether or > not that causes two copies to be loaded into the address space, > I don't know. HELP!!!! I'll see if I can wade through the documentation and offer any help. > To reproduce the problem, you probably have to test on a machine > which doesn't have the CRT redistributable installed centrally > (neither through VS 2008 installation, nor by running the > standalone CRT installer, nor by having installed any other software > that provides an SxS copy of the CRT). That shouldn't be hard - I'll set up a Windows virtual machine with no additional software on it and can use that for testing. If you want me to try anything out, let me know and I can do so in a guaranteed clean environment. Paul. From martin at v.loewis.de Tue Mar 18 15:23:53 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 18 Mar 2008 09:23:53 -0500 Subject: [Python-Dev] Installing Python 2.6 alpha1 on Windows XP In-Reply-To: <79990c6b0803180707o69ca3210n227e0fca1dc2cfb8@mail.gmail.com> References: <47DEEED6.7070904@aon.at> <79990c6b0803171625u5159380at46e4464b2a72dcdd@mail.gmail.com> <47DF0470.5060108@aon.at> <79990c6b0803180239r5167d5b8x8c9d2d74641a323@mail.gmail.com> <47DFC5A9.90401@v.loewis.de> <79990c6b0803180707o69ca3210n227e0fca1dc2cfb8@mail.gmail.com> Message-ID: <47DFD079.2030102@v.loewis.de> > That shouldn't be hard - I'll set up a Windows virtual machine with no > additional software on it and can use that for testing. If you want me > to try anything out, let me know and I can do so in a guaranteed clean > environment. That should be a reasonable setup. Try moving the manifest files and the copies of the CRT around, in the 2.6 installer which (reportedly) doesn't work; you should try to reproduce the error Gregor had first. Don't use the "all users" install, but the per-user one; for the all-users case, I need to add proper SxS support, installing into the assembly cache, which currently isn't implemented. Then, when it does work, try to figure out how to eliminate the extra copy of the CRT. Perhaps the manifest needs to be tweaked? Regards, Martin From martin at v.loewis.de Tue Mar 18 15:25:51 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 18 Mar 2008 09:25:51 -0500 Subject: [Python-Dev] test_support.have_unicode In-Reply-To: <47DFCBD3.9040507@cheimes.de> References: <2a70578d0803180618p2dbd7241s85db8de30d82472a@mail.gmail.com> <47DFC7E9.2060006@v.loewis.de> <47DFCBD3.9040507@cheimes.de> Message-ID: <47DFD0EF.2060007@v.loewis.de> > About two months ago I fixed the most critical bugs but the unicode free > build is treated like a poor cousin at best. It's neither actively > developed nor tested in regular intervals. IMO it's a deprecation candiate. In the sense that 3k won't support it anymore - certainly. In the sense that it will be removed in 2.7: Why? I'd rather say it's unmaintained, not deprecated. Regards, Martin From lists at cheimes.de Tue Mar 18 15:22:18 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 18 Mar 2008 15:22:18 +0100 Subject: [Python-Dev] Installing Python 2.6 alpha1 on Windows XP In-Reply-To: <79990c6b0803180707o69ca3210n227e0fca1dc2cfb8@mail.gmail.com> References: <47DEEED6.7070904@aon.at> <79990c6b0803171625u5159380at46e4464b2a72dcdd@mail.gmail.com> <47DF0470.5060108@aon.at> <79990c6b0803180239r5167d5b8x8c9d2d74641a323@mail.gmail.com> <47DFC5A9.90401@v.loewis.de> <79990c6b0803180707o69ca3210n227e0fca1dc2cfb8@mail.gmail.com> Message-ID: <47DFD01A.6040901@cheimes.de> > That shouldn't be hard - I'll set up a Windows virtual machine with no > additional software on it and can use that for testing. If you want me > to try anything out, let me know and I can do so in a guaranteed clean > environment. I think I've a spare license of XP Home around somewhere. I'm going to install yet another XP in a VM, too. VMware supports snapshots and roll backs. I can try out multiple approaches and roll back the changes easily. Christian From barry at python.org Tue Mar 18 15:47:51 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 18 Mar 2008 09:47:51 -0500 Subject: [Python-Dev] [Python-3000] PEP 361: Python 2.6/3.0 release schedule In-Reply-To: <47DF6989.1010808@ieee.org> References: <47DF6989.1010808@ieee.org> Message-ID: <5F2A88CA-5C89-49C6-9226-B989C46D9D04@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 18, 2008, at 2:04 AM, Travis Oliphant wrote: > > Hey Barry, > > Thanks for putting this PEP together. This is really helpful. Hi Travis... thanks! > I didn't see discussion of PEP 3118 and it's features back-ported to > Python 2.6. I've already back-ported the new buffer API as an > addition > to the old buffer protocol. > > In addition, I've planned to back-port the improvements to the struct > module and the addition of the memoryview object (both in PEP 3118). > > If you have questions, we can talk tomorrow. Let's do that. Neal has put together a list of things that he thinks needs to be backported. We should formalize that list (as best we can given we're still in alpha), and add it to the PEP. I think we should also make sure we have open issues in the tracker for each backporting task. Neal and I talked about this yesterday too and came up with some general guidelines. The view we have is that through conservative use of future imports and backports, we want to start closing the gap between 2.6 and 3.0. It'll still be fairly wide though, so we'll use 2.7/3.1 to close it even further by doing things like backporting more stuff, de-futurizing features in 2.6, etc. It may be that we should take a very conservative approach to new features in the next few major release (in both families), concentrating instead on stabilizing what we've got and helping make the transition less and less painful. So you could imagine that 2.8/3.2 would close the gap far enough that it wouldn't be much more work to move from 2.8 to 3.2 than it would be to move from 2.5 to 2.6. Some other thoughts: we might want to shorten our post 2.6/3.0 release cycles, say to 1 year or less so that we can help speed this transition. If we try hard to keep new features to a minimum, this might be possible, but we also have to avoid churn. Quite a few people expressed to me that it might be 5 years before they switch fully to 3.0. That seems fine to me, given that some big Python shops are still on 2.2 or 1.5 and 1.6. So 3 years from 2.6/3.0 to 2.8/3.2 doesn't seem to wildly outrageous to me. Given this longer view of the transition, we can be more conservative about what we backport to 2.6. A general principle would be anything that is brand new syntax in 3.0 can be backported, because there won't be any existing code in 2.6 that could break. Anything that can be enabled through a future-import might be a candidate, but you have to be careful about tricky semantic differences. For example, changing the meaning of dict.keys() via a future-import is not a good idea. One thing I would like to see is "from __future__ import unicode_strings" or some such. The only thing this would do is allow you to write unicode string literals in Python 2.6 without the u'' prefix. That's a fairly localized change (affecting just the file the literals appear in), but it would go a long way toward closing that gap. One question that came up was whether we have enough bitfields to handle all the future statements we might have, and whether you even want to see Python 2.6 code sprinkled with lots of future statements. Does it make sense to have a "from __future__ import python3" umbrella? Will we really have that many future statements? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR9/WGHEjvBPtnXfVAQLP1AP+OpSuXHDgE1uxifKA6FEKxS1Zms1Pe0ww OimG2kw3afzL5+o1mdrRBUDy/rETpNhlxBTgx+fgI7VJc+Vs+uBi0sQwemCqOZ1I 9qlBFCo8YrmXlCZTdL9E0nEpiBSuanLKJcdNP8VU3QjbOX0EKqNTfK1asSckxvgT H1o5wGqnX5M= =wiJQ -----END PGP SIGNATURE----- From guido at python.org Tue Mar 18 16:51:32 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 18 Mar 2008 10:51:32 -0500 Subject: [Python-Dev] Change in priority fields In-Reply-To: <53B76010-C290-41C8-AC87-076A053ACCFC@python.org> References: <6527F3DE-7122-4570-81A2-B53E5AF8D0C4@python.org> <53B76010-C290-41C8-AC87-076A053ACCFC@python.org> Message-ID: On Tue, Mar 18, 2008 at 8:39 AM, Barry Warsaw wrote: > On Mar 18, 2008, at 12:11 AM, Guido van Rossum wrote: > > Hm... This feels a bit like inflation of priorities to me; there would > > be lots of critical bugs and quite a few release blockers... The > > highest priority just pertains to the upcoming release which could be > > weeks in the future. > > I want a very simple roundup query that I can consult during the > release cycle to know whether everything's fine, or whether I have to > start pestering people and pull the big red "STOP RELEASE" button. > Critical gives us a priority for things we really need to fix, but > maybe not right this second. Sure. My (mild) concern is that (a) the terms chosen sound a bit alarming, and (b) there's no term left for even *more* alarming issues. I also want to express my concern that this sounds like a bit *too* automatic and draconian. The key goal here (well, mine in any case :-) is to manage developers, not to get releases out at all cost. Even though the release schedule is set in stone, I think we should send out a variety of reminders ahead of each scheduled release, and these reminders must come from a human, not from a bot. There should be some kind of consensus that we're all comfortable with releasing the code in the current state -- even for an alpha release -- and that's not necessarily expressed as showstopper bugs. Maybe the reminder mail could include an exhortation to create new showstopper issues for anything that a developer feels should really be addressed before the release, even if it's not thought of a bug. The release reminder emails act as a kind of wake-up call to many developers. Another little trick we're using in my group at Google is to have a bug that tracks a specific release. I've found this is particularly handy once releases are done from release branches, and fixes in the dev branch need to be "cherry-picked". But if you just want to use the mailing list for this that's probably fine too. > > I'd be more comfortable if there were 1-2 > > priorities above that, e.g. one higher for stuff that makes a buildbot > > go red (i.e. breaks a test) and two higher for stuff that affects most > > developers (e.g. stuff checked in that doesn't even build). > > I think neither of these use cases should get that far. Neal and I > talked it over and we're in agreement that if a commit makes the > buildbots go red or breaks the build, we're going to just revert it. > Tough luck Joe Dev, please test your changes more carefully next time. Sure. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Mar 18 16:55:08 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 18 Mar 2008 10:55:08 -0500 Subject: [Python-Dev] test_support.have_unicode In-Reply-To: <47DFD0EF.2060007@v.loewis.de> References: <2a70578d0803180618p2dbd7241s85db8de30d82472a@mail.gmail.com> <47DFC7E9.2060006@v.loewis.de> <47DFCBD3.9040507@cheimes.de> <47DFD0EF.2060007@v.loewis.de> Message-ID: On Tue, Mar 18, 2008 at 9:25 AM, "Martin v. L?wis" wrote: > > About two months ago I fixed the most critical bugs but the unicode free > > build is treated like a poor cousin at best. It's neither actively > > developed nor tested in regular intervals. IMO it's a deprecation candiate. > > In the sense that 3k won't support it anymore - certainly. > > In the sense that it will be removed in 2.7: Why? > > I'd rather say it's unmaintained, not deprecated. Right. It's a Py3k warning candidate. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jmillikin at gmail.com Tue Mar 18 17:17:32 2008 From: jmillikin at gmail.com (John Millikin) Date: Tue, 18 Mar 2008 11:17:32 -0500 Subject: [Python-Dev] PEP 361: Python 2.6/3.0 release schedule In-Reply-To: References: Message-ID: <3283f7fe0803180917o459f52d5x32691b1d6d87a8ef@mail.gmail.com> > Possible features for 2.6 > New modules in the standard library: > - JSON implementation > Have there been any plans made for which one? All of the implementations I'm aware of (cjson, simplejson, jsonlib, demjson, python-json) have serious issues that in my opinion should be fixed before inclusion in the standard library[1]. I am the author of jsonlib, and am willing to write patches for whichever implementation becomes the standard, but it would be very nice to know what to focus on. Apologies if this has been discussed already, but there was no link to a PEP in 361 and a search of python-dev and c.l.p returned no relevant results. [1] * cjson and python-json have almost complete absense of Unicode support. * simplejson is slow and writes incorrect unicode escapes. * demjson is far too forgiving when parsing and accepts all sorts of invalid input. * jsonlib is not PEP 8-compliant and has terrible error handling. * python-json is abandoned, slow, and lacks Unicode support. From theller at ctypes.org Tue Mar 18 17:17:54 2008 From: theller at ctypes.org (Thomas Heller) Date: Tue, 18 Mar 2008 17:17:54 +0100 Subject: [Python-Dev] [Python-3000] PEP 361: Python 2.6/3.0 release schedule In-Reply-To: <5F2A88CA-5C89-49C6-9226-B989C46D9D04@python.org> References: <47DF6989.1010808@ieee.org> <5F2A88CA-5C89-49C6-9226-B989C46D9D04@python.org> Message-ID: Barry Warsaw schrieb: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Mar 18, 2008, at 2:04 AM, Travis Oliphant wrote: >> >> Hey Barry, >> >> Thanks for putting this PEP together. This is really helpful. > > Hi Travis... thanks! > >> I didn't see discussion of PEP 3118 and it's features back-ported to >> Python 2.6. I've already back-ported the new buffer API as an >> addition >> to the old buffer protocol. >> >> In addition, I've planned to back-port the improvements to the struct >> module and the addition of the memoryview object (both in PEP 3118). >> >> If you have questions, we can talk tomorrow. > > Let's do that. Neal has put together a list of things that he thinks > needs to be backported. We should formalize that list (as best we can > given we're still in alpha), and add it to the PEP. I think we should > also make sure we have open issues in the tracker for each backporting > task. > I think that [issue1971] ctypes exposing the pep 3118 buffer interface should / could also be backported to 2.6 (once it is merged into py3k). Thomas From pje at telecommunity.com Tue Mar 18 17:23:29 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 18 Mar 2008 12:23:29 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317161305.22B183A409D@sparrow.telecommunity.com> <47DEA275.4080306@behnel.de> <20080317174542.89A463A409D@sparrow.telecommunity.com> <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> Message-ID: <20080318162331.C820D3A4074@sparrow.telecommunity.com> At 12:31 AM 3/18/2008 -0500, Guido van Rossum wrote: >I am hoping that someone will create a simpler bootstrap module that >is able to download a file of pure Python code and install it, perhaps >by running its setup.py, assuming that it only depends on distutils >(or other things previously installed). I will welcome such a module >into the stdlib. I'm not sure a PEP is even needed, though interested >parties are certainly welcome to write a PEP specifying the behavior >first. With 2.6 and 3.0 slated for release in September, there should >be enough time to get this done before then. Unfortunately, as I've already tried to explain, "download a file ... and install it" is not a sufficiently well-specified requirement to implement a robust tool. Even if it is not to support arbitrary existing distutils sources, there still needs to be a way to document precisely what the tool does and does not support installing, so that users can produce correct files for it to consume, register them properly with PyPI, etc. And as I said before (perhaps not very well) the distutils documentation has already proven to be inadequate as such a specification, both for users to create these files -- and even more important -- for programs to consume them. (For example, distutils source distribution tarball filenames are not unambiguously machine-parseable.) That's why I think some specific "format" (i.e. conventions) have to be defined for this to work, even if it's merely a set of documented restrictions on a distutils-based layout, file naming conventions, versioning, etc. In other words, you can't have your cake and eat it, too. If there's to be a bootstrap tool, you must bless *some* set of packaging conventions, including file naming, version parsing, and so on. Can we use setuptools' version parsing scheme to identify the "latest stable version", for example? What about setuptools' filename component canonicalization and escaping rules? Frankly, I don't care what the conventions are, only that they be unambiguously defined and reasonably implementable for producers and consumers alike. I just want there to be *some* sort of robust, documented, standard installation bootstrap vector in the stdlib, so that setuptools, zc.buildout, and virtualenv don't have to maintain their own (or depend on setuptools maintaining its own). But not only have you rejected the *only* existing robust and well-documented conventions for automated processing of Python libraries, you say you "have no time for this part of the thread" when I ask what conventions you want to bless *instead*. So I'm at a bit of a loss for what we're supposed to do now. From nnorwitz at gmail.com Tue Mar 18 17:43:25 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 18 Mar 2008 11:43:25 -0500 Subject: [Python-Dev] adding json to stdlib (was: Re: PEP 361: Python 2.6/3.0 release schedule) Message-ID: On Tue, Mar 18, 2008 at 11:17 AM, John Millikin wrote: > > Possible features for 2.6 > > New modules in the standard library: > > - JSON implementation > > > Have there been any plans made for which one? All of the No. This was something I added as a nice to have for 2.6. > implementations I'm aware of (cjson, simplejson, jsonlib, demjson, > python-json) have serious issues that in my opinion should be fixed > before inclusion in the standard library[1]. I am the author of > jsonlib, and am willing to write patches for whichever implementation > becomes the standard, but it would be very nice to know what to focus > on. I'm not familiar enough with any of the libraries to comment. If it's premature to add a particular implementation, that's fine, we can wait. > Apologies if this has been discussed already, but there was no link to > a PEP in 361 and a search of python-dev and c.l.p returned no relevant > results. I don't believe it has been discussed before. I've changed the subject and would like to discuss this now. It would be great if you could pull together and get the community behind a single solution that is robust enough to include in the stdlib. If that an be finished for 2.6, great. If it waits for 2.7, that would still be fine. n > > [1] > * cjson and python-json have almost complete absense of Unicode support. > * simplejson is slow and writes incorrect unicode escapes. > * demjson is far too forgiving when parsing and accepts all sorts of > invalid input. > * jsonlib is not PEP 8-compliant and has terrible error handling. > * python-json is abandoned, slow, and lacks Unicode support. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > From douglas at openplans.org Tue Mar 18 17:35:36 2008 From: douglas at openplans.org (Douglas Mayle) Date: Tue, 18 Mar 2008 12:35:36 -0400 Subject: [Python-Dev] PEP 361: Python 2.6/3.0 release schedule In-Reply-To: <3283f7fe0803180917o459f52d5x32691b1d6d87a8ef@mail.gmail.com> References: <3283f7fe0803180917o459f52d5x32691b1d6d87a8ef@mail.gmail.com> Message-ID: I keep forgetting to reply to the list: This is great news. I was getting ready to submit a patch to fix the Unicode handling in simplejson (which I will probably do anyway), but I'm interested in making sure whichever lib will hit the standard library has a correct implementation. Doug On Mar 18, 2008, at 12:17 PM, John Millikin wrote: >> Possible features for 2.6 >> New modules in the standard library: >> - JSON implementation >> > Have there been any plans made for which one? All of the > implementations I'm aware of (cjson, simplejson, jsonlib, demjson, > python-json) have serious issues that in my opinion should be fixed > before inclusion in the standard library[1]. I am the author of > jsonlib, and am willing to write patches for whichever implementation > becomes the standard, but it would be very nice to know what to focus > on. > > Apologies if this has been discussed already, but there was no link to > a PEP in 361 and a search of python-dev and c.l.p returned no relevant > results. > > > [1] > * cjson and python-json have almost complete absense of Unicode > support. > * simplejson is slow and writes incorrect unicode escapes. > * demjson is far too forgiving when parsing and accepts all sorts of > invalid input. > * jsonlib is not PEP 8-compliant and has terrible error handling. > * python-json is abandoned, slow, and lacks Unicode support. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/douglas%40openplans.org > > !DSPAM:4037,47dfeb7b286095409313003! > From mhammond at keypoint.com.au Tue Mar 18 18:05:37 2008 From: mhammond at keypoint.com.au (mhammond at keypoint.com.au) Date: Wed, 19 Mar 2008 02:05:37 +0900 (WST) Subject: [Python-Dev] Consistent platform name for 64bit windows (was: distutils.util.get_platform() for Windows) Message-ID: <3504.63.250.241.10.1205859937.squirrel@mail.eftel.com> I'm reviving a very old thread based on discussions with Martin at pycon. > Sent: Monday, 23 July 2007 5:12 PM > Subject: Re: [Distutils] distutils.util.get_platform() for Windows Rather than forcing everyone to read the context, allow me to summarize: On 64bit Windows versions, we need a "string" that identifies the platform, and this string should ideally be used consistently. This original thread related to the files created by distutils (eg, pywin32-210.win???64??-py2.6.exe) but it seems obvious that we should be consistent wherever Python wants to display the platform (eg, in the startup banner, in platform.py, etc). In the old thread, there was a semi-consensus that 'x86_64' be used by distutils (and indeed, Lib/distutils/util.py in get_platform() has been changed, by me, to use this string), but the Python 'banner', for example, reports AMD64. Platform.py doesn't report much at all in this area, at least when pywin32 isn't installed, but it arguably should. Both Martin and I prefer AMD64 as the string, for various reasons. Firstly, it is less ugly than 'x86_64', and doesn't include an '_'/'-' which might tend to confuse parsing by humans or computers. Martin also made the point that AMD invented the architecture and AMD64 is their preferred name, so we should respect that. So, at the risk of painting a bike-shed, I'd like to propose that we adopt 'AMD64' in distutils (needs a change), platform.py (needs a change to use sys.getwindowsversion() in preference to pywin32, if possible, anyway), and the Python banner (which already uses AMD64). Any objections? Any strong feelings that using 'AMD' will confuse people with Intel processors? Strong feelings about the parsability of the name (PJE? )? Strong feelings about the color ? Thanks, Mark From lists at cheimes.de Tue Mar 18 18:54:08 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 18 Mar 2008 18:54:08 +0100 Subject: [Python-Dev] Consistent platform name for 64bit windows (was: distutils.util.get_platform() for Windows) In-Reply-To: <3504.63.250.241.10.1205859937.squirrel@mail.eftel.com> References: <3504.63.250.241.10.1205859937.squirrel@mail.eftel.com> Message-ID: <47E001C0.4000101@cheimes.de> mhammond at keypoint.com.au schrieb: > So, at the risk of painting a bike-shed, I'd like to propose that we adopt > 'AMD64' in distutils (needs a change), platform.py (needs a change to use > sys.getwindowsversion() in preference to pywin32, if possible, anyway), > and the Python banner (which already uses AMD64). +1 for AMD64 If we ever need names for Itanium and i386 compatible arch I propose IA64 and X86. Christian From martin at v.loewis.de Fri Mar 14 13:00:43 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMwpp3aXMi?=) Date: Fri, 14 Mar 2008 07:00:43 -0500 Subject: [Python-Dev] Windows x64 & bsddb 4.4.20 woes In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE0D@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E1684541D@EXMBX04.exchhosting.com>, <52dc1c820803131900vd889255q4bd6fa0ff52d7c53@mail.gmail.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE0D@EXMBX04.exchhosting.com> Message-ID: <47DA68EB.6050105@v.loewis.de> > The other query I had was whether or not I should try a later version > of BerkeleyDB -- are we committed to 4.4.20 (or 4.4.x?) for 2.6/3.0 > or is it worth investigating newer versions? Martin/Jesus, any > thoughts on this? As Greg says: 4.5.x should work fine. Importing a new version into the build process is a big effort, though, and should only be done if either a) the beta releases are close, so the new version is the one we'll ship, or b) factual improvements can be demonstrated with a new version. Regards, Martin From martin at v.loewis.de Fri Mar 14 12:56:22 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMwpp3aXMi?=) Date: Fri, 14 Mar 2008 06:56:22 -0500 Subject: [Python-Dev] Windows x64 & bsddb 4.4.20 woes In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E1684541D@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E1684541D@EXMBX04.exchhosting.com> Message-ID: <47DA67E6.8050205@v.loewis.de> > Martin, you've changed externals/bsddb-4.4.20 with regards to x64 > builds recently -- have you been able to get things working in your > x64 environments? I changed the project files so that it will use the x64 compilers even if no environment variables were set - the original bsddb files expected that you run VS with /useenv, in an SDK x64 build environment. As a consequence, it now builds; I have never bothered testing that it actually works. Regards, Martin From tnelson at onresolve.com Tue Mar 18 19:02:24 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Tue, 18 Mar 2008 11:02:24 -0700 Subject: [Python-Dev] Consistent platform name for 64bit windows (was: distutils.util.get_platform() for Windows) In-Reply-To: <47E001C0.4000101@cheimes.de> References: <3504.63.250.241.10.1205859937.squirrel@mail.eftel.com>, <47E001C0.4000101@cheimes.de> Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE20@EXMBX04.exchhosting.com> +1 for avoiding a bikeshed, so +1 to AMD64. ________________________________________ From: python-dev-bounces+tnelson=onresolve.com at python.org [python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of Christian Heimes [lists at cheimes.de] Sent: 18 March 2008 13:54 To: mhammond at keypoint.com.au Cc: distutils-sig at python.org; python-dev at python.org Subject: Re: [Python-Dev] Consistent platform name for 64bit windows (was: distutils.util.get_platform() for Windows) mhammond at keypoint.com.au schrieb: > So, at the risk of painting a bike-shed, I'd like to propose that we adopt > 'AMD64' in distutils (needs a change), platform.py (needs a change to use > sys.getwindowsversion() in preference to pywin32, if possible, anyway), > and the Python banner (which already uses AMD64). +1 for AMD64 If we ever need names for Itanium and i386 compatible arch I propose IA64 and X86. Christian _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/tnelson%40onresolve.com From nnorwitz at gmail.com Tue Mar 18 19:11:34 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 18 Mar 2008 13:11:34 -0500 Subject: [Python-Dev] Change in priority fields In-Reply-To: References: <6527F3DE-7122-4570-81A2-B53E5AF8D0C4@python.org> <53B76010-C290-41C8-AC87-076A053ACCFC@python.org> Message-ID: On Tue, Mar 18, 2008 at 10:51 AM, Guido van Rossum wrote: > The key goal here (well, mine in any case :-) > is to manage developers, not to get releases out at all cost. Even > though the release schedule is set in stone, I think we should send > out a variety of reminders ahead of each scheduled release, and these > reminders must come from a human, not from a bot. > ... > Maybe the reminder mail > could include an exhortation to create new showstopper issues for > anything that a developer feels should really be addressed before the > release, even if it's not thought of a bug. The release reminder > emails act as a kind of wake-up call to many developers. I think I did this for 2.5 and plan to do improve communications for 2.6. I'll need to work the details out with Barry, but it is my intention to communicate as much as possible. The next release (in two weeks) will probably be a little haphazard as I try to get up to date after pycon. I'll try to get more organized so all subsequent releases go smoothly. Suggestions to python-dev on how to improve the process are always encouraged. n From jon+python-dev at unequivocal.co.uk Tue Mar 18 19:22:07 2008 From: jon+python-dev at unequivocal.co.uk (Jon Ribbens) Date: Tue, 18 Mar 2008 18:22:07 +0000 Subject: [Python-Dev] Consistent platform name for 64bit windows (was: distutils.util.get_platform() for Windows) In-Reply-To: <3504.63.250.241.10.1205859937.squirrel@mail.eftel.com> References: <3504.63.250.241.10.1205859937.squirrel@mail.eftel.com> Message-ID: <20080318182207.GH29648@snowy.squish.net> On Wed, Mar 19, 2008 at 02:05:37AM +0900, mhammond at keypoint.com.au wrote: > So, at the risk of painting a bike-shed, I'd like to propose that we adopt > 'AMD64' in distutils (needs a change), platform.py (needs a change to use > sys.getwindowsversion() in preference to pywin32, if possible, anyway), > and the Python banner (which already uses AMD64). Debian uses 'amd64', it seems they chose this after a survey of what existing systems used what names, and the consensus came out in favour of amd64. http://lists.debian.org/debian-ctte/2004/06/msg00041.html From brett at python.org Tue Mar 18 20:50:46 2008 From: brett at python.org (Brett Cannon) Date: Tue, 18 Mar 2008 14:50:46 -0500 Subject: [Python-Dev] Introducing the ``make check`` command Message-ID: After having one too many commits be rejected by having trailing whitespace, I wrote a script that runs reindent.py over all .py files that are to be checked in. But since there are other things that one should be aware of when creating a patch I let people know if they edited anything in the Doc directory, Misc/ACKS, and Misc/NEWS. There are a couple of things I would like to see the command gain. One is to detect if any files (outside of Doc) have changed since the last run of regrtest. That would require writing out a file, though, so one can at least stat the file to get the modification time. Do people have any issues with the idea? I would also like to more or less codify whether a patch means someone needs to be added to Misc/ACKS or not. I should be able to run ``svn diff`` to get the output and see if enough lines have changed. Could then write it out to a common file so that one does not need to run the command again if the patch is needed. Lastly, I would like to have it strip trailing whitespace in C files. The only problem is that I don't know if removing trailing whitespace could possibly cause a problem or not. Anyone know? -Brett From jseutter at gmail.com Tue Mar 18 21:00:18 2008 From: jseutter at gmail.com (Jerry Seutter) Date: Tue, 18 Mar 2008 15:00:18 -0500 Subject: [Python-Dev] Introducing test coverage stats Message-ID: <2c8d48d70803181300n15be0956h7e994afd89846c68@mail.gmail.com> I have added a bugtracker issue that adds unit test coverage statistics. Issue 2403: http://bugs.python.org/issue2403 Directions are on the issue page. Titus: Brent suggested I bug you to review this. Test, complain, flame. Feedback welcome. Jerry Seutter -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080318/55284b40/attachment-0001.htm From jeff at taupro.com Tue Mar 18 21:20:19 2008 From: jeff at taupro.com (Jeff Rush) Date: Tue, 18 Mar 2008 15:20:19 -0500 Subject: [Python-Dev] [Distutils] Capsule Summary of Some Packaging/Deployment Technology Concerns In-Reply-To: <20080318192446.GB31013@fridge.pov.lt> References: <47DEEC6B.5040602@taupro.com> <20080318004718.56E173A40FF@sparrow.telecommunity.com> <20080318192446.GB31013@fridge.pov.lt> Message-ID: <47E02403.4040909@taupro.com> Marius Gedminas wrote: > On Mon, Mar 17, 2008 at 08:37:30PM -0400, Phillip J. Eby wrote: >> At 05:10 PM 3/17/2008 -0500, Jeff Rush wrote: >>> People also want a greater variety of file_finders to be included with >>> setuptools. Instead of just CVS and SVN, they want it to comprehend >>> Mercurial, Bazaar, Git and so forth. >> Did you point them to the Cheeseshop? There are plugins already >> available for all the systems you mentioned, plus Darcs and >> Monotone. If you mean "included" as in "bundled", this doesn't make >> a whole lot of sense to me. They knew there were plugins out there, of various quality and availability but wanted them bundled. ;-) It's a pain to track them down. Perhaps if the RPM format were broken out from setuptools, as the inclusion of some formats leads them to believe the set is just incomplete, not intentionally sparse. >> I'd think that if you're using >> setuptools as a developer (the only reason you need the file finders, >> since source distributions include a prebuilt manifest), you'd not >> have a problem saying "easy_install setuptools-git" or adding a >> "setup_requires='setuptools-git'" line to your setup.py. (Although >> the latter would only be needed for *development*, not deployment.) > > setup_requires looks like a solution, but it requires extra attention > from the developers who write the setup.py. Writing a setup.py is > already quite complicated -- I usually end up copying an existing one > and modifying it. As a compromise, of making new formats easily available but not bundled, and not requiring special action within setup.py, setuptools could treat --formats=dpkg as an implicit setup_requires= and pull it from PyPI. And the --list-formats option could query PyPI for the possibilities, just as --list-classifiers does today. If would require a few standards in keywording/classifying those format eggs but we already need those standards for other projects, such as locating recipes for buildout and plugins for trac. -Jeff From stephen at xemacs.org Tue Mar 18 21:45:53 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 19 Mar 2008 05:45:53 +0900 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317161305.22B183A409D@sparrow.telecommunity.com> <47DEA275.4080306@behnel.de> <20080317174542.89A463A409D@sparrow.telecommunity.com> <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> Message-ID: <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> Guido van Rossum writes: > I am hoping that someone will create a simpler bootstrap module FWIW (I've never tried to implement one of these things) I agree with Phillip. This is not possible in the sense you are advocating. Anything "simpler" is simply an invitation to an unending stream of issues, far more so than adopting eggs as a best current practice would. Better to Just Say No to an installer in the stdlib. > I'm not sure a PEP is even needed This I simply don't understand. I *have* participated in bolting on features to such systems, and it's a mess. As features are added, existing users will demand that old packages install exactly as they ever did, except that sometimes (but not always!) they want them upgraded to put things in newly blessed places. Features are easy to add, since on the main thread of control installers are basically just a sequence of single commands, sometimes optional. python-dev has some pretty good controls that will help slow such trends. Nonetheless, PEP-less development of an installer system is scary. Installer complexity is a creeping horror, worse than kudzu.[1] Footnotes: [1] http://en.wikipedia.org/wiki/Kudzu#Invasive_species From guido at python.org Tue Mar 18 21:40:03 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 18 Mar 2008 15:40:03 -0500 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <20080318162331.C820D3A4074@sparrow.telecommunity.com> References: <47DEA275.4080306@behnel.de> <20080317174542.89A463A409D@sparrow.telecommunity.com> <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <20080318162331.C820D3A4074@sparrow.telecommunity.com> Message-ID: On Tue, Mar 18, 2008 at 11:23 AM, Phillip J. Eby wrote: > At 12:31 AM 3/18/2008 -0500, Guido van Rossum wrote: > >I am hoping that someone will create a simpler bootstrap module that > >is able to download a file of pure Python code and install it, perhaps > >by running its setup.py, assuming that it only depends on distutils > >(or other things previously installed). I will welcome such a module > >into the stdlib. I'm not sure a PEP is even needed, though interested > >parties are certainly welcome to write a PEP specifying the behavior > >first. With 2.6 and 3.0 slated for release in September, there should > >be enough time to get this done before then. > > Unfortunately, as I've already tried to explain, "download a file ... > and install it" is not a sufficiently well-specified requirement to > implement a robust tool. > > Even if it is not to support arbitrary existing distutils sources, > there still needs to be a way to document precisely what the tool > does and does not support installing, so that users can produce > correct files for it to consume, register them properly with PyPI, etc. > > And as I said before (perhaps not very well) the distutils > documentation has already proven to be inadequate as such a > specification, both for users to create these files -- and even more > important -- for programs to consume them. (For example, distutils > source distribution tarball filenames are not unambiguously machine-parseable.) > > That's why I think some specific "format" (i.e. conventions) have to > be defined for this to work, even if it's merely a set of documented > restrictions on a distutils-based layout, file naming conventions, > versioning, etc. > > In other words, you can't have your cake and eat it, too. If there's > to be a bootstrap tool, you must bless *some* set of packaging > conventions, including file naming, version parsing, and so on. > > Can we use setuptools' version parsing scheme to identify the "latest > stable version", for example? What about setuptools' filename > component canonicalization and escaping rules? > > Frankly, I don't care what the conventions are, only that they be > unambiguously defined and reasonably implementable for producers and > consumers alike. > > I just want there to be *some* sort of robust, documented, standard > installation bootstrap vector in the stdlib, so that setuptools, > zc.buildout, and virtualenv don't have to maintain their own (or > depend on setuptools maintaining its own). > > But not only have you rejected the *only* existing robust and > well-documented conventions for automated processing of Python > libraries, you say you "have no time for this part of the thread" > when I ask what conventions you want to bless *instead*. > > So I'm at a bit of a loss for what we're supposed to do now. You're welcome to write a PEP as long as it is self-contained (at best referencing other accepted PEPs like the PyPI metadata PEPs). And the PEP better not be 2500 lines long. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Mar 18 21:43:41 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 18 Mar 2008 15:43:41 -0500 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> References: <47DEA275.4080306@behnel.de> <20080317174542.89A463A409D@sparrow.telecommunity.com> <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: There seems to be a misunderstanding about what I am proposing we do instead. The boostrap installer should only be powerful enough to allow it to be used to install a real package manager like setuptools. Maybe my use of Django as an example was confusing; I didn't actually mean that it should be possible to support installing Django directly (although I don't understand all the issure that would make it impossible). Only very few people would care about writing a setup script that works with this bootstrap module; basically only package manager implementers. --Guido On Tue, Mar 18, 2008 at 3:45 PM, Stephen J. Turnbull wrote: > Guido van Rossum writes: > > > I am hoping that someone will create a simpler bootstrap module > > FWIW (I've never tried to implement one of these things) I agree with > Phillip. This is not possible in the sense you are advocating. > Anything "simpler" is simply an invitation to an unending stream of > issues, far more so than adopting eggs as a best current practice > would. Better to Just Say No to an installer in the stdlib. > > > > I'm not sure a PEP is even needed > > This I simply don't understand. I *have* participated in bolting on > features to such systems, and it's a mess. As features are added, > existing users will demand that old packages install exactly as they > ever did, except that sometimes (but not always!) they want them > upgraded to put things in newly blessed places. Features are easy to > add, since on the main thread of control installers are basically just > a sequence of single commands, sometimes optional. > > python-dev has some pretty good controls that will help slow such > trends. Nonetheless, PEP-less development of an installer system is > scary. Installer complexity is a creeping horror, worse than > kudzu.[1] > > > Footnotes: > [1] http://en.wikipedia.org/wiki/Kudzu#Invasive_species -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jseutter at gmail.com Tue Mar 18 21:47:12 2008 From: jseutter at gmail.com (Jerry Seutter) Date: Tue, 18 Mar 2008 15:47:12 -0500 Subject: [Python-Dev] Introducing test coverage stats In-Reply-To: <2c8d48d70803181300n15be0956h7e994afd89846c68@mail.gmail.com> References: <2c8d48d70803181300n15be0956h7e994afd89846c68@mail.gmail.com> Message-ID: <2c8d48d70803181347k4c24800dm315c1547a9162a6f@mail.gmail.com> s/Brent/Brett (sorry Brett. We still love you.) On Tue, Mar 18, 2008 at 3:00 PM, Jerry Seutter wrote: > I have added a bugtracker issue that adds unit test coverage statistics. > Issue 2403: > > http://bugs.python.org/issue2403 > > Directions are on the issue page. > > Titus: Brent suggested I bug you to review this. > > Test, complain, flame. Feedback welcome. > > Jerry Seutter > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080318/48b7aa1a/attachment.htm From wolever at cs.toronto.edu Tue Mar 18 21:54:05 2008 From: wolever at cs.toronto.edu (David Wolever) Date: Tue, 18 Mar 2008 16:54:05 -0400 Subject: [Python-Dev] map, filter, zip in future_builtins Message-ID: <96A6EE23-1300-4ABD-BF6E-A1AC6D2FB742@cs.toronto.edu> I'm working on #2171 -- putting map, filter, zip in 2.6's future_builtins. It has been suggested that it would be simplest to just return itertools.(imap, izip, ifilter), which is what py3k/Python/ bltinmodule.c, revision 61356 did. The advantage of this is that it's really easy and the behaviour seems to be identical. The disadvantage is that the two aren't identical: >>> type(map(lambda x: x, [1, 2, 3])) # Python 3 >>> type(map(lambda x: x, [1, 2, 3])) == map True >>> type(map(lambda x: x, [1, 2, 3])) # Python 2.6, with the patch >>> type(map(lambda x: x, [1, 2, 3])) == map False Recommendations? From guido at python.org Tue Mar 18 22:04:40 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 18 Mar 2008 16:04:40 -0500 Subject: [Python-Dev] adding json to stdlib (was: Re: PEP 361: Python 2.6/3.0 release schedule) In-Reply-To: References: Message-ID: On Tue, Mar 18, 2008 at 11:43 AM, Neal Norwitz wrote: > On Tue, Mar 18, 2008 at 11:17 AM, John Millikin wrote: > > > Possible features for 2.6 > > > New modules in the standard library: > > > - JSON implementation > > > > > Have there been any plans made for which one? All of the > > No. This was something I added as a nice to have for 2.6. I'd like to tentatively elevate it to a must have. There has been overwhelming support for this on web-sig. > > implementations I'm aware of (cjson, simplejson, jsonlib, demjson, > > python-json) have serious issues that in my opinion should be fixed > > before inclusion in the standard library[1]. I am the author of > > jsonlib, and am willing to write patches for whichever implementation > > becomes the standard, but it would be very nice to know what to focus > > on. > > I'm not familiar enough with any of the libraries to comment. If it's > premature to add a particular implementation, that's fine, we can > wait. On web-sig, the overwhelming majority wants simplejson, since that's the API they already use. (All current web frameworks use it.) > > Apologies if this has been discussed already, but there was no link to > > a PEP in 361 and a search of python-dev and c.l.p returned no relevant > > results. > > I don't believe it has been discussed before. I've changed the > subject and would like to discuss this now. > > It would be great if you could pull together and get the community > behind a single solution that is robust enough to include in the > stdlib. If that an be finished for 2.6, great. If it waits for 2.7, > that would still be fine. This is happening on web-sig as we speak. --Guido > n > > > > > [1] > > * cjson and python-json have almost complete absense of Unicode support. > > * simplejson is slow and writes incorrect unicode escapes. > > * demjson is far too forgiving when parsing and accepts all sorts of > > invalid input. > > * jsonlib is not PEP 8-compliant and has terrible error handling. > > * python-json is abandoned, slow, and lacks Unicode support. > > > > > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ijmorlan at cs.uwaterloo.ca Tue Mar 18 22:09:14 2008 From: ijmorlan at cs.uwaterloo.ca (Isaac Morland) Date: Tue, 18 Mar 2008 17:09:14 -0400 (EDT) Subject: [Python-Dev] Introducing the ``make check`` command In-Reply-To: References: Message-ID: On Tue, 18 Mar 2008, Brett Cannon wrote: > Lastly, I would like to have it strip trailing whitespace in C files. > The only problem is that I don't know if removing trailing whitespace > could possibly cause a problem or not. Anyone know? I would be worried about the sequence backslash-space-newline. Off the top of my head I can't see why that would occur in valid code, but dropping the space would give you an escaped newline which could be different from the original. I'd be worried about some weird case related to macro expansion or definition. http://gcc.gnu.org/onlinedocs/cppinternals/Lexer.html has some information related to the Gnu C preprocessor which may be relevant. Have you considered also forcing Mac "\r" and DOS "\r\n" line endings to Unix "\n" style? Isaac Morland CSCF Web Guru DC 2554C, x36650 WWW Software Specialist From guido at python.org Tue Mar 18 22:10:56 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 18 Mar 2008 16:10:56 -0500 Subject: [Python-Dev] map, filter, zip in future_builtins In-Reply-To: <96A6EE23-1300-4ABD-BF6E-A1AC6D2FB742@cs.toronto.edu> References: <96A6EE23-1300-4ABD-BF6E-A1AC6D2FB742@cs.toronto.edu> Message-ID: On Tue, Mar 18, 2008 at 3:54 PM, David Wolever wrote: > I'm working on #2171 -- putting map, filter, zip in 2.6's > future_builtins. > It has been suggested that it would be simplest to just return > itertools.(imap, izip, ifilter), which is what py3k/Python/ > bltinmodule.c, revision 61356 did. > > The advantage of this is that it's really easy and the behaviour > seems to be identical. > The disadvantage is that the two aren't identical: > >>> type(map(lambda x: x, [1, 2, 3])) # Python 3 > > >>> type(map(lambda x: x, [1, 2, 3])) == map > True > > >>> type(map(lambda x: x, [1, 2, 3])) # Python 2.6, with the patch > > >>> type(map(lambda x: x, [1, 2, 3])) == map > False > > Recommendations? Doesn't strike me as a terrible problem. Why is the latter == failing? What's the different between type(map(...)) and map? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steve at holdenweb.com Tue Mar 18 22:13:05 2008 From: steve at holdenweb.com (Steve Holden) Date: Tue, 18 Mar 2008 17:13:05 -0400 Subject: [Python-Dev] 3.0 buildbots all red In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD17D@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD173@EXMBX04.exchhosting.com> <47DD415F.7090109@v.loewis.de> <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD17D@EXMBX04.exchhosting.com> Message-ID: <47E03061.7000906@holdenweb.com> Trent Nelson wrote: >>> New sprint idea: getting all (inc. trunk) the buildbots green by >> Thursday. Anyone interested? >> >> I think the chance to achieve that is close to zero. > > Sounds like a challenge if ever I've heard one -- care to wager a beer on it? (Only applies to buildbots that are connected/online.) > > (FWIW, I've got the x64 Windows build green on my dev server, tcl/tk and bsddb required patching, so did some tests, and so did some C code -- I'm in the process of filtering the efforts back into the tracker.) > Make sure you get a screen shot for OnYourDesktop if/when they *do* go green! regards Steve From steve at holdenweb.com Tue Mar 18 22:13:05 2008 From: steve at holdenweb.com (Steve Holden) Date: Tue, 18 Mar 2008 17:13:05 -0400 Subject: [Python-Dev] 3.0 buildbots all red In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD17D@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD173@EXMBX04.exchhosting.com> <47DD415F.7090109@v.loewis.de> <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD17D@EXMBX04.exchhosting.com> Message-ID: <47E03061.7000906@holdenweb.com> Trent Nelson wrote: >>> New sprint idea: getting all (inc. trunk) the buildbots green by >> Thursday. Anyone interested? >> >> I think the chance to achieve that is close to zero. > > Sounds like a challenge if ever I've heard one -- care to wager a beer on it? (Only applies to buildbots that are connected/online.) > > (FWIW, I've got the x64 Windows build green on my dev server, tcl/tk and bsddb required patching, so did some tests, and so did some C code -- I'm in the process of filtering the efforts back into the tracker.) > Make sure you get a screen shot for OnYourDesktop if/when they *do* go green! regards Steve From nnorwitz at gmail.com Tue Mar 18 22:23:33 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 18 Mar 2008 16:23:33 -0500 Subject: [Python-Dev] changing regrtest to handle import errors Message-ID: [changing to: and subject: ] On Tue, Mar 18, 2008 at 4:09 PM, Guido van Rossum wrote: > On Tue, Mar 18, 2008 at 3:58 PM, neal.norwitz > wrote: > > Get this test to pass (UserList/UserDict no longer exist and caused a skip). > > I think the automatic skip on ImportError is harmful. > > We should add a helper function to test_support so that you can write > > foobar = test_support.import_optional('foobar') > > and it will skip the test if foobar cannot be imported; all other > failing imports should cause the test to *fail*. > > Any takers? This should be an easy two-part task. This would be a great starter project for a new developer. http://bugs.python.org/issue2409 Let me know if you could use some help. Feel free to contact me on or off list. n From wolever at cs.toronto.edu Tue Mar 18 22:26:04 2008 From: wolever at cs.toronto.edu (David Wolever) Date: Tue, 18 Mar 2008 17:26:04 -0400 Subject: [Python-Dev] map, filter, zip in future_builtins In-Reply-To: References: <96A6EE23-1300-4ABD-BF6E-A1AC6D2FB742@cs.toronto.edu> Message-ID: On 18-Mar-08, at 5:10 PM, Guido van Rossum wrote: > On Tue, Mar 18, 2008 at 3:54 PM, David Wolever > wrote: >>>>> type(map(lambda x: x, [1, 2, 3])) # Python 2.6, with the patch >> >>>>> type(map(lambda x: x, [1, 2, 3])) == map >> False > Doesn't strike me as a terrible problem. Excellent, I'll go ahead and do the same thing with filter and zip. > Why is the latter == failing? What's the different between > type(map(...)) and map? Because future_builtins.map imports and returns itertools.imap: def map(*args): from itertools import imap return imap(*args) From brett at python.org Tue Mar 18 22:29:40 2008 From: brett at python.org (Brett Cannon) Date: Tue, 18 Mar 2008 16:29:40 -0500 Subject: [Python-Dev] Introducing the ``make check`` command In-Reply-To: References: Message-ID: On Tue, Mar 18, 2008 at 4:09 PM, Isaac Morland wrote: > On Tue, 18 Mar 2008, Brett Cannon wrote: > > > Lastly, I would like to have it strip trailing whitespace in C files. > > The only problem is that I don't know if removing trailing whitespace > > could possibly cause a problem or not. Anyone know? > > I would be worried about the sequence backslash-space-newline. Off the > top of my head I can't see why that would occur in valid code, but > dropping the space would give you an escaped newline which could be > different from the original. I'd be worried about some weird case related > to macro expansion or definition. > > http://gcc.gnu.org/onlinedocs/cppinternals/Lexer.html has some information > related to the Gnu C preprocessor which may be relevant. > Weird code is not allowed. =) > Have you considered also forcing Mac "\r" and DOS "\r\n" line endings to > Unix "\n" style? > We let svn handle that. -Brett From dpeterson at enthought.com Tue Mar 18 21:40:10 2008 From: dpeterson at enthought.com (Dave Peterson) Date: Tue, 18 Mar 2008 15:40:10 -0500 Subject: [Python-Dev] [Distutils] Capsule Summary of Some Packaging/Deployment Technology Concerns In-Reply-To: <47DEEC6B.5040602@taupro.com> References: <47DEEC6B.5040602@taupro.com> Message-ID: <47E028AA.3070300@enthought.com> I've added your comments to a wiki page (http://wiki.python.org/moin/PackagingBOF) I was working on to summarize some of what went on during these BoF meeting, at least from the Enthought point-of-view. Unfortunately, I wasn't at the first night's event and don't yet have Travis Oliphant's notes on it here in front of me (he's still sprinting) so I only added some more detail to your comments, and also noted some previous issues we'd briefly discussed via e-mail with Phillip. It was great to see so many people interested in sharing their experiences and wanting to help things get better! As you can probably guess as a result of this being a two-night meeting, there wasn't enough time to discuss everything that needed to be brought up. It was suggested that a wiki page be created (see above) and that a new mailing list be setup for those who wanted to discuss further. (Some didn't feel the existing distutils-sig was appropriate.) I'll try to get the latter done shortly. -- Dave Jeff Rush wrote: > I was in a Packaging BoF yesterday and, although not very relevant to the > packager bootstrap thread, Guido has asked me to post some of the concerns. > > The BoF drew about 15 people... > From arnodel at googlemail.com Sun Mar 2 09:58:38 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sun, 02 Mar 2008 08:58:38 -0000 Subject: [Python-Dev] [Python-3000] No releases tonight In-Reply-To: References: <47C9A6DC.9070708@cheimes.de> <3B6D7A5F-2426-485E-BF5F-0BC16DF26A74@python.org> Message-ID: <827383A5-9EB5-4B02-94AB-E33D54481071@gmail.com> On 2 Mar 2008, at 02:00, Alex Martelli wrote: > On Sat, Mar 1, 2008 at 11:11 AM, Barry Warsaw > wrote: > ... >>> I also propose translations of the shorter text to important >>> languages >>> like French, German, Japanese, Portuguese and Spanish. I'm willing >>> to >>> help with the German translation. >> >> Cool, thanks. > > I'd like to volunteer for Italian (and we, the Italian Python > community, do have reasonably good connections to the Italian > technical press, which is covering e.g. the upcoming Pycon Due > conference), and although my French is VERY rusty I can give it a try > if no native French speaker is forthcoming. I'm a native French speaker, and although I am not involved in Python's development I would be happy to help by translating the documents. I have no connections with the French-speaking technical press. -- Arnaud From greg.kochanski at phon.ox.ac.uk Mon Mar 3 18:56:08 2008 From: greg.kochanski at phon.ox.ac.uk (Greg Kochanski) Date: Mon, 03 Mar 2008 17:56:08 -0000 Subject: [Python-Dev] Bug in Pickle protocol involving __setstate__ Message-ID: <47CC12A8.9030803@phon.ox.ac.uk> If we have a hierarchy of classes, and we use __getstate__/__setstate__, the wrong version of __setstate__ gets called. Possibly, this is a documentation problem, but here goes: Take two classes, A and B, where B is the child of A. Construct a B. Pickle it. Unpickle it, and you find that the __setstate__ function for A is called with the result produced by B.__getstate__(). This is wrong. An example follows: import pickle as P class A(object): def __init__(self, a): print 'A.__init__' self.a = a def __getstate__(self): print 'A.__getstate' return self.a def __setstate__(self, upstate): print 'A.__setstate', upstate self.a = upstate class B(A): def __init__(self, a, b): print 'B.__init__' A.__init__(self, a) self.b = b def __getstate__(self): print 'B.__getstate' return (A.__getstate__(self), self.b) def __setstate(self, upstate): # This never gets called! print 'B.__setstate', upstate A.__setstate__(self, upstate[0]) self.b = upstate[1] def __repr__(self): return '' % (self.a, self.b) q = B(1,2) print '---' r = P.loads(P.dumps(q, 0)) print 'q=', q print 'r=', r Now, run it: $ python foo.py B.__init__ A.__init__ --- B.__getstate A.__getstate A.__setstate (1, 2) q= r= Traceback (most recent call last): File "foo.py", line 44, in print 'r=', r File "foo.py", line 37, in __repr__ return '' % (self.a, self.b) AttributeError: 'B' object has no attribute 'b' $ From hvendelbo.dev at gmail.com Wed Mar 5 22:27:32 2008 From: hvendelbo.dev at gmail.com (Henrik Vendelbo) Date: Wed, 5 Mar 2008 21:27:32 +0000 Subject: [Python-Dev] Python 3000: Special type for object attributes & map keys Message-ID: It appears to me that if you can make mapping mechanisms faster in Python you can make significant overall speed improvements. I also think the proposed concept could add flexibility to persistence formats and RMI interfaces. My basic idea is to have a constant string type with an interpreter globally unique hash. If the original constant is created in a manner different from string constants, it can be tracked and handled differently by the interpreter. Obviously most object attribute references are done with the dot operator, so I guess the interpreter already has an efficient mapping mechanism. But there must be a crossover with __getattr__ etc, where a map of some sort is used. I imagine that having a global namespace to translate attribute names into integers could be used for several purposes by the interpreter as well as an application exchanging objects with other applications. I imagine these expressions to be supported: * attrname(string) - creates an attrname value from the string * int(attrname) - gets the hash value * string(attrname) - gets the string value Hope this makes sense Henrik P.S. I originally thought of this in a JavaScript context so forgive me if this would make little difference in Python. From gyorgy.fazekas at elec.qmul.ac.uk Mon Mar 10 13:11:24 2008 From: gyorgy.fazekas at elec.qmul.ac.uk (George Fazekas) Date: Mon, 10 Mar 2008 12:11:24 +0000 Subject: [Python-Dev] embedding in multi threaded C/C++ Message-ID: Hi all, I'm working on embedding Python in a multi threaded application but found mostly old or confusing info on this. Can anyone point me to the right direction or send some working examples? Detail: Python 2.5.1, MacOSX Leopard 10.5.1, using Pytohn/C API The application initializes Python in a shared library, which in turn links in more libraries that may or may not use C API commands in parallel. Generally it all works fine, but when two libraries try to access Python code I get seg fault or similar. The closest I got to resolve this is based on this message: http://groups.google.fi/group/comp.lang.python/msg/fe4e114d1e1a741d which suggests starting a new sub interpreter for each task. However, i still get errors like below. (Thread 0 on it's own works fine.) According to the docs PyObject_HasAttrString should always succeeds so I don't understand what happens. Also I get thread mix-up messages randomly even though I double checked the implementation. 2 Threads accessing Python: ------------------------------- Thread 0 Crashed: 0 org.python.python 0x15a58bcc PyErr_Occurred + 16 (errors.c:77) 1 org.python.python 0x159c642c instance_getattr + 277 (classobject.c:698) 2 org.python.python 0x159f789b PyObject_HasAttrString + 116 (object.c:1069) While Thread 4 is running a process: ---------------------------------------- Thread 4: 0 org.python.python 0x15a43751 PyEval_EvalFrameEx + 794 (ceval.c:852) 1 org.python.python 0x15a49cdc PyEval_EvalCodeEx + 1819 (ceval.c:2831) 2 org.python.python 0x159df537 function_call + 320 (funcobject.c:517) 3 org.python.python 0x159be278 PyObject_Call + 45 (abstract.c:1860) 4 org.python.python 0x159c5ee5 instancemethod_call + 401 (classobject.c:2497) 5 org.python.python 0x159c297c PyObject_CallMethodObjArgs + 223 (abstract.c:1860) Thanks for any advice, George From zooko at zooko.com Mon Mar 17 22:19:31 2008 From: zooko at zooko.com (zooko) Date: Mon, 17 Mar 2008 15:19:31 -0600 Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module) In-Reply-To: <47DEC0EB.3000404@palladion.com> References: <20080317000630.735823A40B0@sparrow.telecommunity.com> <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <47DEC0EB.3000404@palladion.com> Message-ID: <440CD260-F6F6-4484-8326-0EA1D7B83132@zooko.com> Folks: (By the way, it was a pleasure to meet many of you in Real Life for the first time at Pycon.) Here is what I want: 1. The standard Python build tools by default produce machine- parseable package metadata, which can include package dependency information with reasonably well-defined semantics. Oh good! I already have this, since distutils in Python >= 2.5 produces .egg-info metadata in an easy-to-parse format. 2. This machine-parseable metadata is widely supported and understood by the Python community. In retrospect, it's too bad that it isn't named ".pkg-info" instead of ".egg-info", in order to avoid the fraught politics around the concept of "eggs". A concrete example of such a misunderstanding is the sad fact that many Linux distributions were in the habit of deleting this information from their Python packages, perhaps because they were under the impression that it was obviated by their packaging tools. The major distributions have all stopped doing this now. Unifying the created-by-default PKG-INFO files and the created-by- default .egg-info directories would be nice, too. 3. The standard Python library includes a tool to find and parse this metadata, so that I can write programmatic tests of my dependencies, like this: http://allmydata.org/trac/tahoe/browser/_auto_deps.py?rev=2062 This is one of the improvements that I was anticipating from pkg_resources going into the standard library. 4. The standard Python library includes a tool to find and read resources (other than Python modules) that came bundled in a Python package. Consider, for example, this snippets of code in Nevow: http://divmod.org/trac/browser/trunk/Nevow/setup.py?rev=13786#L10 http://divmod.org/trac/browser/trunk/Nevow/setup.py?rev=13786 http://divmod.org/trac/browser/trunk/Nevow/setup_egg.py?rev=2406 When Nevow uses pkg_resources to import its files such as "default.css", then it is able to find at runtime, even if is being imported from a py2exe or py2app zip, or on other platforms where its homegrown setup script and homegrown "find my file" function fail. So using pkg_resources (and setuptools to install it) makes "test_nevow" pass on all of the allmydata.org buildslaves: http://allmydata.org/buildbot/waterfall?show_events=false Regards, Zooko From greg at krypto.org Tue Mar 18 22:43:50 2008 From: greg at krypto.org (Gregory P. Smith) Date: Tue, 18 Mar 2008 16:43:50 -0500 Subject: [Python-Dev] pre-checkin test suggestion for python repository Message-ID: <52dc1c820803181443h8bb13fqef17911314bdf5b9@mail.gmail.com> The tabs/spaces checker that is run before doing a svn ci on the python repository spits out an error message about which files have problems. Could someone please update this error message to say something to the effect of "run Tools/scripts/reindent.py on every file listed above and rerun your tests to fix this before checking in" thanks -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080318/18f1c6b7/attachment.htm From thomas at python.org Tue Mar 18 22:48:26 2008 From: thomas at python.org (Thomas Wouters) Date: Tue, 18 Mar 2008 14:48:26 -0700 Subject: [Python-Dev] Bug in Pickle protocol involving __setstate__ In-Reply-To: <47CC12A8.9030803@phon.ox.ac.uk> References: <47CC12A8.9030803@phon.ox.ac.uk> Message-ID: <9e804ac0803181448o1a95488dl13658caefb8e67b7@mail.gmail.com> On Mon, Mar 3, 2008 at 8:00 AM, Greg Kochanski wrote: > If we have a hierarchy of classes, and we use > __getstate__/__setstate__, the wrong version > of __setstate__ gets called. > > Possibly, this is a documentation problem, but here goes: > No, it's a typo error :) > > Take two classes, A and B, where B is the child of A. > > Construct a B. Pickle it. Unpickle it, and you find > that the __setstate__ function for A is called with the result > produced by B.__getstate__(). > > This is wrong. > > > An example follows: > > import pickle as P > > > class A(object): > def __init__(self, a): > print 'A.__init__' > self.a = a > > def __getstate__(self): > print 'A.__getstate' > return self.a > > def __setstate__(self, upstate): > print 'A.__setstate', upstate > self.a = upstate > > class B(A): > def __init__(self, a, b): > print 'B.__init__' > A.__init__(self, a) > self.b = b > > def __getstate__(self): > print 'B.__getstate' > return (A.__getstate__(self), self.b) > > def __setstate(self, upstate): Try actually calling this method '__setstate__' instead. > # This never gets called! > print 'B.__setstate', upstate > A.__setstate__(self, upstate[0]) > self.b = upstate[1] > > > def __repr__(self): > return '' % (self.a, self.b) > > > q = B(1,2) > print '---' > r = P.loads(P.dumps(q, 0)) > print 'q=', q > print 'r=', r > > > Now, run it: > > $ python foo.py > B.__init__ > A.__init__ > --- > B.__getstate > A.__getstate > A.__setstate (1, 2) > q= > r= Traceback (most recent call last): > File "foo.py", line 44, in > print 'r=', r > File "foo.py", line 37, in __repr__ > return '' % (self.a, self.b) > AttributeError: 'B' object has no attribute 'b' > $ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/thomas%40python.org > -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080318/9f200a4b/attachment.htm From wolever at cs.toronto.edu Tue Mar 18 23:13:11 2008 From: wolever at cs.toronto.edu (David Wolever) Date: Tue, 18 Mar 2008 18:13:11 -0400 Subject: [Python-Dev] map, filter, zip in future_builtins In-Reply-To: <1afaf6160803181501i2354b0dbv55a7e67bc1e2de30@mail.gmail.com> References: <96A6EE23-1300-4ABD-BF6E-A1AC6D2FB742@cs.toronto.edu> <1afaf6160803181432x7a31f451xebf42aac51f59ecd@mail.gmail.com> <66B88059-2F67-40DF-B820-DB9C3F5F372F@cs.toronto.edu> <1afaf6160803181501i2354b0dbv55a7e67bc1e2de30@mail.gmail.com> Message-ID: <0446BBEB-DE1C-42B3-B198-69D2C61AB6DC@cs.toronto.edu> On 18-Mar-08, at 6:01 PM, Benjamin Peterson wrote: >> Couldn't you just import imap as map? > What do you mean? Import imap as map in future_builtins.c? > Like the Python: > import itertools > map = intertools.map > type(map(lambda x: x, range(3))) == map # True Ah, that's a much better idea :P I'll do that. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080318/72efd405/attachment.htm From pje at telecommunity.com Tue Mar 18 23:36:57 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 18 Mar 2008 18:36:57 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <47DEA275.4080306@behnel.de> <20080317174542.89A463A409D@sparrow.telecommunity.com> <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20080318223700.C64123A4074@sparrow.telecommunity.com> At 03:43 PM 3/18/2008 -0500, Guido van Rossum wrote: >Only very few people would care about writing a setup >script that works with this bootstrap module; basically only package >manager implementers. That's true today, sure, but as soon as it is widely available, others are sure to want to use it too. I just want a bright-line distinction between what is and isn't bootstrappable, rather than a murky region of "maybe, if you're not doing anything too complicated". >There seems to be a misunderstanding about what I am proposing we do >instead. The boostrap installer should only be powerful enough to >allow it to be used to install a real package manager like setuptools. Which is why PEP 365 proposed only downloading an archive to a cache directory, and optionally running something from it. It explicitly disavows "installation" of anything, since the downloaded archive wouldn't have been added to sys.path except for the duration of the bootstrap process, and no scripts were to be installed. (Indeed, apart from the methods it would have used to locate the archive on PyPI, and to determine what to run from inside it, there was nothing particularly egg-specific about the proposed bootstrapping process.) So, to fully egg-neutralize the bootstrapping approach, we need only know how to locate an appropriate archive, and how to determine what to run from it. For the latter, we could use the already-in-2.6 convention of running __main__ from a zipfile or directory. (Too bad distutils source distributions have an extra directory name embedded in them, so one can't just execute them directly. Otherwise, we could've just let people drop in a __main__.py next to setup.py. OTOH, maybe it would be enough to use setuptools' algorithm for finding setup.py to locate __main__.py, and I'm fairly sure *that* can be briefly expressed in the PEP.) The other open question is a naming convention and version detection, so that the bootstrap tool can identify which of the files listed on PyPI is suitable for its use. (Both with regard to the version selection, and file type.) However, if PyPI were to grow support for designating the appropriate files and/or versions in some other way, we wouldn't need a naming convention as such. Without one or the other, the bootstrap tool would have to grow a version parsing scheme of some type, and play guessing games with file extensions. (Which is one reason I limited PEP 365's scope to downloading eggs actually *uploaded* to PyPI, rather than arbitrary packages *linked* from PyPI.) So, if I had to propose something right now, I would be inclined to propose: * using setuptools' version parsing semantics for interpretation of alpha/beta/dev/etc. releases * having a bdist_bootstrap format that's essentially a bdist_dumb .zip file with the internal path prefixes stripped off, making it an importable .zip with a different file extension. (Or maybe just .pyboot.zip?) The filename convention would use setuptools' canonicalization and escaping of names and version numbers, to allow unambiguous machine parsing of the filename. A __main__ module would have to be present for the archive to be run, as opposed to just being downloaded to a temporary directory. * calling the bootstrap module 'bootstrap', as in 'python -m bootstrap projectname optionalversion'. The module would expose an API to allow it to be used programmatically as well as the command line, so that bootstrapped packages can use the bootstrap process to locate dependencies if they so desire. (Today's package management tools, at least, are all based on setuptools, so if it's not present they'll need to download that before beginning their own bootstrapping process.) Apart from keeping the PEP self-contained and short, is there anything in this that you think you would object to? (You may reserve the right, of course, to later not like something in the details of setuptools' version/filename rules, after I've put them into the PEP, or really, anything else. I'm just asking if there's anything that's obviously offensive at this point, before I spend time on a new PEP.) From nnorwitz at gmail.com Tue Mar 18 23:51:32 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 18 Mar 2008 17:51:32 -0500 Subject: [Python-Dev] pre-checkin test suggestion for python repository In-Reply-To: <52dc1c820803181443h8bb13fqef17911314bdf5b9@mail.gmail.com> References: <52dc1c820803181443h8bb13fqef17911314bdf5b9@mail.gmail.com> Message-ID: On Tue, Mar 18, 2008 at 4:43 PM, Gregory P. Smith wrote: > The tabs/spaces checker that is run before doing a svn ci on the python > repository spits out an error message about which files have problems. > > Could someone please update this error message to say something to the > effect of > > "run Tools/scripts/reindent.py on every file listed above and rerun your > tests to fix this before checking in" Done. n From jimjjewett at gmail.com Tue Mar 18 23:54:16 2008 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 18 Mar 2008 18:54:16 -0400 Subject: [Python-Dev] logging shutdown (was: Re: [Python-checkins] r61431 - python/trunk/Doc/library/logging.rst) Message-ID: I think (repeatedly) testing an app through IDLE is a reasonable use case. Would it be reasonable for shutdown to remove logging from sys.modules, so that a rerun has some chance of succeeding via its own import? -jJ On 3/16/08, vinay.sajip wrote: > Author: vinay.sajip > Date: Sun Mar 16 22:35:58 2008 > New Revision: 61431 > > Modified: > python/trunk/Doc/library/logging.rst > Log: > Clarified documentation on use of shutdown(). > > Modified: python/trunk/Doc/library/logging.rst > ============================================================================== > --- python/trunk/Doc/library/logging.rst (original) > +++ python/trunk/Doc/library/logging.rst Sun Mar 16 22:35:58 2008 > @@ -732,7 +732,8 @@ > .. function:: shutdown() > > Informs the logging system to perform an orderly shutdown by flushing and > - closing all handlers. > + closing all handlers. This should be called at application exit and no > + further use of the logging system should be made after this call. > > > .. function:: setLoggerClass(klass) > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > From aahz at pythoncraft.com Wed Mar 19 00:44:46 2008 From: aahz at pythoncraft.com (Aahz) Date: Tue, 18 Mar 2008 16:44:46 -0700 Subject: [Python-Dev] embedding in multi threaded C/C++ In-Reply-To: References: Message-ID: <20080318234446.GA6160@panix.com> On Mon, Mar 10, 2008, George Fazekas wrote: > > I'm working on embedding Python in a multi threaded application but > found mostly old or confusing info on this. Can anyone point me to the > right direction or send some working examples? You should ask on comp.lang.python or the capi-sig list. python-dev is for people working on improving the Python core. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "It is easier to optimize correct code than to correct optimized code." --Bill Harlan From nnorwitz at gmail.com Wed Mar 19 00:54:47 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 18 Mar 2008 18:54:47 -0500 Subject: [Python-Dev] Python 3000: Special type for object attributes & map keys In-Reply-To: References: Message-ID: On Wed, Mar 5, 2008 at 4:27 PM, Henrik Vendelbo wrote: > It appears to me that if you can make mapping mechanisms faster in > Python you can make significant > overall speed improvements. I also think the proposed concept could > add flexibility to persistence formats > and RMI interfaces. > > My basic idea is to have a constant string type with an interpreter > globally unique hash. If the original constant > is created in a manner different from string constants, it can be > tracked and handled differently by the interpreter. Part of this is done, but very differently in that all strings used in code objects are interned (stored in a dictionary so we don't increase memory by storing multiple string objects which contain the same string) . So there is partially a mechanism to do what you suggest. But there would be many places that would need to be modified. I think we briefly discussed this in the past. > P.S. I originally thought of this in a JavaScript context so forgive > me if this would make little difference in Python. Your message was a little confusing at first because the terminology is a little different, but the idea makes sense. It's not clear how much this would speed up the interpreter. The best way to test your theory would be to create a patch and measure the performance difference. First, you should measure the current speed difference. Something like: $ ./python.exe -m timeit -s 'd = {1: None}' 'd[1]' 1000000 loops, best of 3: 0.793 usec per loop $ ./python.exe -m timeit -s 'd = {"1": None}' 'd["1"]' 1000000 loops, best of 3: 0.728 usec per loop My python is a debug version, so a release version might be faster for ints. If not, the first task would be to speed up int lookups. :-) (You should verify more with real world dict sizes.) It is possible to optimize dicts with int keys since string keys are specialized in dicts, but ints are not. You would need to look in Objects/dictobject.c. See http://python.org/dev/faq/ for general hints on how to get started. If ints were faster, some of the next steps would be: * keep the globally unique number (very easy) * update the source that generates byte codes to use the globally unique number * store ints in dicts and update all the places for how they use attributes * update byte code when a module is imported to use the globally unique number Feel free to ask questions. Cheers, n From janssen at parc.com Wed Mar 19 00:59:45 2008 From: janssen at parc.com (Bill Janssen) Date: Tue, 18 Mar 2008 16:59:45 PDT Subject: [Python-Dev] platform management In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE20@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE20@EXMBX04.exchhosting.com> Message-ID: <08Mar18.165949pdt."58696"@synergy1.parc.xerox.com> I don't think this is bike-shedding. The debate about "AMD64" vs. "amd64" vs. "x86_64" reminded me that I've been bit more and more frequently by bits of platform-specific knowledge scattered around the standard library. The latest is the code in distutils.unixccompiler that tries to figure out what flags to send to the linker in order to add a dynamic library path lookup to a shared library. There are lots of different ways of figuring out which platform is being used, and they're all over the place. The code in distutils.unixccompiler uses "sys.platform[:6]", and looks for "darwin"; the code in urllib.py uses "os.name", and expects it to be "mac", but later looks for "sys.platform == 'darwin'; posixfile believes that platforms come with version numbers ("linux2", "aix4"); pydoc and tarfile have tests for "sys.platform == 'mac'". tempfile looks at os.sys.platform *and* os.name. Could well be that all of these are correct (I believe that "mac", for instance, refers to the generations of the Mac OS before OS X). But it means that when someone tries to update "Python" to a new major version release for, say, OS X, it's really easy to miss things. And the fact that the platform version is sometimes included and sometimes not is also problematic; darwin9 is different from darwin8 in some important aspects. It would be nice to (a) come up with a list of standard platform symbols, (b) put those symbols somewhere in the documentation, (c) have one way of discovering them, via sys or os or platform or whichever module, (d) add a standard way of discovering the platform version, and (e) stamp out all the other ways that have crept in over the years. Bill From dpeterson at enthought.com Wed Mar 19 01:27:01 2008 From: dpeterson at enthought.com (Dave Peterson) Date: Tue, 18 Mar 2008 19:27:01 -0500 Subject: [Python-Dev] [Distutils] Capsule Summary of Some Packaging/Deployment Technology Concerns In-Reply-To: <20080318004718.56E173A40FF@sparrow.telecommunity.com> References: <47DEEC6B.5040602@taupro.com> <20080318004718.56E173A40FF@sparrow.telecommunity.com> Message-ID: <47E05DD5.1090707@enthought.com> Phillip J. Eby wrote: > At 05:10 PM 3/17/2008 -0500, Jeff Rush wrote: > > >> 1. Many felt the existing dependency resolver was not correct. They wanted a >> full tree traversal resulting in an intersection of all restrictions, >> instead of a first-acceptable-solution approach taking now, which can >> result in top-level dependencies not being enforced upon lower >> levels. The >> latter is faster however. One solution would be to make the resolver >> pluggable. >> > > Patches welcome, on both counts. Personally, Bob and I originally > wanted a full-tree intersection, too, but it turned out to be hairier > to implement than it seems at first. My guess is that none of the > people who want it, have actually tried to implement it without a > factorial or exponential O(). But that doesn't mean I'll be unhappy > if somebody succeeds. :) > I think we'd make significant progress by just intersecting the dependencies we know about as we progress through the dependency tree. For example, if A requires B==2 and C==3, and if B requires C>=2,<=4, then at the time we install A we'd pick C==3 and also at the time we install B we'd pick C==3. As opposed to the current scheme that would choose C==4 for the latter case. This would allow dependent projects (think applications here) to better control the versions of the full set of libraries they use. Things would still fail (like they do now) if you ran across dependencies that had no intersection or if you encountered a new requirement after the target projected was already installed. If you really wanted to do a full-tree intersection, it seems to me that the problem is detecting all the dependencies without having to spend significant time downloading/building in order to find them out. This could be solved by simply extending the cheeseshop interface to export the set of requirements outside of the egg / tarball / etc. We've done this for our own egg repository by extracting the appropriate meta-data files out of EGG-INFO and putting it into a separate file. This info is also useful for users as it gives them an idea of how much *new* stuff is going to be installed (a la yum, apt-get, etc.) > In other words, we attempt to achieve heuristically what's being > proposed to do algorithmically. And my guess is that whatever cases > the heuristic is failing at, would probably not be helped by an > algorithmic approach either. But I would welcome some actual data, either way. > With our ETS projects, we've run into problems with the current heuristic. Perhaps we just don't know how to make it work like we want? We have a set of projects that we want to be individually installable (to the extent that we limit cross-project dependencies) but we also want to make it easy to install the complete set. We use a meta-egg for the latter. It's purpose is only to specify the exact versions of each project that have been explicitly tested to work together -- you could almost think of it as a source control system tag. Whereas on the individual projects, we explicitly want to ensure that people get the latest possible release of each required API so the version requirements are wider here. This setup causes problems whenever we release new versions of projects because it seems easy_install ignores the meta-egg exact versions when it gets down into a project and comes across a wider cross-project dependency. We ended up having to give up on the ranges in the cross-project dependencies and synchronize them to the same values in the meta-egg dependencies. There are numerous side-effects of this that we don't like but we haven't found a way around it. > Again, though, patches are welcome. :) (Specifically, for the > trunk; I don't see a resolver overhaul as being suitable for the 0.6 > stable branch.) > We're planning to pursue this (for the above mentioned strategy) as soon as we work ourselves out of a bit of a backlog of other things to do. >> 2. People want a solution for the handling of documentation. The distutils >> module has had commented out sections related to this for several years. >> > > As with so many other things, this gets tossed around the > distutils-sig every now and then. A couple of times I've thrown out > some options for how this might be done, but then the conversation > peters out around the time anybody would have to actually do some > work on it. (Me included, since I don't have an itch that needs > scratching in this area.) > > In particular, if somebody wants to come up with a metadata standard > for including documentation in eggs, we've got a boatload of hooks by > which it could be done. Nothing's stopping anybody from proposing a > standard and building a tool, here. (e.g. using the setuptools > command hook, .egg-info writer hook, etc.) Enthought has started an effort (it's currently one of two things in our ETSProjectTools project at https://svn.enthought.com/svn/enthought/ETSProjectTools/trunk) and we're experimenting with our solution before proposing it as a patch. We'd love some more help if anyone wants to participate. >> 3. A more flexible internal handing of the different types of files is needed. >> Currently the code, data, lib, etc. files are aggregated at >> build time and >> people would like them to be kept separate until install/packaging time. >> > > I don't know what this means, exactly. > A number of projects want to provide various types of files besides code in their distributable, and they'd like these to end up in standard locations for that type of file. Think documentation, sample data, web templates, configuration settings, etc. Each of these should be treated differently at installation time depending on platform. On *nix, docs should go in /usr/share/doc whereas we might need to create a C:\Python2.5\docs on Windows. With sample data and templates, you probably just want it accessible outside of the zipped egg so users can easily look at it, add to it, edit it, etc. Configuration settings should be installed with some defaults into a standard configuration directory like /etc on *nix, etc. Basically the issue is that it needs to be easier to include different sets of files into an egg for different actions to be taken during installation or packaging into an OS-specific distribution format. >> The other is the use of a single .pth file to control the list >> of activated >> packages. Those who produce distributions would prefer a magic directory >> into which links to distributions could be dropped, similar to >> the current >> best practices for Linux, with /etc/conf.d/, /etc/profile.d/, >> /etc/xinetd.d/ and so forth. >> > > site-packages is that directory, and has been since long before > setuptools. Just drop uniquely-named .pth files there, and you're good to go. > But the docs for easy_install claim that the list of active eggs is maintained in easy-install.pth. Also, if I create my own .pth file, and the user tries to update my version to a new one, will the easy_install tool modify my .pth file to remove the mention of the old version from my sys.path and put the new version in the same .pth file? Or will it now be listed in both places? Or will it only in easy-install.pth? >> 7. Many wanted to ability to install files anywhere in the install tree and >> not just under the Python package. Under distutils this was possible but >> it was removed in setuptools for security reasons. >> > > It wasn't security, it was manageability. Egg-based installation > means containment, (analagous to GNU stow) and therefore portability > and disposability of plugins. (Which again is what eggs were really > developed for in the first place.) > Yes, but as you've already pointed out, they've escaped into a larger ecosystem and this restriction is a severe limitation -- leading to significant frustration. Especially as projects evolve and want to do something more complex than simply install pure Python code. Here at Enthought, we use and ship a number of projects that have extensions and thus dynamic libraries that need to either be modified during installation to work from the user's installed location, or copied elsewhere on the system to avoid the need to modify (which we also can't do via an egg install) env variables, registries, etc. We'd also love to be able to ship end-user enterprise-scale applications via eggs so that bug fixes and updates don't require downloading a monolithic 100MB+ installer. But doing that requires the ability to update desktop icons, menus, etc. which we also can't do automatically via an egg. If you don't want the burden on setuptools to support, much less track, all these options, then perhaps it could just support automatic execution of a post-install script (and pre-uninstall script if uninstallation ever happens) that allows individual project developers to do what they need to do? Let the burden of describing how those things happen and how to uninstall/relocate/update them fall to the provider of the projects that do them. Also, IIUC, stow only tries to "contain" the hard files. It puts links in multiple standard locations (for man pages, executables, libraries, etc.) If setuptools supported these options, I don't think there'd be any discussion here except for things like "how do I extend the set of things the tool supports so that my foo-type files get linked into the standard /os/path/to/foo for the X os?" >> Custom code can still >> be written to do this explicitly but this is not popular. >> > > No kidding. :) Current best practice is to include a script or > module in the package that can install other files to a designated > location. Personally, though, I tend to view applications and > libraries that target specific install locations to be overreaching > their bounds, and stepping into sysadmin territory. Give me the > tools to install the data, don't just dump it somewhere on my system > where *you* think it should go, in other words. > I should have read ahead. This sounds close to what I've been describing except that this leads me to picture a script that prompts for install locations and allows the user to customize the destinations rather than one that assumes everything goes in a standard place. I'm all for this, and the continuation of the ability to install an egg into a user-environment vs. a system-environment. The only thing missing here is the ability for the installer to automatically run that script so that installation isn't a disjointed, two-step manual process that a user is prone to forgot to complete. One of the features of Enthought's Enstaller extension to easy_install was that it looks for a post_install.py script in EGG-INFO and if one is found, runs it. I would think that getting this into setuptools would be a significant step forward but I believe you previously rejected that idea. We'll take a stab at creating a patch for you if you're more receptive to that idea now. Just let me know. > On the other hand, I've been puzzling over how to handle legitimate > post-install features. On Windows, both wx and pywin32 have a real > need to do some actuall "install" operations. Some is just copying > files, but pywin32 also has to do some registry stuff. I don't know > how to allow just what's sensible, without opening up a huge can of > worms, though. > I think there are lots of situations that are legitimate (projects with extensions, projects that want to put icons on the desktop or in menus, projects that need to interact with a registry, projects that want to put configuration information somewhere other than in a zip file in a site-packages dir, etc.) I think we should worry less about preventing developers from shooting themselves in the foot and more about ensuring that they can hunt for food for their survival. We can always tighten things down after seeing the usecases that develop, right? -- Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080318/ac9b10a3/attachment-0001.htm From tnelson at onresolve.com Wed Mar 19 02:16:33 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Tue, 18 Mar 2008 18:16:33 -0700 Subject: [Python-Dev] [Python-checkins] r61577 - in python/trunk: Include/code.h Include/compile.h Include/parsetok.h Include/pythonrun.h Lib/__future__.py Lib/test/test_print.py Misc/ACKS Misc/NEWS Parser/parser.c Parser/parsetok.c Python/bltinmodule.c Python/future.c ... In-Reply-To: <20080318234550.096EE1E4011@bag.python.org> References: <20080318234550.096EE1E4011@bag.python.org> Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE23@EXMBX04.exchhosting.com> This change breaks all the trunk buildbots: ====================================================================== ERROR: testCompileLibrary (test.test_compiler.CompilerTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "S:\buildbots\python\trunk.nelson-windows\build\lib\test\test_compiler.py", line 52, in testCompileLibrary compiler.compile(buf, basename, "exec") File "S:\buildbots\python\trunk.nelson-windows\build\lib\compiler\pycodegen.py", line 64, in compile gen.compile() File "S:\buildbots\python\trunk.nelson-windows\build\lib\compiler\pycodegen.py", line 112, in compile gen = ModuleCodeGenerator(tree) File "S:\buildbots\python\trunk.nelson-windows\build\lib\compiler\pycodegen.py", line 1275, in __init__ self.futures = future.find_futures(tree) File "S:\buildbots\python\trunk.nelson-windows\build\lib\compiler\future.py", line 59, in find_futures walk(node, p1) File "S:\buildbots\python\trunk.nelson-windows\build\lib\compiler\visitor.py", line 106, in walk walker.preorder(tree, visitor) File "S:\buildbots\python\trunk.nelson-windows\build\lib\compiler\visitor.py", line 63, in preorder self.dispatch(tree, *args) # XXX *args make sense? File "S:\buildbots\python\trunk.nelson-windows\build\lib\compiler\visitor.py", line 57, in dispatch return meth(node, *args) File "S:\buildbots\python\trunk.nelson-windows\build\lib\compiler\future.py", line 27, in visitModule if not self.check_stmt(s): File "S:\buildbots\python\trunk.nelson-windows\build\lib\compiler\future.py", line 37, in check_stmt "future feature %s is not defined" % name SyntaxError: future feature print_function is not defined ________________________________________ From: python-checkins-bounces+tnelson=onresolve.com at python.org [python-checkins-bounces+tnelson=onresolve.com at python.org] On Behalf Of eric.smith [python-checkins at python.org] Sent: 18 March 2008 19:45 To: python-checkins at python.org Subject: [Python-checkins] r61577 - in python/trunk: Include/code.h Include/compile.h Include/parsetok.h Include/pythonrun.h Lib/__future__.py Lib/test/test_print.py Misc/ACKS Misc/NEWS Parser/parser.c Parser/parsetok.c Python/bltinmodule.c Python/future.c Pyth... Author: eric.smith Date: Wed Mar 19 00:45:49 2008 New Revision: 61577 Added: python/trunk/Lib/test/test_print.py Modified: python/trunk/Include/code.h python/trunk/Include/compile.h python/trunk/Include/parsetok.h python/trunk/Include/pythonrun.h python/trunk/Lib/__future__.py python/trunk/Misc/ACKS python/trunk/Misc/NEWS python/trunk/Parser/parser.c python/trunk/Parser/parsetok.c python/trunk/Python/bltinmodule.c python/trunk/Python/future.c python/trunk/Python/pythonrun.c Log: Backport of the print function, using a __future__ import. This work is substantially Anthony Baxter's, from issue 1633807. I just freshened it, made a few minor tweaks, and added the test cases. I also created issue 2412, which is to check for 2to3's behavior with the print function. I also added myself to ACKS. Modified: python/trunk/Include/code.h ============================================================================== --- python/trunk/Include/code.h (original) +++ python/trunk/Include/code.h Wed Mar 19 00:45:49 2008 @@ -48,11 +48,12 @@ #define CO_FUTURE_DIVISION 0x2000 #define CO_FUTURE_ABSOLUTE_IMPORT 0x4000 /* do absolute imports by default */ #define CO_FUTURE_WITH_STATEMENT 0x8000 +#define CO_FUTURE_PRINT_FUNCTION 0x10000 /* This should be defined if a future statement modifies the syntax. For example, when a keyword is added. */ -#if 0 +#if 1 #define PY_PARSER_REQUIRES_FUTURE_KEYWORD #endif Modified: python/trunk/Include/compile.h ============================================================================== --- python/trunk/Include/compile.h (original) +++ python/trunk/Include/compile.h Wed Mar 19 00:45:49 2008 @@ -24,6 +24,8 @@ #define FUTURE_DIVISION "division" #define FUTURE_ABSOLUTE_IMPORT "absolute_import" #define FUTURE_WITH_STATEMENT "with_statement" +#define FUTURE_PRINT_FUNCTION "print_function" + struct _mod; /* Declare the existence of this type */ PyAPI_FUNC(PyCodeObject *) PyAST_Compile(struct _mod *, const char *, Modified: python/trunk/Include/parsetok.h ============================================================================== --- python/trunk/Include/parsetok.h (original) +++ python/trunk/Include/parsetok.h Wed Mar 19 00:45:49 2008 @@ -27,6 +27,10 @@ #define PyPARSE_WITH_IS_KEYWORD 0x0003 #endif +#define PyPARSE_PRINT_IS_FUNCTION 0x0004 + + + PyAPI_FUNC(node *) PyParser_ParseString(const char *, grammar *, int, perrdetail *); PyAPI_FUNC(node *) PyParser_ParseFile (FILE *, const char *, grammar *, int, Modified: python/trunk/Include/pythonrun.h ============================================================================== --- python/trunk/Include/pythonrun.h (original) +++ python/trunk/Include/pythonrun.h Wed Mar 19 00:45:49 2008 @@ -8,7 +8,7 @@ #endif #define PyCF_MASK (CO_FUTURE_DIVISION | CO_FUTURE_ABSOLUTE_IMPORT | \ - CO_FUTURE_WITH_STATEMENT) + CO_FUTURE_WITH_STATEMENT|CO_FUTURE_PRINT_FUNCTION) #define PyCF_MASK_OBSOLETE (CO_NESTED) #define PyCF_SOURCE_IS_UTF8 0x0100 #define PyCF_DONT_IMPLY_DEDENT 0x0200 Modified: python/trunk/Lib/__future__.py ============================================================================== --- python/trunk/Lib/__future__.py (original) +++ python/trunk/Lib/__future__.py Wed Mar 19 00:45:49 2008 @@ -53,6 +53,7 @@ "division", "absolute_import", "with_statement", + "print_function", ] __all__ = ["all_feature_names"] + all_feature_names @@ -66,6 +67,7 @@ CO_FUTURE_DIVISION = 0x2000 # division CO_FUTURE_ABSOLUTE_IMPORT = 0x4000 # perform absolute imports by default CO_FUTURE_WITH_STATEMENT = 0x8000 # with statement +CO_FUTURE_PRINT_FUNCTION = 0x10000 # print function class _Feature: def __init__(self, optionalRelease, mandatoryRelease, compiler_flag): @@ -114,3 +116,7 @@ with_statement = _Feature((2, 5, 0, "alpha", 1), (2, 6, 0, "alpha", 0), CO_FUTURE_WITH_STATEMENT) + +print_function = _Feature((2, 6, 0, "alpha", 2), + (3, 0, 0, "alpha", 0), + CO_FUTURE_PRINT_FUNCTION) Added: python/trunk/Lib/test/test_print.py ============================================================================== --- (empty file) +++ python/trunk/Lib/test/test_print.py Wed Mar 19 00:45:49 2008 @@ -0,0 +1,129 @@ +"""Test correct operation of the print function. +""" + +from __future__ import print_function + +import unittest +from test import test_support + +import sys +try: + # 3.x + from io import StringIO +except ImportError: + # 2.x + from StringIO import StringIO + +from contextlib import contextmanager + +NotDefined = object() + +# A dispatch table all 8 combinations of providing +# sep, end, and file +# I use this machinery so that I'm not just passing default +# values to print, I'm eiher passing or not passing in the +# arguments +dispatch = { + (False, False, False): + lambda args, sep, end, file: print(*args), + (False, False, True): + lambda args, sep, end, file: print(file=file, *args), + (False, True, False): + lambda args, sep, end, file: print(end=end, *args), + (False, True, True): + lambda args, sep, end, file: print(end=end, file=file, *args), + (True, False, False): + lambda args, sep, end, file: print(sep=sep, *args), + (True, False, True): + lambda args, sep, end, file: print(sep=sep, file=file, *args), + (True, True, False): + lambda args, sep, end, file: print(sep=sep, end=end, *args), + (True, True, True): + lambda args, sep, end, file: print(sep=sep, end=end, file=file, *args), + } + + at contextmanager +def stdout_redirected(new_stdout): + save_stdout = sys.stdout + sys.stdout = new_stdout + try: + yield None + finally: + sys.stdout = save_stdout + +# Class used to test __str__ and print +class ClassWith__str__: + def __init__(self, x): + self.x = x + def __str__(self): + return self.x + +class TestPrint(unittest.TestCase): + def check(self, expected, args, + sep=NotDefined, end=NotDefined, file=NotDefined): + # Capture sys.stdout in a StringIO. Call print with args, + # and with sep, end, and file, if they're defined. Result + # must match expected. + + # Look up the actual function to call, based on if sep, end, and file + # are defined + fn = dispatch[(sep is not NotDefined, + end is not NotDefined, + file is not NotDefined)] + + t = StringIO() + with stdout_redirected(t): + fn(args, sep, end, file) + + self.assertEqual(t.getvalue(), expected) + + def test_print(self): + def x(expected, args, sep=NotDefined, end=NotDefined): + # Run the test 2 ways: not using file, and using + # file directed to a StringIO + + self.check(expected, args, sep=sep, end=end) + + # When writing to a file, stdout is expected to be empty + o = StringIO() + self.check('', args, sep=sep, end=end, file=o) + + # And o will contain the expected output + self.assertEqual(o.getvalue(), expected) + + x('\n', ()) + x('a\n', ('a',)) + x('None\n', (None,)) + x('1 2\n', (1, 2)) + x('1 2\n', (1, ' ', 2)) + x('1*2\n', (1, 2), sep='*') + x('1 s', (1, 's'), end='') + x('a\nb\n', ('a', 'b'), sep='\n') + x('1.01', (1.0, 1), sep='', end='') + x('1*a*1.3+', (1, 'a', 1.3), sep='*', end='+') + x('a\n\nb\n', ('a\n', 'b'), sep='\n') + x('\0+ +\0\n', ('\0', ' ', '\0'), sep='+') + + x('a\n b\n', ('a\n', 'b')) + x('a\n b\n', ('a\n', 'b'), sep=None) + x('a\n b\n', ('a\n', 'b'), end=None) + x('a\n b\n', ('a\n', 'b'), sep=None, end=None) + + x('*\n', (ClassWith__str__('*'),)) + x('abc 1\n', (ClassWith__str__('abc'), 1)) + + # 2.x unicode tests + x(u'1 2\n', ('1', u'2')) + x(u'u\1234\n', (u'u\1234',)) + x(u' abc 1\n', (' ', ClassWith__str__(u'abc'), 1)) + + # errors + self.assertRaises(TypeError, print, '', sep=3) + self.assertRaises(TypeError, print, '', end=3) + self.assertRaises(AttributeError, print, '', file='') + +def test_main(): + test_support.run_unittest(TestPrint) + +if __name__ == "__main__": + test_main() Modified: python/trunk/Misc/ACKS ============================================================================== --- python/trunk/Misc/ACKS (original) +++ python/trunk/Misc/ACKS Wed Mar 19 00:45:49 2008 @@ -622,6 +622,7 @@ J. Sipprell Kragen Sitaker Christopher Smith +Eric V. Smith Gregory P. Smith Rafal Smotrzyk Dirk Soede Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Wed Mar 19 00:45:49 2008 @@ -12,6 +12,9 @@ Core and builtins ----------------- +- Issue 1745. Backport print function with: + from __future__ import print_function + - Issue 2332: add new attribute names for instance method objects. The two changes are: im_self -> __self__ and im_func -> __func__ Modified: python/trunk/Parser/parser.c ============================================================================== --- python/trunk/Parser/parser.c (original) +++ python/trunk/Parser/parser.c Wed Mar 19 00:45:49 2008 @@ -149,12 +149,10 @@ strcmp(l->lb_str, s) != 0) continue; #ifdef PY_PARSER_REQUIRES_FUTURE_KEYWORD - if (!(ps->p_flags & CO_FUTURE_WITH_STATEMENT)) { - if (s[0] == 'w' && strcmp(s, "with") == 0) - break; /* not a keyword yet */ - else if (s[0] == 'a' && strcmp(s, "as") == 0) - break; /* not a keyword yet */ - } + if (ps->p_flags & CO_FUTURE_PRINT_FUNCTION && + s[0] == 'p' && strcmp(s, "print") == 0) { + break; /* no longer a keyword */ + } #endif D(printf("It's a keyword\n")); return n - i; @@ -208,6 +206,10 @@ strcmp(STR(CHILD(cch, 0)), "with_statement") == 0) { ps->p_flags |= CO_FUTURE_WITH_STATEMENT; break; + } else if (NCH(cch) >= 1 && TYPE(CHILD(cch, 0)) == NAME && + strcmp(STR(CHILD(cch, 0)), "print_function") == 0) { + ps->p_flags |= CO_FUTURE_PRINT_FUNCTION; + break; } } } Modified: python/trunk/Parser/parsetok.c ============================================================================== --- python/trunk/Parser/parsetok.c (original) +++ python/trunk/Parser/parsetok.c Wed Mar 19 00:45:49 2008 @@ -123,8 +123,8 @@ return NULL; } #ifdef PY_PARSER_REQUIRES_FUTURE_KEYWORD - if (flags & PyPARSE_WITH_IS_KEYWORD) - ps->p_flags |= CO_FUTURE_WITH_STATEMENT; + if (flags & PyPARSE_PRINT_IS_FUNCTION) + ps->p_flags |= CO_FUTURE_PRINT_FUNCTION; #endif for (;;) { @@ -167,26 +167,6 @@ str[len] = '\0'; #ifdef PY_PARSER_REQUIRES_FUTURE_KEYWORD - /* This is only necessary to support the "as" warning, but - we don't want to warn about "as" in import statements. */ - if (type == NAME && - len == 6 && str[0] == 'i' && strcmp(str, "import") == 0) - handling_import = 1; - - /* Warn about with as NAME */ - if (type == NAME && - !(ps->p_flags & CO_FUTURE_WITH_STATEMENT)) { - if (len == 4 && str[0] == 'w' && strcmp(str, "with") == 0) - warn(with_msg, err_ret->filename, tok->lineno); - else if (!(handling_import || handling_with) && - len == 2 && str[0] == 'a' && - strcmp(str, "as") == 0) - warn(as_msg, err_ret->filename, tok->lineno); - } - else if (type == NAME && - (ps->p_flags & CO_FUTURE_WITH_STATEMENT) && - len == 4 && str[0] == 'w' && strcmp(str, "with") == 0) - handling_with = 1; #endif if (a >= tok->line_start) col_offset = a - tok->line_start; Modified: python/trunk/Python/bltinmodule.c ============================================================================== --- python/trunk/Python/bltinmodule.c (original) +++ python/trunk/Python/bltinmodule.c Wed Mar 19 00:45:49 2008 @@ -1486,6 +1486,78 @@ equivalent to (x**y) % z, but may be more efficient (e.g. for longs)."); +static PyObject * +builtin_print(PyObject *self, PyObject *args, PyObject *kwds) +{ + static char *kwlist[] = {"sep", "end", "file", 0}; + static PyObject *dummy_args; + PyObject *sep = NULL, *end = NULL, *file = NULL; + int i, err; + + if (dummy_args == NULL) { + if (!(dummy_args = PyTuple_New(0))) + return NULL; + } + if (!PyArg_ParseTupleAndKeywords(dummy_args, kwds, "|OOO:print", + kwlist, &sep, &end, &file)) + return NULL; + if (file == NULL || file == Py_None) { + file = PySys_GetObject("stdout"); + /* sys.stdout may be None when FILE* stdout isn't connected */ + if (file == Py_None) + Py_RETURN_NONE; + } + + if (sep && sep != Py_None && !PyString_Check(sep) && + !PyUnicode_Check(sep)) { + PyErr_Format(PyExc_TypeError, + "sep must be None, str or unicode, not %.200s", + sep->ob_type->tp_name); + return NULL; + } + if (end && end != Py_None && !PyString_Check(end) && + !PyUnicode_Check(end)) { + PyErr_Format(PyExc_TypeError, + "end must be None, str or unicode, not %.200s", + end->ob_type->tp_name); + return NULL; + } + + for (i = 0; i < PyTuple_Size(args); i++) { + if (i > 0) { + if (sep == NULL || sep == Py_None) + err = PyFile_WriteString(" ", file); + else + err = PyFile_WriteObject(sep, file, + Py_PRINT_RAW); + if (err) + return NULL; + } + err = PyFile_WriteObject(PyTuple_GetItem(args, i), file, + Py_PRINT_RAW); + if (err) + return NULL; + } + + if (end == NULL || end == Py_None) + err = PyFile_WriteString("\n", file); + else + err = PyFile_WriteObject(end, file, Py_PRINT_RAW); + if (err) + return NULL; + + Py_RETURN_NONE; +} + +PyDoc_STRVAR(print_doc, +"print(value, ..., sep=' ', end='\\n', file=sys.stdout)\n\ +\n\ +Prints the values to a stream, or to sys.stdout by default.\n\ +Optional keyword arguments:\n\ +file: a file-like object (stream); defaults to the current sys.stdout.\n\ +sep: string inserted between values, default a space.\n\ +end: string appended after the last value, default a newline."); + /* Return number of items in range (lo, hi, step), when arguments are * PyInt or PyLong objects. step > 0 required. Return a value < 0 if @@ -2424,6 +2496,7 @@ {"open", (PyCFunction)builtin_open, METH_VARARGS | METH_KEYWORDS, open_doc}, {"ord", builtin_ord, METH_O, ord_doc}, {"pow", builtin_pow, METH_VARARGS, pow_doc}, + {"print", (PyCFunction)builtin_print, METH_VARARGS | METH_KEYWORDS, print_doc}, {"range", builtin_range, METH_VARARGS, range_doc}, {"raw_input", builtin_raw_input, METH_VARARGS, raw_input_doc}, {"reduce", builtin_reduce, METH_VARARGS, reduce_doc}, Modified: python/trunk/Python/future.c ============================================================================== --- python/trunk/Python/future.c (original) +++ python/trunk/Python/future.c Wed Mar 19 00:45:49 2008 @@ -33,6 +33,8 @@ ff->ff_features |= CO_FUTURE_ABSOLUTE_IMPORT; } else if (strcmp(feature, FUTURE_WITH_STATEMENT) == 0) { ff->ff_features |= CO_FUTURE_WITH_STATEMENT; + } else if (strcmp(feature, FUTURE_PRINT_FUNCTION) == 0) { + ff->ff_features |= CO_FUTURE_PRINT_FUNCTION; } else if (strcmp(feature, "braces") == 0) { PyErr_SetString(PyExc_SyntaxError, "not a chance"); Modified: python/trunk/Python/pythonrun.c ============================================================================== --- python/trunk/Python/pythonrun.c (original) +++ python/trunk/Python/pythonrun.c Wed Mar 19 00:45:49 2008 @@ -738,18 +738,19 @@ } } +#if 0 /* compute parser flags based on compiler flags */ #define PARSER_FLAGS(flags) \ ((flags) ? ((((flags)->cf_flags & PyCF_DONT_IMPLY_DEDENT) ? \ PyPARSE_DONT_IMPLY_DEDENT : 0)) : 0) - -#if 0 +#endif +#if 1 /* Keep an example of flags with future keyword support. */ #define PARSER_FLAGS(flags) \ ((flags) ? ((((flags)->cf_flags & PyCF_DONT_IMPLY_DEDENT) ? \ PyPARSE_DONT_IMPLY_DEDENT : 0) \ - | ((flags)->cf_flags & CO_FUTURE_WITH_STATEMENT ? \ - PyPARSE_WITH_IS_KEYWORD : 0)) : 0) + | ((flags)->cf_flags & CO_FUTURE_PRINT_FUNCTION ? \ + PyPARSE_PRINT_IS_FUNCTION : 0)) : 0) #endif int _______________________________________________ Python-checkins mailing list Python-checkins at python.org http://mail.python.org/mailman/listinfo/python-checkins From pje at telecommunity.com Wed Mar 19 02:20:44 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 18 Mar 2008 21:20:44 -0400 Subject: [Python-Dev] [Distutils] Capsule Summary of Some Packaging/Deployment Technology Concerns In-Reply-To: <47E05DD5.1090707@enthought.com> References: <47DEEC6B.5040602@taupro.com> <20080318004718.56E173A40FF@sparrow.telecommunity.com> <47E05DD5.1090707@enthought.com> Message-ID: <20080319012046.7B1233A4074@sparrow.telecommunity.com> We should probably move this off of Python-Dev, as we're getting into deep details now... At 07:27 PM 3/18/2008 -0500, Dave Peterson wrote: >If you really wanted to do a full-tree intersection, it seems to me >that the problem is detecting all the dependencies without having to >spend significant time downloading/building in order to find them >out. This could be solved by simply extending the cheeseshop >interface to export the set of requirements outside of the egg / >tarball / etc. We've done this for our own egg repository by >extracting the appropriate meta-data files out of EGG-INFO and >putting it into a separate file. This info is also useful for users >as it gives them an idea of how much *new* stuff is going to be >installed (a la yum, apt-get, etc.) ...and now we're more directly competing with them, too. The original idea Bob and I had was to do XML files ala Eclipse feature repositories, but then later I realized that for what we were doing, HTML was both adequate and already available. However, I don't see a problem in principle with having "header" files available for this sort of thing. >With our ETS projects, we've run into problems with the current >heuristic. Perhaps we just don't know how to make it work like we want? > >We have a set of projects that we want to be individually >installable (to the extent that we limit cross-project dependencies) >but we also want to make it easy to install the complete set. We >use a meta-egg for the latter. It's purpose is only to specify the >exact versions of each project that have been explicitly tested to >work together -- you could almost think of it as a source control system tag. I would think that as long as that meta-egg specifies *all* the required versions (right down to recursive dependencies), then there shouldn't be any problem. Maybe it's me who's not understanding something? I would think that you could get the appropriate data by running the tl.eggdeps tool. >A number of projects want to provide various types of files besides >code in their distributable, and they'd like these to end up in >standard locations for that type of file. Think documentation, >sample data, web templates, configuration settings, etc. Each of >these should be treated differently at installation time depending >on platform. On *nix, docs should go in /usr/share/doc whereas we >might need to create a C:\Python2.5\docs on Windows. With sample >data and templates, you probably just want it accessible outside of >the zipped egg so users can easily look at it, add to it, edit it, >etc. Configuration settings should be installed with some defaults >into a standard configuration directory like /etc on *nix, etc. > >Basically the issue is that it needs to be easier to include >different sets of files into an egg for different actions to be >taken during installation or packaging into an OS-specific distribution format. Yes, it would be nice to define a metadata standard for including installable "datasets" either through copying or symlinking, optionally with entry points for running some code, too. When you install an egg, these things could get added to a "post-install to-do" list, that you could then read to find out what steps to do, or invoke a tool on to actually do some of those steps. >But the docs for easy_install claim that the list of active eggs is >maintained in easy-install.pth. Also, if I create my own .pth file, >and the user tries to update my version to a new one, will the >easy_install tool modify my .pth file to remove the mention of the >old version from my sys.path and put the new version in the same >.pth file? Or will it now be listed in both places? Or will it >only in easy-install.pth? My understanding of the context of the question was that it applied to *system* packaging tools, which would be exclusively maintaining the .pth entries for the packages they installed. i.e., a scenario with *no* easy-install.pth. Setuptools will still detect the presence of their eggs, regardless of the means by which they're added to sys.path. But it would not *maintain* those .pth files. >Yes, but as you've already pointed out, they've escaped into a >larger ecosystem and this restriction is a severe limitation -- >leading to significant frustration. Especially as projects evolve >and want to do something more complex than simply install pure >Python code. Here at Enthought, we use and ship a number of >projects that have extensions and thus dynamic libraries that need >to either be modified during installation to work from the user's >installed location, or copied elsewhere on the system to avoid the >need to modify (which we also can't do via an egg install) env >variables, registries, etc. By the way, there *is* experimental shared library building support in setuptools, and I recently heard from Andi Vajda that he was successful in using it in his JCC project to make available a C++ library for linkage from JCC-built projects. (I'm also sitting on his patch that makes it work...) I'm not sure that it actually fixes the larger problem, in that e.g., if the main project is installed by the system, and then you build or install an egg yourself. But I think those problems are solvable. > We'd also love to be able to ship end-user enterprise-scale > applications via eggs so that bug fixes and updates don't require > downloading a monolithic 100MB+ installer. But doing that requires > the ability to update desktop icons, menus, etc. which we also > can't do automatically via an egg. Yep... a good post-install mechanism would be handy for wx and pywin32 as well. >If you don't want the burden on setuptools to support, much less >track, all these options, then perhaps it could just support >automatic execution of a post-install script (and pre-uninstall >script if uninstallation ever happens) that allows individual >project developers to do what they need to do? Let the burden of >describing how those things happen and how to >uninstall/relocate/update them fall to the provider of the projects >that do them. Yeah, that's what I really *don't* want. I'd like to enable a more trustable mechanism than a blindly-executed script. I'd rather see a standard that makes a developer document more, and have to at least *convince* the user that their post-install is worthwhile, even if the tool then makes it easy to run. Better still, I'd rather have those post-install parts done in such a way that things like icons, menus, manifests, registry stuff, etc., have to get explicitly listed instead of being done programatically. >Also, IIUC, stow only tries to "contain" the hard files. It puts >links in multiple standard locations (for man pages, executables, >libraries, etc.) If setuptools supported these options, I don't >think there'd be any discussion here except for things like "how do >I extend the set of things the tool supports so that my foo-type >files get linked into the standard /os/path/to/foo for the X os?" Yep. Having that would be a worthwhile thing, I think. Discussion leading to specs is most welcome. >I should have read ahead. This sounds close to what I've been >describing except that this leads me to picture a script that >prompts for install locations and allows the user to customize the >destinations rather than one that assumes everything goes in a >standard place. I'm all for this, and the continuation of the >ability to install an egg into a user-environment vs. a system-environment. +1. >The only thing missing here is the ability for the installer to >automatically run that script so that installation isn't a >disjointed, two-step manual process that a user is prone to forgot >to complete. I don't see a problem with a prompting process, backed by a log file that records what post-install steps are pending, finished, or explicitly rejected by the user. One possibility, by the way, is that we could overload "extras" for this purpose. Entry points (such as those for scripts) can require extras; if extras could mean post-install components like docs or icons or what-have-you, then trying to run the script could result in an error message telling you you need to "easy_install foo_package[icons]" or whatever. >One of the features of Enthought's Enstaller extension to >easy_install was that it looks for a post_install.py script in >EGG-INFO and if one is found, runs it. I would think that getting >this into setuptools would be a significant step forward but I >believe you previously rejected that idea. We'll take a stab at >creating a patch for you if you're more receptive to that idea >now. Just let me know. No -- I'm not happy with a straight-up executable hook for post-install steps. My evaluation of the state of PyPI is that I don't trust the community to write non-hazardous setup.py files, let alone post-install scripts. There should be a high technical and social barrier to including post-install hooks with arbitrary code. For example, if there was a required separation between installer tools and the things they install, such that any post-install operation had to be performed strictly by providing some human-readable data that will be passed to a separately-installed tool, and there was a high social stigma associated with writing your own post-install tool, then that might work. So, for example, if the community creates an icons and menus installer tool for the various platforms, and then anybody can use it in their project by adding the right data, then the user doesn't have to fully trust arbitrary package authors, only the authors of the post-install tools. I'm not saying that model is perfect; in fact I can see some potential pitfalls. But once an automatic post-install hole is opened it will be *very* hard to close, because it will always be *easier* to roll your own crappy post-installer instead of contributing to a set of robust cross-project/cross-platform tools. So I'd rather keep this particular "itch" in play and try to build up the scratching pressure until some people get together and pay attention long enough to solve the problem in a less hacky way. :) >>On the other hand, I've been puzzling over how to handle legitimate >>post-install features. On Windows, both wx and pywin32 have a real >>need to do some actuall "install" operations. Some is just copying >>files, but pywin32 also has to do some registry stuff. I don't know >>how to allow just what's sensible, without opening up a huge can of >>worms, though. >> > >I think there are lots of situations that are legitimate (projects >with extensions, projects that want to put icons on the desktop or >in menus, projects that need to interact with a registry, projects >that want to put configuration information somewhere other than in a >zip file in a site-packages dir, etc.) I think we should worry >less about preventing developers from shooting themselves in the foot It's the users' feet that I'm concerned with. Some people are already paranoid about the fact that PyPI doesn't use SSL and code signing, or that easy_install uses the intarwebs at all. I can just see the witch hunt when we start executing arbitrary code. Unh unh. No way am I letting that happen. Nope. > and more about ensuring that they can hunt for food for their survival. Right now, if you have a post-install script that's essential, you'll just have to convince your users to run it. Which nicely keeps easy_install out of what should be a conversation between developer and user. Enstaller is a different case - you are presumably installing an application, and the user is trusting your installer. easy_install is something else altogether, and is used by other programs such as buildout. Actually, I wonder if instead of trying to enhance setuptools for post-install, if maybe we should be looking at buildout recipes and maybe having a way for setuptools dependencies to point to buildout specs. IIRC, buildout specs can be remotely retrieved from a single URL, too. > We can always tighten things down after seeing the usecases that > develop, right? Actually, no, we can't, since backward compatibility would keep us from removing the hook, once people rely on it. I really feel yours (and others) pain on this issue, but it's one place where the users have to come first, and they need protection from the wilds of PyPI. Distribution and installation issues are not first on most developers' minds, so the fact that someone writes a great library on PyPI doesn't mean they can write installers worth a crap. Frankly, I wouldn't trust myself to write a correct post-installer on the first go -- perhaps *because* I have seen so many "simple" things go wrong. From tnelson at onresolve.com Wed Mar 19 02:18:01 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Tue, 18 Mar 2008 18:18:01 -0700 Subject: [Python-Dev] 3.0 buildbots all red In-Reply-To: <47E03061.7000906@holdenweb.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD173@EXMBX04.exchhosting.com> <47DD415F.7090109@v.loewis.de> <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD17D@EXMBX04.exchhosting.com>, <47E03061.7000906@holdenweb.com> Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE24@EXMBX04.exchhosting.com> > > Sounds like a challenge if ever I've heard one -- care to wager a beer on it? > > (Only applies to buildbots that are connected/online.) > Make sure you get a screen shot for OnYourDesktop if/when they *do* go > green! Screenshot? I'm going to buy a pack of iron-on transfers and sell t-shirts of it online. "All the buildbots were green momentarily after PyCon 2008... ....and all I got was this lousy t-shirt." Trent. From eric+python-dev at trueblade.com Wed Mar 19 02:23:29 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Tue, 18 Mar 2008 21:23:29 -0400 Subject: [Python-Dev] [Python-checkins] r61577 - in python/trunk: Include/code.h Include/compile.h Include/parsetok.h Include/pythonrun.h Lib/__future__.py Lib/test/test_print.py Misc/ACKS Misc/NEWS Parser/parser.c Parser/parsetok.c Python/bltinmodule.c Python/future.c ... In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE23@EXMBX04.exchhosting.com> References: <20080318234550.096EE1E4011@bag.python.org> <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE23@EXMBX04.exchhosting.com> Message-ID: <47E06B11.80904@trueblade.com> Yes, I know, and I'm looking at it. It doesn't fail on my Linux or Mac OS X boxes. I'm trying to duplicate the problem. I'm going to try it on my Windows box when I get home in about an hour. I'll fix it tonight. I realize there's a beer riding on the buildbots being green! Eric. Trent Nelson wrote: > This change breaks all the trunk buildbots: > > ====================================================================== > ERROR: testCompileLibrary (test.test_compiler.CompilerTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "S:\buildbots\python\trunk.nelson-windows\build\lib\test\test_compiler.py", line 52, in testCompileLibrary > compiler.compile(buf, basename, "exec") > File "S:\buildbots\python\trunk.nelson-windows\build\lib\compiler\pycodegen.py", line 64, in compile > gen.compile() > File "S:\buildbots\python\trunk.nelson-windows\build\lib\compiler\pycodegen.py", line 112, in compile > gen = ModuleCodeGenerator(tree) > File "S:\buildbots\python\trunk.nelson-windows\build\lib\compiler\pycodegen.py", line 1275, in __init__ > self.futures = future.find_futures(tree) > File "S:\buildbots\python\trunk.nelson-windows\build\lib\compiler\future.py", line 59, in find_futures > walk(node, p1) > File "S:\buildbots\python\trunk.nelson-windows\build\lib\compiler\visitor.py", line 106, in walk > walker.preorder(tree, visitor) > File "S:\buildbots\python\trunk.nelson-windows\build\lib\compiler\visitor.py", line 63, in preorder > self.dispatch(tree, *args) # XXX *args make sense? > File "S:\buildbots\python\trunk.nelson-windows\build\lib\compiler\visitor.py", line 57, in dispatch > return meth(node, *args) > File "S:\buildbots\python\trunk.nelson-windows\build\lib\compiler\future.py", line 27, in visitModule > if not self.check_stmt(s): > File "S:\buildbots\python\trunk.nelson-windows\build\lib\compiler\future.py", line 37, in check_stmt > "future feature %s is not defined" % name > SyntaxError: future feature print_function is not defined > > ________________________________________ > From: python-checkins-bounces+tnelson=onresolve.com at python.org [python-checkins-bounces+tnelson=onresolve.com at python.org] On Behalf Of eric.smith [python-checkins at python.org] > Sent: 18 March 2008 19:45 > To: python-checkins at python.org > Subject: [Python-checkins] r61577 - in python/trunk: Include/code.h Include/compile.h Include/parsetok.h Include/pythonrun.h Lib/__future__.py Lib/test/test_print.py Misc/ACKS Misc/NEWS Parser/parser.c Parser/parsetok.c Python/bltinmodule.c Python/future.c Pyth... > > Author: eric.smith > Date: Wed Mar 19 00:45:49 2008 > New Revision: 61577 > > Added: > python/trunk/Lib/test/test_print.py > Modified: > python/trunk/Include/code.h > python/trunk/Include/compile.h > python/trunk/Include/parsetok.h > python/trunk/Include/pythonrun.h > python/trunk/Lib/__future__.py > python/trunk/Misc/ACKS > python/trunk/Misc/NEWS > python/trunk/Parser/parser.c > python/trunk/Parser/parsetok.c > python/trunk/Python/bltinmodule.c > python/trunk/Python/future.c > python/trunk/Python/pythonrun.c > Log: > Backport of the print function, using a __future__ import. > This work is substantially Anthony Baxter's, from issue > 1633807. I just freshened it, made a few minor tweaks, > and added the test cases. I also created issue 2412, > which is to check for 2to3's behavior with the print > function. I also added myself to ACKS. > > Modified: python/trunk/Include/code.h > ============================================================================== > --- python/trunk/Include/code.h (original) > +++ python/trunk/Include/code.h Wed Mar 19 00:45:49 2008 > @@ -48,11 +48,12 @@ > #define CO_FUTURE_DIVISION 0x2000 > #define CO_FUTURE_ABSOLUTE_IMPORT 0x4000 /* do absolute imports by default */ > #define CO_FUTURE_WITH_STATEMENT 0x8000 > +#define CO_FUTURE_PRINT_FUNCTION 0x10000 > > /* This should be defined if a future statement modifies the syntax. > For example, when a keyword is added. > */ > -#if 0 > +#if 1 > #define PY_PARSER_REQUIRES_FUTURE_KEYWORD > #endif > > > Modified: python/trunk/Include/compile.h > ============================================================================== > --- python/trunk/Include/compile.h (original) > +++ python/trunk/Include/compile.h Wed Mar 19 00:45:49 2008 > @@ -24,6 +24,8 @@ > #define FUTURE_DIVISION "division" > #define FUTURE_ABSOLUTE_IMPORT "absolute_import" > #define FUTURE_WITH_STATEMENT "with_statement" > +#define FUTURE_PRINT_FUNCTION "print_function" > + > > struct _mod; /* Declare the existence of this type */ > PyAPI_FUNC(PyCodeObject *) PyAST_Compile(struct _mod *, const char *, > > Modified: python/trunk/Include/parsetok.h > ============================================================================== > --- python/trunk/Include/parsetok.h (original) > +++ python/trunk/Include/parsetok.h Wed Mar 19 00:45:49 2008 > @@ -27,6 +27,10 @@ > #define PyPARSE_WITH_IS_KEYWORD 0x0003 > #endif > > +#define PyPARSE_PRINT_IS_FUNCTION 0x0004 > + > + > + > PyAPI_FUNC(node *) PyParser_ParseString(const char *, grammar *, int, > perrdetail *); > PyAPI_FUNC(node *) PyParser_ParseFile (FILE *, const char *, grammar *, int, > > Modified: python/trunk/Include/pythonrun.h > ============================================================================== > --- python/trunk/Include/pythonrun.h (original) > +++ python/trunk/Include/pythonrun.h Wed Mar 19 00:45:49 2008 > @@ -8,7 +8,7 @@ > #endif > > #define PyCF_MASK (CO_FUTURE_DIVISION | CO_FUTURE_ABSOLUTE_IMPORT | \ > - CO_FUTURE_WITH_STATEMENT) > + CO_FUTURE_WITH_STATEMENT|CO_FUTURE_PRINT_FUNCTION) > #define PyCF_MASK_OBSOLETE (CO_NESTED) > #define PyCF_SOURCE_IS_UTF8 0x0100 > #define PyCF_DONT_IMPLY_DEDENT 0x0200 > > Modified: python/trunk/Lib/__future__.py > ============================================================================== > --- python/trunk/Lib/__future__.py (original) > +++ python/trunk/Lib/__future__.py Wed Mar 19 00:45:49 2008 > @@ -53,6 +53,7 @@ > "division", > "absolute_import", > "with_statement", > + "print_function", > ] > > __all__ = ["all_feature_names"] + all_feature_names > @@ -66,6 +67,7 @@ > CO_FUTURE_DIVISION = 0x2000 # division > CO_FUTURE_ABSOLUTE_IMPORT = 0x4000 # perform absolute imports by default > CO_FUTURE_WITH_STATEMENT = 0x8000 # with statement > +CO_FUTURE_PRINT_FUNCTION = 0x10000 # print function > > class _Feature: > def __init__(self, optionalRelease, mandatoryRelease, compiler_flag): > @@ -114,3 +116,7 @@ > with_statement = _Feature((2, 5, 0, "alpha", 1), > (2, 6, 0, "alpha", 0), > CO_FUTURE_WITH_STATEMENT) > + > +print_function = _Feature((2, 6, 0, "alpha", 2), > + (3, 0, 0, "alpha", 0), > + CO_FUTURE_PRINT_FUNCTION) > > Added: python/trunk/Lib/test/test_print.py > ============================================================================== > --- (empty file) > +++ python/trunk/Lib/test/test_print.py Wed Mar 19 00:45:49 2008 > @@ -0,0 +1,129 @@ > +"""Test correct operation of the print function. > +""" > + > +from __future__ import print_function > + > +import unittest > +from test import test_support > + > +import sys > +try: > + # 3.x > + from io import StringIO > +except ImportError: > + # 2.x > + from StringIO import StringIO > + > +from contextlib import contextmanager > + > +NotDefined = object() > + > +# A dispatch table all 8 combinations of providing > +# sep, end, and file > +# I use this machinery so that I'm not just passing default > +# values to print, I'm eiher passing or not passing in the > +# arguments > +dispatch = { > + (False, False, False): > + lambda args, sep, end, file: print(*args), > + (False, False, True): > + lambda args, sep, end, file: print(file=file, *args), > + (False, True, False): > + lambda args, sep, end, file: print(end=end, *args), > + (False, True, True): > + lambda args, sep, end, file: print(end=end, file=file, *args), > + (True, False, False): > + lambda args, sep, end, file: print(sep=sep, *args), > + (True, False, True): > + lambda args, sep, end, file: print(sep=sep, file=file, *args), > + (True, True, False): > + lambda args, sep, end, file: print(sep=sep, end=end, *args), > + (True, True, True): > + lambda args, sep, end, file: print(sep=sep, end=end, file=file, *args), > + } > + > + at contextmanager > +def stdout_redirected(new_stdout): > + save_stdout = sys.stdout > + sys.stdout = new_stdout > + try: > + yield None > + finally: > + sys.stdout = save_stdout > + > +# Class used to test __str__ and print > +class ClassWith__str__: > + def __init__(self, x): > + self.x = x > + def __str__(self): > + return self.x > + > +class TestPrint(unittest.TestCase): > + def check(self, expected, args, > + sep=NotDefined, end=NotDefined, file=NotDefined): > + # Capture sys.stdout in a StringIO. Call print with args, > + # and with sep, end, and file, if they're defined. Result > + # must match expected. > + > + # Look up the actual function to call, based on if sep, end, and file > + # are defined > + fn = dispatch[(sep is not NotDefined, > + end is not NotDefined, > + file is not NotDefined)] > + > + t = StringIO() > + with stdout_redirected(t): > + fn(args, sep, end, file) > + > + self.assertEqual(t.getvalue(), expected) > + > + def test_print(self): > + def x(expected, args, sep=NotDefined, end=NotDefined): > + # Run the test 2 ways: not using file, and using > + # file directed to a StringIO > + > + self.check(expected, args, sep=sep, end=end) > + > + # When writing to a file, stdout is expected to be empty > + o = StringIO() > + self.check('', args, sep=sep, end=end, file=o) > + > + # And o will contain the expected output > + self.assertEqual(o.getvalue(), expected) > + > + x('\n', ()) > + x('a\n', ('a',)) > + x('None\n', (None,)) > + x('1 2\n', (1, 2)) > + x('1 2\n', (1, ' ', 2)) > + x('1*2\n', (1, 2), sep='*') > + x('1 s', (1, 's'), end='') > + x('a\nb\n', ('a', 'b'), sep='\n') > + x('1.01', (1.0, 1), sep='', end='') > + x('1*a*1.3+', (1, 'a', 1.3), sep='*', end='+') > + x('a\n\nb\n', ('a\n', 'b'), sep='\n') > + x('\0+ +\0\n', ('\0', ' ', '\0'), sep='+') > + > + x('a\n b\n', ('a\n', 'b')) > + x('a\n b\n', ('a\n', 'b'), sep=None) > + x('a\n b\n', ('a\n', 'b'), end=None) > + x('a\n b\n', ('a\n', 'b'), sep=None, end=None) > + > + x('*\n', (ClassWith__str__('*'),)) > + x('abc 1\n', (ClassWith__str__('abc'), 1)) > + > + # 2.x unicode tests > + x(u'1 2\n', ('1', u'2')) > + x(u'u\1234\n', (u'u\1234',)) > + x(u' abc 1\n', (' ', ClassWith__str__(u'abc'), 1)) > + > + # errors > + self.assertRaises(TypeError, print, '', sep=3) > + self.assertRaises(TypeError, print, '', end=3) > + self.assertRaises(AttributeError, print, '', file='') > + > +def test_main(): > + test_support.run_unittest(TestPrint) > + > +if __name__ == "__main__": > + test_main() > > Modified: python/trunk/Misc/ACKS > ============================================================================== > --- python/trunk/Misc/ACKS (original) > +++ python/trunk/Misc/ACKS Wed Mar 19 00:45:49 2008 > @@ -622,6 +622,7 @@ > J. Sipprell > Kragen Sitaker > Christopher Smith > +Eric V. Smith > Gregory P. Smith > Rafal Smotrzyk > Dirk Soede > > Modified: python/trunk/Misc/NEWS > ============================================================================== > --- python/trunk/Misc/NEWS (original) > +++ python/trunk/Misc/NEWS Wed Mar 19 00:45:49 2008 > @@ -12,6 +12,9 @@ > Core and builtins > ----------------- > > +- Issue 1745. Backport print function with: > + from __future__ import print_function > + > - Issue 2332: add new attribute names for instance method objects. > The two changes are: im_self -> __self__ and im_func -> __func__ > > > Modified: python/trunk/Parser/parser.c > ============================================================================== > --- python/trunk/Parser/parser.c (original) > +++ python/trunk/Parser/parser.c Wed Mar 19 00:45:49 2008 > @@ -149,12 +149,10 @@ > strcmp(l->lb_str, s) != 0) > continue; > #ifdef PY_PARSER_REQUIRES_FUTURE_KEYWORD > - if (!(ps->p_flags & CO_FUTURE_WITH_STATEMENT)) { > - if (s[0] == 'w' && strcmp(s, "with") == 0) > - break; /* not a keyword yet */ > - else if (s[0] == 'a' && strcmp(s, "as") == 0) > - break; /* not a keyword yet */ > - } > + if (ps->p_flags & CO_FUTURE_PRINT_FUNCTION && > + s[0] == 'p' && strcmp(s, "print") == 0) { > + break; /* no longer a keyword */ > + } > #endif > D(printf("It's a keyword\n")); > return n - i; > @@ -208,6 +206,10 @@ > strcmp(STR(CHILD(cch, 0)), "with_statement") == 0) { > ps->p_flags |= CO_FUTURE_WITH_STATEMENT; > break; > + } else if (NCH(cch) >= 1 && TYPE(CHILD(cch, 0)) == NAME && > + strcmp(STR(CHILD(cch, 0)), "print_function") == 0) { > + ps->p_flags |= CO_FUTURE_PRINT_FUNCTION; > + break; > } > } > } > > Modified: python/trunk/Parser/parsetok.c > ============================================================================== > --- python/trunk/Parser/parsetok.c (original) > +++ python/trunk/Parser/parsetok.c Wed Mar 19 00:45:49 2008 > @@ -123,8 +123,8 @@ > return NULL; > } > #ifdef PY_PARSER_REQUIRES_FUTURE_KEYWORD > - if (flags & PyPARSE_WITH_IS_KEYWORD) > - ps->p_flags |= CO_FUTURE_WITH_STATEMENT; > + if (flags & PyPARSE_PRINT_IS_FUNCTION) > + ps->p_flags |= CO_FUTURE_PRINT_FUNCTION; > #endif > > for (;;) { > @@ -167,26 +167,6 @@ > str[len] = '\0'; > > #ifdef PY_PARSER_REQUIRES_FUTURE_KEYWORD > - /* This is only necessary to support the "as" warning, but > - we don't want to warn about "as" in import statements. */ > - if (type == NAME && > - len == 6 && str[0] == 'i' && strcmp(str, "import") == 0) > - handling_import = 1; > - > - /* Warn about with as NAME */ > - if (type == NAME && > - !(ps->p_flags & CO_FUTURE_WITH_STATEMENT)) { > - if (len == 4 && str[0] == 'w' && strcmp(str, "with") == 0) > - warn(with_msg, err_ret->filename, tok->lineno); > - else if (!(handling_import || handling_with) && > - len == 2 && str[0] == 'a' && > - strcmp(str, "as") == 0) > - warn(as_msg, err_ret->filename, tok->lineno); > - } > - else if (type == NAME && > - (ps->p_flags & CO_FUTURE_WITH_STATEMENT) && > - len == 4 && str[0] == 'w' && strcmp(str, "with") == 0) > - handling_with = 1; > #endif > if (a >= tok->line_start) > col_offset = a - tok->line_start; > > Modified: python/trunk/Python/bltinmodule.c > ============================================================================== > --- python/trunk/Python/bltinmodule.c (original) > +++ python/trunk/Python/bltinmodule.c Wed Mar 19 00:45:49 2008 > @@ -1486,6 +1486,78 @@ > equivalent to (x**y) % z, but may be more efficient (e.g. for longs)."); > > > +static PyObject * > +builtin_print(PyObject *self, PyObject *args, PyObject *kwds) > +{ > + static char *kwlist[] = {"sep", "end", "file", 0}; > + static PyObject *dummy_args; > + PyObject *sep = NULL, *end = NULL, *file = NULL; > + int i, err; > + > + if (dummy_args == NULL) { > + if (!(dummy_args = PyTuple_New(0))) > + return NULL; > + } > + if (!PyArg_ParseTupleAndKeywords(dummy_args, kwds, "|OOO:print", > + kwlist, &sep, &end, &file)) > + return NULL; > + if (file == NULL || file == Py_None) { > + file = PySys_GetObject("stdout"); > + /* sys.stdout may be None when FILE* stdout isn't connected */ > + if (file == Py_None) > + Py_RETURN_NONE; > + } > + > + if (sep && sep != Py_None && !PyString_Check(sep) && > + !PyUnicode_Check(sep)) { > + PyErr_Format(PyExc_TypeError, > + "sep must be None, str or unicode, not %.200s", > + sep->ob_type->tp_name); > + return NULL; > + } > + if (end && end != Py_None && !PyString_Check(end) && > + !PyUnicode_Check(end)) { > + PyErr_Format(PyExc_TypeError, > + "end must be None, str or unicode, not %.200s", > + end->ob_type->tp_name); > + return NULL; > + } > + > + for (i = 0; i < PyTuple_Size(args); i++) { > + if (i > 0) { > + if (sep == NULL || sep == Py_None) > + err = PyFile_WriteString(" ", file); > + else > + err = PyFile_WriteObject(sep, file, > + Py_PRINT_RAW); > + if (err) > + return NULL; > + } > + err = PyFile_WriteObject(PyTuple_GetItem(args, i), file, > + Py_PRINT_RAW); > + if (err) > + return NULL; > + } > + > + if (end == NULL || end == Py_None) > + err = PyFile_WriteString("\n", file); > + else > + err = PyFile_WriteObject(end, file, Py_PRINT_RAW); > + if (err) > + return NULL; > + > + Py_RETURN_NONE; > +} > + > +PyDoc_STRVAR(print_doc, > +"print(value, ..., sep=' ', end='\\n', file=sys.stdout)\n\ > +\n\ > +Prints the values to a stream, or to sys.stdout by default.\n\ > +Optional keyword arguments:\n\ > +file: a file-like object (stream); defaults to the current sys.stdout.\n\ > +sep: string inserted between values, default a space.\n\ > +end: string appended after the last value, default a newline."); > + > > /* Return number of items in range (lo, hi, step), when arguments are > * PyInt or PyLong objects. step > 0 required. Return a value < 0 if > @@ -2424,6 +2496,7 @@ > {"open", (PyCFunction)builtin_open, METH_VARARGS | METH_KEYWORDS, open_doc}, > {"ord", builtin_ord, METH_O, ord_doc}, > {"pow", builtin_pow, METH_VARARGS, pow_doc}, > + {"print", (PyCFunction)builtin_print, METH_VARARGS | METH_KEYWORDS, print_doc}, > {"range", builtin_range, METH_VARARGS, range_doc}, > {"raw_input", builtin_raw_input, METH_VARARGS, raw_input_doc}, > {"reduce", builtin_reduce, METH_VARARGS, reduce_doc}, > > Modified: python/trunk/Python/future.c > ============================================================================== > --- python/trunk/Python/future.c (original) > +++ python/trunk/Python/future.c Wed Mar 19 00:45:49 2008 > @@ -33,6 +33,8 @@ > ff->ff_features |= CO_FUTURE_ABSOLUTE_IMPORT; > } else if (strcmp(feature, FUTURE_WITH_STATEMENT) == 0) { > ff->ff_features |= CO_FUTURE_WITH_STATEMENT; > + } else if (strcmp(feature, FUTURE_PRINT_FUNCTION) == 0) { > + ff->ff_features |= CO_FUTURE_PRINT_FUNCTION; > } else if (strcmp(feature, "braces") == 0) { > PyErr_SetString(PyExc_SyntaxError, > "not a chance"); > > Modified: python/trunk/Python/pythonrun.c > ============================================================================== > --- python/trunk/Python/pythonrun.c (original) > +++ python/trunk/Python/pythonrun.c Wed Mar 19 00:45:49 2008 > @@ -738,18 +738,19 @@ > } > } > > +#if 0 > /* compute parser flags based on compiler flags */ > #define PARSER_FLAGS(flags) \ > ((flags) ? ((((flags)->cf_flags & PyCF_DONT_IMPLY_DEDENT) ? \ > PyPARSE_DONT_IMPLY_DEDENT : 0)) : 0) > - > -#if 0 > +#endif > +#if 1 > /* Keep an example of flags with future keyword support. */ > #define PARSER_FLAGS(flags) \ > ((flags) ? ((((flags)->cf_flags & PyCF_DONT_IMPLY_DEDENT) ? \ > PyPARSE_DONT_IMPLY_DEDENT : 0) \ > - | ((flags)->cf_flags & CO_FUTURE_WITH_STATEMENT ? \ > - PyPARSE_WITH_IS_KEYWORD : 0)) : 0) > + | ((flags)->cf_flags & CO_FUTURE_PRINT_FUNCTION ? \ > + PyPARSE_PRINT_IS_FUNCTION : 0)) : 0) > #endif > > int > _______________________________________________ > Python-checkins mailing list > Python-checkins at python.org > http://mail.python.org/mailman/listinfo/python-checkins > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/eric%2Bpython-dev%40trueblade.com > From greg.ewing at canterbury.ac.nz Wed Mar 19 02:51:48 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 19 Mar 2008 13:51:48 +1200 Subject: [Python-Dev] Python 3000: Special type for object attributes & map keys In-Reply-To: References: Message-ID: <47E071B4.8030503@canterbury.ac.nz> Neal Norwitz wrote: > Part of this is done, but very differently in that all strings used in > code objects are interned (stored in a dictionary And since two interned strings can be compared by pointer identity, I don't see how this differs significantly from the "unique integer" idea. If the integers were used to directly index an array instead of being used as dict keys, it might make a difference. The cost would be that every namespace would need to be as big as the number of names in existence, with most of them being extremely sparse. -- Greg From eric+python-dev at trueblade.com Wed Mar 19 02:57:46 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Tue, 18 Mar 2008 21:57:46 -0400 Subject: [Python-Dev] [Python-checkins] r61577 - in python/trunk: Include/code.h Include/compile.h Include/parsetok.h Include/pythonrun.h Lib/__future__.py Lib/test/test_print.py Misc/ACKS Misc/NEWS Parser/parser.c Parser/parsetok.c Python/bltinmodule.c Python/future.c ... In-Reply-To: <47E06B11.80904@trueblade.com> References: <20080318234550.096EE1E4011@bag.python.org> <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE23@EXMBX04.exchhosting.com> <47E06B11.80904@trueblade.com> Message-ID: <47E0731A.4030703@trueblade.com> I see the problem. Without -ucompiler, test_compiler doesn't compile everything. I'll fix the breakage shortly. Apologies. Eric Smith wrote: > Yes, I know, and I'm looking at it. It doesn't fail on my Linux or Mac > OS X boxes. I'm trying to duplicate the problem. I'm going to try it > on my Windows box when I get home in about an hour. I'll fix it tonight. > > I realize there's a beer riding on the buildbots being green! > > Eric. > > Trent Nelson wrote: >> This change breaks all the trunk buildbots: >> >> ====================================================================== >> ERROR: testCompileLibrary (test.test_compiler.CompilerTest) >> ---------------------------------------------------------------------- From musiccomposition at gmail.com Wed Mar 19 04:07:18 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 18 Mar 2008 22:07:18 -0500 Subject: [Python-Dev] PyErr_Warn or PyErr_WarnEx Message-ID: <1afaf6160803182007n3428087fncbd2dc275009ec6e@mail.gmail.com> Now that we're aggressively adding Py3k warnings to the trunk, I think it's a good time to get this straightened out. The docs [1] say PyErr_Warn is deprecated in favor of PyErr_WarnEx. However, I have seen both in recent checkins. What is preferred? [1] http://docs.python.org/dev/c-api/exceptions.html Cheers, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080318/4d348e41/attachment.htm From eric+python-dev at trueblade.com Wed Mar 19 05:11:01 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Wed, 19 Mar 2008 00:11:01 -0400 Subject: [Python-Dev] PEP 3127 (Integer Literal Support and Syntax): %o and %b Message-ID: <47E09255.8090907@trueblade.com> I've been double checking the PEP 3127 implementation in py3k and the backport I did to 2.6. The PEP says this about the % operator: "The string (and unicode in 2.6) % operator will have 'b' format specifier added for binary, and the alternate syntax of the 'o' option will need to be updated to add '0o' in front, instead of '0'." The %b operator was not added to 3.0, so I'll look into doing that in both 2.6 and 3.0 (which I opened as issue 2416). What should be done for '%#o' formatting in 2.6? The above sentence from the PEP implies it should be modified to add '0o' instead of '0', even in 2.6. But that seems like a bad idea to me. Maybe it should stay as-is, but add a -3 warning? Unfortunately, there'd be no way to change your code to get rid of the warning, short of switching to str.format() or adding a __future__ import (shudder). In 3.0, '%#o' already adds the leading '0o'. From nnorwitz at gmail.com Wed Mar 19 05:13:22 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 18 Mar 2008 23:13:22 -0500 Subject: [Python-Dev] PyErr_Warn or PyErr_WarnEx In-Reply-To: <1afaf6160803182007n3428087fncbd2dc275009ec6e@mail.gmail.com> References: <1afaf6160803182007n3428087fncbd2dc275009ec6e@mail.gmail.com> Message-ID: On Tue, Mar 18, 2008 at 10:07 PM, Benjamin Peterson wrote: > Now that we're aggressively adding Py3k warnings to the trunk, I think it's > a good time to get this straightened out. The docs [1] say PyErr_Warn is > deprecated in favor of PyErr_WarnEx. However, I have seen both in recent > checkins. What is preferred? PyErr_WarnEx should be used. PyErr_Warn is just (see from Include/pyerrors.h): #define PyErr_Warn(category, msg) PyErr_WarnEx(category, msg, 1) If someone wants to remove the macro in 3k, go for it. There are many of these compatibility macros and stub functions left around for binary compatibility. We should try to get rid of those in 3k and shrink the API. n From its.jeff.balogh at gmail.com Wed Mar 19 05:45:37 2008 From: its.jeff.balogh at gmail.com (Jeff Balogh) Date: Wed, 19 Mar 2008 00:45:37 -0400 Subject: [Python-Dev] changing regrtest to handle import errors In-Reply-To: References: Message-ID: <1205901273-sup-694@archie> Neal Norwitz wrote: > [changing to: and subject: ] > > On Tue, Mar 18, 2008 at 4:09 PM, Guido van Rossum wrote: > > On Tue, Mar 18, 2008 at 3:58 PM, neal.norwitz > > wrote: > > > Get this test to pass (UserList/UserDict no longer exist and caused a skip). > > > > I think the automatic skip on ImportError is harmful. > > > > We should add a helper function to test_support so that you can write > > > > foobar = test_support.import_optional('foobar') > > > > and it will skip the test if foobar cannot be imported; all other > > failing imports should cause the test to *fail*. > > > > Any takers? This should be an easy two-part task. > > This would be a great starter project for a new developer. > http://bugs.python.org/issue2409 > Let me know if you could use some help. Feel free to contact me on or off list. > > n This is available in the form of four patches on http://bugs.python.org/issue2409. The first adds test_support.optional_import, which I now see is the opposite of Guido's suggestion (blame the dyslexia). I actually prefer optional_import, though, since it puts 'import' next to the imported name, but I can add a fix if it's a problem. The next two patches refactor the imports of test_{sunaudiodev,winreg}.py to make the imports easier to work with in the new scheme. The last patch fixes the stdlib tests to use optional_import at the spots where I was getting ImportErrors on my box (x86 Linux). Please test these on your boxes so we can discover all the ImportErrors before the buildbots do. Thanks, Jeff Balogh From jackdied at jackdied.com Wed Mar 19 06:14:33 2008 From: jackdied at jackdied.com (Jack Diederich) Date: Wed, 19 Mar 2008 01:14:33 -0400 Subject: [Python-Dev] Not backporting PEP 3115 (metaclass __prepare__) Message-ID: <20080319051433.GA16598@performancedrivers.com> We can't backport the __prepare__ syntax without requiring metaclass definition on the 'class' line. Because the __metaclass__ definition can be at the end of the class in 2.6 we can't find it until after we execute the class and that is too late to use a custom dictionary. I wish I had thought of that yesterday, -Jack From grantgm at mcmaster.ca Wed Mar 19 08:24:49 2008 From: grantgm at mcmaster.ca (Gabriel Grant) Date: Wed, 19 Mar 2008 02:24:49 -0500 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses Message-ID: Hi all, This gem from unittest.py is pretty much the opposite of "one obvious way": # Synonyms for assertion methods assertEqual = assertEquals = failUnlessEqual assertNotEqual = assertNotEquals = failIfEqual assertAlmostEqual = assertAlmostEquals = failUnlessAlmostEqual assertNotAlmostEqual = assertNotAlmostEquals = failIfAlmostEqual assertRaises = failUnlessRaises assert_ = assertTrue = failUnless assertFalse = failIf Could these be removed for 3k? There was a short discussion about this among some of those those present in the Python Core sprint room at PyCon today and most preferred the "assertEqual" form for [Not][Almost]Equal and Raises. With assertFalse vs. failIf (and assertTrue vs. failUnless) there was far less agreement. JUnit uses assertTrue exclusively, and most people said they feel that using assertTrue would be more consistent, but many (myself included) still think failUnless and failIf are much more natural. Another issue with assertTrue is that it doesn't actually test for 'True', strictly speaking, since it is based on equality, not identity. Its also interesting to note the original commit message: > r34209 | purcell | 2003-09-22 06:08:12 -0500 (Mon, 22 Sep 2003) > [...] > - New assertTrue and assertFalse aliases for comfort of JUnit users > [...] assertEqual (and its cousins) were already present at that point. In any case, if the decision is made to not use failUnless, something still needs to be done with assert_ vs. assertTrue. assert_ seems somewhat better to me, in that it has fewer characters, but I think that a case could certainly be made to keep both of these. I certainly don't have the authority to make a call on any of this, but if someone else decides what colour to paint this bike shed, I can try to get it done (hopefully with 2.6 warnings) tomorrow. Cheers, -Gabriel P.S. If you were in the sprint room and feel terribly misrepresented, please feel free to give me a swift kick both on-list and in person tomorrow morning. From jyasskin at gmail.com Wed Mar 19 09:20:56 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Wed, 19 Mar 2008 03:20:56 -0500 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: References: Message-ID: <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> +1 to assert* from me. the fail* variants always feel like double-negatives. I also always use assertTrue instead of assert_. But I don't care enough to argue about it. :) On Wed, Mar 19, 2008 at 2:24 AM, Gabriel Grant wrote: > Hi all, > > This gem from unittest.py is pretty much the opposite of "one obvious way": > > # Synonyms for assertion methods > > assertEqual = assertEquals = failUnlessEqual > > assertNotEqual = assertNotEquals = failIfEqual > > assertAlmostEqual = assertAlmostEquals = failUnlessAlmostEqual > > assertNotAlmostEqual = assertNotAlmostEquals = failIfAlmostEqual > > assertRaises = failUnlessRaises > > assert_ = assertTrue = failUnless > > assertFalse = failIf > > Could these be removed for 3k? > > There was a short discussion about this among some of those those > present in the Python Core sprint room at PyCon today and most > preferred the "assertEqual" form for [Not][Almost]Equal and Raises. > > With assertFalse vs. failIf (and assertTrue vs. failUnless) there was > far less agreement. JUnit uses assertTrue exclusively, and most people > said they feel that using assertTrue would be more consistent, but > many (myself included) still think failUnless and failIf are much more > natural. Another issue with assertTrue is that it doesn't actually > test for 'True', strictly speaking, since it is based on equality, not > identity. > > Its also interesting to note the original commit message: > > > r34209 | purcell | 2003-09-22 06:08:12 -0500 (Mon, 22 Sep 2003) > > [...] > > - New assertTrue and assertFalse aliases for comfort of JUnit users > > [...] > > assertEqual (and its cousins) were already present at that point. > > In any case, if the decision is made to not use failUnless, something > still needs to be done with assert_ vs. assertTrue. assert_ seems > somewhat better to me, in that it has fewer characters, but I think > that a case could certainly be made to keep both of these. > > I certainly don't have the authority to make a call on any of this, > but if someone else decides what colour to paint this bike shed, I can > try to get it done (hopefully with 2.6 warnings) tomorrow. > > Cheers, > > -Gabriel > > P.S. If you were in the sprint room and feel terribly misrepresented, > please feel free to give me a swift kick both on-list and in person > tomorrow morning. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jyasskin%40gmail.com > -- Namast?, Jeffrey Yasskin http://jeffrey.yasskin.info/ From mail at timgolden.me.uk Wed Mar 19 09:39:25 2008 From: mail at timgolden.me.uk (Tim Golden) Date: Wed, 19 Mar 2008 08:39:25 +0000 Subject: [Python-Dev] 3.0 buildbots all red In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE24@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD173@EXMBX04.exchhosting.com> <47DD415F.7090109@v.loewis.de> <87D3F9C72FBF214DB39FA4E3FE618CDC6E168AD17D@EXMBX04.exchhosting.com>, <47E03061.7000906@holdenweb.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE24@EXMBX04.exchhosting.com> Message-ID: <47E0D13D.8030004@timgolden.me.uk> Trent Nelson wrote: >>> Sounds like a challenge if ever I've heard one -- care to wager a beer on it? >>> (Only applies to buildbots that are connected/online.) > >> Make sure you get a screen shot for OnYourDesktop if/when they *do* go >> green! > > Screenshot? I'm going to buy a pack of iron-on transfers and sell t-shirts of it online. > > "All the buildbots were green momentarily after PyCon 2008... > ....and all I got was this lousy t-shirt." Momentarily? You mean they were only up for a few seconds? From jeff at taupro.com Wed Mar 19 09:57:21 2008 From: jeff at taupro.com (Jeff Rush) Date: Wed, 19 Mar 2008 03:57:21 -0500 Subject: [Python-Dev] Capsule Summary of Some Packaging/Deployment Technology Concerns In-Reply-To: <20080318004718.56E173A40FF@sparrow.telecommunity.com> References: <47DEEC6B.5040602@taupro.com> <20080318004718.56E173A40FF@sparrow.telecommunity.com> Message-ID: <47E0D571.2070004@taupro.com> Phillip J. Eby wrote: > > I'm actually happy to hear that there's this much energy available -- > hopefully some of it can be harnessed towards positive solutions. > > When I began developing setuptools, I often asked for the input of > packagers, developers, etc., through the distutils-sig... and was met > with overwhelming silence. So the fact that there is now a group of > people who are ready to work for some solutions seems like a positive > change, to me. I can appreciate how frustrating silence is when you call for input. Let's see if we can keep the volunteer energy going this time around. > It's hard to make design decisions regarding itches you don't personally > have, and which other people won't help scratch. Unfortunately, a lot > of the proposals from packaging system people have been of the form of, > "fix this for us by breaking things for other people". Not all of them, > though. Many have been very helpful, contributing troubleshooting help > and good patches. > > That some of those good patches took nearly a year to get into > setuptools (some from Fedora just got into 0.6c8 that were sent to me > almost a year ago) is because I'm the only person reviewing setuptools > patches, and I've spent only a few days in the last year doing focused > development work on setuptools (as opposed to answering questions about > it on the SIG). > > It's never a good thing when people's patches sit around, regardless of > where they come from. But that's not the same thing as *rejecting* the > patches. I and others appreciate your call for more patches on various topics. However a long delay in applying them will discourage contribution. Are you open to giving certain others patch view/commit privileges to setuptools? I'd be willing to help out, and keep a carefully balanced hand in what is accepted. -Jeff From jeff at taupro.com Wed Mar 19 10:10:27 2008 From: jeff at taupro.com (Jeff Rush) Date: Wed, 19 Mar 2008 04:10:27 -0500 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? Message-ID: <47E0D883.9030402@taupro.com> As I'm digging into packaging issues here at PyCon, a couple of Python 3000 related matters occur to me. As I'm new to the Python 3000 development, if these have already been addressed in prior discussions, I apologize for your time. 1. What is the plan for PyPI when Python 3.0 comes out and dependencies start getting satisfied from distribution across the great divide, e.g. a 3.0-specific package pulls from PyPI a 2.x-specific package to meet some need? Are there plans to fork PyPI, apply special tags to uploads or what? While binary distributions are tagged with the Python version, source distributions are not. And of course a dependency expression as it stands today for "SomePackage > 2.4" may pull 3.0 to satisfy it. 2. There have been attempts over the years to fix distutils, with the last one being in 2006 by Anthony Baxter. He stated that a major hurdle was the strong demand to respect backward compatibility and he finally gave up. One of the purposes of Python 3.0 was the freedom to break backward compatibility for the sake of "doing the right thing". So is it now permissible to give distutils a good reworking and stop letting compatibility issues hold us back? -Jeff From p.f.moore at gmail.com Wed Mar 19 10:52:19 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 19 Mar 2008 09:52:19 +0000 Subject: [Python-Dev] [Distutils] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <47E0D883.9030402@taupro.com> References: <47E0D883.9030402@taupro.com> Message-ID: <79990c6b0803190252w566d81n971a81df616c8cae@mail.gmail.com> On 19/03/2008, Jeff Rush wrote: > 1. What is the plan for PyPI when Python 3.0 comes out and > dependencies start getting satisfied from distribution > across the great divide, e.g. a 3.0-specific package > pulls from PyPI a 2.x-specific package to meet some > need? As distutils (and core Python) doesn't do any automatic dependency management, this is a setuptools issue. As such, it's up to setuptools to deal with it. There may be infrastructure changes that would be generally useful, but there's nothing *needed* for the core. > 2. There have been attempts over the years to fix distutils, > with the last one being in 2006 by Anthony Baxter. He > stated that a major hurdle was the strong demand to > respect backward compatibility and he finally gave up. > One of the purposes of Python 3.0 was the freedom to > break backward compatibility for the sake of "doing > the right thing". So is it now permissible to give > distutils a good reworking and stop letting > compatibility issues hold us back? Sounds reasonable. I'm sure patches would be considered, but past discussions around "including setuptools" have been controversial and generally not reached consensus (for reasons other than pure backward compatibility). Also, while compatibility isn't as important for 3.0, smooth migration *is* - so any incompatible proposal must include some consideration of how to assist people with huge, complex setup.py files which use distutils internals in complex ways. So be prepared to do some work :-) (But I'd be happy to see distutils improved. I just don't have any need for such improvement, personally). Paul. From vinay_sajip at red-dove.com Wed Mar 19 09:36:52 2008 From: vinay_sajip at red-dove.com (Vinay Sajip) Date: Wed, 19 Mar 2008 08:36:52 -0000 Subject: [Python-Dev] logging shutdown (was: Re: [Python-checkins] r61431 - python/trunk/Doc/library/logging.rst) References: Message-ID: <001901c8899c$65d01ba0$0200a8c0@alpha> > I think (repeatedly) testing an app through IDLE is a reasonable use case. I don't disagree, but cleanup of logging may not be all that trivial in some scenarios: for example, if multiple threads have references to handlers, then after shutdown() was called, logging by those threads would fail due to I/O errors - a reasonable outcome. However, logging cannot know if threads contain references to loggers or handlers, and so I do not e.g. remove loggers on shutdown(). > Would it be reasonable for shutdown to remove logging from > sys.modules, so that a rerun has some chance of succeeding via its own > import? > I'm not sure that would be enough in the scenario I mentioned above - would removing a module from sys.modules be a guarantee of removing it from memory? If so, what if still-running threads contained references to loggers etc. - wouldn't they potentially segfault? It's safer, in my view, for the developer of an application to do cleanup of their app if they want to test repeatedly in IDLE. After all, this could involve cleanup of other resources which are nothing to do with logging; and a developer can certainly remove handlers from loggers using the existing API, after ensuring all their threads are done and there are no unaccounted-for references to loggers and handlers. Regards, Vinay Sajip From ncoghlan at gmail.com Wed Mar 19 12:51:23 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 19 Mar 2008 21:51:23 +1000 Subject: [Python-Dev] Checking for the -3 flag from Python code Message-ID: <47E0FE3B.8030006@gmail.com> This flag is exposed to python code as sys.flags.py3k_warning So the hack added to some of the test code that I saw go by on python-checkins isn't needed :) Cheers, Nick -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From s.r at visotech.at Tue Mar 18 08:29:10 2008 From: s.r at visotech.at (Stefan Ring) Date: Tue, 18 Mar 2008 08:29:10 +0100 Subject: [Python-Dev] Improved thread switching Message-ID: <20080319115107.YPWO11090.viefep31-int.chello.at@zion.visotech.at> The company I work for has over the last couple of years created an application server for use in most of our customer projects. It embeds Python and most project code is written in Python by now. It is quite resource-hungry (several GB of RAM, MySQL databases of 50-100GB). And of course it is multi-threaded and, at least originally, we hoped to make it utilize multiple processor cores. Which, as we all know, doesn't sit very well with Python. Our application runs heavy background calculations most of the time (in Python) and has to service multiple (few) GUI clients at the same time, also using Python. The problem was that a single background thread would increase the response time of the client threads by a factor of 10 or (usually) more. This led me to add a dirty hack to the Python core to make it switch threads more frequently. While this hack greatly improved response time for the GUI clients, it also slowed down the background threads quite a bit. top would often show significantly less CPU usage -- 80% instead of the more usual 100%. The problem with thread switching in Python is that the global semaphore used for the GIL is regularly released and immediately reacquired. Unfortunately, most of the time this leads to the very same thread winning the race on the semaphore again and thus more wait time for the other threads. This is where my dirty patch intervened and just did a nanosleep() for a short amount of time (I used 1000 nsecs). I have then created a better scheduling scheme and written a small test program that nicely mimics what Python does for some statistics. I call the scheduling algorithm the round-robin semaphore because threads can now run in a more or less round-robin fashion. Actually, it's just a semaphore with FIFO semantics. The implementation problem with the round-robin semaphore is the __thread variable I had to use because I did not want to change the signature of the Enter() and Leave() methods. For CPython, I have replaced this thread-local allocation with an additional field in the PyThreadState. Because of that, the patch for CPython I have already created is a bit more involved than the simple nanosleep() hack. Consequently, it's not very polished yet and not at all as portable as the rest of the Python core. I now show you the results from the test program which compares all three scheduling mechanisms -- standard python, my dirty hack and the new round-robin semaphore. I also show you the test program containing the three implementations nicely encapsulated. The program was run on a quad-core Xeon 1.86 GHz on Fedora 5 x86_64. The first three lines from the output (including the name of the algorithm) should be self-explanatory. The fourth and the fifth show a distribution of wait times for the individual threads. The ideal distribution would be everything on the number of threads (2 in this case) and zero everywhere else. As you can see, the round-robin semaphore is pretty close to that. Also, because of the high thread switching frequency, we could lower Python's checkinterval -- the jury is still out on the actual value, likely something between 1000 and 10000. I can post my Python patch if there is enough interest. Thanks for your attention. -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: results.txt Url: http://mail.python.org/pipermail/python-dev/attachments/20080318/4b3d5d53/attachment.txt -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: p.cpp Url: http://mail.python.org/pipermail/python-dev/attachments/20080318/4b3d5d53/attachment-0001.txt From ncoghlan at gmail.com Wed Mar 19 12:55:08 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 19 Mar 2008 21:55:08 +1000 Subject: [Python-Dev] [Python-3000-checkins] r61522 - python/branches/py3k/Lib/test/test_print.py In-Reply-To: <20080318153528.20E4D1E402B@bag.python.org> References: <20080318153528.20E4D1E402B@bag.python.org> Message-ID: <47E0FF1C.4020709@gmail.com> eric.smith wrote: > + at contextmanager > +def stdout_redirected(new_stdout): > + save_stdout = sys.stdout > + sys.stdout = new_stdout > + try: > + yield None > + finally: > + sys.stdout = save_stdout I think this test could easily be tweaked to use test.test_support.captured_stdout rather than reinventing the wheel :) (cc'ing python-dev for visibility since the sprints are generating a lot of python-checkins traffic) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From eric+python-dev at trueblade.com Wed Mar 19 13:04:17 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Wed, 19 Mar 2008 08:04:17 -0400 Subject: [Python-Dev] [Python-3000-checkins] r61522 - python/branches/py3k/Lib/test/test_print.py In-Reply-To: <47E0FF1C.4020709@gmail.com> References: <20080318153528.20E4D1E402B@bag.python.org> <47E0FF1C.4020709@gmail.com> Message-ID: <47E10141.3090702@trueblade.com> Nick Coghlan wrote: > eric.smith wrote: >> + at contextmanager >> +def stdout_redirected(new_stdout): >> + save_stdout = sys.stdout >> + sys.stdout = new_stdout >> + try: >> + yield None >> + finally: >> + sys.stdout = save_stdout > > I think this test could easily be tweaked to use > test.test_support.captured_stdout rather than reinventing the wheel :) Who reinvented the wheel? I stole this code from the PEP! :) I'll modify it. Thanks for noticing. Eric. From musiccomposition at gmail.com Wed Mar 19 14:13:48 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Wed, 19 Mar 2008 08:13:48 -0500 Subject: [Python-Dev] Checking for the -3 flag from Python code In-Reply-To: <47E0FE3B.8030006@gmail.com> References: <47E0FE3B.8030006@gmail.com> Message-ID: <1afaf6160803190613p5ccd7818u4cc6bc066987c7a4@mail.gmail.com> On Wed, Mar 19, 2008 at 6:51 AM, Nick Coghlan wrote: > This flag is exposed to python code as sys.flags.py3k_warning > > So the hack added to some of the test code that I saw go by on > python-checkins isn't needed :) Somebody can remove TODO in Lib/test/test_py3kwarn.py. > > > Cheers, > Nick > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > http://www.boredomandlaziness.org > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080319/209ea4a4/attachment.htm From martin at v.loewis.de Wed Mar 19 14:34:17 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 19 Mar 2008 08:34:17 -0500 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <47E0D883.9030402@taupro.com> References: <47E0D883.9030402@taupro.com> Message-ID: <47E11659.9000307@v.loewis.de> > 1. What is the plan for PyPI when Python 3.0 comes out and > dependencies start getting satisfied from distribution > across the great divide, e.g. a 3.0-specific package > pulls from PyPI a 2.x-specific package to meet some > need? Are there plans to fork PyPI, apply special > tags to uploads or what? I don't see the need to for PyPI. For packages (or "distributions", to avoid confusion with Python packages), I see two options: a) provide a single release that supports both 2.x and 3.x. The precise strategy to do so might vary. If one is going for a single source version, have setup.py run 2to3 (or perhaps 3to2). For dual-source packages, have setup.py just install the one for the right version; setup.py itself needs to be written so it runs on both versions (which is easy to do). b) switch to Python 3 at some point (i.e. burn your bridges). You seem to be implying that some projects may release separate source distributions. I cannot imagine why somebody would want to do that. > 2. There have been attempts over the years to fix distutils, > with the last one being in 2006 by Anthony Baxter. He > stated that a major hurdle was the strong demand to > respect backward compatibility and he finally gave up. Can you kindly refer to some archived discussion for that? > One of the purposes of Python 3.0 was the freedom to > break backward compatibility for the sake of "doing > the right thing". So is it now permissible to give > distutils a good reworking and stop letting > compatibility issues hold us back? I don't know what the proposed changes are, but for some changes; in general, I feel that the need for backwards compatibility is exaggerated. Regards, Martin From barry at python.org Wed Mar 19 14:54:12 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 19 Mar 2008 08:54:12 -0500 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> References: <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> Message-ID: <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 19, 2008, at 3:20 AM, Jeffrey Yasskin wrote: > +1 to assert* from me. the fail* variants always feel like > double-negatives. I also always use assertTrue instead of assert_. But > I don't care enough to argue about it. :) I'm in the camp that Gabriel describes. I prefer assertEqual/ assertRaises and failIf/failUnless. I like the latter because it reads nicely: fail unless [this thing is true], fail if [this thing is true]. OTOH, I'd rather there be OOWTDI so whatever the consensus is is fine with me. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR+EbBXEjvBPtnXfVAQIWDAQAi3/aoOhxeeeY85J4GEAW8hk3ONBQTUi8 jSdm62ooDndcuROZbC2EqfEGJJvX/JnbwstT195HD1EpsOohtA9tObZ294BO5vpg 4lEQhqqXlkQsZEgwM6+pcW8xUI3mv0HPiT/HZZZj/+71FpToSElist/l/sLYIvZv At7qT4DFKeo= =jyxp -----END PGP SIGNATURE----- From barry at python.org Wed Mar 19 15:00:10 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 19 Mar 2008 09:00:10 -0500 Subject: [Python-Dev] Capsule Summary of Some Packaging/Deployment Technology Concerns In-Reply-To: <47E0D571.2070004@taupro.com> References: <47DEEC6B.5040602@taupro.com> <20080318004718.56E173A40FF@sparrow.telecommunity.com> <47E0D571.2070004@taupro.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 19, 2008, at 3:57 AM, Jeff Rush wrote: > > I and others appreciate your call for more patches on various > topics. However > a long delay in applying them will discourage contribution. Are you > open to > giving certain others patch view/commit privileges to setuptools? > I'd be > willing to help out, and keep a carefully balanced hand in what is > accepted. The Python sandbox has a setuptools directory. Is this the canonical location for the code? If so, then anybody who has Python commit privileges can commit to it and help further develop setuptools. If not, why not and what is the sandbox setuptools used for? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR+Eca3EjvBPtnXfVAQLabwP9F8NtQX6YsDXJMHiByCGILPAQ2NgtaIzg en6yYbhl5IAweTr0DtWzxRXjSGMifK/D4PmtRSWWUTy3VY+8cRUkYuBjIxPOHJRF 4TA4dYoW4f2+qM1IO/l59FIAJgUyrXKhv3aznpXBFl+PaRCW9qP9G1lur3lolipB h4i8ya+I7zU= =2/iq -----END PGP SIGNATURE----- From murman at gmail.com Wed Mar 19 15:21:53 2008 From: murman at gmail.com (Michael Urman) Date: Wed, 19 Mar 2008 09:21:53 -0500 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> References: <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> Message-ID: > OTOH, I'd rather there be OOWTDI so whatever the consensus is is fine > with me. This strikes me as a gratuitous API change of the kind Guido was warning about in his recent post: "Don't change your APIs incompatibly when porting to Py3k" Yes it removes redundancy, but it really doesn't change the cognitive load (at least for native speakers). If the blessed set were restricted to assert*, what would users of fail* do when trying to test their packages on py3k? Search and replace, or monkey patch unittest? I'm guessing monkey patch unittest, which means the change saves nothing, and costs plenty. Note the acronym is OOWTDI, not OONTDI - using a different name does not necessarily make it a different way. -- Michael Urman From jek-gmane at kleckner.net Wed Mar 19 16:12:04 2008 From: jek-gmane at kleckner.net (Jim Kleckner) Date: Wed, 19 Mar 2008 08:12:04 -0700 Subject: [Python-Dev] Installing Python 2.6 alpha1 on Windows XP In-Reply-To: <47DFC5A9.90401@v.loewis.de> References: <47DEEED6.7070904@aon.at> <79990c6b0803171625u5159380at46e4464b2a72dcdd@mail.gmail.com> <47DF0470.5060108@aon.at> <79990c6b0803180239r5167d5b8x8c9d2d74641a323@mail.gmail.com> <47DFC5A9.90401@v.loewis.de> Message-ID: Martin v. L?wis wrote: >> That's odd. In theory, having msvcr90.dll in C:\Python26 should be >> sufficient, as once python.exe is loaded, its directory is added to >> the DLL search path. Maybe it's something to do with the "side by side >> manifest installation" stuff (or whatever it's called). > > Yes, with VS 2008, the DLL search path becomes irrelevant (or so it > seems). > >> Martin, can you comment? It looks like the 3.0 installer uses 2 copies >> of msvcr90.dll, where the 2.6 one doesn't. I would have thought that >> only one is necessary, but Gregor's experiments seem to demonstrate >> otherwise. > > I haven't figured it out myself; it's a complete mess, and Microsoft > is heavily wasting our time. > > It seems that you absolutely *must* have the manifest file in each > directory that has a DLL which links with the CRT. Whether or not > separate copies of the DLL are then also necessary, and whether or > not that causes two copies to be loaded into the address space, > I don't know. HELP!!!! > > To reproduce the problem, you probably have to test on a machine > which doesn't have the CRT redistributable installed centrally > (neither through VS 2008 installation, nor by running the > standalone CRT installer, nor by having installed any other software > that provides an SxS copy of the CRT). FYI, here is a related bug report on this: http://bugs.python.org/issue2256 If you send me private email, I can test installs on that machine. From pje at telecommunity.com Wed Mar 19 16:21:00 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 19 Mar 2008 11:21:00 -0400 Subject: [Python-Dev] Capsule Summary of Some Packaging/Deployment Technology Concerns In-Reply-To: <47E0D571.2070004@taupro.com> References: <47DEEC6B.5040602@taupro.com> <20080318004718.56E173A40FF@sparrow.telecommunity.com> <47E0D571.2070004@taupro.com> Message-ID: <20080319152101.730BC3A40A5@sparrow.telecommunity.com> At 03:57 AM 3/19/2008 -0500, Jeff Rush wrote: >Are you open to giving certain others patch view/commit privileges >to setuptools? Jim Fulton has such already. I'm open to extending that to others who have a good grasp of the subtleties involved. Truthfully, if we can just get 0.6 put to bed, I could probably open up the trunk a lot wider. One of the things that slows me down is that patches usually don't come with tests, so I usually have to manually smoke-test them for scenarios I think they'll effect. There isn't really any automated procedure. Probably the most frustrating thing (or "chief amongst the most frustrating things") about setuptools development is that it's a black hole. By which I mean that backward compatibility and cruft accretion make it difficult to get out of. In the beginning, there was the distutils. Distutils begat setuptools, and setuptools begat virtualenv and zc.buildout and source control plugins. Etc., etc. What I think is really needed in the long run is to keep eggs, but get rid of setuptools and the distutils in their current form. There's a lot of brokenness there, and also a lot of accumulated cruft. We really need a distutils 3000, and it needs to be built on a better approach. In truth, my *real* motivation for PEP 365's bootstrap tool isn't so much to support the package management tools we have today, as it is to support a new one tomorrow. I have a few ideas for ways to shift the paradigm of how individual projects get built, to incorporate many scenarios that don't work well now. But to implement those things in such a next-generation tool, I will not want to be restricted to just what's in the stdlib or what can be bundled in the tool. (Btw, by "real" motivation, I don't mean I've been deceptive about my intentions, I mean that my strong intuition that such a bootstrap facility is needed, is probably being fueled by the long term desire to replace the entire distutils-based infrastructure with something better.) > I'd be willing to help out, and keep a carefully balanced hand in > what is accepted. And I think it's probably getting close to time I stepped down from day-to-day management of the codebase (which is more like month-to-month or quarter-to-quarter for me lately). It will probably be a lot easier for me to step back and critique stuff that goes in, after the fact, than to go over the stuff beforehand. :) I'm not sure exactly how to go about such a handoff though. My guess is that we need a bug/patch tracker, and a few people to review, test, and apply. Maybe a transitional period during which I just say yea or nay and let others do the test and apply, before opening it up entirely. That way, we can perhaps solidify a few principles that I'd like to have stay in place. (Like no arbitrary post-install code hooks.) btw, offtopic question: are you by any chance the same Jeff Rush who invented EchoMail? From stephen at xemacs.org Wed Mar 19 16:44:20 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 20 Mar 2008 00:44:20 +0900 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: References: <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> Message-ID: <87hcf2afcb.fsf@uwakimon.sk.tsukuba.ac.jp> Michael Urman writes: > Yes it removes redundancy, but it really doesn't change the cognitive > load (at least for native speakers). Actually, OONTDI is important to me, cognitively. Multiple names implies the possibility of multiple semantics, often unintentionally. Here that can't be the case by construction, but I wouldn't have known that if I weren't reading this thread. And I still won't be sure for any given one of them, since there are so many remembering the whole list is "hard." > If the blessed set were restricted to assert*, what would users of > fail* do when trying to test their packages on py3k? Search and > replace, or monkey patch unittest? So we should add this to 2to3, no? They're going to run that anyway. > I'm guessing monkey patch unittest, which means I can flag them as people in too much of a hurry to do things right. > the change saves nothing, and costs plenty. That's a bit short-sighted, no? It saves nothing for old working code. But 90% of the Python code in use 5 years from now will be written between now and then. From rhamph at gmail.com Wed Mar 19 16:59:42 2008 From: rhamph at gmail.com (Adam Olsen) Date: Wed, 19 Mar 2008 09:59:42 -0600 Subject: [Python-Dev] Improved thread switching In-Reply-To: <20080319115107.YPWO11090.viefep31-int.chello.at@zion.visotech.at> References: <20080319115107.YPWO11090.viefep31-int.chello.at@zion.visotech.at> Message-ID: On Tue, Mar 18, 2008 at 1:29 AM, Stefan Ring wrote: > The company I work for has over the last couple of years created an > application server for use in most of our customer projects. It embeds Python > and most project code is written in Python by now. It is quite resource-hungry > (several GB of RAM, MySQL databases of 50-100GB). And of course it is > multi-threaded and, at least originally, we hoped to make it utilize multiple > processor cores. Which, as we all know, doesn't sit very well with Python. Our > application runs heavy background calculations most of the time (in Python) > and has to service multiple (few) GUI clients at the same time, also using > Python. The problem was that a single background thread would increase the > response time of the client threads by a factor of 10 or (usually) more. > > This led me to add a dirty hack to the Python core to make it switch threads > more frequently. While this hack greatly improved response time for the GUI > clients, it also slowed down the background threads quite a bit. top would > often show significantly less CPU usage -- 80% instead of the more usual 100%. > > The problem with thread switching in Python is that the global semaphore used > for the GIL is regularly released and immediately reacquired. Unfortunately, > most of the time this leads to the very same thread winning the race on the > semaphore again and thus more wait time for the other threads. This is where > my dirty patch intervened and just did a nanosleep() for a short amount of > time (I used 1000 nsecs). Can you try with a call to sched_yield(), rather than nanosleep()? It should have the same benefit but without as much performance hit. If it works, but is still too much hit, try tuning the checkinterval to see if you can find an acceptable throughput/responsiveness balance. -- Adam Olsen, aka Rhamphoryncus From s.r at visotech.at Wed Mar 19 17:09:11 2008 From: s.r at visotech.at (Stefan Ring) Date: Wed, 19 Mar 2008 16:09:11 +0000 (UTC) Subject: [Python-Dev] Improved thread switching References: <20080319115107.YPWO11090.viefep31-int.chello.at@zion.visotech.at> Message-ID: Adam Olsen gmail.com> writes: > Can you try with a call to sched_yield(), rather than nanosleep()? It > should have the same benefit but without as much performance hit. > > If it works, but is still too much hit, try tuning the checkinterval > to see if you can find an acceptable throughput/responsiveness > balance. > I tried that, and it had no effect whatsoever. I suppose it would make an effect on a single CPU or an otherwise heavily loaded SMP system but that's not the secnario we care about. From facundobatista at gmail.com Wed Mar 19 17:21:14 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Wed, 19 Mar 2008 11:21:14 -0500 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> References: <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> Message-ID: 2008/3/19, Barry Warsaw : > > +1 to assert* from me. the fail* variants always feel like > > double-negatives. I also always use assertTrue instead of assert_. But > > I don't care enough to argue about it. :) +1 to the plain affirmative propositions (assert*) instead of the kind-of-double-negative stuff. It helps a lot specially if you're not a native english speaker. +1 to remove them in Py3k. Questions: - 2to3 should "fix" them? - should we add a warning in 2.6 and remove them in 2.7? Or 2.7/2.8? Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From rhamph at gmail.com Wed Mar 19 17:24:21 2008 From: rhamph at gmail.com (Adam Olsen) Date: Wed, 19 Mar 2008 10:24:21 -0600 Subject: [Python-Dev] Improved thread switching In-Reply-To: References: <20080319115107.YPWO11090.viefep31-int.chello.at@zion.visotech.at> Message-ID: On Wed, Mar 19, 2008 at 10:09 AM, Stefan Ring wrote: > Adam Olsen gmail.com> writes: > > > Can you try with a call to sched_yield(), rather than nanosleep()? It > > should have the same benefit but without as much performance hit. > > > > If it works, but is still too much hit, try tuning the checkinterval > > to see if you can find an acceptable throughput/responsiveness > > balance. > > > > I tried that, and it had no effect whatsoever. I suppose it would make an effect > on a single CPU or an otherwise heavily loaded SMP system but that's not the > secnario we care about. So you've got a lightly loaded SMP system? Multiple threads all blocked on the GIL, multiple CPUs to run them, but only one CPU is active? I that case I can imagine how sched_yield() might finish before the other CPUs wake up a thread. A FIFO scheduler would be the right thing here, but it's only a short term solution. Care for a long term solution? ;) http://code.google.com/p/python-safethread/ -- Adam Olsen, aka Rhamphoryncus From fuzzyman at voidspace.org.uk Wed Mar 19 17:37:37 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 19 Mar 2008 16:37:37 +0000 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: References: Message-ID: <47E14151.1040107@voidspace.org.uk> Gabriel Grant wrote: > Hi all, > > This gem from unittest.py is pretty much the opposite of "one obvious way": > > # Synonyms for assertion methods > > assertEqual = assertEquals = failUnlessEqual > > assertNotEqual = assertNotEquals = failIfEqual > > assertAlmostEqual = assertAlmostEquals = failUnlessAlmostEqual > > assertNotAlmostEqual = assertNotAlmostEquals = failIfAlmostEqual > > assertRaises = failUnlessRaises > > assert_ = assertTrue = failUnless > > assertFalse = failIf > > Could these be removed for 3k? > > There was a short discussion about this among some of those those > present in the Python Core sprint room at PyCon today and most > preferred the "assertEqual" form for [Not][Almost]Equal and Raises. > > With assertFalse vs. failIf (and assertTrue vs. failUnless) there was > far less agreement. JUnit uses assertTrue exclusively, and most people > said they feel that using assertTrue would be more consistent, but > many (myself included) still think failUnless and failIf are much more > natural. Another issue with assertTrue is that it doesn't actually > test for 'True', strictly speaking, since it is based on equality, not > identity. > +1 on standardising on 'assert*' and removing 'fail*'. +1 on making 'assertTrue' test for True rather than any non-false object (and vice versa for assertFalse) For migration a simple subclass of TestCase that provides the old methods/semantics is trivial to write. No need for monkey-patching. Michael Foord > Its also interesting to note the original commit message: > > >> r34209 | purcell | 2003-09-22 06:08:12 -0500 (Mon, 22 Sep 2003) >> [...] >> - New assertTrue and assertFalse aliases for comfort of JUnit users >> [...] >> > > assertEqual (and its cousins) were already present at that point. > > In any case, if the decision is made to not use failUnless, something > still needs to be done with assert_ vs. assertTrue. assert_ seems > somewhat better to me, in that it has fewer characters, but I think > that a case could certainly be made to keep both of these. > > I certainly don't have the authority to make a call on any of this, > but if someone else decides what colour to paint this bike shed, I can > try to get it done (hopefully with 2.6 warnings) tomorrow. > > Cheers, > > -Gabriel > > P.S. If you were in the sprint room and feel terribly misrepresented, > please feel free to give me a swift kick both on-list and in person > tomorrow morning. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > From s.r at visotech.at Wed Mar 19 17:59:00 2008 From: s.r at visotech.at (Stefan Ring) Date: Wed, 19 Mar 2008 16:59:00 +0000 (UTC) Subject: [Python-Dev] Improved thread switching References: <20080319115107.YPWO11090.viefep31-int.chello.at@zion.visotech.at> Message-ID: Adam Olsen gmail.com> writes: > > On Wed, Mar 19, 2008 at 10:09 AM, Stefan Ring visotech.at> wrote: > > Adam Olsen gmail.com> writes: > > > > > Can you try with a call to sched_yield(), rather than nanosleep()? It > > > should have the same benefit but without as much performance hit. > > > > > > If it works, but is still too much hit, try tuning the checkinterval > > > to see if you can find an acceptable throughput/responsiveness > > > balance. > > > > > > > I tried that, and it had no effect whatsoever. I suppose it would make an effect > > on a single CPU or an otherwise heavily loaded SMP system but that's not the > > secnario we care about. > > So you've got a lightly loaded SMP system? Multiple threads all > blocked on the GIL, multiple CPUs to run them, but only one CPU is > active? I that case I can imagine how sched_yield() might finish > before the other CPUs wake up a thread. > > A FIFO scheduler would be the right thing here, but it's only a short > term solution. Care for a long term solution? ;) > > http://code.google.com/p/python-safethread/ > I've already seen that but it would not help us in our current situation. The performance penalty really is too heavy. Our system is slow enough already ;). And it would be very difficult bordering on impossible to parallelize Plus, I can imagine that all extension modules (and our own code) would have to be adapted. The FIFO scheduler is perfect for us because the load is typically quite low. It's mostly at those times when someone runs a lengthy calculation that all other users suffer greatly increased response times. From rhamph at gmail.com Wed Mar 19 18:03:02 2008 From: rhamph at gmail.com (Adam Olsen) Date: Wed, 19 Mar 2008 11:03:02 -0600 Subject: [Python-Dev] Improved thread switching In-Reply-To: <799854773.351205944939606.OPEN-XCHANGE.WebMail.tomcat@morpheus.visotech.at> References: <20080319115107.YPWO11090.viefep31-int.chello.at@zion.visotech.at> <799854773.351205944939606.OPEN-XCHANGE.WebMail.tomcat@morpheus.visotech.at> Message-ID: On Wed, Mar 19, 2008 at 10:42 AM, Stefan Ring wrote: > > On Mar 19, 2008 05:24 PM, Adam Olsen wrote: > > > On Wed, Mar 19, 2008 at 10:09 AM, Stefan Ring wrote: > > > Adam Olsen gmail.com> writes: > > > > > > > Can you try with a call to sched_yield(), rather than nanosleep()? > > > > It > > > > should have the same benefit but without as much performance hit. > > > > > > > > If it works, but is still too much hit, try tuning the > > > > checkinterval > > > > to see if you can find an acceptable throughput/responsiveness > > > > balance. > > > > > > > > > > I tried that, and it had no effect whatsoever. I suppose it would > > > make an effect > > > on a single CPU or an otherwise heavily loaded SMP system but that's > > > not the > > > secnario we care about. > > > > So you've got a lightly loaded SMP system? Multiple threads all > > blocked on the GIL, multiple CPUs to run them, but only one CPU is > > active? I that case I can imagine how sched_yield() might finish > > before the other CPUs wake up a thread. > > > > A FIFO scheduler would be the right thing here, but it's only a short > > term solution. Care for a long term solution? ;) > > > > http://code.google.com/p/python-safethread/ > > I've already seen that but it would not help us in our current > situation. The performance penalty really is too heavy. Our system is > slow enough already ;). And it would be very difficult bordering on > impossible to parallelize Plus, I can imagine that all extension modules > (and our own code) would have to be adapted. > > The FIFO scheduler is perfect for us because the load is typically quite > low. It's mostly at those times when someone runs a lengthy calculation > that all other users suffer greatly increased response times. So you want responsiveness when idle but throughput when busy? Are those calculations primarily python code, or does a C library do the grunt work? If it's a C library you shouldn't be affected by safethread's increased overhead. -- Adam Olsen, aka Rhamphoryncus From murman at gmail.com Wed Mar 19 18:12:17 2008 From: murman at gmail.com (Michael Urman) Date: Wed, 19 Mar 2008 12:12:17 -0500 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <87hcf2afcb.fsf@uwakimon.sk.tsukuba.ac.jp> References: <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <87hcf2afcb.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Wed, Mar 19, 2008 at 10:44 AM, Stephen J. Turnbull wrote: > So we should add this to 2to3, no? They're going to run that anyway. If 2to3 can handle this, that removes the larger half of my objection. I was under the impression that this kind of semantic inferencing was beyond its capabilities. But even if so, maybe it's safe to assume that those names aren't used in other contexts. My remaining smaller half of the objection is that these aliases appear to have been added to reduce the friction when moving from another unit test system. Since the exact names are as much a matter of muscle memory as anything else being changed by py3k, that's not very important in this context. I still don't see the benefit paying for the cost. Are people genuinely confused by the plethora of names for the operations (instead of by their occasional "misuse")? But I'm not the one offering a patch here, so I'll pipe down now. :) -- Michael Urman From doko at cs.tu-berlin.de Wed Mar 19 18:05:42 2008 From: doko at cs.tu-berlin.de (Matthias Klose) Date: Wed, 19 Mar 2008 18:05:42 +0100 Subject: [Python-Dev] Capsule Summary of Some Packaging/Deployment Technology Concerns In-Reply-To: <47DEEC6B.5040602@taupro.com> References: <47DEEC6B.5040602@taupro.com> Message-ID: <18401.18406.521359.387045@gargle.gargle.HOWL> Jeff Rush writes: > I was in a Packaging BoF yesterday and, although not very relevant to the > packager bootstrap thread, Guido has asked me to post some of the concerns. We did address many topics on both days, I added the following topics which were addressed on the Friday BoF only, see http://wiki.python.org/moin/PackagingBOF - Linux distributions try to ship only one version of a package/egg/module in one release, only shipping more than one version if necessary. eggs (as least as shipped with Debian, Fedora, Ubuntu) are all built using --single-version-externally-managed. - import foo should work wether installed as an egg or installed with distutils, and without using pkg_resources.require - pkg_resources should handle the situation of one egg version installed as --single-version-externally-managed (default version) and one or more eggs installed not using --single-version-externally-managed. Currently these additional versions cannot be imported. - It would be useful if setuptools could handle separate build and install steps like most configure/make/make install systems do. Access to external resources should optionally be disabled during a build. - The idea was brought up to use a to-be-defined api-version to describe dependencies between eggs. Version numbers are generally used for more than api changes; the idea follows existing practice for shared object names, only changing when the API is changed. From guido at python.org Wed Mar 19 18:25:22 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 19 Mar 2008 10:25:22 -0700 Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module) In-Reply-To: <440CD260-F6F6-4484-8326-0EA1D7B83132@zooko.com> References: <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <47DEC0EB.3000404@palladion.com> <440CD260-F6F6-4484-8326-0EA1D7B83132@zooko.com> Message-ID: On Mon, Mar 17, 2008 at 2:19 PM, zooko wrote: > 4. The standard Python library includes a tool to find and read > resources (other than Python modules) that came bundled in a Python > package. > > Consider, for example, this snippets of code in Nevow: > > http://divmod.org/trac/browser/trunk/Nevow/setup.py?rev=13786#L10 > http://divmod.org/trac/browser/trunk/Nevow/setup.py?rev=13786 > http://divmod.org/trac/browser/trunk/Nevow/setup_egg.py?rev=2406 > > When Nevow uses pkg_resources to import its files such as > "default.css", then it is able to find at runtime, even if is being > imported from a py2exe or py2app zip, or on other platforms where its > homegrown setup script and homegrown "find my file" function fail. > So using pkg_resources (and setuptools to install it) makes > "test_nevow" pass on all of the allmydata.org buildslaves: > > http://allmydata.org/buildbot/waterfall?show_events=false I think we're pretty close to this already. PEP 302 defines a getdata() method. Hopefully most PEP 302 implementations support it. The only thing missing IMO is a little function that does what getdata() does when there is no __loader__ object (i.e. when the default "import-from-filesystem" import method is used). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From s.r at visotech.at Wed Mar 19 18:25:16 2008 From: s.r at visotech.at (Stefan Ring) Date: Wed, 19 Mar 2008 17:25:16 +0000 (UTC) Subject: [Python-Dev] Improved thread switching References: <20080319115107.YPWO11090.viefep31-int.chello.at@zion.visotech.at> <799854773.351205944939606.OPEN-XCHANGE.WebMail.tomcat@morpheus.visotech.at> Message-ID: Adam Olsen gmail.com> writes: > So you want responsiveness when idle but throughput when busy? Exactly ;) > Are those calculations primarily python code, or does a C library do > the grunt work? If it's a C library you shouldn't be affected by > safethread's increased overhead. > It's Python code all the way. Frankly, it's a huge mess, but it would be very very hard to come up with a scalable solution that would allow to optimize certain hotspots and redo them in C or C++. There isn't even anything left to optimize in particular because all those low hanging fruit have already been taken care of. So it's just ~30kloc Python code over which the total time spent is quite uniformly distributed :(. From doko at cs.tu-berlin.de Wed Mar 19 18:37:37 2008 From: doko at cs.tu-berlin.de (Matthias Klose) Date: Wed, 19 Mar 2008 18:37:37 +0100 Subject: [Python-Dev] Capsule Summary of Some Packaging/Deployment Technology Concerns In-Reply-To: <20080318004718.56E173A40FF@sparrow.telecommunity.com> References: <47DEEC6B.5040602@taupro.com> <20080318004718.56E173A40FF@sparrow.telecommunity.com> Message-ID: <18401.20321.441120.809380@gargle.gargle.HOWL> Phillip J. Eby writes: > >7. Many wanted to ability to install files anywhere in the install tree and > > not just under the Python package. Under distutils this was possible but > > it was removed in setuptools for security reasons. > > It wasn't security, it was manageability. Egg-based installation > means containment, (analagous to GNU stow) and therefore portability > and disposability of plugins. (Which again is what eggs were really > developed for in the first place.) defining containment this way doesn't help when preparing eggs for inclusion in a linux distribution. E.g. users on these distributions are used to find log files in /var/log (maybe in a subdir), documentation in /usr/share/doc/. You probably will get different views about manageability depending on your background (used to linux distribution standards or used to standards set by setuptools/cheeseshop). Packagers currently move these files manually to the standard locations and often have to keep symlinks in the egg dirs to these locations. Installation on linux distributions is handled by existing package tools which is unlikely to change. So it would be nice to find a common layer which can be used for both distribution methods, optionally enabling this with some kind of option like --install-files-in-places-not-handled-by-setuptools ;) Matthias From guido at python.org Wed Mar 19 18:48:02 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 19 Mar 2008 10:48:02 -0700 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <20080318223700.C64123A4074@sparrow.telecommunity.com> References: <20080317174542.89A463A409D@sparrow.telecommunity.com> <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> Message-ID: On Tue, Mar 18, 2008 at 3:36 PM, Phillip J. Eby wrote: > At 03:43 PM 3/18/2008 -0500, Guido van Rossum wrote: > >Only very few people would care about writing a setup > >script that works with this bootstrap module; basically only package > >manager implementers. > > That's true today, sure, but as soon as it is widely available, > others are sure to want to use it too. I just want a bright-line > distinction between what is and isn't bootstrappable, rather than a > murky region of "maybe, if you're not doing anything too complicated". How about "anything that uses only distutils in its setup.py and doesn't have external dependencies"? See a (horribly incomplete) prototype I added as sandbox/bootstrap/bootstrap.py. I wrote this on the plane last night and have only tested it with file:/// URLs; it needs to add the ability to consult PyPI to find the download URL, and probably more. (PS: just now I also managed to successfully install setuptools from source by giving it the URL to the gar.gz file.) > >There seems to be a misunderstanding about what I am proposing we do > >instead. The boostrap installer should only be powerful enough to > >allow it to be used to install a real package manager like setuptools. > > Which is why PEP 365 proposed only downloading an archive to a cache > directory, and optionally running something from it. It explicitly > disavows "installation" of anything, since the downloaded archive > wouldn't have been added to sys.path except for the duration of the > bootstrap process, and no scripts were to be installed. (Indeed, > apart from the methods it would have used to locate the archive on > PyPI, and to determine what to run from inside it, there was nothing > particularly egg-specific about the proposed bootstrapping process.) My bootstrap.py does exactly that: it downloads and unzips/untars a file and runs its setup.py with "install" as the only command line argument. (It currently looks for setup.py at the toplevel and one level deep in the unpacked archive.) Of course you will likely have to be root or administrator to run it effectively. > So, to fully egg-neutralize the bootstrapping approach, we need only > know how to locate an appropriate archive, and how to determine what > to run from it. Right. > For the latter, we could use the already-in-2.6 convention of running > __main__ from a zipfile or directory. (Too bad distutils source > distributions have an extra directory name embedded in them, so one > can't just execute them directly. Otherwise, we could've just let > people drop in a __main__.py next to setup.py. OTOH, maybe it would > be enough to use setuptools' algorithm for finding setup.py to locate > __main__.py, and I'm fairly sure *that* can be briefly expressed in the PEP.) What's wrong with just running "setup.py install"? I'd rather continue existing standards / conventions. Of course, it won't work when setup.py requires setuptools; but "old style" setup.py files that use only distutils work great (I managed to install Django from a file:/// URL). > The other open question is a naming convention and version detection, > so that the bootstrap tool can identify which of the files listed on > PyPI is suitable for its use. (Both with regard to the version > selection, and file type.) However, if PyPI were to grow support for > designating the appropriate files and/or versions in some other way, > we wouldn't need a naming convention as such. I don't understand PyPI all that well; it seems poor design that the browsing via keywords is emphasized but there is no easy way to *search* for a keyword (the list of all packages is not emphasized enough on the main page -- it occurs in the side bar but not in the main text). I assume there's a programmatic API (XML-RPC?) but I haven't found it yet. > Without one or the other, the bootstrap tool would have to grow a > version parsing scheme of some type, and play guessing games with > file extensions. (Which is one reason I limited PEP 365's scope to > downloading eggs actually *uploaded* to PyPI, rather than arbitrary > packages *linked* from PyPI.) There are two version parsers in distutils, referenced by PEP 345, the PyPI 1.2 metadata standard. > So, if I had to propose something right now, I would be inclined to propose: > > * using setuptools' version parsing semantics for interpretation of > alpha/beta/dev/etc. releases Can you point me to the code for this? What is its advantage over distutils.version? > * having a bdist_bootstrap format that's essentially a bdist_dumb > .zip file with the internal path prefixes stripped off, making it an > importable .zip with a different file extension. (Or maybe just > .pyboot.zip?) The filename convention would use setuptools' > canonicalization and escaping of names and version numbers, to allow > unambiguous machine parsing of the filename. A __main__ module would > have to be present for the archive to be run, as opposed to just > being downloaded to a temporary directory. Hm. Why not just use the existing convention for running setup.py after unpacking? This works great in my experience, and has the advantage of having an easy fallback if you end up having to do this manually for whatever reason. > * calling the bootstrap module 'bootstrap', as in 'python -m > bootstrap projectname optionalversion'. The module would expose an > API to allow it to be used programmatically as well as the command > line, so that bootstrapped packages can use the bootstrap process to > locate dependencies if they so desire. (Today's package management > tools, at least, are all based on setuptools, so if it's not present > they'll need to download that before beginning their own > bootstrapping process.) This sounds like going beyond bootstrapping. My vision is that you use the bootstrap module (with the command line you suggest above) once to install setuptools or the alternate package manager of your choice, and then you can use easy_install (or whatever alternative) to install the rest. > Apart from keeping the PEP self-contained and short, is there > anything in this that you think you would object to? (You may > reserve the right, of course, to later not like something in the > details of setuptools' version/filename rules, after I've put them > into the PEP, or really, anything else. I'm just asking if there's > anything that's obviously offensive at this point, before I spend > time on a new PEP.) I'd love it if you could write or point me to code that takes a package name and optional version and returns the URL for the source archive, and the type (in case it can't be guessed from the filename or the Content-type header). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rhamph at gmail.com Wed Mar 19 18:49:04 2008 From: rhamph at gmail.com (Adam Olsen) Date: Wed, 19 Mar 2008 11:49:04 -0600 Subject: [Python-Dev] Improved thread switching In-Reply-To: References: <20080319115107.YPWO11090.viefep31-int.chello.at@zion.visotech.at> <799854773.351205944939606.OPEN-XCHANGE.WebMail.tomcat@morpheus.visotech.at> Message-ID: On Wed, Mar 19, 2008 at 11:25 AM, Stefan Ring wrote: > Adam Olsen gmail.com> writes: > > > > So you want responsiveness when idle but throughput when busy? > > Exactly ;) > > > > Are those calculations primarily python code, or does a C library do > > the grunt work? If it's a C library you shouldn't be affected by > > safethread's increased overhead. > > > > It's Python code all the way. Frankly, it's a huge mess, but it would be very > very hard to come up with a scalable solution that would allow to optimize > certain hotspots and redo them in C or C++. There isn't even anything left to > optimize in particular because all those low hanging fruit have already been > taken care of. So it's just ~30kloc Python code over which the total time spent > is quite uniformly distributed :(. I see. Well, at this point I think the most you can do is file a bug so the problem doesn't get forgotten. If nothing else, if my safethread stuff goes in it'll very likely include a --with-gil option, so I may put together a FIFO scheduler. -- Adam Olsen, aka Rhamphoryncus From wolever at cs.toronto.edu Wed Mar 19 19:01:47 2008 From: wolever at cs.toronto.edu (David Wolever) Date: Wed, 19 Mar 2008 14:01:47 -0400 Subject: [Python-Dev] Order of Fixers Message-ID: <6BDF4C9F-AF89-43C2-B769-4C40398832FD@cs.toronto.edu> At the moment, fixers are run in alphabetical order -- but this poses a problem, because some depend on others (for example, fix_print will need to be run _before_ fix_future, because fix_print looks for the 'from __future__ import ...' statement. I'm tempted to simply change fix_future to fix_zz_future... But that has some obvious drawbacks. Alternately, if future is the only dependent module, it might be marginally cleaner to simply special-case it in refactor.get_all_fix_names. So, any better suggestions? From musiccomposition at gmail.com Wed Mar 19 19:18:59 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Wed, 19 Mar 2008 13:18:59 -0500 Subject: [Python-Dev] Order of Fixers In-Reply-To: <6BDF4C9F-AF89-43C2-B769-4C40398832FD@cs.toronto.edu> References: <6BDF4C9F-AF89-43C2-B769-4C40398832FD@cs.toronto.edu> Message-ID: <1afaf6160803191118hfb152ej3f32ae527aadef67@mail.gmail.com> On Wed, Mar 19, 2008 at 1:01 PM, David Wolever wrote: > At the moment, fixers are run in alphabetical order -- but this poses > a problem, because some depend on others (for example, fix_print will > need to be run _before_ fix_future, because fix_print looks for the > 'from __future__ import ...' statement. > > I'm tempted to simply change fix_future to fix_zz_future... But that > has some obvious drawbacks. > Alternately, if future is the only dependent module, it might be > marginally cleaner to simply special-case it in > refactor.get_all_fix_names. > > So, any better suggestions? I would create a list of fixers that need to go first in refactor.py and run those in order. If you wanted to get complex, you could add a requires member to fixes, but that is probably overkill. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080319/6427d102/attachment.htm From p.f.moore at gmail.com Wed Mar 19 19:34:02 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 19 Mar 2008 18:34:02 +0000 Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317142225.7C7D83A40B1@sparrow.telecommunity.com> <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <47DEC0EB.3000404@palladion.com> <440CD260-F6F6-4484-8326-0EA1D7B83132@zooko.com> Message-ID: <79990c6b0803191134had5def2uff0742cc5188b125@mail.gmail.com> On 19/03/2008, Guido van Rossum wrote: > On Mon, Mar 17, 2008 at 2:19 PM, zooko wrote: > > 4. The standard Python library includes a tool to find and read > > resources (other than Python modules) that came bundled in a Python > > package. > > I think we're pretty close to this already. PEP 302 defines a > getdata() method. Hopefully most PEP 302 implementations support it. > The only thing missing IMO is a little function that does what > getdata() does when there is no __loader__ object (i.e. when the > default "import-from-filesystem" import method is used). I'm currently working on an addition to pkgutil to provide this type of function. I'm considering going a little further (adding functions to get a file-like object, test for existence, and list available resources, modelled on the pkg_resources functions - but these extra ones are not supported by the loader protocol, so I'm undecided as to whether it's worth it, I'll see how complex the code gets). Once I have a patch, I'll post it to the tracker. What's the best approach? Code a patch for 3.0 and backport, or code for 2.6 and let the merging process do its stuff? Paul. From wolever at cs.toronto.edu Wed Mar 19 19:41:31 2008 From: wolever at cs.toronto.edu (David Wolever) Date: Wed, 19 Mar 2008 14:41:31 -0400 Subject: [Python-Dev] Order of Fixers In-Reply-To: <1afaf6160803191118hfb152ej3f32ae527aadef67@mail.gmail.com> References: <6BDF4C9F-AF89-43C2-B769-4C40398832FD@cs.toronto.edu> <1afaf6160803191118hfb152ej3f32ae527aadef67@mail.gmail.com> Message-ID: <676F5A5B-B2CD-40BC-8B5C-E12D1B25BC23@cs.toronto.edu> On 19-Mar-08, at 2:18 PM, Benjamin Peterson wrote: > So, any better suggestions? > I would create a list of fixers that need to go first in > refactor.py and run those in order. If you wanted to get complex, > you could add a requires member to fixes, but that is probably > overkill. Ok, so I was digging around a bit and there is the option to traverse the tree in preorder or postorder. While the actual order does not matter in this case, what does matter is that the preorder fixers are run first -- so I've just dropped the print fixer in there (and commented everything). This isn't a great general solution... But, at the moment, there is no need for it to be. If the need for a general solution arises, that can be added :) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080319/4dd7cc55/attachment-0001.htm From dpeterson at enthought.com Wed Mar 19 19:55:47 2008 From: dpeterson at enthought.com (Dave Peterson) Date: Wed, 19 Mar 2008 13:55:47 -0500 Subject: [Python-Dev] [Distutils] Capsule Summary of Some Packaging/Deployment Technology Concerns In-Reply-To: <20080319152101.730BC3A40A5@sparrow.telecommunity.com> References: <47DEEC6B.5040602@taupro.com> <20080318004718.56E173A40FF@sparrow.telecommunity.com> <47E0D571.2070004@taupro.com> <20080319152101.730BC3A40A5@sparrow.telecommunity.com> Message-ID: <47E161B3.9070208@enthought.com> Phillip J. Eby wrote: > At 03:57 AM 3/19/2008 -0500, Jeff Rush wrote: > >> I'd be willing to help out, and keep a carefully balanced hand in >> what is accepted. >> > > > I'm not sure exactly how to go about such a handoff though. My guess > is that we need a bug/patch tracker, and a few people to review, > test, and apply. Maybe a transitional period during which I just say > yea or nay and let others do the test and apply, before opening it up > entirely. That way, we can perhaps solidify a few principles that > I'd like to have stay in place. (Like no arbitrary post-install code hooks.) > +1 to blessing more people to commit. +1 to the transition period idea. These two ought to enable things to move a bit quicker than taking a year to accept a patch. :-) In addition to a bug tracker and patch manager, seems like perhaps a wiki to help document some of these solidified principles and other notes would be a good thing. (Like a patch should almost always include at least one test, possibly more.) Given that the source for setuptools is in the python.org svn, couldn't we just use the python.org roundup and wiki for these facilities? Though looking at the list of components, it seems that things in the sandbox generally aren't tracked in this infrastructure. In which case, I'm sure we could use sf, launchpad, or some such external provider. Enthought could even host this stuff. Like Jeff Rush, I'm also willing to help out as both a writer and reviewer of patches. As you can see from my earlier posts there are a number of things (besides running an arbitrary post-install script) that we'd like to be able to get into the codebase. -- Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080319/797705e9/attachment.htm From wolever at cs.toronto.edu Wed Mar 19 20:04:31 2008 From: wolever at cs.toronto.edu (David Wolever) Date: Wed, 19 Mar 2008 15:04:31 -0400 Subject: [Python-Dev] 2to3 and print function Message-ID: <2A4672AB-A1FC-4CB6-A9C4-20B5059513B4@cs.toronto.edu> At the moment, fix_print.py does the Right Thing when it finds ``from __future__ import print_function``... But the 2to3 parser gets upset when print() is passed kwargs: $ cat x.py from __future__ import print_function print("Hello, world!", end=' ') $ 2to3 x.py ... RefactoringTool: Can't parse x.py: ParseError: bad input: type=22, value='=', context=('', (2, 26)) What would be the best way to start fixing this? #2412 is the related bug. From guido at python.org Wed Mar 19 20:06:28 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 19 Mar 2008 12:06:28 -0700 Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module) In-Reply-To: <79990c6b0803191134had5def2uff0742cc5188b125@mail.gmail.com> References: <47DE8405.2030201@v.loewis.de> <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <47DEC0EB.3000404@palladion.com> <440CD260-F6F6-4484-8326-0EA1D7B83132@zooko.com> <79990c6b0803191134had5def2uff0742cc5188b125@mail.gmail.com> Message-ID: On Wed, Mar 19, 2008 at 11:34 AM, Paul Moore wrote: > On 19/03/2008, Guido van Rossum wrote: > > On Mon, Mar 17, 2008 at 2:19 PM, zooko wrote: > > > 4. The standard Python library includes a tool to find and read > > > resources (other than Python modules) that came bundled in a Python > > > package. > > I think we're pretty close to this already. PEP 302 defines a > > getdata() method. Hopefully most PEP 302 implementations support it. > > The only thing missing IMO is a little function that does what > > getdata() does when there is no __loader__ object (i.e. when the > > default "import-from-filesystem" import method is used). > > I'm currently working on an addition to pkgutil to provide this type > of function. I'm considering going a little further (adding functions > to get a file-like object, test for existence, and list available > resources, modelled on the pkg_resources functions - but these extra > ones are not supported by the loader protocol, so I'm undecided as to > whether it's worth it, I'll see how complex the code gets). I'd only do what __loader__ offers. People can always wrap a StringIO around it. > Once I have a patch, I'll post it to the tracker. What's the best > approach? Code a patch for 3.0 and backport, or code for 2.6 and let > the merging process do its stuff? Code for 2.6, let the merge do its thing. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ntoronto at cs.byu.edu Wed Mar 19 19:54:49 2008 From: ntoronto at cs.byu.edu (Neil Toronto) Date: Wed, 19 Mar 2008 12:54:49 -0600 Subject: [Python-Dev] Python 3000: Special type for object attributes & map keys In-Reply-To: <47E071B4.8030503@canterbury.ac.nz> References: <47E071B4.8030503@canterbury.ac.nz> Message-ID: <47E16179.7090305@cs.byu.edu> Greg Ewing wrote: > Neal Norwitz wrote: >> Part of this is done, but very differently in that all strings used in >> code objects are interned (stored in a dictionary > > And since two interned strings can be compared > by pointer identity, I don't see how this differs > significantly from the "unique integer" idea. > > If the integers were used to directly index an > array instead of being used as dict keys, it > might make a difference. The cost would be that > every namespace would need to be as big as > the number of names in existence, with most > of them being extremely sparse. And we already have a data structure for sparse arrays... it's called a "dict". :) If every attribute name were guaranteed to be an interned string (not currently the case - attribute names can be any kind of object), it might be possible to add another dict specialization for interned string lookup. The wins would be that lookup could assume the string's hash is valid, and equality comparison could be done via pointer comparison. HOWEVER... I think that the attribute cache patches that went into 2.6 and 3.0 mostly take care of lookup speed issues. They both assume strings are interned already. A cache hit consists of calculating a cache index from the string hash, ensuring that the attribute name at that index is identical (via pointer comparison) to the attribute name to be looked up, and returning the associated value. Lookup with an attribute that's not a string or not interned is automatically a cache miss, and it happens very rarely. Specializing dicts for interned strings would optimize the cache miss. (When I was making the 3.0 patch, I found that happened rarely on the benchmarks and regression tests. It was somewhere around 5%.) The cache miss isn't expensive just because of the dict lookup. The attribute has to be searched for in every type and super-type of the object. The occasional string equality test probably doesn't register. I'd be happy to be shown to be wrong, though. Neil From jimjjewett at gmail.com Wed Mar 19 20:15:20 2008 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 19 Mar 2008 15:15:20 -0400 Subject: [Python-Dev] [Python-checkins] logging shutdown (was: Re: r61431 - python/trunk/Doc/library/logging.rst) In-Reply-To: <001901c8899c$65d01ba0$0200a8c0@alpha> References: <001901c8899c$65d01ba0$0200a8c0@alpha> Message-ID: On 3/19/08, Vinay Sajip wrote: > > I think (repeatedly) testing an app through IDLE is a reasonable use case. > [other threads may still have references to loggers or handlers] > > Would it be reasonable for shutdown to remove logging from > > sys.modules, so that a rerun has some chance of succeeding via its own > > import? > I'm not sure that would be enough in the scenario I mentioned above - would > removing a module from sys.modules be a guarantee of removing it from > memory? No. It will explicitly not be removed from memory while anything holds a live reference. Removing it from sys.modules just means that the next time a module does "import logging", the logging initialization code will run again. It is true that this could cause contention if the old version is still holding an exclusive lock on some output file. > It's safer, in my view, for the developer of an application to do cleanup of > their app if they want to test repeatedly in IDLE. Depending on the issue just fixed, the app may not have a clean shutdown. -jJ From pje at telecommunity.com Wed Mar 19 20:54:37 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 19 Mar 2008 15:54:37 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317174542.89A463A409D@sparrow.telecommunity.com> <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> Message-ID: <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> At 10:48 AM 3/19/2008 -0700, Guido van Rossum wrote: >I don't understand PyPI all that well; it seems poor design that the >browsing via keywords is emphasized but there is no easy way to >*search* for a keyword (the list of all packages is not emphasized >enough on the main page -- it occurs in the side bar but not in the >main text). I assume there's a programmatic API (XML-RPC?) but I >haven't found it yet. http://wiki.python.org/moin/CheeseShopXmlRpc There's also a REST API that setuptools uses: http://peak.telecommunity.com/DevCenter/EasyInstall#package-index-api The API was originally designed for screen-scraping an older version of PyPI, but that has been replaced with a "lite" version served from: http://pypi.python.org/simple/ The "lite" version is intended for tools such as easy_install to process, as it consists strictly of links and can be statically cached. Zope Corp., for example, maintains a static mirror of this API, to guard themselves against PyPI outages and slowdowns, since their buildouts can involve huge numbers of eggs, both their own and external dependencies. >I'd love it if you could write or point me to code that takes a >package name and optional version and returns the URL for the source >archive, and the type (in case it can't be guessed from the filename >or the Content-type header). You can probably do that with the XML-RPC API. There's a function to get the versions of a package, given a (case-sensitive) name, and there's a function to get information for uploaded archives, given a name and a version. I originally intended to use it for the PEP 365 approach, but you can get the necessary information in just one static roundtrip using the REST (/simple) HTML API, if you're willing to parse the URLs for version information. (The catch of course being that distutils source distributions don't have unambiguously parseable filenames.) >Hm. Why not just use the existing convention for running setup.py >after unpacking? This works great in my experience, and has the >advantage of having an easy fallback if you end up having to do this >manually for whatever reason. Because I want bootstrap-ees to be able to use the bootstrap mechanism. For example, I expect at some point that setuptools will use other, non-self-contained packages, and other package managers such as zc.buildout et al also want to depend on setuptools without bundling it. > > * calling the bootstrap module 'bootstrap', as in 'python -m > > bootstrap projectname optionalversion'. The module would expose an > > API to allow it to be used programmatically as well as the command > > line, so that bootstrapped packages can use the bootstrap process to > > locate dependencies if they so desire. (Today's package management > > tools, at least, are all based on setuptools, so if it's not present > > they'll need to download that before beginning their own > > bootstrapping process.) > >This sounds like going beyond bootstrapping. My vision is that you use >the bootstrap module (with the command line you suggest above) once to >install setuptools or the alternate package manager of your choice, >and then you can use easy_install (or whatever alternative) to install >the rest. Well, I noticed that the other package managers were writing bootstrap scripts that then download setuptools' bootstrap script and run it as part of *their* bootstrap process... and then I got to thinking that it sure would be nice for setuptools to not have to be a giant monolithic download if I wanted to start using other packages in it... and that it sure would be nice to get rid of all these bootstrap scripts downloading other bootstrap scripts... and then I wrote PEP 365. :) One other thing that PEP 365 does for these use cases that your approach doesn't, is that pkg_resources could detect whether a desired package of a usable version was *already* installed, and skip it if so. So, we've already scaled back the intended use cases quite a bit, as people will have to write their own "is it already there?" and "is it the right version?" checks. > > Without one or the other, the bootstrap tool would have to grow a > > version parsing scheme of some type, and play guessing games with > > file extensions. (Which is one reason I limited PEP 365's scope to > > downloading eggs actually *uploaded* to PyPI, rather than arbitrary > > packages *linked* from PyPI.) > >There are two version parsers in distutils, referenced by PEP 345, the >PyPI 1.2 metadata standard. Yes, and StrictVersion doesn't parse release candidates. And neither LooseVersion nor StrictVersion supports handling multiple pre/post-release tags correctly. (E.g. "1.1a1dev-r2753") > > So, if I had to propose something right now, I would be inclined > to propose: > > > > * using setuptools' version parsing semantics for interpretation of > > alpha/beta/dev/etc. releases > >Can you point me to the code for this? What is its advantage over >distutils.version? It implements version comparison semantics that are closer to programmer expectations. It has also been far more widely used and exposed to more feedback. distutils.version, as far as I know, is really only used by the PEP 345 metadata standard -- which isn't used by *any* automated tools as far as I know, and I'm not sure how many packages bother declaring it. In addition to alpha/beta/candidate/dev versions, it also supports post-release (patchlevel) tags such as svn revision or date-based tags. Here is the code; the docstring is actually longer than the bits that do anything: def parse_version(s): """Convert a version string to a chronologically-sortable key This is a rough cross between distutils' StrictVersion and LooseVersion; if you give it versions that would work with StrictVersion, then it behaves the same; otherwise it acts like a slightly-smarter LooseVersion. It is *possible* to create pathological version coding schemes that will fool this parser, but they should be very rare in practice. The returned value will be a tuple of strings. Numeric portions of the version are padded to 8 digits so they will compare numerically, but without relying on how numbers compare relative to strings. Dots are dropped, but dashes are retained. Trailing zeros between alpha segments or dashes are suppressed, so that e.g. "2.4.0" is considered the same as "2.4". Alphanumeric parts are lower-cased. The algorithm assumes that strings like "-" and any alpha string that alphabetically follows "final" represents a "patch level". So, "2.4-1" is assumed to be a branch or patch of "2.4", and therefore "2.4.1" is considered newer than "2.4-1", which in turn is newer than "2.4". Strings like "a", "b", "c", "alpha", "beta", "candidate" and so on (that come before "final" alphabetically) are assumed to be pre-release versions, so that the version "2.4" is considered newer than "2.4a1". Finally, to handle miscellaneous cases, the strings "pre", "preview", and "rc" are treated as if they were "c", i.e. as though they were release candidates, and therefore are not as new as a version string that does not contain them, and "dev" is replaced with an '@' so that it sorts lower than than any other pre-release tag. """ parts = [] for part in _parse_version_parts(s.lower()): if part.startswith('*'): if part<'*final': # remove '-' before a prerelease tag while parts and parts[-1]=='*final-': parts.pop() # remove trailing zeros from each series of numeric parts while parts and parts[-1]=='00000000': parts.pop() parts.append(part) return tuple(parts) component_re = re.compile(r'(\d+ | [a-z]+ | \.| -)', re.VERBOSE) replace = {'pre':'c', 'preview':'c','-':'final-','rc':'c','dev':'@'}.get def _parse_version_parts(s): for part in component_re.split(s): part = replace(part,part) if not part or part=='.': continue if part[:1] in '0123456789': yield part.zfill(8) # pad for numeric comparison else: yield '*'+part yield '*final' # ensure that alpha/beta/candidate are before final To check a parse_version() value for stability, you can just loop over it looking for any part <"*foo" where "foo" is the desired minimum stability. That is, if you find a '*a' and you don't want alphas, then this version's no good. This lets you also distinguish between a beta that you might accept, from an in-development snapshot of a beta, that you wouldn't. >What's wrong with just running "setup.py install"? I'd rather continue >existing standards / conventions. Of course, it won't work when >setup.py requires setuptools; Actually, it will, if the setup script uses the current ez_setup bootstrapping method for setuptools. However, I'd like to get *rid* of that bootstrapping method, and replace it with this one. That's why I'd prefer that the bootstrap approach use a different entry point for launching, and why I want the module to expose an API, and why I don't really want the bootstrapper to actually "install" anything. For one thing, it means dealing with installation *options*. Your prototype doesn't pass through any command-line options to the script, so people would have to use a ~/.pydistutils.cfg file in order to control the installation options, for example. (Which then can break if the packager included a setup.cfg that was supposed to be overridden on the command line...) Probably this seems a lot more messy to me, because I've had my face directly planted in the mess for a number of years now, and I know that, for example, people bitched and moaned excessively about not being able to use --prefix with easy_install, the way they could with 'setup.py install'. And maybe my experiences aren't all relevant here; I'm just not very good at turning them off. My skepticism for the setup.py-based approach is at close to "new scheme for removing the GIL" level, because I've gone through a lot of pain to get easy_install from the stage where it looked a lot like your bootstrap prototype, to something that actually works, most of the time, for arbitrary distutils packages. :) And unfortunately, some of the hurdles will require a few release cycles to show up. And hey, if you're okay with that, cool. I just think that as soon as it gets out in the field, people will use it far outside anything we expect it to be used for, and if there's not a bright line for the *packager* to cross, I think we'll have people unhappy with the tool. If you have to do a special step to make something bootstrappable, then when the tool doesn't work, the user will ask the packager to take the special step. However, if the tool allows the user to *point* it at any package, and it randomly (from the user's POV) fails, then the tool (and Python) will be blamed for the failure. Because even though the bootstrap tool is "not a package manager", if it's close enough to look like "a simpler easy_install", people will try to use it as one, and blog about how bootstrap is broken and should support installation options, etc. (I suppose at this point easy_install is something of a counter-example to this worry; people can and do now give packagers patches to make their setup scripts more compatible with easy_install, in cases where the package does extensive distutils modification. OTOH, easy_install is a de facto standard, where bootstrap will be de jure. What does that mean in practice? Heck if I know. :) I guess people will hate on you instead of me, then, so maybe I should view that as an improvement. :) (It also makes it easier to understand your reluctance to be in any way associated with eggs, but there's a big difference between eggs and easy_install, and IMO your approach leans more towards the relative vices of easy_install than the relative virtues of eggs. But oh well.)) From dpeterson at enthought.com Wed Mar 19 21:13:27 2008 From: dpeterson at enthought.com (Dave Peterson) Date: Wed, 19 Mar 2008 15:13:27 -0500 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <47E11659.9000307@v.loewis.de> References: <47E0D883.9030402@taupro.com> <47E11659.9000307@v.loewis.de> Message-ID: <47E173E7.9060300@enthought.com> Martin v. L?wis wrote: >> 1. What is the plan for PyPI when Python 3.0 comes out and >> dependencies start getting satisfied from distribution >> across the great divide, e.g. a 3.0-specific package >> pulls from PyPI a 2.x-specific package to meet some >> need? Are there plans to fork PyPI, apply special >> tags to uploads or what? >> > > I don't see the need to for PyPI. For packages (or "distributions", > to avoid confusion with Python packages), I see two options: > > a) provide a single release that supports both 2.x and 3.x. > The precise strategy to do so might vary. If one is going > for a single source version, have setup.py run 2to3 > (or perhaps 3to2). For dual-source packages, have setup.py > just install the one for the right version; setup.py itself > needs to be written so it runs on both versions (which is > easy to do). > b) switch to Python 3 at some point (i.e. burn your bridges). > > You seem to be implying that some projects may release separate > source distributions. I cannot imagine why somebody would want > to do that. > While not quite to the same scale as the 2 to 3 transition, this problem seems like one that would generally already exist. If one writes code that uses newer 2.4/2.5 features (say decorators for example,) it won't build/run on 2.3 or earlier installs. How have people been handling that sort of situation? Is it only by not using the newer features or is there some situation I'm just not seeing in a brief review of a few projects pages on PyPI where there is only one source tarball? -- Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080319/40a13e12/attachment.htm From fumanchu at aminus.org Wed Mar 19 21:34:50 2008 From: fumanchu at aminus.org (Robert Brewer) Date: Wed, 19 Mar 2008 13:34:50 -0700 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <47E11659.9000307@v.loewis.de> References: <47E0D883.9030402@taupro.com> <47E11659.9000307@v.loewis.de> Message-ID: Martin v. L?wis wrote: > I don't see the need to for PyPI. For packages (or "distributions", > to avoid confusion with Python packages), I see two options: > > a) provide a single release that supports both 2.x and 3.x. > The precise strategy to do so might vary. If one is going > for a single source version, have setup.py run 2to3 > (or perhaps 3to2). For dual-source packages, have setup.py > just install the one for the right version; setup.py itself > needs to be written so it runs on both versions (which is > easy to do). > b) switch to Python 3 at some point (i.e. burn your bridges). > > You seem to be implying that some projects may release separate > source distributions. I cannot imagine why somebody would want > to do that. That's odd. I can't imagine why anybody would *not* want to do that. Given the number of issues 2to3 can't fix (because it would be too dangerous to guess), I certainly can't imagine a just-in-time porting solution that would work reliably. Making two releases means I can migrate once and only once and be done with it. Making a single release work on 2.x and 3.x means I have to keep all of the details of both Python 2 and 3 in my head all the time as I code? not to mention litter my codebase with "# the following ugly hack lets us work with Python 2 and 3" comments so someone else doesn't undo all my hard work when they run the tests on Python 3 but not 2? No thanks. My brain is too small. Robert Brewer fumanchu at aminus.org From thomas at python.org Wed Mar 19 21:46:19 2008 From: thomas at python.org (Thomas Wouters) Date: Wed, 19 Mar 2008 13:46:19 -0700 Subject: [Python-Dev] Order of Fixers In-Reply-To: <6BDF4C9F-AF89-43C2-B769-4C40398832FD@cs.toronto.edu> References: <6BDF4C9F-AF89-43C2-B769-4C40398832FD@cs.toronto.edu> Message-ID: <9e804ac0803191346jad5181bl97a6ac6183af3756@mail.gmail.com> On Wed, Mar 19, 2008 at 11:01 AM, David Wolever wrote: > At the moment, fixers are run in alphabetical order -- but this poses > a problem, because some depend on others (for example, fix_print will > need to be run _before_ fix_future, because fix_print looks for the > 'from __future__ import ...' statement. > > I'm tempted to simply change fix_future to fix_zz_future... But that > has some obvious drawbacks. > Alternately, if future is the only dependent module, it might be > marginally cleaner to simply special-case it in > refactor.get_all_fix_names. > > So, any better suggestions? > I would fix the from-future fixer to not remove futures that are specific to 3.0, and let the fixers specific to those features remove them. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080319/afab0e67/attachment.htm From glyph at divmod.com Wed Mar 19 22:15:56 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Wed, 19 Mar 2008 21:15:56 -0000 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: References: <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> Message-ID: <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> On 02:21 pm, murman at gmail.com wrote: >>OTOH, I'd rather there be OOWTDI so whatever the consensus is is fine >>with me. > >This strikes me as a gratuitous API change of the kind Guido was >warning about in his recent post: "Don't change your APIs incompatibly >when porting to Py3k" I agree emphatically. Actually I think this is the most extreme case. The unit test stuff should be as stable as humanly possible between 2 and 3, moreso than any other library. It's one thing to have a test that fails because the functionality is broken. It's much worse to have a test that fails because the meaning of the test has changed - and this is going to happen anyway with e.g. assertEquals(foo.keys(), ['some', 'list']) which is a common enough assertion. Mixin in changes to the test framework creates even more confusion and pain. Granted, this change is apparently trivial and easy to understand, but each new traceback requires more thinking and there is going to be quite enough thinking going on in 2->3. I say this from experience. Twisted's "trial" has been burned repeatedly both by introducing apparently trivial changes of our own to our testing tool and by apparently trivial changes to the Python stdlib unittest module, upon which we depend. Nothing we haven't been able to handle, but the 2->3 transition exacerbates this as it does all potential compatibility problems. Whatever the stdlib does in this regard we'll definitely continue to provide an insulating layer in trial and keep both types of assertions for compatibility. So maybe the answer is to use Twisted for your unit tests rather than worry about your own compatibility lib ;-). From guido at python.org Wed Mar 19 22:23:26 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 19 Mar 2008 14:23:26 -0700 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> References: <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> Message-ID: On Wed, Mar 19, 2008 at 12:54 PM, Phillip J. Eby wrote: [a long message] I'm back at Google and *really* busy for another week or so, so I'll have to postpone the rest of this discussion for a while. If other people want to chime in please do so; if this is just a dialog between Phillip and me I might incorrectly assume that nobody besides Phillip really cares. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From glyph at divmod.com Wed Mar 19 22:56:15 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Wed, 19 Mar 2008 21:56:15 -0000 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: References: <47E0D883.9030402@taupro.com> <47E11659.9000307@v.loewis.de> Message-ID: <20080319215615.21558.287992911.divmod.xquotient.8259@joule.divmod.com> On 08:34 pm, fumanchu at aminus.org wrote: >Martin v. L?wis wrote: >>You seem to be implying that some projects may release separate >>source distributions. I cannot imagine why somebody would want >>to do that. > >That's odd. I can't imagine why anybody would *not* want to do that. Python 2 is going to be around for a long time. No user is going to want to pay the migration cost all at once. Users of library packages will loudly demand this continued support. Long-term maintenance of a complete fork of your software in 2 very very subtly different languages, and backporting every single change effectively doubles the amount of work, in the best case. I certainly can't afford to do that with Twisted. Inserting a few hacks here and there (and annotating your code with some extra metadata, in the py3 case for 2to3) is something we _already_ have to do to maintain compatibility for multiple Python versions in one piece of software. That is why Guido has personally explained, in at least 2 keynote speeches, several blog posts, and several mailing list messages, that maintaining a single source release and translating it is *the* way that you manage the Python 3 transition for anything but a small project, or an application that's a true leaf in the dependency graph. (The "burn your bridges" solution is not available to anyone who has more than one other person using their code as code and not simply a UI.) I am happy to have found a reason to emphatically agree with you, Martin :-). From jml at mumak.net Wed Mar 19 23:02:07 2008 From: jml at mumak.net (Jonathan Lange) Date: Thu, 20 Mar 2008 09:02:07 +1100 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: References: Message-ID: On Wed, Mar 19, 2008 at 6:24 PM, Gabriel Grant wrote: > Hi all, > > This gem from unittest.py is pretty much the opposite of "one obvious way": > > # Synonyms for assertion methods > [snip] > > Could these be removed for 3k? > I agree with others who say that we shouldn't do this for Python 3k. If we want to get rid of them, we should deprecate them, wait a release or so, *then* remove them. That said, can we axe one of (assertEqual, assertEquals) too? jml From jeff at taupro.com Wed Mar 19 23:15:43 2008 From: jeff at taupro.com (Jeff Rush) Date: Wed, 19 Mar 2008 17:15:43 -0500 Subject: [Python-Dev] Capsule Summary of Some Packaging/Deployment Technology Concerns In-Reply-To: <20080319152101.730BC3A40A5@sparrow.telecommunity.com> References: <47DEEC6B.5040602@taupro.com> <20080318004718.56E173A40FF@sparrow.telecommunity.com> <47E0D571.2070004@taupro.com> <20080319152101.730BC3A40A5@sparrow.telecommunity.com> Message-ID: <47E1908F.5030501@taupro.com> Phillip J. Eby wrote: > At 03:57 AM 3/19/2008 -0500, Jeff Rush wrote: >> Are you open to giving certain others patch view/commit privileges to >> setuptools? > > Jim Fulton has such already. I'm open to extending that to others who > have a good grasp of the subtleties involved. > > Truthfully, if we can just get 0.6 put to bed, I could probably open up > the trunk a lot wider. What is needed to put 0.6 to bed? How can we help accelerate this? > Probably the most frustrating thing (or "chief amongst the most > frustrating things") about setuptools development is that it's a black > hole. By which I mean that backward compatibility and cruft accretion > make it difficult to get out of. > > In the beginning, there was the distutils. Distutils begat setuptools, > and setuptools begat virtualenv and zc.buildout and source control > plugins. Etc., etc. I've found in the past a revisiting of basic principles and objectives, communicated in enhanced documentation, can help to clear out such black holes. ;-) I'm pulling something together, from the recent emails and some archived threads -- it definitely is tangled though, I'll agree. > What I think is really needed in the long run is to keep eggs, but get > rid of setuptools and the distutils in their current form. There's a > lot of brokenness there, and also a lot of accumulated cruft. We really > need a distutils 3000, and it needs to be built on a better approach. That will require a lot of concensus building as well as collection of use cases so that the architecture team can encompass aspects they are not personally aware of. As you've said, it's hard to address itches that are not your own. It certainly is possible for someone to create a parallel packaging moduleset that uses the existing eggs format and PyPI but without the currently codebase, and then, once proven to work, lobby for it as distutils 3000. Frankly I'd like to see setuptools exploded, with those parts of general use folded back into the standard library, the creation of a set of non-implementation-specific documents of the distribution formats and behavior, leaving a small core of one implementation of how to do it and the door open for others to compete with their own implementation. > In truth, my *real* motivation for PEP 365's bootstrap tool isn't so > much to support the package management tools we have today, as it is to > support a new one tomorrow. I have a few ideas for ways to shift the > paradigm of how individual projects get built, to incorporate many > scenarios that don't work well now. But to implement those things in > such a next-generation tool, I will not want to be restricted to just > what's in the stdlib or what can be bundled in the tool. You should document those ideas someplace and start getting community input. There are a lot of diverse opinions on the right way to do this and the way ahead is quite unclear. > And I think it's probably getting close to time I stepped down from > day-to-day management of the codebase (which is more like month-to-month > or quarter-to-quarter for me lately). It will probably be a lot easier > for me to step back and critique stuff that goes in, after the fact, > than to go over the stuff beforehand. :) > > I'm not sure exactly how to go about such a handoff though. My guess is > that we need a bug/patch tracker, and a few people to review, test, and > apply. Maybe a transitional period during which I just say yea or nay > and let others do the test and apply, before opening it up entirely. > That way, we can perhaps solidify a few principles that I'd like to have > stay in place. (Like no arbitrary post-install code hooks.) I'll see about a tracker and identify some people to help out. > btw, offtopic question: are you by any chance the same Jeff Rush who > invented EchoMail? Yep, that's me. Not many remember the Fidonet days. I designed EchoMail on a napkin during a DFW Sysop pizza party during a conversation on what to do with the unused capability of inter-BBS private file transfers. It too escaped its ecosystem and spread like wildfire, almost getting banned from Fidonet. ;-) -Jeff From fuzzyman at voidspace.org.uk Wed Mar 19 23:21:55 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 19 Mar 2008 22:21:55 +0000 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: References: Message-ID: <47E19203.8000108@voidspace.org.uk> Jonathan Lange wrote: > On Wed, Mar 19, 2008 at 6:24 PM, Gabriel Grant wrote: > >> Hi all, >> >> This gem from unittest.py is pretty much the opposite of "one obvious way": >> >> # Synonyms for assertion methods >> >> > [snip] > >> Could these be removed for 3k? >> >> > > I agree with others who say that we shouldn't do this for Python 3k. > If we want to get rid of them, we should deprecate them, wait a > release or so, *then* remove them. > > That said, can we axe one of (assertEqual, assertEquals) too? > We should keep 'assertEquals'. Michael > jml > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > From collinw at gmail.com Wed Mar 19 23:44:10 2008 From: collinw at gmail.com (Collin Winter) Date: Wed, 19 Mar 2008 15:44:10 -0700 Subject: [Python-Dev] 2to3 and print function In-Reply-To: <2A4672AB-A1FC-4CB6-A9C4-20B5059513B4@cs.toronto.edu> References: <2A4672AB-A1FC-4CB6-A9C4-20B5059513B4@cs.toronto.edu> Message-ID: <43aa6ff70803191544n2ef94e60ye89312dd3243051e@mail.gmail.com> On Wed, Mar 19, 2008 at 12:04 PM, David Wolever wrote: > At the moment, fix_print.py does the Right Thing when it finds ``from > __future__ import print_function``... But the 2to3 parser gets upset > when print() is passed kwargs: > $ cat x.py > from __future__ import print_function > print("Hello, world!", end=' ') > $ 2to3 x.py > ... > RefactoringTool: Can't parse x.py: ParseError: bad input: type=22, > value='=', context=('', (2, 26)) > > What would be the best way to start fixing this? > > #2412 is the related bug. You can pass -p to refactor.py to fix this on a per-run basis. See r58002 (and the revisions it mentions) for a failed attempt to do this automatically. From collinw at gmail.com Wed Mar 19 23:51:17 2008 From: collinw at gmail.com (Collin Winter) Date: Wed, 19 Mar 2008 15:51:17 -0700 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: References: <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <87hcf2afcb.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <43aa6ff70803191551h328f9bb4v7e2acfd05cc421c4@mail.gmail.com> On Wed, Mar 19, 2008 at 10:12 AM, Michael Urman wrote: > On Wed, Mar 19, 2008 at 10:44 AM, Stephen J. Turnbull > wrote: > > So we should add this to 2to3, no? They're going to run that anyway. > > If 2to3 can handle this, that removes the larger half of my objection. > I was under the impression that this kind of semantic inferencing was > beyond its capabilities. But even if so, maybe it's safe to assume > that those names aren't used in other contexts. 2to3 can indeed handle this, but I'm not sure I would want it run automatically (rather have it be opt-in, the way several other fixers are). Solid test suites are critical to the transition process, and changing method names around may upset that. It's unlikely, sure, but it may add to general unease. The way I'd see such a fixer working is that people would run it over their 2.x codebase, commit that change, then transition the rest of their code at release-time, without having to worry about gratuitous code changes in their test suite. Collin Winter From grantgm at mcmaster.ca Wed Mar 19 23:54:24 2008 From: grantgm at mcmaster.ca (Gabriel Grant) Date: Wed, 19 Mar 2008 17:54:24 -0500 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: References: Message-ID: On Wed, Mar 19, 2008 at 5:02 PM, Jonathan Lange wrote: > On Wed, Mar 19, 2008 at 6:24 PM, Gabriel Grant wrote: > > Hi all, > > > > This gem from unittest.py is pretty much the opposite of "one obvious way": > > > > # Synonyms for assertion methods > > > [snip] > > > Could these be removed for 3k? > > I agree with others who say that we shouldn't do this for Python 3k. > If we want to get rid of them, we should deprecate them, wait a > release or so, *then* remove them. It seems to me if we want to remove this at some point, the time is now. I'm not aware of anything starting off deprecated in 3k - my impression is the whole point of 3k is to clean house. Deprecation warnings can be put into 2.6 if people think thats necessary, but the more important thing will be including it in 2to3. I'm working on that right now, so if/when the actual wording is finalized, it should just be a matter of changing around the order of the function names in a dict. -Gabriel From jeff at taupro.com Wed Mar 19 23:59:10 2008 From: jeff at taupro.com (Jeff Rush) Date: Wed, 19 Mar 2008 17:59:10 -0500 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <47E11659.9000307@v.loewis.de> References: <47E0D883.9030402@taupro.com> <47E11659.9000307@v.loewis.de> Message-ID: <47E19ABE.2040800@taupro.com> Martin v. L?wis wrote: > > I don't see the need to for PyPI. For packages (or "distributions", > to avoid confusion with Python packages), I see two options: > > a) provide a single release that supports both 2.x and 3.x. > b) switch to Python 3 at some point (i.e. burn your bridges). > > You seem to be implying that some projects may release separate > source distributions. I cannot imagine why somebody would want > to do that. Yes, I am assuming that existing projects would at some point introduce a 3.x version and maybe continue a 2.x version as separate distros, similar to Python itself. Then the large number of existing unqualified dependencies on, say SQLObject, would pull in the higher 3.x version and crash. It's the older projects that don't get updated often that are at risk of being destabilized by the arrival of 3.x specific code in PyPI. Are developers for Python 3.x encouraged in 3.x guidelines to release 'fat' distributions that combine 2.x and 3.x usable versions? There is also some hassle with 2.x programmers browsing PyPI for useful modules to incorporate in their programs, downloading them (w/easy_install so they don't see the project website) and getting streams of errors because they unknowningly hit a 3.x-specific version. Perhaps a convention of a keyword or more likely a new trove classifier that spells outs 3.x stuff, with indicators on package info pages and query filters on PyPI against that? >> 2. There have been attempts over the years to fix distutils, >> with the last one being in 2006 by Anthony Baxter. He >> stated that a major hurdle was the strong demand to >> respect backward compatibility and he finally gave up. > > Can you kindly refer to some archived discussion for that? http://mail.python.org/pipermail/python-dev/2006-April/063943.html "I started looking at this. The number of complaints I got when I started on this that it would break the existing distutils based installers totally discouraged me. In addition, the existing distutils codebase is ... not good. It is flatly not possible to "fix" distutils and preserve backwards compatibility." -Anthony Baxter >> One of the purposes of Python 3.0 was the freedom to >> break backward compatibility for the sake of "doing >> the right thing". So is it now permissible to give >> distutils a good reworking and stop letting >> compatibility issues hold us back? > > I don't know what the proposed changes are, but for some > changes; in general, I feel that the need for backwards > compatibility is exaggerated. A controversial point, I'm afraid. Perhaps it is time for a parallel rewrite, so that those setup.py who import distutils get the old behavior, and those who import distutils2 get the new, and we let attrition and the community decide which is standard. -Jeff From p.f.moore at gmail.com Thu Mar 20 00:05:35 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 19 Mar 2008 23:05:35 +0000 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: References: <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> Message-ID: <79990c6b0803191605u3a4d0e4brc9c348e1d8b0fd16@mail.gmail.com> On 19/03/2008, Michael Urman wrote: > > OTOH, I'd rather there be OOWTDI so whatever the consensus is is fine > > with me. > > > This strikes me as a gratuitous API change of the kind Guido was > warning about in his recent post: "Don't change your APIs incompatibly > when porting to Py3k" This seems compelling to me. And as Glyph mentioned, the testing APIs are the most critical ones to keep working when moving from 2 to 3. -1 on this change as part of 3.0. Either do it in 2.6 (which wouldn't satisfy backward compatibility requirements) or defer it to 3.1 or later. OK, starting 3.0 with deprecated methods is a nuisance, but the testing framework seems to me to be a valid special case. Or just don't bother at all. It's not *that* bad. Paul. From pje at telecommunity.com Thu Mar 20 00:10:00 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 19 Mar 2008 19:10:00 -0400 Subject: [Python-Dev] Capsule Summary of Some Packaging/Deployment Technology Concerns In-Reply-To: <47E1908F.5030501@taupro.com> References: <47DEEC6B.5040602@taupro.com> <20080318004718.56E173A40FF@sparrow.telecommunity.com> <47E0D571.2070004@taupro.com> <20080319152101.730BC3A40A5@sparrow.telecommunity.com> <47E1908F.5030501@taupro.com> Message-ID: <20080319231002.07DF43A40A5@sparrow.telecommunity.com> At 05:15 PM 3/19/2008 -0500, Jeff Rush wrote: >Phillip J. Eby wrote: > > At 03:57 AM 3/19/2008 -0500, Jeff Rush wrote: > >> Are you open to giving certain others patch view/commit privileges to > >> setuptools? > > > > Jim Fulton has such already. I'm open to extending that to others who > > have a good grasp of the subtleties involved. > > > > Truthfully, if we can just get 0.6 put to bed, I could probably open up > > the trunk a lot wider. > >What is needed to put 0.6 to bed? How can we help accelerate this? Get a tracker set up. I'm already in the main Python one, might as well use that. >It certainly is possible for someone to create a parallel packaging moduleset >that uses the existing eggs format and PyPI but without the currently >codebase, and then, once proven to work, lobby for it as distutils 3000. Yep. And I believe that something will look rather more like zc.buildout than setuptools, actually. Specifically in being data-driven rather than script-driven, and in the flexibility of what sort of parts get build and by what methods. Setuptools is still too rooted in distutils' world, the world where you can't depend on any other components being around to build things with. >Frankly I'd like to see setuptools exploded, with those parts of general use >folded back into the standard library, the creation of a set of >non-implementation-specific documents of the distribution formats and >behavior, leaving a small core of one implementation of how to do it and the >door open for others to compete with their own implementation. Apart from the exploding part, there are already documents. The only thing that makes them implementation-specific is that they haven't passed through any magic blessing process to make them standards. >You should document those ideas someplace and start getting community input. >There are a lot of diverse opinions on the right way to do this and the way >ahead is quite unclear. We might be talking about different things, as I'm more concerned with replacing setuptools and distutils on the build-and-distribute side. What's needed there is more the weeding out of too many ways to do simple things, and fixing the complete absence of ways to do complex things. :) For simple things the distutils are too hard, and for slightly-more-complex things, the entry barrier encourages people to abandon and replace them. On the package management side, I'm somewhat more inclined to agree with the need for a community approach, though. > > btw, offtopic question: are you by any chance the same Jeff Rush who > > invented EchoMail? > >Yep, that's me. Not many remember the Fidonet days. I designed >EchoMail on a >napkin during a DFW Sysop pizza party during a conversation on what >to do with >the unused capability of inter-BBS private file transfers. It too >escaped its >ecosystem and spread like wildfire, almost getting banned from Fidonet. ;-) Ah, so you *do* know what it's like to develop setuptools, then. I might even have met you at the one DFW sysop pizza party I ever attended. Back then, I ran the FreeZone, and before that, "Ferris Bueller's Fine Arts Forum", back in the late 80's and early 90's. My wife met me through the D/FW BBS list in the back of Computer Shopper, with a modem she bought at Software, Etc., up in Allen or wherever that place was. Not the chain store, the little consignment shop. Those were the days. But now we're *really* getting off-topic. :) From greg.ewing at canterbury.ac.nz Thu Mar 20 00:05:47 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 20 Mar 2008 11:05:47 +1200 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <47E19ABE.2040800@taupro.com> References: <47E0D883.9030402@taupro.com> <47E11659.9000307@v.loewis.de> <47E19ABE.2040800@taupro.com> Message-ID: <47E19C4B.4020505@canterbury.ac.nz> Jeff Rush wrote: > Perhaps it is time for a parallel rewrite, > so that those setup.py who import distutils get the old behavior, and those > who import distutils2 get the new, That sounds good to me. If anyone wants to have a go at this, I have some ideas on how to structure it that I'd be happy to discuss. -- Greg From tnelson at onresolve.com Wed Mar 19 23:57:44 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Wed, 19 Mar 2008 15:57:44 -0700 Subject: [Python-Dev] First green Windows x64 buildbots! Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2C@EXMBX04.exchhosting.com> We've just experienced our first 2.6 green x64 Windows builds on the build slaves! Well, almost green. Thomas's 'amd64 XP trunk' ran out of disk: 304 tests OK. 1 test failed: test_largefile ====================================================================== ERROR: test_seek (test.test_largefile.TestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_largefile.py", line 42, in test_seek f.flush() IOError: [Errno 28] No space left on device Sorry about that Thomas ;-) Trent. From lists at cheimes.de Thu Mar 20 00:40:36 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 20 Mar 2008 00:40:36 +0100 Subject: [Python-Dev] PyCon: please review miy pending patches Message-ID: Dear sprinters! I've a batch of pending patches I like to get into Python before the next alphas are send out. The math, epoll/kqueue and shell folder patches just need a review. The memory management patches are more complex. Please refer to the thread "int/float freelists vs pymalloc", too. epoll and kqueue patch: http://bugs.python.org/issue1657 Mark's and my math branch: trunk$ svnmerge.py merge -S svn+ssh://pythondev at svn.python.org/python/branches/trunk-math Windows shell folder patch for os.path: http://bugs.python.org/issue1763 Memory management of ints, floats and longs http://bugs.python.org/issue2039 http://bugs.python.org/issue2013 I've also two pending PEPs. I like to see at least PEP 370 in Python 2.6 and 3.0. It's not as intrusive as PEP 370 and IMHO very useful for deployment of Python applications. Post import hooks http://www.python.org/dev/peps/pep-0369/ Per user site-packages directory http://www.python.org/dev/peps/pep-0370/ Christian From nnorwitz at gmail.com Thu Mar 20 00:42:16 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 19 Mar 2008 18:42:16 -0500 Subject: [Python-Dev] First green Windows x64 buildbots! In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2C@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2C@EXMBX04.exchhosting.com> Message-ID: Great work Trent! You'll need to take a picture of Martin buying you the beer once you get the rest green. :-) n On Wed, Mar 19, 2008 at 5:57 PM, Trent Nelson wrote: > We've just experienced our first 2.6 green x64 Windows builds on the build slaves! Well, almost green. Thomas's 'amd64 XP trunk' ran out of disk: > > 304 tests OK. > 1 test failed: > test_largefile > ====================================================================== > ERROR: test_seek (test.test_largefile.TestCase) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_largefile.py", line 42, in test_seek > f.flush() > IOError: [Errno 28] No space left on device > > Sorry about that Thomas ;-) > > > Trent. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > From mithrandi-python-dev at mithrandi.za.net Thu Mar 20 00:30:11 2008 From: mithrandi-python-dev at mithrandi.za.net (Tristan Seligmann) Date: Thu, 20 Mar 2008 01:30:11 +0200 Subject: [Python-Dev] Python 3000: Special type for object attributes & map keys In-Reply-To: References: Message-ID: <20080319233011.GA18693@mithrandi.net> * Neal Norwitz [2008-03-18 18:54:47 -0500]: > First, you should measure the current speed difference. Something like: > > $ ./python.exe -m timeit -s 'd = {1: None}' 'd[1]' > 1000000 loops, best of 3: 0.793 usec per loop > $ ./python.exe -m timeit -s 'd = {"1": None}' 'd["1"]' > 1000000 loops, best of 3: 0.728 usec per loop > > My python is a debug version, so a release version might be faster for > ints. If not, the first task would be to speed up int lookups. :-) mithrandi at elvardein:~> python -V Python 2.4.5 mithrandi at elvardein:~> python -m timeit -s 'd = {1: None}' 'd[1]' 10000000 loops, best of 3: 0.142 usec per loop mithrandi at elvardein:~> python -m timeit -s 'd = {"1": None}' 'd["1"]' 10000000 loops, best of 3: 0.138 usec per loop mithrandi at elvardein:~> python2.5 -V Python 2.5.2 mithrandi at elvardein:~> python2.5 -m timeit -s 'd = {1: None}' 'd[1]' 10000000 loops, best of 3: 0.136 usec per loop mithrandi at elvardein:~> python2.5 -m timeit -s 'd = {"1": None}' 'd["1"]' 10000000 loops, best of 3: 0.126 usec per loop -- mithrandi, i Ainil en-Balandor, a faer Ambar -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: Digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20080320/e9a1e61f/attachment.pgp From janssen at parc.com Thu Mar 20 01:02:10 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 19 Mar 2008 17:02:10 PDT Subject: [Python-Dev] how to build extensions for Windows? Message-ID: <08Mar19.170220pdt."58696"@synergy1.parc.xerox.com> I've set up a Parallels virtual machine on my Mac, and have succeeded in getting Windows XP running in it! And I've installed MinGW, as well. Now I'd like to learn how to build the SSL module from source on Windows for Python 2.5.2. Is there any documentation on the process of building an extension from scratch that's appropriate for someone who doesn't know much about Windows? I'm looking for step-by-step. What about this? http://www.mingw.org/MinGWiki/index.php/Python%20extensions Bill From jyasskin at gmail.com Thu Mar 20 01:16:01 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Wed, 19 Mar 2008 17:16:01 -0700 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> References: <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <20080319211556.21558.2085930557.divmod.xquotient.8213@joule.divmod.com> Message-ID: <5d44f72f0803191716k330e1249kce353d173fe503@mail.gmail.com> On Wed, Mar 19, 2008 at 2:15 PM, wrote: > > On 02:21 pm, murman at gmail.com wrote: > >>OTOH, I'd rather there be OOWTDI so whatever the consensus is is fine > >>with me. > > > >This strikes me as a gratuitous API change of the kind Guido was > >warning about in his recent post: "Don't change your APIs incompatibly > >when porting to Py3k" > > I agree emphatically. Actually I think this is the most extreme case. > The unit test stuff should be as stable as humanly possible between 2 > and 3, moreso than any other library. This is convincing for me. Move my +1 back to 3.1. -- Namast?, Jeffrey Yasskin From tnelson at onresolve.com Thu Mar 20 01:23:14 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Wed, 19 Mar 2008 17:23:14 -0700 Subject: [Python-Dev] how to build extensions for Windows? In-Reply-To: <08Mar19.170220pdt."58696"@synergy1.parc.xerox.com> References: <08Mar19.170220pdt."58696"@synergy1.parc.xerox.com> Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2D@EXMBX04.exchhosting.com> Having recently sunk a lot of time into the Windows build process, I'd recommend going with Visual C++ Express 2008 rather than MinGW, as this is the official compiler for 2.6/3.0. (You can download a free copy.) FWIW, I've probably been working on the Windows build side of things on and off for the past month or so, and we've only just reached a point where 32bit and 64bit Windows builds are compiling with all extension modules (bsddb, tcl/tk, ssl etc) and passing all tests (most work has gone into the x64 builds though, the 32-bit ones were already green on XP and below for 32bit). Using MinGW/gcc on Windows hasn't seen anywhere near so much attention, so, YMWV. In terms of my Windows-oriented priorities, they are as follows: - Get 3.0 32/64 Windows builds actually compiling successfully and then passing all tests (given that all build slaves for 3.0 are red that's not exactly a quick action). - Move on to the MSI installer improvements for 2.6/3.0, specifically with regards to the VCRT9 runtime and signing of the installer/binaries. - Maybe putting some cycles into Python builds on MinGW. To be honest though, the main motivation for doing that will be to demonstrate that a Python executable compiled with Visual Studio 2008 Professional with Profile Guided Optimisation will outperform a MinGW/gcc build ;-) Trent. ________________________________________ From: python-dev-bounces+tnelson=onresolve.com at python.org [python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of Bill Janssen [janssen at parc.com] Sent: 19 March 2008 20:02 To: python-dev at python.org Subject: [Python-Dev] how to build extensions for Windows? I've set up a Parallels virtual machine on my Mac, and have succeeded in getting Windows XP running in it! And I've installed MinGW, as well. Now I'd like to learn how to build the SSL module from source on Windows for Python 2.5.2. Is there any documentation on the process of building an extension from scratch that's appropriate for someone who doesn't know much about Windows? I'm looking for step-by-step. What about this? http://www.mingw.org/MinGWiki/index.php/Python%20extensions Bill _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/tnelson%40onresolve.com From tnelson at onresolve.com Thu Mar 20 01:26:31 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Wed, 19 Mar 2008 17:26:31 -0700 Subject: [Python-Dev] trunk buildbot status Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2E@EXMBX04.exchhosting.com> Quick update on the status of the trunk buildbots: Failing: [x86 gentoo trunk (Neal Norwitz)] This has been failing at the same point for the past couple of days now: test_sqlite command timed out: 1800 seconds without output, killing pid 15168 process killed by signal 9 program finished with exit code -1 None of the other buildbots seem to be encountering the same problem. Neal, got any idea what's going on with this one? [alpha True64 5.1 trunk (Neal Norwitz)] test_tarfile started failing recently (within the past few days) with CRC checks. See http://www.python.org/dev/buildbot/trunk/alpha%20Tru64%205.1%20trunk/builds/2712/step-test/0. Greg updated the test such that it prints out some more detail about the failure so we're waiting on that at the moment. [hppa Ubuntu trunk (Matthias Klose)] This has been consistently failing in test_socketserver for as long as I can remember: test_socketserver make: *** [buildbottest] Alarm clock program finished with exit code 2 I just updated that test such that it waits 20 seconds instead of 3 seconds at the end of the test if the server hasn't shutdown -- waiting for the test results of this still. [x86 XP trunk (Joseph Armbruster)] This box didn't survive the recent build changes, but I can't figure out why, none of the other Windows boxes encounter this error: The following error has occurred during XML parsing: File: C:\python\buildarea\trunk.armbruster-windows\build\PCbuild\_bsddb.vcproj Line: 179 Column: 1 Error Message: Illegal qualified name character. The file 'C:\python\buildarea\trunk.armbruster-windows\build\PCbuild\_bsddb.vcproj' has failed to load. Can someone check a clean trunk build on a Windows system that *only* has Visual C++ Express 2008? The latest build system updates don't rely on any features of Visual Studio Professional, but the tools use a lot of common files, and perhaps a Service Pack needs to be applied or something. [amd64 XP trunk (Thomas Heller)] Builds fine, all tests pass except for test_largefile, which is failing as there's no more space left on the drive ;-) [x86 XP-4 trunk (David Bolen)] This is currently flagged as having failed test, but I don't think it's finished building since the finalised build updates, so hopefully the BSDDB errors in the last run will be resolved when it finished the latest build. [x86 FreeBSD 2 trunk (Jeroen Ruigrok van der Werven)] This is a FreeBSD 6.3-STABLE box (which switched to curses 5.6 from 5.2) -- there's been an ongoing thread with regards to why curses has started failing, Jeroen can probably provide more info on that. Either way I don't anticipate a quick fix for this particular slave, unfortuantely. Neal/Martin, I'd like to promote the following slaves to the stable list: [g4 osx.4] [x86 W2k8] [AMD64 W2k8] [ppc Debian unstable] [sparc Ubuntu] [sparc Debian] [PPC64 Debian] [S-390 Debian] [x86 XP-3] [amd64 XP] [x86 FreeBSD] [x86 FreeBSD 3] The trunk builds of these slaves have been the most reliable since I've been tracking. Trent. From janssen at parc.com Thu Mar 20 01:33:44 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 19 Mar 2008 17:33:44 PDT Subject: [Python-Dev] how to build extensions for Windows? In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2D@EXMBX04.exchhosting.com> References: <08Mar19.170220pdt."58696"@synergy1.parc.xerox.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2D@EXMBX04.exchhosting.com> Message-ID: <08Mar19.173345pdt."58696"@synergy1.parc.xerox.com> > Having recently sunk a lot of time into the Windows build process, I'd recommend going with Visual C++ Express 2008 rather than MinGW, as this is the official compiler for 2.6/3.0. (You can download a free copy.) Thanks for the advice, but it's sort of Greek to me. Is there a step-by-step guide somewhere? Bill From martin at v.loewis.de Thu Mar 20 01:48:21 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 19 Mar 2008 19:48:21 -0500 Subject: [Python-Dev] Capsule Summary of Some Packaging/Deployment Technology Concerns In-Reply-To: References: <47DEEC6B.5040602@taupro.com> <20080318004718.56E173A40FF@sparrow.telecommunity.com> <47E0D571.2070004@taupro.com> Message-ID: <47E1B455.9030900@v.loewis.de> > The Python sandbox has a setuptools directory. Is this the canonical > location for the code? Yes, it is. > If so, then anybody who has Python commit > privileges can commit to it and help further develop setuptools. They can, but they shouldn't. Nothing should be committed there without pje's approval (in whatever form he choses to give such approval). > If not, why not and what is the sandbox setuptools used for? I think it shouldn't be in sandbox, but toplevel, but that's a minor detail. Maybe I misunderstand the English word "sandbox". Regards, Martin From eric+python-dev at trueblade.com Thu Mar 20 01:49:41 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Wed, 19 Mar 2008 20:49:41 -0400 Subject: [Python-Dev] trunk buildbot status In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2E@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2E@EXMBX04.exchhosting.com> Message-ID: <47E1B4A5.8050409@trueblade.com> Trent Nelson wrote: > Quick update on the status of the trunk buildbots: > > [x86 XP trunk (Joseph Armbruster)] > This box didn't survive the recent build changes, but I can't figure out why, none of the other Windows boxes encounter this error: > The following error has occurred during XML parsing: > File: C:\python\buildarea\trunk.armbruster-windows\build\PCbuild\_bsddb.vcproj > Line: 179 > Column: 1 > Error Message: > Illegal qualified name character. > The file 'C:\python\buildarea\trunk.armbruster-windows\build\PCbuild\_bsddb.vcproj' has failed to load. > > Can someone check a clean trunk build on a Windows system that *only* has Visual C++ Express 2008? The latest build system updates don't rely on any features of Visual Studio Professional, but the tools use a lot of common files, and perhaps a Service Pack needs to be applied or something. I just built the trunk on a Windows XP x86 box that only has Visual C++ Express 2008 installed. I got a bunch of errors with sqlite, tcl, db-4.4.20, and ssl, but the interpreter built and appears to run ok. But since I don't have bsddb installed, I don't think I'm executing the portion of the build process that you find failing. I don't have time to install bsddb tonight, but I can do that in about 24 hours if you still need me to. Eric. From adlaiff6 at gmail.com Thu Mar 20 01:49:37 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Wed, 19 Mar 2008 20:49:37 -0400 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: <79990c6b0803191605u3a4d0e4brc9c348e1d8b0fd16@mail.gmail.com> References: <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <79990c6b0803191605u3a4d0e4brc9c348e1d8b0fd16@mail.gmail.com> Message-ID: On Wed, Mar 19, 2008 at 7:05 PM, Paul Moore wrote: > > This strikes me as a gratuitous API change of the kind Guido was > > warning about in his recent post: "Don't change your APIs incompatibly > > when porting to Py3k" > > This seems compelling to me. And as Glyph mentioned, the testing APIs > are the most critical ones to keep working when moving from 2 to 3. It seems as though this declaration, in this case, is in direct conflict with the overarching theme of "one obvious way to do things". That statement, of course, is the reason so many things are being changed in the move to 3k already, and I don't see why this shouldn't be one of them (particularly, because it's so easy to account for this in 2to3). As one is a global statement, and the other is fairly local, I vote for the change. -- Cheers, Leif From steve at holdenweb.com Thu Mar 20 02:02:00 2008 From: steve at holdenweb.com (Steve Holden) Date: Wed, 19 Mar 2008 21:02:00 -0400 Subject: [Python-Dev] First green Windows x64 buildbots! In-Reply-To: References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2C@EXMBX04.exchhosting.com> Message-ID: <47E1B788.4090704@holdenweb.com> Then put the picture on your screen and SEND ME A SCREENSHOT! regards Steve Neal Norwitz wrote: > Great work Trent! You'll need to take a picture of Martin buying you > the beer once you get the rest green. :-) > > n > > On Wed, Mar 19, 2008 at 5:57 PM, Trent Nelson wrote: >> We've just experienced our first 2.6 green x64 Windows builds on the build slaves! Well, almost green. Thomas's 'amd64 XP trunk' ran out of disk: >> >> 304 tests OK. >> 1 test failed: >> test_largefile >> ====================================================================== >> ERROR: test_seek (test.test_largefile.TestCase) >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> File "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_largefile.py", line 42, in test_seek >> f.flush() >> IOError: [Errno 28] No space left on device >> >> Sorry about that Thomas ;-) >> >> >> Trent. >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com >> > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org > From steve at holdenweb.com Thu Mar 20 02:02:00 2008 From: steve at holdenweb.com (Steve Holden) Date: Wed, 19 Mar 2008 21:02:00 -0400 Subject: [Python-Dev] First green Windows x64 buildbots! In-Reply-To: References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2C@EXMBX04.exchhosting.com> Message-ID: <47E1B788.4090704@holdenweb.com> Then put the picture on your screen and SEND ME A SCREENSHOT! regards Steve Neal Norwitz wrote: > Great work Trent! You'll need to take a picture of Martin buying you > the beer once you get the rest green. :-) > > n > > On Wed, Mar 19, 2008 at 5:57 PM, Trent Nelson wrote: >> We've just experienced our first 2.6 green x64 Windows builds on the build slaves! Well, almost green. Thomas's 'amd64 XP trunk' ran out of disk: >> >> 304 tests OK. >> 1 test failed: >> test_largefile >> ====================================================================== >> ERROR: test_seek (test.test_largefile.TestCase) >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> File "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_largefile.py", line 42, in test_seek >> f.flush() >> IOError: [Errno 28] No space left on device >> >> Sorry about that Thomas ;-) >> >> >> Trent. >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com >> > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org > From tnelson at onresolve.com Thu Mar 20 02:01:16 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Wed, 19 Mar 2008 18:01:16 -0700 Subject: [Python-Dev] trunk buildbot status In-Reply-To: <47E1B4A5.8050409@trueblade.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2E@EXMBX04.exchhosting.com>, <47E1B4A5.8050409@trueblade.com> Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2F@EXMBX04.exchhosting.com> I'd recommend cd'ing to your trunk root directory and running Tool\buildbot\build.bat from there -- it'll automatically check out all the dependencies and build via command line with vcbuild (building via Visual Studio usually always Does The Right Thing, command line builds often take a bit more coercing). ________________________________________ From: Eric Smith [eric+python-dev at trueblade.com] Sent: 19 March 2008 20:49 To: Trent Nelson Cc: python-dev at python.org; doko at ubuntu.com; joe at joevial.com; theller at ctypes.org; db3l.net at gmail.com; nnortwitz at gmail.org Subject: Re: [Python-Dev] trunk buildbot status Trent Nelson wrote: > Quick update on the status of the trunk buildbots: > > [x86 XP trunk (Joseph Armbruster)] > This box didn't survive the recent build changes, but I can't figure out why, none of the other Windows boxes encounter this error: > The following error has occurred during XML parsing: > File: C:\python\buildarea\trunk.armbruster-windows\build\PCbuild\_bsddb.vcproj > Line: 179 > Column: 1 > Error Message: > Illegal qualified name character. > The file 'C:\python\buildarea\trunk.armbruster-windows\build\PCbuild\_bsddb.vcproj' has failed to load. > > Can someone check a clean trunk build on a Windows system that *only* has Visual C++ Express 2008? The latest build system updates don't rely on any features of Visual Studio Professional, but the tools use a lot of common files, and perhaps a Service Pack needs to be applied or something. I just built the trunk on a Windows XP x86 box that only has Visual C++ Express 2008 installed. I got a bunch of errors with sqlite, tcl, db-4.4.20, and ssl, but the interpreter built and appears to run ok. But since I don't have bsddb installed, I don't think I'm executing the portion of the build process that you find failing. I don't have time to install bsddb tonight, but I can do that in about 24 hours if you still need me to. Eric. From martin at v.loewis.de Thu Mar 20 02:05:28 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 19 Mar 2008 20:05:28 -0500 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317174542.89A463A409D@sparrow.telecommunity.com> <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> Message-ID: <47E1B858.1090107@v.loewis.de> > I don't understand PyPI all that well; it seems poor design that the > browsing via keywords is emphasized but there is no easy way to > *search* for a keyword (the list of all packages is not emphasized > enough on the main page -- it occurs in the side bar but not in the > main text). I don't understand. What is "browsing via keywords" and how is that emphasized? (one I know that, I can look into ways for searching for keywords) > I assume there's a programmatic API (XML-RPC?) but I > haven't found it yet. The recommended "programmatic" API is http://pypi.python.org/simple/ Not sure what you were trying to achieve programmatically; "typically" people know what they want to install (e.g. "threadedcomments"), and then the tool goes directly to http://pypi.python.org/simple/threadedcomments/ Regards, Martin From martin at v.loewis.de Thu Mar 20 02:14:58 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 19 Mar 2008 20:14:58 -0500 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <47E173E7.9060300@enthought.com> References: <47E0D883.9030402@taupro.com> <47E11659.9000307@v.loewis.de> <47E173E7.9060300@enthought.com> Message-ID: <47E1BA92.8050709@v.loewis.de> > While not quite to the same scale as the 2 to 3 transition, this problem > seems like one that would generally already exist. If one writes code > that uses newer 2.4/2.5 features (say decorators for example,) it won't > build/run on 2.3 or earlier installs. How have people been handling > that sort of situation? Is it only by not using the newer features or > is there some situation I'm just not seeing in a brief review of a few > projects pages on PyPI where there is only one source tarball? I think packages have taken all sorts of responses to this issue. Some will list the minimum required Python version in their README, some might put a test in setup.py that aborts installation if the Python version is too old, some may just install and let the user find out at runtime. Typically, packages try to support all the Python versions that their users still use. If a user of an older Python version comes along, they'll just need to fetch the older release (which hopefully is still online, or can be extracted from the source repository). Regards, Martin From martin at v.loewis.de Thu Mar 20 02:17:17 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 19 Mar 2008 20:17:17 -0500 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: References: <47E0D883.9030402@taupro.com> <47E11659.9000307@v.loewis.de> Message-ID: <47E1BB1D.4060303@v.loewis.de> >> You seem to be implying that some projects may release separate >> source distributions. I cannot imagine why somebody would want to >> do that. > > That's odd. I can't imagine why anybody would *not* want to do that. > Given the number of issues 2to3 can't fix (because it would be too > dangerous to guess) Like which one specifically? >, I certainly can't imagine a just-in-time porting > solution that would work reliably. I can imagine that absolutely. > Making two releases means I can > migrate once and only once and be done with it. No, you won't be done. You have to maintain two releases in parallel now. > Making a single > release work on 2.x and 3.x means I have to keep all of the details > of both Python 2 and 3 in my head all the time as I code? not to > mention litter my codebase with "# the following ugly hack lets us > work with Python 2 and 3" comments so someone else doesn't undo all > my hard work when they run the tests on Python 3 but not 2? No > thanks. My brain is too small. So you rather put more work into maintenance sequentially. Fair enough. Regards, Martin From martin at v.loewis.de Thu Mar 20 02:25:40 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 19 Mar 2008 20:25:40 -0500 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <47E19ABE.2040800@taupro.com> References: <47E0D883.9030402@taupro.com> <47E11659.9000307@v.loewis.de> <47E19ABE.2040800@taupro.com> Message-ID: <47E1BD14.2030805@v.loewis.de> > Yes, I am assuming that existing projects would at some point introduce a 3.x > version and maybe continue a 2.x version as separate distros, similar to > Python itself. Then the large number of existing unqualified dependencies on, > say SQLObject, would pull in the higher 3.x version and crash. It's the older > projects that don't get updated often that are at risk of being destabilized > by the arrival of 3.x specific code in PyPI. Are developers for Python 3.x > encouraged in 3.x guidelines to release 'fat' distributions that combine 2.x > and 3.x usable versions? Passive voice is misleading here: encouraged by whom? *I* encourage people to consider that option, rather than assuming it couldn't possibly work. Whether it actually works, I don't know. I hope it would work, and I hope it would not be fat at all. > Perhaps a convention of a keyword or more likely a new trove classifier that > spells outs 3.x stuff, with indicators on package info pages and query filters > on PyPI against that? I'm fine with adding more trove classifiers if that solves the problem (although I still assume the problem doesn't actually exist). As always, a classifier should not be added until there actually are two packages that want to use it. >> Can you kindly refer to some archived discussion for that? > > http://mail.python.org/pipermail/python-dev/2006-April/063943.html > > "I started looking at this. The number of complaints I > got when I started on this that it would break the > existing distutils based installers totally discouraged > me. In addition, the existing distutils codebase is ... > not good. > > It is flatly not possible to "fix" distutils and > preserve backwards compatibility." -Anthony Baxter Thanks. I still have the same position as I had then - if distutils is broken, it should be fixed, not ignored. > A controversial point, I'm afraid. Perhaps it is time for a parallel rewrite, > so that those setup.py who import distutils get the old behavior, and those > who import distutils2 get the new, and we let attrition and the community > decide which is standard. Is there a list of the problems with distutils somewhere? It always worked fine for me, so I see no reason to fix it in the first place. Regards, Martin From steve at holdenweb.com Thu Mar 20 02:29:27 2008 From: steve at holdenweb.com (Steve Holden) Date: Wed, 19 Mar 2008 21:29:27 -0400 Subject: [Python-Dev] unittest's redundant assertions: asserts vs. failIf/Unlesses In-Reply-To: References: <5d44f72f0803190120n700de143m5d67b8164f7c57ae@mail.gmail.com> <644FD103-F31C-4A7E-8172-374B57BF3BC4@python.org> <79990c6b0803191605u3a4d0e4brc9c348e1d8b0fd16@mail.gmail.com> Message-ID: Leif Walsh wrote: > On Wed, Mar 19, 2008 at 7:05 PM, Paul Moore wrote: >> > This strikes me as a gratuitous API change of the kind Guido was >> > warning about in his recent post: "Don't change your APIs incompatibly >> > when porting to Py3k" >> >> This seems compelling to me. And as Glyph mentioned, the testing APIs >> are the most critical ones to keep working when moving from 2 to 3. > > It seems as though this declaration, in this case, is in direct > conflict with the overarching theme of "one obvious way to do things". > That statement, of course, is the reason so many things are being > changed in the move to 3k already, and I don't see why this shouldn't > be one of them (particularly, because it's so easy to account for this > in 2to3). As one is a global statement, and the other is fairly > local, I vote for the change. > As Guido(?) pointed out, this would be acceptable because it's simply different spellings of the same way. regards Steve From martin at v.loewis.de Thu Mar 20 02:41:42 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 19 Mar 2008 20:41:42 -0500 Subject: [Python-Dev] how to build extensions for Windows? In-Reply-To: <08Mar19.170220pdt."58696"@synergy1.parc.xerox.com> References: <08Mar19.170220pdt."58696"@synergy1.parc.xerox.com> Message-ID: <47E1C0D6.7050202@v.loewis.de> > I've set up a Parallels virtual machine on my Mac, and have succeeded > in getting Windows XP running in it! And I've installed MinGW, as > well. Now I'd like to learn how to build the SSL module from source > on Windows for Python 2.5.2. Is there any documentation on the > process of building an extension from scratch that's appropriate for > someone who doesn't know much about Windows? I'm looking for > step-by-step. I'll make a custom Bill-Janssen-Step-By-Step-List: try: 1. Write a setup.py file. See distutils documentation for details. 2. Install Python 2.5.2 3. Run c:\python25\python.exe setup.py build --compiler=mingw32 4. Run c:\python25\python.exe setup.py build --compiler=mingw32 bdist_wininst except Exception, e: A. Post e to python-dev else: 5. Upload dist/* to PyPI (you can use setup.py upload for that also) HTH, Martin From aleaxit at gmail.com Thu Mar 20 02:52:48 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Wed, 19 Mar 2008 18:52:48 -0700 Subject: [Python-Dev] Improved thread switching In-Reply-To: References: <20080319115107.YPWO11090.viefep31-int.chello.at@zion.visotech.at> <799854773.351205944939606.OPEN-XCHANGE.WebMail.tomcat@morpheus.visotech.at> Message-ID: Hmmm, sorry if I'm missing something obvious, but, if the occasional background computations are sufficiently heavy -- why not fork, do said computations in the child thread, and return the results via any of the various available IPC approaches? I've recently (at Pycon, mostly) been playing devil's advocate (i.e., being PRO-threads, for once) on the subject of utilizing multiple cores effectively -- but the classic approach (using multiple _processes_ instead) actually works quite well in many cases, and this application server would appear to be one. (the pyProcessing package appears to offer an easy way to migrate threaded code to multiple-processes approaches, although I've only played around with it, not [yet] used it for production code). Alex On Wed, Mar 19, 2008 at 10:49 AM, Adam Olsen wrote: > On Wed, Mar 19, 2008 at 11:25 AM, Stefan Ring wrote: > > Adam Olsen gmail.com> writes: > > > > > > > So you want responsiveness when idle but throughput when busy? > > > > Exactly ;) > > > > > > > Are those calculations primarily python code, or does a C library do > > > the grunt work? If it's a C library you shouldn't be affected by > > > safethread's increased overhead. > > > > > > > It's Python code all the way. Frankly, it's a huge mess, but it would be very > > very hard to come up with a scalable solution that would allow to optimize > > certain hotspots and redo them in C or C++. There isn't even anything left to > > optimize in particular because all those low hanging fruit have already been > > taken care of. So it's just ~30kloc Python code over which the total time spent > > is quite uniformly distributed :(. > > I see. Well, at this point I think the most you can do is file a bug > so the problem doesn't get forgotten. If nothing else, if my > safethread stuff goes in it'll very likely include a --with-gil > option, so I may put together a FIFO scheduler. > > > -- > Adam Olsen, aka Rhamphoryncus > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com > From janssen at parc.com Thu Mar 20 03:00:46 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 19 Mar 2008 19:00:46 PDT Subject: [Python-Dev] how to build extensions for Windows? In-Reply-To: <47E1C0D6.7050202@v.loewis.de> References: <08Mar19.170220pdt."58696"@synergy1.parc.xerox.com> <47E1C0D6.7050202@v.loewis.de> Message-ID: <08Mar19.190050pdt."58696"@synergy1.parc.xerox.com> Nice and simple, thanks, Martin! Works OK after I upgraded to MinGW 5.1.3 (5.0.0 is what I had, and the gcc build didn't work there with that). I think I've got it! Bill From nnorwitz at gmail.com Thu Mar 20 04:14:18 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 19 Mar 2008 22:14:18 -0500 Subject: [Python-Dev] trunk buildbot status In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2E@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2E@EXMBX04.exchhosting.com> Message-ID: Unfortunately, I don't have ssh access from my hotel room. This means I won't be able to help much until I get home. On Wed, Mar 19, 2008 at 7:26 PM, Trent Nelson wrote: > Quick update on the status of the trunk buildbots: > > Failing: > [x86 gentoo trunk (Neal Norwitz)] > This has been failing at the same point for the past couple of days now: > test_sqlite > command timed out: 1800 seconds without output, killing pid 15168 > process killed by signal 9 > program finished with exit code -1 > > None of the other buildbots seem to be encountering the same problem. Neal, got any idea what's going on with this one? Last status was here: http://mail.python.org/pipermail/python-checkins/2008-March/066824.html I haven't logged in to check what's going on. Gerhard had some ideas in the same thread: http://mail.python.org/pipermail/python-checkins/2008-March/066863.html I just need to have some time on the machine and look into the problem. If I determine the problem is with the underlying sqlite, I'll try to upgrade it. > [alpha True64 5.1 trunk (Neal Norwitz)] > [hppa Ubuntu trunk (Matthias Klose)] I can probably diagnose both of these too when I get back from Chicago. > Neal/Martin, I'd like to promote the following slaves to the stable list: > [g4 osx.4] > [x86 W2k8] > [AMD64 W2k8] > [ppc Debian unstable] > [sparc Ubuntu] > [sparc Debian] > [PPC64 Debian] > [S-390 Debian] > [x86 XP-3] > [amd64 XP] > [x86 FreeBSD] > [x86 FreeBSD 3] > > The trunk builds of these slaves have been the most reliable since I've been tracking. Most of these have already been promoted to stable. I just didn't restart the buildbot master since making the change. It requires a restart, not just a reconfig. I was waiting for a quiet time when the bots weren't busy, but that hasn't happened yet. :-) I can make more changes and restart when I get home. n From zooko at zooko.com Thu Mar 20 05:18:47 2008 From: zooko at zooko.com (zooko) Date: Wed, 19 Mar 2008 22:18:47 -0600 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> Message-ID: On Mar 19, 2008, at 3:23 PM, Guido van Rossum wrote: > If other people want to chime in please do so; if this is just a > dialog between Phillip and me I might incorrectly assume > that nobody besides Phillip really cares. I really care. I've used setuptools, easy_install, eggs, and pkg_resources extensively for the past year or so (and contributed a few small patches). There have been plenty of problems, but I find them to be overall useful tools. It is a great boon to a programming community to lower the costs of re-using other people's code. The Python community will benefit greatly once a way to do that becomes widely enough accepted to reach a tipping point and become ubiquitous. Setuptools is already the de facto standard, but it hasn't become ubiquitous, possibly in part because of "egg hatred", about which more below. I've interviewed several successful Python hackers who "hate eggs" in order to understand what they hate about them, and I've taken notes from some of these interviews. (The list includes MvL, whose name was invoked earlier in this thread.) After filtering out yer basic complaining about bugs (which complaints are of course legitimate, but which don't indict setuptools as worse than other software of comparable scope and maturity), their objections seem to fall into two categories: 1. "The very notion of package dependency resolution and programmable or command-line installation of packages at the language level is a bad notion." This can't really be the case. If the existence of such functionality at the programming language level were an inherently bad notion, then we would be hearing some complaints from the Ruby folks, where the Gems system is standard and ubiquitous. We hear no complaints -- only murmurs of satisfaction. One person recently reported to me that while there are more packages in Python, he finds himself re-using other people's code more often when he works in Ruby, because almost all Ruby software is Gemified, but only a fraction of Python software is Eggified. Often this complaint comes with the idea that eggs conflict with their system-level package management tools. (These are usually Debian/Ubuntu users.) Note that Ruby software is not too hard to include in operating system packaging schemes -- my Ubuntu Hardy apt-cache shows plenty of Ruby software. A sufficiently mature and widely supported setuptools could actually make it easier to integrate Python software into Debian -- see stdeb [1]. 2. "Setuptools/eggs give me grief." What can really be the case is that setuptools causes a host of small, unnecessary problems for people who prefer to do things differently than PJE does. Personally, I prefer to use GNU stow, and setuptools causes unnecessary, but avoidable, problems for me. Many people object (rightly enough) to a "./setup.py install" automatically fetching new software over the Internet by default. The fact that easy_install creates a site.py that changes the semantics of PYTHONPATH is probably the most widely and deservedly hated example of this kind of thing [2]. I could go on with a few other common technical complaints of this kind. These type-2 problems can be fixed by changing setuptools or they can be grudgingly accepted by users, while retaining compatibility with the large and growing ecosystem of eggy software. Certainly fixing setuptools to play better with others is a more likely path to success than setting out to invent a non-egg-compatible alternative. Such a project might never be implemented well enough to serve, and if it were it would probably never overtake eggs's lead in the Python ecosystem, and if it did it would probably not turn out to be a better tool. So, since you asked for my chime, I advise you to publically bless eggs, setuptools, and easy_install as plausible future standards and solicit patches which address the complaints. For that matter, soliciting specific complaints would be a good start. I've done so in private many times with only partial success as to the "specific" part. One promising approach is to request objections in the form of automated tests that setuptools fails, e.g. [3]. Regards, Zooko O'Whielacronx [1] http://stdeb.python-hosting.com/ [2] http://www.rittau.org/blog/20070726-02 And no, PJE's suggested "trivial fix" does not satisfy the objectors, as it can't support the use case of "cd somepkg ; python ./ setup.py install ; cd .. ; python -c 'import somepkg'". [3] http://twistedmatrix.com/trac/ticket/2308#comment:5 From jeff at taupro.com Thu Mar 20 05:41:08 2008 From: jeff at taupro.com (Jeff Rush) Date: Wed, 19 Mar 2008 23:41:08 -0500 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <47E1BD14.2030805@v.loewis.de> References: <47E0D883.9030402@taupro.com> <47E11659.9000307@v.loewis.de> <47E19ABE.2040800@taupro.com> <47E1BD14.2030805@v.loewis.de> Message-ID: <47E1EAE4.90802@taupro.com> Martin v. L?wis wrote: >> specific code in PyPI. Are developers for Python 3.x encouraged in >> 3.x guidelines to release 'fat' distributions that combine 2.x and 3.x >> usable versions? > > Passive voice is misleading here: encouraged by whom? "... encouraged in __3.x guidelines__ to ...": I presume although I've not found them yet that there is some kind of document for developers titled something like, "how to migrate your Python code from 2.x to 3.x". That document would be a logical place for advice and consideration of the tradeoffs of jumping to 3.x, maintaining two synced versions using 2to3 or 3to2, and the risks of keeping two independent releases. Identifying best practices would help them make good choices for the community. > *I* encourage people to consider that option, rather than assuming it > couldn't possibly work. Whether it actually works, I don't know. > I hope it would work, and I hope it would not be fat at all. So we don't have an actual success story of a dual-version distribution, even as a prototype, using 2to3 within a distutils package? I would not encourage a practice without at least one such example. >>> Can you kindly refer to some archived discussion for that? >> >> http://mail.python.org/pipermail/python-dev/2006-April/063943.html >> > Thanks. I still have the same position as I had then - if > distutils is broken, it should be fixed, not ignored. Since the precise API was not documented well and many people began to make use of ambiguous internal interfaces, such fixes would indeed break them. So your vote would be to do the right thing, even if it results in some breakage. I can respect that philosophy. >> A controversial point, I'm afraid. Perhaps it is time for a parallel >> rewrite, so that those setup.py who import distutils get the old >> behavior, and those who import distutils2 get the new, and we let >> attrition and the community decide which is standard. > > Is there a list of the problems with distutils somewhere? Unfortunately no. Much of it is anecdotal, much of it occurs on lists outside the Python community by those attempting to package things. And some of it are comments by developers who peeked into the distutils source and blanched. And some of the problems are not bugs, per se, but disagreement on scope of functionality and a lack of well-known alternatives. So just "fix it if broken" doesn't work when there is no agreement on how to expand that scope. I am working on pulling together such a list however, and getting it into the tracker, so that debate with a record of decisions can occur. I agree that it is hard to fix problems if no one is _clearly_ reporting them to us. Too much smoke, not enough light. > It always worked fine for me, so I see no reason to fix it in the > first place. Pardon my lack of knowledge of your background; when you say it always worked fine for you, are you referring to personal experiences using it to _install_ software or to experiences as a packager in actually distributing complex collections of modules on different platforms? -Jeff From tseaver at palladion.com Thu Mar 20 05:58:14 2008 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 20 Mar 2008 00:58:14 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> Message-ID: <47E1EEE6.5050608@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Guido van Rossum wrote: > On Wed, Mar 19, 2008 at 12:54 PM, Phillip J. Eby wrote: > [a long message] > > I'm back at Google and *really* busy for another week or so, so I'll > have to postpone the rest of this discussion for a while. If other > people want to chime in please do so; if this is just a dialog between > Phillip and me I might incorrectly assume that nobody besides Phillip > really cares. I care, a lot, enough to have volunteered to help with maintenance / development of setuptols back in September 2007. I think that, warts an all, setuptools is a *huge* improvement over bare distutils for nearly every use case I know about. A lot of setuptools warts are driven by related design problems in the distutils, such as the choice to use imperative / procedural code for everything: a declarative approach, with hooks for cases which actually need them (likely 5% of existing packages) would have made writing tools on top of the framework much simpler. It is ironic that Python is *too powerful* a tool for the tasks normally done by distutils / setuptools: a more restricted, and thererfore introspectable, configuration-driven approoach seems much cleaner. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFH4e7m+gerLs4ltQ4RAt+hAKDBqIrashlgf8U6XRtfMHjTOaiy4gCeO1Zn UfdjDYIb2P6vDCcUGSjITTo= =JTok -----END PGP SIGNATURE----- From wolever at cs.toronto.edu Thu Mar 20 06:04:01 2008 From: wolever at cs.toronto.edu (David Wolever) Date: Thu, 20 Mar 2008 01:04:01 -0400 Subject: [Python-Dev] Pretty-printing 2to3 Nodes Message-ID: <4E9D632B-7998-4BD2-9FE2-8E51BA57BE35@cs.toronto.edu> Would anyone be averse to changing pytree.Node's __repr__ so it includes the name of the name of the symbol the node represents? The only downside is that it makes the __reprs__ longer... But I think its worth the length: Node(313:simple_stmt, [Node(298:import_name, [Leaf(1, 'import'), Node (279:dotted_as_name, [Node(281:dotted_name, [Leaf(1, 'foo'), Leaf(23, '.'), Leaf(1, 'bar')]), Leaf(1, 'as'), Leaf(1, 'bang')])]), Leaf(4, '\n')]) OR just names: Node(import_name, [Leaf(1, 'import'), Node(dotted_as_name, [Node (dotted_name, [Leaf(1, 'foo'), Leaf(23, '.'), Leaf(1, 'bar')]), Leaf (1, 'as'), Leaf(1, 'bang')])]) OR the original: Node(313, [Node(298, [Leaf(1, 'import'), Node(279, [Node(281, [Leaf (1, 'foo'), Leaf(23, '.'), Leaf(1, 'bar')]), Leaf(1, 'as'), Leaf(1, 'bang')])]), Leaf(4, '\n')]) From guido at python.org Thu Mar 20 06:08:26 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 19 Mar 2008 22:08:26 -0700 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <47E1B858.1090107@v.loewis.de> References: <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <47E1B858.1090107@v.loewis.de> Message-ID: I was using the human interface at python.org/pypi. There are two prominent links at the top of the page: "Browse the tree of packages" and "Submit package information" followed by the 30 most recently changed packages. What I was looking for was the page for a specific package. The "Browse the tree of packages" link was no help. Finally I realized that in the side bar, in a small unobtrusive font, is a link to "List packages" which links to a list of *all* packages, in alphabetical order. I found my package there. I think repeating that link right below "browse the tree" would have been sufficient. But it would have been cool if there had been a search box (also in the start page) where I could type (part of) the name of the package and it would have given me the nearest matches. On Wed, Mar 19, 2008 at 6:05 PM, "Martin v. L?wis" wrote: > > I don't understand PyPI all that well; it seems poor design that the > > browsing via keywords is emphasized but there is no easy way to > > *search* for a keyword (the list of all packages is not emphasized > > enough on the main page -- it occurs in the side bar but not in the > > main text). > > I don't understand. What is "browsing via keywords" and how is that > emphasized? (one I know that, I can look into ways for searching > for keywords) > > > > I assume there's a programmatic API (XML-RPC?) but I > > haven't found it yet. > > The recommended "programmatic" API is > > > http://pypi.python.org/simple/ > > Not sure what you were trying to achieve programmatically; > "typically" people know what they want to install (e.g. > "threadedcomments"), and then the tool goes directly to > > http://pypi.python.org/simple/threadedcomments/ > > Regards, > Martin > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From wolever at cs.toronto.edu Thu Mar 20 06:27:11 2008 From: wolever at cs.toronto.edu (David Wolever) Date: Thu, 20 Mar 2008 01:27:11 -0400 Subject: [Python-Dev] 2to3 and print function In-Reply-To: <43aa6ff70803191544n2ef94e60ye89312dd3243051e@mail.gmail.com> References: <2A4672AB-A1FC-4CB6-A9C4-20B5059513B4@cs.toronto.edu> <43aa6ff70803191544n2ef94e60ye89312dd3243051e@mail.gmail.com> Message-ID: <18762F56-39FC-45C3-9DCD-D60DCB1E541C@cs.toronto.edu> On 19-Mar-08, at 6:44 PM, Collin Winter wrote: > You can pass -p to refactor.py to fix this on a per-run basis. See > r58002 (and the revisions it mentions) for a failed attempt to do this > automatically. So, correct me if I'm wrong, but the relevant code is this: - try: - tree = self.driver.parse_string(data) - except parse.ParseError, e: - if e.type == token.EQUAL: - tree = self.printless_driver.parse_string(data) - else: - raise Why not, instead of trying both parsers, scan for a __future__ import, then do the Right Thing based on that? From guido at python.org Thu Mar 20 07:02:05 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 19 Mar 2008 23:02:05 -0700 Subject: [Python-Dev] Pretty-printing 2to3 Nodes In-Reply-To: <4E9D632B-7998-4BD2-9FE2-8E51BA57BE35@cs.toronto.edu> References: <4E9D632B-7998-4BD2-9FE2-8E51BA57BE35@cs.toronto.edu> Message-ID: Sounds good! On Wed, Mar 19, 2008 at 10:04 PM, David Wolever wrote: > Would anyone be averse to changing pytree.Node's __repr__ so it > includes the name of the name of the symbol the node represents? > > The only downside is that it makes the __reprs__ longer... But I > think its worth the length: > > Node(313:simple_stmt, [Node(298:import_name, [Leaf(1, 'import'), Node > (279:dotted_as_name, [Node(281:dotted_name, [Leaf(1, 'foo'), Leaf(23, > '.'), Leaf(1, 'bar')]), Leaf(1, 'as'), Leaf(1, 'bang')])]), Leaf(4, > '\n')]) > OR just names: > Node(import_name, [Leaf(1, 'import'), Node(dotted_as_name, [Node > (dotted_name, [Leaf(1, 'foo'), Leaf(23, '.'), Leaf(1, 'bar')]), Leaf > (1, 'as'), Leaf(1, 'bang')])]) > OR the original: > Node(313, [Node(298, [Leaf(1, 'import'), Node(279, [Node(281, [Leaf > (1, 'foo'), Leaf(23, '.'), Leaf(1, 'bar')]), Leaf(1, 'as'), Leaf(1, > 'bang')])]), Leaf(4, '\n')]) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Mar 20 07:15:20 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 19 Mar 2008 23:15:20 -0700 Subject: [Python-Dev] 2to3 and print function In-Reply-To: <18762F56-39FC-45C3-9DCD-D60DCB1E541C@cs.toronto.edu> References: <2A4672AB-A1FC-4CB6-A9C4-20B5059513B4@cs.toronto.edu> <43aa6ff70803191544n2ef94e60ye89312dd3243051e@mail.gmail.com> <18762F56-39FC-45C3-9DCD-D60DCB1E541C@cs.toronto.edu> Message-ID: On Wed, Mar 19, 2008 at 10:27 PM, David Wolever wrote: > On 19-Mar-08, at 6:44 PM, Collin Winter wrote: > > You can pass -p to refactor.py to fix this on a per-run basis. See > > r58002 (and the revisions it mentions) for a failed attempt to do this > > automatically. > > So, correct me if I'm wrong, but the relevant code is this: > - try: > - tree = self.driver.parse_string(data) > - except parse.ParseError, e: > - if e.type == token.EQUAL: > - tree = self.printless_driver.parse_string(data) > - else: > - raise > > Why not, instead of trying both parsers, scan for a __future__ > import, then do the Right Thing based on that? Different use cases. Auto-detection based on __future__ would be a good thing to have (assuming that __future__ statement has actually been implemented :-). But the -p lag is for code that has already been converted to 3.0 (as far as print statements are concerned anyway) and hence doesn't have __future__ statements. This is mostly used in the standard library, where sometimes it is useful to run selected fixers again after a merge. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Mar 20 07:57:31 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 19 Mar 2008 23:57:31 -0700 Subject: [Python-Dev] platform management In-Reply-To: <-3692898589838545524@unknownmsgid> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE20@EXMBX04.exchhosting.com> <-3692898589838545524@unknownmsgid> Message-ID: Great idea! Sounds like a PEP (informational, probably) would be good idea. On Tue, Mar 18, 2008 at 4:59 PM, Bill Janssen wrote: > I don't think this is bike-shedding. > > The debate about "AMD64" vs. "amd64" vs. "x86_64" reminded me that > I've been bit more and more frequently by bits of platform-specific > knowledge scattered around the standard library. The latest is the > code in distutils.unixccompiler that tries to figure out what flags to > send to the linker in order to add a dynamic library path lookup to a > shared library. > > There are lots of different ways of figuring out which platform is > being used, and they're all over the place. The code in > distutils.unixccompiler uses "sys.platform[:6]", and looks for > "darwin"; the code in urllib.py uses "os.name", and expects it to be > "mac", but later looks for "sys.platform == 'darwin'; posixfile > believes that platforms come with version numbers ("linux2", "aix4"); > pydoc and tarfile have tests for "sys.platform == 'mac'". tempfile > looks at os.sys.platform *and* os.name. > > Could well be that all of these are correct (I believe that "mac", for > instance, refers to the generations of the Mac OS before OS X). But > it means that when someone tries to update "Python" to a new major > version release for, say, OS X, it's really easy to miss things. And > the fact that the platform version is sometimes included and sometimes > not is also problematic; darwin9 is different from darwin8 in some > important aspects. > > It would be nice to > > (a) come up with a list of standard platform symbols, > (b) put those symbols somewhere in the documentation, > (c) have one way of discovering them, via sys or os or platform or > whichever module, > (d) add a standard way of discovering the platform version, and > (e) stamp out all the other ways that have crept in over the years. > > Bill > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From eric+python-dev at trueblade.com Thu Mar 20 07:55:55 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Thu, 20 Mar 2008 02:55:55 -0400 Subject: [Python-Dev] trunk buildbot status In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2F@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2E@EXMBX04.exchhosting.com>, <47E1B4A5.8050409@trueblade.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2F@EXMBX04.exchhosting.com> Message-ID: <47E20A7B.6070600@trueblade.com> Trent Nelson wrote: > I'd recommend cd'ing to your trunk root directory and running Tool\buildbot\build.bat from there -- it'll automatically check out all the dependencies and build via command line with vcbuild (building via Visual Studio usually always Does The Right Thing, command line builds often take a bit more coercing). Okay, that's extremely helpful. With that (and installing nasmw.exe), a trunk checkout builds correctly and passes all tests (although skipping test_tcl) on my box. As I said, it's XP x86 with 2008 Express Edition only. Let me know if I can provide any other information. Unfortunately I don't have access to this box during the work day (EDT), and I'm leaving for vacation tomorrow (Friday). But I'll help as best I can. Eric. > > ________________________________________ > From: Eric Smith [eric+python-dev at trueblade.com] > Sent: 19 March 2008 20:49 > To: Trent Nelson > Cc: python-dev at python.org; doko at ubuntu.com; joe at joevial.com; theller at ctypes.org; db3l.net at gmail.com; nnortwitz at gmail.org > Subject: Re: [Python-Dev] trunk buildbot status > > Trent Nelson wrote: >> Quick update on the status of the trunk buildbots: >> >> [x86 XP trunk (Joseph Armbruster)] >> This box didn't survive the recent build changes, but I can't figure out why, none of the other Windows boxes encounter this error: >> The following error has occurred during XML parsing: >> File: C:\python\buildarea\trunk.armbruster-windows\build\PCbuild\_bsddb.vcproj >> Line: 179 >> Column: 1 >> Error Message: >> Illegal qualified name character. >> The file 'C:\python\buildarea\trunk.armbruster-windows\build\PCbuild\_bsddb.vcproj' has failed to load. >> >> Can someone check a clean trunk build on a Windows system that *only* has Visual C++ Express 2008? The latest build system updates don't rely on any features of Visual Studio Professional, but the tools use a lot of common files, and perhaps a Service Pack needs to be applied or something. > > I just built the trunk on a Windows XP x86 box that only has Visual C++ > Express 2008 installed. I got a bunch of errors with sqlite, tcl, > db-4.4.20, and ssl, but the interpreter built and appears to run ok. > > But since I don't have bsddb installed, I don't think I'm executing the > portion of the build process that you find failing. > > I don't have time to install bsddb tonight, but I can do that in about > 24 hours if you still need me to. > > Eric. > From tnelson at onresolve.com Thu Mar 20 08:13:32 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Thu, 20 Mar 2008 00:13:32 -0700 Subject: [Python-Dev] trunk buildbot status In-Reply-To: <47E20A7B.6070600@trueblade.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2E@EXMBX04.exchhosting.com>, <47E1B4A5.8050409@trueblade.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2F@EXMBX04.exchhosting.com>, <47E20A7B.6070600@trueblade.com> Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE33@EXMBX04.exchhosting.com> Thanks Eric, very useful to know. I guess it's just that particular build slave... ________________________________________ From: Eric Smith [eric+python-dev at trueblade.com] Sent: 20 March 2008 02:55 To: Trent Nelson Cc: python-dev at python.org Subject: Re: [Python-Dev] trunk buildbot status Trent Nelson wrote: > I'd recommend cd'ing to your trunk root directory and running Tool\buildbot\build.bat from there -- it'll automatically check out all the dependencies and build via command line with vcbuild (building via Visual Studio usually always Does The Right Thing, command line builds often take a bit more coercing). Okay, that's extremely helpful. With that (and installing nasmw.exe), a trunk checkout builds correctly and passes all tests (although skipping test_tcl) on my box. As I said, it's XP x86 with 2008 Express Edition only. Let me know if I can provide any other information. Unfortunately I don't have access to this box during the work day (EDT), and I'm leaving for vacation tomorrow (Friday). But I'll help as best I can. Eric. > > ________________________________________ > From: Eric Smith [eric+python-dev at trueblade.com] > Sent: 19 March 2008 20:49 > To: Trent Nelson > Cc: python-dev at python.org; doko at ubuntu.com; joe at joevial.com; theller at ctypes.org; db3l.net at gmail.com; nnortwitz at gmail.org > Subject: Re: [Python-Dev] trunk buildbot status > > Trent Nelson wrote: >> Quick update on the status of the trunk buildbots: >> >> [x86 XP trunk (Joseph Armbruster)] >> This box didn't survive the recent build changes, but I can't figure out why, none of the other Windows boxes encounter this error: >> The following error has occurred during XML parsing: >> File: C:\python\buildarea\trunk.armbruster-windows\build\PCbuild\_bsddb.vcproj >> Line: 179 >> Column: 1 >> Error Message: >> Illegal qualified name character. >> The file 'C:\python\buildarea\trunk.armbruster-windows\build\PCbuild\_bsddb.vcproj' has failed to load. >> >> Can someone check a clean trunk build on a Windows system that *only* has Visual C++ Express 2008? The latest build system updates don't rely on any features of Visual Studio Professional, but the tools use a lot of common files, and perhaps a Service Pack needs to be applied or something. > > I just built the trunk on a Windows XP x86 box that only has Visual C++ > Express 2008 installed. I got a bunch of errors with sqlite, tcl, > db-4.4.20, and ssl, but the interpreter built and appears to run ok. > > But since I don't have bsddb installed, I don't think I'm executing the > portion of the build process that you find failing. > > I don't have time to install bsddb tonight, but I can do that in about > 24 hours if you still need me to. > > Eric. > From theller at ctypes.org Thu Mar 20 08:24:43 2008 From: theller at ctypes.org (Thomas Heller) Date: Thu, 20 Mar 2008 08:24:43 +0100 Subject: [Python-Dev] First green Windows x64 buildbots! In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2C@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2C@EXMBX04.exchhosting.com> Message-ID: Trent Nelson schrieb: > We've just experienced our first 2.6 green x64 Windows builds on the > build slaves! Well, almost green. Thomas's 'amd64 XP trunk' ran out > of disk: > > 304 tests OK. 1 test failed: test_largefile > ====================================================================== > ERROR: test_seek (test.test_largefile.TestCase) > ---------------------------------------------------------------------- > Traceback (most recent call last): File > "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_largefile.py", > line 42, in test_seek f.flush() IOError: [Errno 28] No space left on > device > > Sorry about that Thomas ;-) Trent - that's great. I'll try to free some diskspace. Thomas From theller at ctypes.org Thu Mar 20 08:30:10 2008 From: theller at ctypes.org (Thomas Heller) Date: Thu, 20 Mar 2008 08:30:10 +0100 Subject: [Python-Dev] trunk buildbot status In-Reply-To: References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2E@EXMBX04.exchhosting.com> Message-ID: >> Neal/Martin, I'd like to promote the following slaves to the stable list: >> [g4 osx.4] >> [x86 W2k8] >> [AMD64 W2k8] >> [ppc Debian unstable] >> [sparc Ubuntu] >> [sparc Debian] >> [PPC64 Debian] >> [S-390 Debian] >> [x86 XP-3] >> [amd64 XP] >> [x86 FreeBSD] >> [x86 FreeBSD 3] >> >> The trunk builds of these slaves have been the most reliable since I've been tracking. > > Most of these have already been promoted to stable. I just didn't > restart the buildbot master since making the change. It requires a > restart, not just a reconfig. I was waiting for a quiet time when the > bots weren't busy, but that hasn't happened yet. :-) I can make more > changes and restart when I get home. Hm, what is this 'stable list' anyway? BTW: Although the x86 XP-3 and x86 W2k8 bots are green, there's still a problem: test_tcl test_tcl skipped -- DLL load failed: The specified module could not be found. AFAIK, this is because tcl86g.dll and tk86g.dll fail to load because they cannot find MSVCR90D.DLL; do these dlls need a manifest? Thomas From p.f.moore at gmail.com Thu Mar 20 10:33:53 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 20 Mar 2008 09:33:53 +0000 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> Message-ID: <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> On 20/03/2008, zooko wrote: > On Mar 19, 2008, at 3:23 PM, Guido van Rossum wrote: > > > If other people want to chime in please do so; if this is just a > > dialog between Phillip and me I might incorrectly assume > > that nobody besides Phillip really cares. > > I really care. I've used setuptools, easy_install, eggs, and > pkg_resources extensively for the past year or so (and contributed a > few small patches). There have been plenty of problems, but I find > them to be overall useful tools. I'll chime in here, too. I really want to like setuptools/easy_install, but I don't. I'll try to be specific in my reasons, in the hope that they can be addressed. I know some of these are "known about", but one of my meta-dislikes of setuptools is that known issues never seem to get addressed (I know, patches accepted, but I haven't got the time either...) 1. No integration with the system packager (Windows, in my case). If I do easy_install nose, then nose does not show up in add/remove programs. That significantly affects the way I manage my PC. 2. No uninstaller. After easy_install nose, how do I get rid of it later? Searching for files to delete (even if there are only a few obviously named ones) is not good enough. 3. The pkg_resources documentation (in particular, that's the one I've tried to follow) is extremely hard to read. Partly this is just style, but it's partly because it is couched in very unfamiliar terms (distributions, working sets, interfaces, providers, etc). It's also *huge*. A tutorial style overview, supported by API detail, would be far better. 4. Hard to use with limited connectivity. At work, I *only* have access to the internet via Internet Explorer (MS based proxy). There are workarounds, but ultimately "download an installer, then run it" is a far simpler approach for me. 5. Auto-discovery doesn't always work. I'm sorry, I really can't recall the example at the moment, but sometimes easy_install says it can't find a package I *know* is available. 6. Splitting the community. Windows users rely heavily on binary installers (at least, I do). We're starting to get a situation where some projects provide .egg files, and some provide traditional (bdist_wininst/bdist_msi) installers. This is bad. One way to do it, and all that :-) But if these problems are solved, then I have no problem with seeing the features of setuptools added to the standard library - resource APIs, plugin/entry point APIs, ways to create executable scripts, and such things *should* be standardised. Dependency resolution and automatic installation isn't something I like (probably because as a Windows user I've never used such a system, so I mistrust it) but if it works *with* the system and not against it, I don't mind. I hope this helps, Paul. From glyph at divmod.com Thu Mar 20 11:38:23 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Thu, 20 Mar 2008 10:38:23 -0000 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> Message-ID: <20080320103823.21558.491968946.divmod.xquotient.8438@joule.divmod.com> On 09:33 am, p.f.moore at gmail.com wrote: >I'll chime in here, too. I really want to like >setuptools/easy_install, but I don't. I'll try to be specific in my >reasons, in the hope that they can be addressed. I know some of these >are "known about", but one of my meta-dislikes of setuptools is that >known issues never seem to get addressed (I know, patches accepted, >but I haven't got the time either...) I agree with almost everything that Paul says, and he put it quite well, so I'll spare the "me too", but I do have some additional gripes to add. setuptools (or, for that matter, distutils) has no business installing stuff in the system directory on a Linux box with a package manager. The *major* benefit I can see to a tool like easy_install is providing a feature that system packagers do not: allowing developers to quickly pull down all their dependencies into a *user directory* without worrying about system administration. However, not only does setuptools not do this on its own, it actively fights me when I try to do it. Admittedly, my user directory is a little messed up. Combinator, the Divmod path management / developer deployment tool, does some inadvisable things to attempt to trick distutils into doing local installation. However, setuptools does have some pretty clear bugs in this area; for example, it hard-codes a copy of a list that's present in site.py to try to figure out where .pth files will be honored, rather than looking at what's actually on sys.path. Every time I've tried to install a package for development using setuptools - and I am speaking across a wide range of versions here, so this isn't a specific bug report - it's either emitted a completely inscrutable traceback or printed an error message telling me that it couldn't or wouldn't install to the provided location. >But if these problems are solved, then I have no problem with seeing >the features of setuptools added to the standard library - resource >APIs, plugin/entry point APIs, ways to create executable scripts, and >such things *should* be standardised. Dependency resolution and >automatic installation isn't something I like (probably because as a >Windows user I've never used such a system, so I mistrust it) but if >it works *with* the system and not against it, I don't mind. This is more of a vague impression than a specific technical criticism, but it really seems like the implementation of these features face a lot of unnecessary coupling in setuptools. Twisted (Hey, did you guys know I work on Twisted? It seems I hardly ever mention it!), for example, implements resource APIs (twisted.python.modules), plugins (twisted.plugin, which is a bit like some of the uses of entrypoints), and the zip-file agnosticism of both (twisted.python.zipstream) without needing any packaging metadata or ini files. It just introspects the Python path and adds a little frosting to importers. I could be wrong about setuptools' actual design; this could be a documentation or UI issue, because I haven't read the code. But when interacting with it as a user and perusing its API, it definitely seems as though things are woven too tightly together, and the whole thing is very centered around the concept of a "build", i.e. running some kind of compilation or deployment step with a setup.py. One of my favorite things about python is that stuff just runs without needing that normally. I know that "setup.py develop" is supposed to avoid that - but even that itself is a deployment step which generates some metadata. With the setuptools-free libraries I use, it's just check out, then run; no other intermediary step. I'm spoiled, of course, having apt to do the package-management work for me on the majority of my dependencies, and combinator mostly handling the rest. easy_install also definitely has problems with security. It automatically downloads links from plain-HTTP connections, some of them, I believe, from publicly editable wiki pages, and installs them with no verification of signatures (as root! because how else are you going to get them to the only supported installation directory!). I believe that this is possibly the easiest issue to fix though, and I hope that my information here is already out of date. I realize that people are already doing this (insecure installation) with their web browsers, but there are tons of UI cues in a web browser looking at a link on a wiki page which you don't get from an automated command-line tool. As others have said, I wanted to like setuptools. I wanted to believe we could be saved from the unfortunate issues with distutils. But the extremely high degree of frustration I've encountered every time I've tried to use it, its general inscrutability, its awful UI ("python -c "import setuptools; execfile('setup.py')" develop", seriously? you couldn't write a command-line tool to make that look like 'setuptool develop' or something?) and now the availability of my own libraries which do the things in setuptools that interest me, have served to strongly discourage me from trying to closely inspect or fix it. I just kind of vaguely hope that it will be overhauled if it's ever really considered for inclusion in the standard library and I try not to think too hard about it. I'm not actively opposing it, for those who want to use it - c.f. http://twistedmatrix.com/trac/ticket/1286 - but it definitely doesn't work for me. Just for the record: I wrote my own zip-file-friendly resource-loading library after trying to use setuptools. I had to get some code working on an embedded device, and it needed to load Twisted plugins (which predates setuptools by a long while, I believe - or at least I didn't know about them at the time). setuptools somehow simultaneously broke the path requirements of the twisted plugin system and blew my memory budget. I attempted to investigate but didn't get far - it was quite a lot easier to just write some libraries that performed one task at a time rather than trying to manage the whole deployment. From mal at egenix.com Thu Mar 20 11:42:48 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 20 Mar 2008 11:42:48 +0100 Subject: [Python-Dev] Consistent platform name for 64bit windows (was: distutils.util.get_platform() for Windows) In-Reply-To: <3504.63.250.241.10.1205859937.squirrel@mail.eftel.com> References: <3504.63.250.241.10.1205859937.squirrel@mail.eftel.com> Message-ID: <47E23FA8.3090104@egenix.com> On 2008-03-18 18:05, mhammond at keypoint.com.au wrote: > I'm reviving a very old thread based on discussions with Martin at pycon. > >> Sent: Monday, 23 July 2007 5:12 PM >> Subject: Re: [Distutils] distutils.util.get_platform() for Windows > > Rather than forcing everyone to read the context, allow me to summarize: > On 64bit Windows versions, we need a "string" that identifies the > platform, and this string should ideally be used consistently. This > original thread related to the files created by distutils (eg, > pywin32-210.win???64??-py2.6.exe) but it seems obvious that we should be > consistent wherever Python wants to display the platform (eg, in the > startup banner, in platform.py, etc). > > In the old thread, there was a semi-consensus that 'x86_64' be used by > distutils (and indeed, Lib/distutils/util.py in get_platform() has been > changed, by me, to use this string), but the Python 'banner', for example, > reports AMD64. Platform.py doesn't report much at all in this area, at > least when pywin32 isn't installed, but it arguably should. > > Both Martin and I prefer AMD64 as the string, for various reasons. > Firstly, it is less ugly than 'x86_64', and doesn't include an '_'/'-' > which might tend to confuse parsing by humans or computers. Martin also > made the point that AMD invented the architecture and AMD64 is their > preferred name, so we should respect that. > > So, at the risk of painting a bike-shed, I'd like to propose that we adopt > 'AMD64' in distutils (needs a change), platform.py (needs a change to use > sys.getwindowsversion() in preference to pywin32, if possible, anyway), > and the Python banner (which already uses AMD64). > > Any objections? Any strong feelings that using 'AMD' will confuse people > with Intel processors? Strong feelings about the parsability of the name > (PJE? )? Strong feelings about the color ? Not really an object, but Microsoft itself uses the term "x64" for the 64-bit variants of their OS, e.g. http://www.microsoft.com/windowsxp/64bit/default.mspx Since the platform name is targeting Windows, I think we should avoid confusing Windows users more than Intel users ;-) About the platform.py changes: if someone could provide the return values of sys.getwindowsversion() for 64bit versions of Windows XP and Vista, I could add support for it (don't have a 64bit version of Windows available to check myself). Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 20 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From martin at v.loewis.de Thu Mar 20 12:37:32 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 20 Mar 2008 06:37:32 -0500 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <47E1EAE4.90802@taupro.com> References: <47E0D883.9030402@taupro.com> <47E11659.9000307@v.loewis.de> <47E19ABE.2040800@taupro.com> <47E1BD14.2030805@v.loewis.de> <47E1EAE4.90802@taupro.com> Message-ID: <47E24C7C.6080209@v.loewis.de> > "... encouraged in __3.x guidelines__ to ...": I presume although I've not > found them yet that there is some kind of document for developers titled > something like, "how to migrate your Python code from 2.x to 3.x". That > document would be a logical place for advice and consideration of the > tradeoffs of jumping to 3.x, maintaining two synced versions using 2to3 or > 3to2, and the risks of keeping two independent releases. Identifying best > practices would help them make good choices for the community. I don't think any of the core committers is qualified to write such a document. Instead, it would have to be written by people who *actually* ported a project from 2 to 3 in some form. That is not to say that such a document couldn't be part of the 3k release, or shouldn't be reviewed by core committers. [Also, it might turn out that some of the core committers writes such a document, with the theoretical background of what *could* work for projects. That would be a lot like all those books giving advise written by people who never followed their own advise because they never had a chance to]. > So we don't have an actual success story of a dual-version distribution, even > as a prototype, using 2to3 within a distutils package? I would not encourage > a practice without at least one such example. We don't have any success story for Python 3, period. Nobody has ever attempted to run a significant code base in Python 3, other than the test suite, AFAIK. >> It always worked fine for me, so I see no reason to fix it in the >> first place. > > Pardon my lack of knowledge of your background; when you say it always worked > fine for you, are you referring to personal experiences using it to _install_ > software or to experiences as a packager in actually distributing complex > collections of modules on different platforms? I've been maintaining a larger project (PyXML) for several years, and have written/maintained a few smaller projects (iconv, partial, python-fam), which all used distutils. I have also extended distutils in the core, with the upload and bdist_msi commands. And then there is the experience with installing distutils-based packages, which is usually pleasant (although I prefer to use the Debian package where available) Regards, Martin From martin at v.loewis.de Thu Mar 20 12:38:46 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 20 Mar 2008 06:38:46 -0500 Subject: [Python-Dev] Pretty-printing 2to3 Nodes In-Reply-To: <4E9D632B-7998-4BD2-9FE2-8E51BA57BE35@cs.toronto.edu> References: <4E9D632B-7998-4BD2-9FE2-8E51BA57BE35@cs.toronto.edu> Message-ID: <47E24CC6.1020107@v.loewis.de> > OR just names: > Node(import_name, [Leaf(1, 'import'), Node(dotted_as_name, [Node > (dotted_name, [Leaf(1, 'foo'), Leaf(23, '.'), Leaf(1, 'bar')]), Leaf > (1, 'as'), Leaf(1, 'bang')])]) This is the one I prefer. Regards, Martin From andymac at bullseye.apana.org.au Thu Mar 20 13:36:33 2008 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Thu, 20 Mar 2008 22:36:33 +1000 Subject: [Python-Dev] PyCon: please review miy pending patches In-Reply-To: References: Message-ID: <47E25A51.9080307@bullseye.andymac.org> Christian Heimes wrote: > Memory management of ints, floats and longs > http://bugs.python.org/issue2039 > http://bugs.python.org/issue2013 wrt 2039 - I would like to see the free list compaction called from gc.collect() rather than a function in sys... something you suggested. As I noted in the comments to 2039, in the presence of pymalloc there is almost no value to floats having a freelist as far as I can test - other than in microbenchmarks. I see from a comment in 2013 that you were testing that patch with a debug build, which skews the timings. If your performance evaluation of 2039 was also done with a debug build, I suggest you try it with a non-debug build (which is what I used for all my testing). -- ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac at bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac at pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From theller at ctypes.org Thu Mar 20 13:42:23 2008 From: theller at ctypes.org (Thomas Heller) Date: Thu, 20 Mar 2008 13:42:23 +0100 Subject: [Python-Dev] Consistent platform name for 64bit windows (was: distutils.util.get_platform() for Windows) In-Reply-To: <47E23FA8.3090104@egenix.com> References: <3504.63.250.241.10.1205859937.squirrel@mail.eftel.com> <47E23FA8.3090104@egenix.com> Message-ID: M.-A. Lemburg schrieb: > About the platform.py changes: if someone could provide the return > values of sys.getwindowsversion() for 64bit versions of Windows > XP and Vista, I could add support for it (don't have a 64bit version > of Windows available to check myself). This is the output of a 32-bit Python running on "Windows XP Professional x64 Edition, Version 2003, Service Pack 2": C:\Python24>ver Microsoft Windows [Version 5.2.3790] C:\Python24>python Python 2.4.4 (#71, Oct 18 2006, 08:34:43) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.getwindowsversion() (5, 2, 3790, 2, 'Service Pack 2') >>> Thomas From skip at pobox.com Thu Mar 20 13:09:19 2008 From: skip at pobox.com (skip at pobox.com) Date: Thu, 20 Mar 2008 07:09:19 -0500 Subject: [Python-Dev] trunk buildbot status In-Reply-To: References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE2E@EXMBX04.exchhosting.com> Message-ID: <18402.21487.559570.659282@montanaro-dyndns-org.local> Neal> Unfortunately, I don't have ssh access from my hotel room. This Neal> means I won't be able to help much until I get home. There's always the hideously priced access at O'Hare. ;-) Skip From mal at egenix.com Thu Mar 20 13:55:21 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 20 Mar 2008 13:55:21 +0100 Subject: [Python-Dev] Consistent platform name for 64bit windows (was: distutils.util.get_platform() for Windows) In-Reply-To: References: <3504.63.250.241.10.1205859937.squirrel@mail.eftel.com> <47E23FA8.3090104@egenix.com> Message-ID: <47E25EB9.3000301@egenix.com> On 2008-03-20 13:42, Thomas Heller wrote: > M.-A. Lemburg schrieb: >> About the platform.py changes: if someone could provide the return >> values of sys.getwindowsversion() for 64bit versions of Windows >> XP and Vista, I could add support for it (don't have a 64bit version >> of Windows available to check myself). > > This is the output of a 32-bit Python running on "Windows XP Professional > x64 Edition, Version 2003, Service Pack 2": > > C:\Python24>ver > > Microsoft Windows [Version 5.2.3790] > > C:\Python24>python > Python 2.4.4 (#71, Oct 18 2006, 08:34:43) [MSC v.1310 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> import sys >>>> sys.getwindowsversion() > (5, 2, 3790, 2, 'Service Pack 2') Thank you ! Anyone with a 64bit Vista ? Or even better: a page documenting what to expect from the system call behind the sys.getwindowsversion() API ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 20 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From tseaver at palladion.com Thu Mar 20 14:44:00 2008 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 20 Mar 2008 09:44:00 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> Message-ID: <47E26A20.1090402@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Paul Moore wrote: > I'll chime in here, too. I really want to like > setuptools/easy_install, but I don't. I'll try to be specific in my > reasons, in the hope that they can be addressed. I know some of these > are "known about", but one of my meta-dislikes of setuptools is that > known issues never seem to get addressed (I know, patches accepted, > but I haven't got the time either...) Thanks for feedback from the Windows world, from whence I have been blissfully exiled now for years. > 1. No integration with the system packager (Windows, in my case). If I > do easy_install nose, then nose does not show up in add/remove > programs. That significantly affects the way I manage my PC. Point taken. Of course, it isn't really a "program" at that point: it is an installed "add-on" to Python. However, if Windows users expect such add-ons to show up in the "system" list, that is good to know. I'll note that I use easy_install *only* to work in *non-system* locations: if I want to install Python packages to /usr/lib/python2.x/, I use the standard system installer, e.g. 'apt-get install python-frobnatz'. But I routinely create non-system Python environments for development, using either alternate Pythons or virtualenv: in those environments, it works very well for me. > 2. No uninstaller. After easy_install nose, how do I get rid of it > later? Searching for files to delete (even if there are only a few > obviously named ones) is not good enough. People ask for this on Unix platforms as well, often adding a request that pacakges installed only as dependencies of the package-being-removed go away as well. If you install everything in a way that works with system package manager, of course, you don't need this. ;) Deleting the 'lib/python2.x/site-packages/foo-X.Y.X.egg' directory is all that is actually required to uninstall an egg that was previouly added via easy_install. Cleaning out the equivalent entry in 'easy_install.pth' in that directory is not strictly required. I wonder if a GUI for managing the add-ons would fit the bill, as an alternative to packaging them as though they were standalone programs? > 3. The pkg_resources documentation (in particular, that's the one I've > tried to follow) is extremely hard to read. Partly this is just style, > but it's partly because it is couched in very unfamiliar terms > (distributions, working sets, interfaces, providers, etc). It's also > *huge*. A tutorial style overview, supported by API detail, would be > far better. Many of those terms are distutils jargon, actually. I think Jeff Rush' recent work looks like a good start here. > 4. Hard to use with limited connectivity. At work, I *only* have > access to the internet via Internet Explorer (MS based proxy). There > are workarounds, but ultimately "download an installer, then run it" > is a far simpler approach for me. I don't know how to make this requirement compatible with using shared dependencies, except to make it easier for folks to download *all* the requirements, and later install from the local "distribution cache" (a directory full of .zip / .egg / .tgs files). It does turn out to be quite easy to build a PyPI-style "simple" index for such a cache. Your use case would then require: 1. Run some command to fetch the desired package and the transitive closure of its dependencies into a working directory (the cache). 2. Run another command to build an index for that directory. 3. Run 'easy_install', pointing to the local index. > 5. Auto-discovery doesn't always work. I'm sorry, I really can't > recall the example at the moment, but sometimes easy_install says it > can't find a package I *know* is available. Usually this indicates that there are incompatible dependencies between packages already installed and those on the index. E.g., if I already have package foo installed, but its version is not compatible with the requirements for package bar, then I can't install bar, even though the distribution is "available." Because PyPI is not a centrally-managed index of packages, such conflicts are pretty much inevitable over time for those who don't subset it in some form (what we've been calling the "known good set" strategy in Zope-land). > 6. Splitting the community. Windows users rely heavily on binary > installers (at least, I do). We're starting to get a situation where > some projects provide .egg files, and some provide traditional > (bdist_wininst/bdist_msi) installers. This is bad. One way to do it, > and all that :-) If it weren't for the "Add / Remove Programs" requirement you mentioned above, we would be better off if authors of pure Python packages uploaded only 'sdist' distributions, which can be cleanly converted to platform-local eggs at install time, even on Windows. Packages which contain C extensions typically must upload the 'bdist_win' version for the benefit of the vast majority of Windows users who can't bulid the extensions locally. Uploading any other binary distribution is pretty much a lose, because the underlying platform dependencies (UCS2 vs UCS4, i386 vs x64, framework vs. universal vs. MacPorts vs. Fink, etc) lead to combinatorial expolosions and or segfaults. Better to let the installer fetch the source and build it locally. > But if these problems are solved, then I have no problem with seeing > the features of setuptools added to the standard library - resource > APIs, plugin/entry point APIs, ways to create executable scripts, and > such things *should* be standardised. Dependency resolution and > automatic installation isn't something I like (probably because as a > Windows user I've never used such a system, so I mistrust it) but if > it works *with* the system and not against it, I don't mind. > > I hope this helps, Very much, thanks. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFH4mof+gerLs4ltQ4RApBLAJwI0Be1CtSKgpAYDEyH2qd0K+a+6QCeN/cf 5Pg43ot4H954A87ZWIouwLo= =S4yF -----END PGP SIGNATURE----- From bkline at rksystems.com Thu Mar 20 14:56:02 2008 From: bkline at rksystems.com (Bob Kline) Date: Thu, 20 Mar 2008 09:56:02 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <47E26A20.1090402@palladion.com> References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E26A20.1090402@palladion.com> Message-ID: <47E26CF2.1090701@rksystems.com> Tres Seaver wrote: > Point taken. Of course, it isn't really a "program" at that point: it > is an installed "add-on" to Python. However, if Windows users expect > such add-ons to show up in the "system" list, that is good to know. > Are things really that different in the non-Windows worlds? If I want python-nose, I run "sudo apt-get install python-nose" (and that means I can always remove it with "sudo apt-get remove ..."). Seems more similar than different (ignoring the silliness of Microsoft's insistence on "the GUI is the OOWTDI" even for such administrative tasks as installing system-wide software). -- Bob Kline http://www.rksystems.com mailto:bkline at rksystems.com From jnoller at gmail.com Thu Mar 20 14:58:46 2008 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 20 Mar 2008 09:58:46 -0400 Subject: [Python-Dev] Improved thread switching In-Reply-To: References: <20080319115107.YPWO11090.viefep31-int.chello.at@zion.visotech.at> <799854773.351205944939606.OPEN-XCHANGE.WebMail.tomcat@morpheus.visotech.at> Message-ID: <4222a8490803200658t5410422dn925f32d6eb6ec280@mail.gmail.com> FYI: I shot an email to stdlib-sig about the fact I am proposing the inclusion of the pyProcessing module into the stdlib. Comments and thoughts regarding that would be welcome. I've got a rough outline of the PEP, but I need to spend more time with the code examples. -jesse On Wed, Mar 19, 2008 at 9:52 PM, Alex Martelli wrote: > Hmmm, sorry if I'm missing something obvious, but, if the occasional > background computations are sufficiently heavy -- why not fork, do > said computations in the child thread, and return the results via any > of the various available IPC approaches? I've recently (at Pycon, > mostly) been playing devil's advocate (i.e., being PRO-threads, for > once) on the subject of utilizing multiple cores effectively -- but > the classic approach (using multiple _processes_ instead) actually > works quite well in many cases, and this application server would > appear to be one. (the pyProcessing package appears to offer an easy > way to migrate threaded code to multiple-processes approaches, > although I've only played around with it, not [yet] used it for > production code). > > > Alex > > > > On Wed, Mar 19, 2008 at 10:49 AM, Adam Olsen wrote: > > On Wed, Mar 19, 2008 at 11:25 AM, Stefan Ring wrote: > > > Adam Olsen gmail.com> writes: > > > > > > > > > > So you want responsiveness when idle but throughput when busy? > > > > > > Exactly ;) > > > > > > > > > > Are those calculations primarily python code, or does a C library do > > > > the grunt work? If it's a C library you shouldn't be affected by > > > > safethread's increased overhead. > > > > > > > > > > It's Python code all the way. Frankly, it's a huge mess, but it would be very > > > very hard to come up with a scalable solution that would allow to optimize > > > certain hotspots and redo them in C or C++. There isn't even anything left to > > > optimize in particular because all those low hanging fruit have already been > > > taken care of. So it's just ~30kloc Python code over which the total time spent > > > is quite uniformly distributed :(. > > > > I see. Well, at this point I think the most you can do is file a bug > > so the problem doesn't get forgotten. If nothing else, if my > > safethread stuff goes in it'll very likely include a --with-gil > > option, so I may put together a FIFO scheduler. > > > > > > -- > > Adam Olsen, aka Rhamphoryncus > > > > > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com > From janssen at parc.com Thu Mar 20 15:07:45 2008 From: janssen at parc.com (Bill Janssen) Date: Thu, 20 Mar 2008 07:07:45 PDT Subject: [Python-Dev] platform management In-Reply-To: References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE20@EXMBX04.exchhosting.com> <-3692898589838545524@unknownmsgid> Message-ID: <08Mar20.070752pdt."58696"@synergy1.parc.xerox.com> Looking at http://docs.python.org/lib/module-os.html, I find the following: name The name of the operating system dependent module imported. The following names have currently been registered: 'posix', 'nt', 'mac', 'os2', 'ce', 'java', 'riscos'. This implies that there's a registry somewhere? Bill > Great idea! Sounds like a PEP (informational, probably) would be good idea. > > On Tue, Mar 18, 2008 at 4:59 PM, Bill Janssen wrote: > > I don't think this is bike-shedding. > > > > The debate about "AMD64" vs. "amd64" vs. "x86_64" reminded me that > > I've been bit more and more frequently by bits of platform-specific > > knowledge scattered around the standard library. The latest is the > > code in distutils.unixccompiler that tries to figure out what flags to > > send to the linker in order to add a dynamic library path lookup to a > > shared library. > > > > There are lots of different ways of figuring out which platform is > > being used, and they're all over the place. The code in > > distutils.unixccompiler uses "sys.platform[:6]", and looks for > > "darwin"; the code in urllib.py uses "os.name", and expects it to be > > "mac", but later looks for "sys.platform == 'darwin'; posixfile > > believes that platforms come with version numbers ("linux2", "aix4"); > > pydoc and tarfile have tests for "sys.platform == 'mac'". tempfile > > looks at os.sys.platform *and* os.name. > > > > Could well be that all of these are correct (I believe that "mac", for > > instance, refers to the generations of the Mac OS before OS X). But > > it means that when someone tries to update "Python" to a new major > > version release for, say, OS X, it's really easy to miss things. And > > the fact that the platform version is sometimes included and sometimes > > not is also problematic; darwin9 is different from darwin8 in some > > important aspects. > > > > It would be nice to > > > > (a) come up with a list of standard platform symbols, > > (b) put those symbols somewhere in the documentation, > > (c) have one way of discovering them, via sys or os or platform or > > whichever module, > > (d) add a standard way of discovering the platform version, and > > (e) stamp out all the other ways that have crept in over the years. > > > > Bill > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) From mail at timgolden.me.uk Thu Mar 20 15:10:41 2008 From: mail at timgolden.me.uk (Tim Golden) Date: Thu, 20 Mar 2008 14:10:41 +0000 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <47E26CF2.1090701@rksystems.com> References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E26A20.1090402@palladion.com> <47E26CF2.1090701@rksystems.com> Message-ID: <47E27061.8080709@timgolden.me.uk> Bob Kline wrote: > Are things really that different in the non-Windows worlds? If I want > python-nose, I run "sudo apt-get install python-nose" (and that means I > can always remove it with "sudo apt-get remove ..."). Seems more > similar than different (ignoring the silliness of Microsoft's insistence > on "the GUI is the OOWTDI" even for such administrative tasks as > installing system-wide software). I was going to -- pointedly -- drop in here the help output for msiexec, which is the commandline version of the MSI installation graphical stuff. Only... when I did msiexec /?, the result was that a Window popped up with the information in it. (Sort of agrees with your point a bit!) Still, here's the info (cut-and-pasted from that window): ----- Windows ? Installer. V 3.01.4000.1823 msiexec /Option [Optional Parameter] Install Options Installs or configures a product /a Administrative install - Installs a product on the network /j [/t ] [/g ] Advertises a product - m to all users, u to current user Uninstalls the product [... snip lots of other options ...] TJG From martin at v.loewis.de Thu Mar 20 15:28:18 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 20 Mar 2008 09:28:18 -0500 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <47E1B858.1090107@v.loewis.de> Message-ID: <47E27482.9090908@v.loewis.de> Guido van Rossum schrieb: > I was using the human interface at python.org/pypi. There are two > prominent links at the top of the page: "Browse the tree of packages" > and "Submit package information" followed by the 30 most recently > changed packages. Ah, ok. In PyPI parlance, these are "classifiers" (Trove classifiers, although the word "trove" means nothing to me), not keywords. They are different from keywords in the sense that they form a hierarchy. I personally consider trove classifiers over-valued, but apparently, some people really love them (probably the ones who are more organized than I am). Developers continuously request addition of new classifiers; I don't have any statistics whether users actually use them to locate stuff. > What I was looking for was the page for a specific > package. The "Browse the tree of packages" link was no help. Finally I > realized that in the side bar, in a small unobtrusive font, is a link > to "List packages" which links to a list of *all* packages, in > alphabetical order. I found my package there. I think repeating that > link right below "browse the tree" would have been sufficient. I can't change that right now, but created http://sourceforge.net/tracker/index.php?func=detail&aid=1921108&group_id=66150&atid=513503 > But it > would have been cool if there had been a search box (also in the start > page) where I could type (part of) the name of the package and it > would have given me the nearest matches. Did you try the search box in the top-right, and did did not work? What search term did you enter, and what package did you expect to get? Regards, Martin From pje at telecommunity.com Thu Mar 20 16:07:55 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 20 Mar 2008 11:07:55 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <47E1EEE6.5050608@palladion.com> References: <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <47E1EEE6.5050608@palladion.com> Message-ID: <20080320150754.9F6503A40A5@sparrow.telecommunity.com> At 12:58 AM 3/20/2008 -0400, Tres Seaver wrote: >A lot of setuptools warts are driven by related design problems in the >distutils, such as the choice to use imperative / procedural code for >everything: a declarative approach, with hooks for cases which actually >need them (likely 5% of existing packages) would have made writing tools >on top of the framework much simpler. It is ironic that Python is *too >powerful* a tool for the tasks normally done by distutils / setuptools: > a more restricted, and thererfore introspectable, configuration-driven >approoach seems much cleaner. +1 From pje at telecommunity.com Thu Mar 20 16:18:04 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 20 Mar 2008 11:18:04 -0400 Subject: [Python-Dev] [Distutils-SIG] PYTHONPATH installation (was Re: PEP 365 (Adding the pkg_resources module)) In-Reply-To: References: <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> Message-ID: <20080320151806.F0A223A40A5@sparrow.telecommunity.com> At 10:18 PM 3/19/2008 -0600, zooko wrote: >The fact that easy_install creates a site.py that changes the >semantics of PYTHONPATH is probably the most widely and deservedly >hated example of this kind of thing [2]. Yep, this was an unfortunate side effect of eggs growing outside their original ecological niche. Without the 'site' hack, it was impossible to install eggs to user directories and avoid installation conflicts. Specifically, if someone installed a package to PYTHONPATH with the distutils, and then installed a later version using setuptools, the setuptools-installed version would always end up on sys.path *after* the distutils-installed version. Detecting this condition and handling it properly was a major problem for users of easy_install, who wanted it to "just work". Standardization of a PEP 262-style installation database is still needed to address these problems, not to mention uninstallation. Maybe now with some package manager folks paying some attention here, we can do something about that. >[2] http://www.rittau.org/blog/20070726-02 > And no, PJE's suggested "trivial fix" does not satisfy the >objectors, as it can't support the use case of "cd somepkg ; python >./ setup.py install ; cd .. ; python -c 'import somepkg'". Well, it replaces the hack being complained about, with the problem that the hack was introduced to fix. :) Again, to properly fix this, we need a metadata standard for who owns what packages -- and it should probably include information about the *tool* that did the installation, so that system packagers can either tell Python-level tools to keep their hands off, or tell Python how to run the tool in question. From pje at telecommunity.com Thu Mar 20 16:25:18 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 20 Mar 2008 11:25:18 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.co m> References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> Message-ID: <20080320152518.B79263A40A5@sparrow.telecommunity.com> At 09:33 AM 3/20/2008 +0000, Paul Moore wrote: >1. No integration with the system packager (Windows, in my case). If I >do easy_install nose, then nose does not show up in add/remove >programs. That significantly affects the way I manage my PC. The long-term fix here is probably to have a platform-specific installer capable of either turning eggs into .msi or .exe installers, or of doing the add/remove programs integration directly. (Someone, of course, will have to step up to create such a tool.) >5. Auto-discovery doesn't always work. I'm sorry, I really can't >recall the example at the moment, but sometimes easy_install says it >can't find a package I *know* is available. Sometimes it does that to me, too. But then I look at the project's page in PyPI, and they don't have a link to a download page. Usually, they've got a link to a page on their site with instructions about downloading, but that doesn't directly link to any tarballs or anything. So I grab the link of the real download page and paste it into a -f option to easy_install, so it knows where to find the link from. From pje at telecommunity.com Thu Mar 20 16:31:20 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 20 Mar 2008 11:31:20 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <47E26A20.1090402@palladion.com> References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E26A20.1090402@palladion.com> Message-ID: <20080320153120.7384B3A40A5@sparrow.telecommunity.com> At 09:44 AM 3/20/2008 -0400, Tres Seaver wrote: >I don't know how to make this requirement compatible with using shared >dependencies, except to make it easier for folks to download *all* the >requirements, and later install from the local "distribution cache" (a >directory full of .zip / .egg / .tgs files). It does turn out to be >quite easy to build a PyPI-style "simple" index for such a cache. Your >use case would then require: > > 1. Run some command to fetch the desired package and the transitive > closure of its dependencies into a working directory (the cache). > > 2. Run another command to build an index for that directory. > > 3. Run 'easy_install', pointing to the local index. Actually, if someone were to develop a patch for PyPI to do this, we could perhaps have a "display download dependencies" link for eggs shown on PyPI. That way, someone who wants to do a manual download could get a page with links for all the required eggs, and manually download them. (Of course, the other alternative would be for someone to provide an IE-controlling extension to urllib2 so that easy_install wouldn't be proxy-bound on such machines.) From asmodai at in-nomine.org Thu Mar 20 16:43:55 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Thu, 20 Mar 2008 16:43:55 +0100 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <47E27482.9090908@v.loewis.de> References: <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <47E1B858.1090107@v.loewis.de> <47E27482.9090908@v.loewis.de> Message-ID: <20080320154355.GI60713@nexus.in-nomine.org> -On [20080320 15:29], "Martin v. L?wis" (martin at v.loewis.de) wrote: >(Trove classifiers, >although the word "trove" means nothing to me) Isn't that something lifted from SourceForge? -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ The Eyes of Truth are always watching you... From asmodai at in-nomine.org Thu Mar 20 16:45:51 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Thu, 20 Mar 2008 16:45:51 +0100 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <47E1EEE6.5050608@palladion.com> References: <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <47E1EEE6.5050608@palladion.com> Message-ID: <20080320154551.GJ60713@nexus.in-nomine.org> -On [20080320 05:58], Tres Seaver (tseaver at palladion.com) wrote: >I think that, warts an all, setuptools is a *huge* improvement over bare >distutils for nearly every use case I know about. Agreed. I see setuptools (along with PyPI - hopefully much better in near future though) as the Python equivalent to CPAN and RubyGems. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Sadness is the inner beauty of the untouched tear... From amcnabb at mcnabbs.org Thu Mar 20 16:53:20 2008 From: amcnabb at mcnabbs.org (Andrew McNabb) Date: Thu, 20 Mar 2008 09:53:20 -0600 Subject: [Python-Dev] Improved thread switching In-Reply-To: <4222a8490803200658t5410422dn925f32d6eb6ec280@mail.gmail.com> References: <20080319115107.YPWO11090.viefep31-int.chello.at@zion.visotech.at> <799854773.351205944939606.OPEN-XCHANGE.WebMail.tomcat@morpheus.visotech.at> <4222a8490803200658t5410422dn925f32d6eb6ec280@mail.gmail.com> Message-ID: <20080320155320.GB8933@mcnabbs.org> On Thu, Mar 20, 2008 at 09:58:46AM -0400, Jesse Noller wrote: > FYI: I shot an email to stdlib-sig about the fact I am proposing the > inclusion of the pyProcessing module into the stdlib. Comments and > thoughts regarding that would be welcome. I've got a rough outline of > the PEP, but I need to spend more time with the code examples. Since we officially encourage people to spawn processes instead of threads, I think that this would be a great idea. The processing module has a similar API to threading. It's easy to use, works well, and most importantly, gives us some place to point people to when they complain about the GIL. -- Andrew McNabb http://www.mcnabbs.org/andrew/ PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20080320/80cd87f0/attachment.pgp From steven.bethard at gmail.com Thu Mar 20 16:54:58 2008 From: steven.bethard at gmail.com (Steven Bethard) Date: Thu, 20 Mar 2008 09:54:58 -0600 Subject: [Python-Dev] Checking for the -3 flag from Python code In-Reply-To: <47E0FE3B.8030006@gmail.com> References: <47E0FE3B.8030006@gmail.com> Message-ID: On Wed, Mar 19, 2008 at 5:51 AM, Nick Coghlan wrote: > This flag is exposed to python code as sys.flags.py3k_warning > > So the hack added to some of the test code that I saw go by on > python-checkins isn't needed :) Excellent. I asked around at the sprints and everyone thought it was unexposed. If no one else has already done it, I'll remove the hacks from test_py3kwarn and the regrtest skipping mechanism. Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From phd at phd.pp.ru Thu Mar 20 16:55:33 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Thu, 20 Mar 2008 18:55:33 +0300 Subject: [Python-Dev] Trove classifiers In-Reply-To: <20080320154355.GI60713@nexus.in-nomine.org> References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <47E1B858.1090107@v.loewis.de> <47E27482.9090908@v.loewis.de> <20080320154355.GI60713@nexus.in-nomine.org> Message-ID: <20080320155533.GB12099@phd.pp.ru> On Thu, Mar 20, 2008 at 04:43:55PM +0100, Jeroen Ruigrok van der Werven wrote: > -On [20080320 15:29], "Martin v. L??wis" (martin at v.loewis.de) wrote: > >(Trove classifiers, >although the word "trove" means nothing to me) > > Isn't that something lifted from SourceForge? Yes, exactly. Eric Raymond claims to be the inventor, but there are different voices against him: http://damagestudios.net/blog/2005/08/15/sourceforge-founders Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From pje at telecommunity.com Thu Mar 20 17:03:55 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 20 Mar 2008 12:03:55 -0400 Subject: [Python-Dev] The "autoinstall" package just uploaded to PyPI Message-ID: <20080320160357.8F2463A40AE@sparrow.telecommunity.com> I just wanted to throw in a quick note that this package: http://pypi.python.org/pypi/autoinstall which was just uploaded by Daniel Krech, is a lot closer in spirit to what I was trying to accomplish with PEP 365 than Guido's bootstrap proposal. Perhaps there's room for both in the stdlib? (And note that even though the examples use eggs, it does not do anything egg-specific; any zipfile importable by Python works with autoinstall.) There are a number of changes I would suggest making to autoinstall, like making it possible to access information about files in the cache, supporting non-toplevel modules, programmatic and environment-level control of the cache directory, that sort of thing. Heck, it'd be nice (although not essential) for it to support finding the right URL from PyPI. I also suspect that users might want to have some way to disable it or restrict it to certain hosts, etc., perhaps through a configuration file. It should probably also default the cache to a temporary directory, in the absence of other input. (Experience with pkg_resources' caching approach suggests that using the current directory or a home-directory-based location by default was a bad idea, at least without a fallback to a tempdir on write failure.) Any thoughts? From acapnotic at foobox.net Thu Mar 20 17:08:31 2008 From: acapnotic at foobox.net (Kevin Turner) Date: Thu, 20 Mar 2008 09:08:31 -0700 Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> Message-ID: <1206029311.9502.91.camel@grinky> On Wed, 2008-03-19 at 22:18 -0600, zooko wrote: > 1. "The very notion of package dependency resolution and > programmable or command-line installation of packages at the language > level is a bad notion." > > This can't really be the case. If the existence of such > functionality at the programming language level were an inherently > bad notion, then we would be hearing some complaints from the Ruby > folks, where the Gems system is standard and ubiquitous. We hear no > complaints -- only murmurs of satisfaction. Okay then, just to fill out your sample -- as the maintainer of a Python library which is ported to Ruby, I complain equally about eggs and gems. This isn't really the place for it, but as near as I can tell, the use of gems requires you to know whether the user has installed your dependency in the system install or through a gem *at the time you write your code*, so you know whether to write "require 'dep'" or "require 'rubygems'; gem 'dep'". This is, IMHO, even worse than the "setuptools breaks PYTHONPATH" complaint you cited. > Note that Ruby software is not too hard to include in operating > system packaging schemes -- my Ubuntu Hardy apt-cache shows plenty of > Ruby software. Yes, but that software is not installed using the gem management system, as I confirmed with a recent conversation with my package manager while we were talking about http://bugs.debian.org/470282 , a quirk which was hopefully a one-time API breakage, but certainly has not endeared me to rubygems any further. I'm sure we could find other people to complain if we look around a little more. I know I have commiserators out there. But, stepping back a bit: You're right in believing that it is neigh impossible to distribute Ruby software without providing gems. So much of your userbase expects it, especially when you're distributing a library which their applications will in turn depend on, because *their* users will expect gems, and they need to be able to use gems to install the dependency. setuptools seems to perform slightly better here, as, by merely making sure my pypi entry has a reachable download_url, my package seems to be available for installation by setuptools users. Nonetheless, I get a recurring stream of requests for egg distribution from people who believe eggs have manifest destiny, and as we heard recently, that "the controversy is over." Meanwhile, I beg their continued forgiveness for being hesitant to require my users to use something not in the standard library for something as fundamental as "setup.py install." These folks are the same who gave me bug reports when I put a .tar.bz2 link to my pypi entry, because apparently -- even though bz2 extraction has been a feature of GNU tar for years -- setuptools (which uses the standard library tarfile module) on some platforms cannot uncompress bz2 packages. the conclusion I am trying to reach here is this: as a Python package maintainer, I have no idea what the hell to do to satisfy my users, from those who are using python 2.3 and have no desire for any new packaging or import semantics, to those who don't mind having a new ez_setup downloaded on install. The people who have found advantages to using the egg-based distribution system are not going away. Providing something in the standard library will provide clear guidance for me, and relieve me of the fear that I am pushing surprising (.pth) or non-standard installation behavior on my users. so, I hope you work something out. Love, - Kevin From janssen at parc.com Thu Mar 20 17:12:51 2008 From: janssen at parc.com (Bill Janssen) Date: Thu, 20 Mar 2008 09:12:51 PDT Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module) In-Reply-To: <47E27482.9090908@v.loewis.de> References: <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <47E1B858.1090107@v.loewis.de> <47E27482.9090908@v.loewis.de> Message-ID: <08Mar20.091252pdt."58696"@synergy1.parc.xerox.com> > although the word "trove" means nothing to me http://www.askoxford.com/concise_oed/trove?view=uk "a store of valuable or delightful things" Bill From lxander.m at gmail.com Thu Mar 20 14:50:58 2008 From: lxander.m at gmail.com (Alexander Michael) Date: Thu, 20 Mar 2008 09:50:58 -0400 Subject: [Python-Dev] [Distutils] Capsule Summary of Some Packaging/Deployment Technology Concerns In-Reply-To: <47E1908F.5030501@taupro.com> References: <47DEEC6B.5040602@taupro.com> <20080318004718.56E173A40FF@sparrow.telecommunity.com> <47E0D571.2070004@taupro.com> <20080319152101.730BC3A40A5@sparrow.telecommunity.com> <47E1908F.5030501@taupro.com> Message-ID: <525f23e80803200650w555fd368va19d749b429d3969@mail.gmail.com> On Wed, Mar 19, 2008 at 6:15 PM, Jeff Rush wrote: > Frankly I'd like to see setuptools exploded, with those parts of general use > folded back into the standard library, the creation of a set of > non-implementation-specific documents of the distribution formats and > behavior, leaving a small core of one implementation of how to do it and the > door open for others to compete with their own implementation. If I hazard an opinion seconding this sentiment. In my use of setuptools, it definitely feels like it wants to be three (mostly) independent projects: 1) The project that standardizes the concept now embodied by eggs and provides the basic machinery to work with them (find them, introspect metadata, "import" them, etc.), but not install them per se. This is generally useful as common plug-in framework, if nothing else. Currently, this "run-time support" functionality is in pkg_resources. 2) The tool you can use to build eggs (but not install them per se). Currently this is the setuptools extension to distutils. 3) The tool for installing eggs (or their equivalent) and (optionally) their dependencies (optionally using remote hosts) as well as uninstalling. Currently this is easy_install (well, except for uninstalling, which is understandable quite difficult). Finally, there is the fourth and already separate project of PyPI: 4) The hosted repository of publicly available eggs (or their equivalent). This should export any metadata required to resolve dependencies relatively cheeply. Breaking them apart will make it easier to have two separate projects for building eggs (or their equivalents) -- one based on distutils and the other replacing it. Even more importantly, it will make it possible for multiple installers to be developed that scratch particular itches. Hopefully one would eventually emerge as the de-facto standard, but this will ultimately be decided by community adoption. Alex From martin at v.loewis.de Thu Mar 20 17:31:50 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 20 Mar 2008 11:31:50 -0500 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <47E26A20.1090402@palladion.com> References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E26A20.1090402@palladion.com> Message-ID: <47E29176.9050109@v.loewis.de> > I'll note that I use easy_install *only* to work in *non-system* > locations: if I want to install Python packages to /usr/lib/python2.x/, > I use the standard system installer, e.g. 'apt-get install > python-frobnatz'. This is probably not the Windows way of doing things (at least not how I use Windows). Windows doesn't really have the notion of "system location" (or multiple levels of them, where \Windows and \Windows\system32 is "more system" than \Program Files, say). Windows users typically view the entire system as "theirs", and have no concerns at all about putting things into Program Files, system32, or, for that matter, \python25. In fact, they don't care or even know where stuff ends up - they expect that the system will find it, and they expect that they can remove anything they installed even without known where it is - because there is a standard place to look for uninstalling things. Of course, setuptools is not the only piece of software that doesn't play well, so Windows users collect all kinds of cruft over time. Eventually, C: will run out of disk space, and they either get a new machine, or reinstall from scratch. > I wonder if a GUI for managing the add-ons would fit the bill, as an > alternative to packaging them as though they were standalone programs? On Windows, it is fairly easy to have an uninstaller registered. There are wrappers to managing that (such as MSI), but it's really only a set of registry keys that needs to get written on installation time, one of them being the command to run on uninstallation. Assuming that you uninstall the package before uninstalling Python, that uninstall program could be a Python script (although using a cmd.exe batch file would probably be more resilient). The concern with "you just need to delete the folder" is "how am I supposed to know that? and can I be really sure?". If you run the official uninstall procedure, and it messes things up, you can complain to setuptools, or the package author that uninstallation "doesn't work". If you delete stuff manually, and you forgot to remove something in a remote location you didn't even know it existed, you still think it's your own fault. So people are hesitant to actually execute the procedure. Of course, once you *do* provide an entry to "Add/Remove Programs", uninstalling won't be mere deletion, as mere deletion would still leave these registry keys behind (although Windows got more resilient over time to provide cleanup in that case: I believe it offers to remove the ARP entry if the uninstall program has been removed) Regards, Martin From martin at v.loewis.de Thu Mar 20 17:42:26 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 20 Mar 2008 11:42:26 -0500 Subject: [Python-Dev] platform management In-Reply-To: <08Mar20.070752pdt."58696"@synergy1.parc.xerox.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEE20@EXMBX04.exchhosting.com> <-3692898589838545524@unknownmsgid> <08Mar20.070752pdt."58696"@synergy1.parc.xerox.com> Message-ID: <47E293F2.2070000@v.loewis.de> > Looking at http://docs.python.org/lib/module-os.html, I find the following: > > name > > The name of the operating system dependent module imported. The > following names have currently been registered: 'posix', 'nt', 'mac', > 'os2', 'ce', 'java', 'riscos'. > > This implies that there's a registry somewhere? This is actually the list of names that the "os" module may take. There are different implementations of the os module, so instead of "import os", you could write "import posix", "import nt", "import ce" (assuming you run on one of these systems). So it really has not much to do with the name of the operating system, but rather with the name Python gives to the API. Regards, Martin From fdrake at acm.org Thu Mar 20 17:43:49 2008 From: fdrake at acm.org (Fred Drake) Date: Thu, 20 Mar 2008 12:43:49 -0400 Subject: [Python-Dev] Trove classifiers In-Reply-To: <20080320155533.GB12099@phd.pp.ru> References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <47E1B858.1090107@v.loewis.de> <47E27482.9090908@v.loewis.de> <20080320154355.GI60713@nexus.in-nomine.org> <20080320155533.GB12099@phd.pp.ru> Message-ID: On Mar 20, 2008, at 11:55 AM, Oleg Broytmann wrote: > Yes, exactly. Eric Raymond claims to be the inventor, but there are > different voices against him: > http://damagestudios.net/blog/2005/08/15/sourceforge-founders That contests that Raymond was an "architectural granddaddy of SourceForge", not that he invented Trove. My understanding is that he did start the efforts to define the Trove classifiers as part of a larger effort that never panned out, but that defining the classifiers was not a solo effort. -Fred -- Fred Drake From martin at v.loewis.de Thu Mar 20 17:48:59 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 20 Mar 2008 11:48:59 -0500 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <20080320153120.7384B3A40A5@sparrow.telecommunity.com> References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E26A20.1090402@palladion.com> <20080320153120.7384B3A40A5@sparrow.telecommunity.com> Message-ID: <47E2957B.50501@v.loewis.de> > Actually, if someone were to develop a patch for PyPI to do this, we > could perhaps have a "display download dependencies" link for eggs > shown on PyPI. That way, someone who wants to do a manual download > could get a page with links for all the required eggs, and manually > download them. Just to make this position a bit more official (as one of the PyPI maintainers): it would be fully within the scope of PyPI to integrate dependency tracking into its database, and present it in any form that is desired. Any such feature would have to be contributed. Regards, Martin From wolever at cs.toronto.edu Thu Mar 20 16:49:13 2008 From: wolever at cs.toronto.edu (David Wolever) Date: Thu, 20 Mar 2008 11:49:13 -0400 Subject: [Python-Dev] 2to3 and print function In-Reply-To: References: <2A4672AB-A1FC-4CB6-A9C4-20B5059513B4@cs.toronto.edu> <43aa6ff70803191544n2ef94e60ye89312dd3243051e@mail.gmail.com> <18762F56-39FC-45C3-9DCD-D60DCB1E541C@cs.toronto.edu> Message-ID: <14056503-C776-4D07-A931-1F9F70723A2F@cs.toronto.edu> On 20-Mar-08, at 2:15 AM, Guido van Rossum wrote: > On Wed, Mar 19, 2008 at 10:27 PM, David Wolever > wrote: >> Why not, instead of trying both parsers, scan for a __future__ >> import, then do the Right Thing based on that? > Different use cases. > Auto-detection based on __future__ would be a good thing to have > (assuming that __future__ statement has actually been implemented :-). The __future__ statement has been implemented, so __future__ auto- detection will come shortly :) From ncoghlan at gmail.com Thu Mar 20 18:36:38 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 21 Mar 2008 03:36:38 +1000 Subject: [Python-Dev] Checking for the -3 flag from Python code In-Reply-To: References: <47E0FE3B.8030006@gmail.com> Message-ID: <47E2A0A6.9090802@gmail.com> Steven Bethard wrote: > On Wed, Mar 19, 2008 at 5:51 AM, Nick Coghlan wrote: >> This flag is exposed to python code as sys.flags.py3k_warning >> >> So the hack added to some of the test code that I saw go by on >> python-checkins isn't needed :) > > Excellent. I asked around at the sprints and everyone thought it was > unexposed. If no one else has already done it, I'll remove the hacks > from test_py3kwarn and the regrtest skipping mechanism. Brett's subsequent checkin pointed out that that particular flag is exposed even more directly as sys.py3kwarning, in addition to being accessible via the general 'command line flags' object. The downside of the module level attribute is that it gives the illusion of being writable without actually having any effect when writing to it. The inconsistent spelling between sys.py3kwarning and sys.flags.py3k_warning is also a minor irritation - are we actually gaining anything by having both mechanisms for accessing the flag value? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From facundobatista at gmail.com Thu Mar 20 18:37:43 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 20 Mar 2008 12:37:43 -0500 Subject: [Python-Dev] Improved thread switching In-Reply-To: <20080320155320.GB8933@mcnabbs.org> References: <20080319115107.YPWO11090.viefep31-int.chello.at@zion.visotech.at> <799854773.351205944939606.OPEN-XCHANGE.WebMail.tomcat@morpheus.visotech.at> <4222a8490803200658t5410422dn925f32d6eb6ec280@mail.gmail.com> <20080320155320.GB8933@mcnabbs.org> Message-ID: 2008/3/20, Andrew McNabb : > Since we officially encourage people to spawn processes instead of > threads, I think that this would be a great idea. The processing module > has a similar API to threading. It's easy to use, works well, and most > importantly, gives us some place to point people to when they complain > about the GIL. I'm +1 to include the processing module in the stdlib. just avoid confussions, with these libraries with alike names, I'm meaning this [1] module, the one that emulates the semantics of threading module. Does anybody has strong reasons for this module to not get included? Regards, [1] http://pypi.python.org/pypi/processing -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From mal at egenix.com Thu Mar 20 18:43:37 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 20 Mar 2008 18:43:37 +0100 Subject: [Python-Dev] Consistent platform name for 64bit windows (was: distutils.util.get_platform() for Windows) In-Reply-To: <47E25EB9.3000301@egenix.com> References: <3504.63.250.241.10.1205859937.squirrel@mail.eftel.com> <47E23FA8.3090104@egenix.com> <47E25EB9.3000301@egenix.com> Message-ID: <47E2A249.3010109@egenix.com> On 2008-03-20 13:55, M.-A. Lemburg wrote: > On 2008-03-20 13:42, Thomas Heller wrote: >> M.-A. Lemburg schrieb: >>> About the platform.py changes: if someone could provide the return >>> values of sys.getwindowsversion() for 64bit versions of Windows >>> XP and Vista, I could add support for it (don't have a 64bit version >>> of Windows available to check myself). >> This is the output of a 32-bit Python running on "Windows XP Professional >> x64 Edition, Version 2003, Service Pack 2": >> >> C:\Python24>ver >> >> Microsoft Windows [Version 5.2.3790] >> >> C:\Python24>python >> Python 2.4.4 (#71, Oct 18 2006, 08:34:43) [MSC v.1310 32 bit (Intel)] on win32 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> import sys >>>>> sys.getwindowsversion() >> (5, 2, 3790, 2, 'Service Pack 2') > > Thank you ! > > Anyone with a 64bit Vista ? > > Or even better: a page documenting what to expect from the system call > behind the sys.getwindowsversion() API ? FYI: I added winreg and sys.getwindowsversion() support in r61674. platform.machine() and .processor() will now use the environment variables PROCESSOR_ARCHITECTURE and PROCESSOR_IDENTIFIER where available (should work on Windows XP and later). According to http://support.microsoft.com/kb/888731 platform.machine() will return "AMD64", so I guess the "x64" string is just a marketing name for 64-bit platforms on Windows and the underlying system uses "AMD64" as machine type name. For x86 processors, you'll now get "x86" on Windows XP and later. For Itanium processors, you should get "IA64" according to this WOW64 page: http://msdn2.microsoft.com/en-us/library/aa384274(VS.85).aspx -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 20 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From jnoller at gmail.com Thu Mar 20 18:45:37 2008 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 20 Mar 2008 13:45:37 -0400 Subject: [Python-Dev] Improved thread switching In-Reply-To: References: <20080319115107.YPWO11090.viefep31-int.chello.at@zion.visotech.at> <799854773.351205944939606.OPEN-XCHANGE.WebMail.tomcat@morpheus.visotech.at> <4222a8490803200658t5410422dn925f32d6eb6ec280@mail.gmail.com> <20080320155320.GB8933@mcnabbs.org> Message-ID: <4222a8490803201045wfedfb32sa48215283d62c2cf@mail.gmail.com> Even I, as a strong advocate for it's inclusion think I should finish the PEP and outline all of the questions/issues that may come out of it. On Thu, Mar 20, 2008 at 1:37 PM, Facundo Batista wrote: > 2008/3/20, Andrew McNabb : > > > > Since we officially encourage people to spawn processes instead of > > threads, I think that this would be a great idea. The processing module > > has a similar API to threading. It's easy to use, works well, and most > > importantly, gives us some place to point people to when they complain > > about the GIL. > > I'm +1 to include the processing module in the stdlib. > > just avoid confussions, with these libraries with alike names, I'm > meaning this [1] module, the one that emulates the semantics of > threading module. > > Does anybody has strong reasons for this module to not get included? > > Regards, > > [1] http://pypi.python.org/pypi/processing > > -- > . Facundo > > Blog: http://www.taniquetil.com.ar/plog/ > PyAr: http://www.python.org/ar/ > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com > From ncoghlan at gmail.com Thu Mar 20 19:04:45 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 21 Mar 2008 04:04:45 +1000 Subject: [Python-Dev] Improved thread switching In-Reply-To: References: <20080319115107.YPWO11090.viefep31-int.chello.at@zion.visotech.at> <799854773.351205944939606.OPEN-XCHANGE.WebMail.tomcat@morpheus.visotech.at> <4222a8490803200658t5410422dn925f32d6eb6ec280@mail.gmail.com> <20080320155320.GB8933@mcnabbs.org> Message-ID: <47E2A73D.6060303@gmail.com> Facundo Batista wrote: > 2008/3/20, Andrew McNabb : > >> Since we officially encourage people to spawn processes instead of >> threads, I think that this would be a great idea. The processing module >> has a similar API to threading. It's easy to use, works well, and most >> importantly, gives us some place to point people to when they complain >> about the GIL. > > I'm +1 to include the processing module in the stdlib. > > just avoid confussions, with these libraries with alike names, I'm > meaning this [1] module, the one that emulates the semantics of > threading module. > > Does anybody has strong reasons for this module to not get included? Other than the pre-release version number and the fact that doing such a thing would require R. Oudkerk to actually make the offer rather than anyone else? There would also need to be the usual thing of at least a couple of people stepping up and being willing to maintain it. I also wouldn't mind seeing some performance figures for an application that was limited to making good use of only one CPU when run with the threading module, but was able to exploit multiple processors to obtain a speed improvements when run with the processing module. That said, I'm actually +1 on the general idea, since I always write my threaded Python code using worker threads that I communicate with via Queue objects. Pyprocessing would be a great way for me to scale to multiple processors if I was running CPU intensive tasks rather than potentially long-running hardware IO operations (I've been meaning to check it out for a long time, but have never actually needed to for either work or any home projects). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From stephen at xemacs.org Thu Mar 20 19:14:37 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 21 Mar 2008 03:14:37 +0900 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <47E24C7C.6080209@v.loewis.de> References: <47E0D883.9030402@taupro.com> <47E11659.9000307@v.loewis.de> <47E19ABE.2040800@taupro.com> <47E1BD14.2030805@v.loewis.de> <47E1EAE4.90802@taupro.com> <47E24C7C.6080209@v.loewis.de> Message-ID: <871w65gt4i.fsf@uwakimon.sk.tsukuba.ac.jp> "Martin v. L?wis" writes: > I don't think any of the core committers is qualified to write such > a document. Instead, it would have to be written by people who > *actually* ported a project from 2 to 3 in some form. Note that this is precisely the kind of experience Skip Montanaro is talking about trying to generate in the context of SpamBayes in a thread on the python-3000 list entitled "Strategy for porting to 3.0?" I don't know if he plans to write a HOWTO himself, but he certainly intends to keep a lab notebook that will be of help in writing such a document. From collinw at gmail.com Thu Mar 20 19:17:44 2008 From: collinw at gmail.com (Collin Winter) Date: Thu, 20 Mar 2008 11:17:44 -0700 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <47E1BB1D.4060303@v.loewis.de> References: <47E0D883.9030402@taupro.com> <47E11659.9000307@v.loewis.de> <47E1BB1D.4060303@v.loewis.de> Message-ID: <43aa6ff70803201117y46e09c5frd6e5be6e4902b23c@mail.gmail.com> On Wed, Mar 19, 2008 at 6:17 PM, "Martin v. L?wis" wrote: > >> You seem to be implying that some projects may release separate > >> source distributions. I cannot imagine why somebody would want to > >> do that. > > > > That's odd. I can't imagine why anybody would *not* want to do that. > > Given the number of issues 2to3 can't fix (because it would be too > > dangerous to guess) > > Like which one specifically? Anything having to do with the str->bytes/unicode->str move is so far off-limits to 2to3. Collin Winter From steve at holdenweb.com Thu Mar 20 19:23:24 2008 From: steve at holdenweb.com (Steve Holden) Date: Thu, 20 Mar 2008 14:23:24 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <47E29176.9050109@v.loewis.de> References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E26A20.1090402@palladion.com> <47E29176.9050109@v.loewis.de> Message-ID: Martin v. L?wis wrote: Martin v. L?wis wrote: >> I'll note that I use easy_install *only* to work in *non-system* >> locations: if I want to install Python packages to /usr/lib/python2.x/, >> I use the standard system installer, e.g. 'apt-get install >> python-frobnatz'. > > This is probably not the Windows way of doing things (at least not how > I use Windows). Windows doesn't really have the notion of "system > location" (or multiple levels of them, where \Windows and > \Windows\system32 is "more system" than \Program Files, say). > > Windows users typically view the entire system as "theirs", and > have no concerns at all about putting things into Program Files, > system32, or, for that matter, \python25. In fact, they don't care > or even know where stuff ends up - they expect that the system will > find it, and they expect that they can remove anything they installed > even without known where it is - because there is a standard place > to look for uninstalling things. > > Of course, setuptools is not the only piece of software that doesn't > play well, so Windows users collect all kinds of cruft over time. > Eventually, C: will run out of disk space, and they either get a > new machine, or reinstall from scratch. > >> I wonder if a GUI for managing the add-ons would fit the bill, as an >> alternative to packaging them as though they were standalone programs? > > On Windows, it is fairly easy to have an uninstaller registered. There > are wrappers to managing that (such as MSI), but it's really only a > set of registry keys that needs to get written on installation time, > one of them being the command to run on uninstallation. > > Assuming that you uninstall the package before uninstalling Python, that > uninstall program could be a Python script (although using a cmd.exe > batch file would probably be more resilient). > > The concern with "you just need to delete the folder" is "how am I > supposed to know that? and can I be really sure?". If you run the > official uninstall procedure, and it messes things up, you can complain > to setuptools, or the package author that uninstallation "doesn't work". > > If you delete stuff manually, and you forgot to remove something in > a remote location you didn't even know it existed, you still think > it's your own fault. So people are hesitant to actually execute the > procedure. > > Of course, once you *do* provide an entry to "Add/Remove Programs", > uninstalling won't be mere deletion, as mere deletion would still > leave these registry keys behind (although Windows got more resilient > over time to provide cleanup in that case: I believe it offers to > remove the ARP entry if the uninstall program has been removed) > > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org > >> I'll note that I use easy_install *only* to work in *non-system* >> locations: if I want to install Python packages to /usr/lib/python2.x/, >> I use the standard system installer, e.g. 'apt-get install >> python-frobnatz'. > > This is probably not the Windows way of doing things (at least not how > I use Windows). Windows doesn't really have the notion of "system > location" (or multiple levels of them, where \Windows and > \Windows\system32 is "more system" than \Program Files, say). > > Windows users typically view the entire system as "theirs", and > have no concerns at all about putting things into Program Files, > system32, or, for that matter, \python25. In fact, they don't care > or even know where stuff ends up - they expect that the system will > find it, and they expect that they can remove anything they installed > even without known where it is - because there is a standard place > to look for uninstalling things. > In point of fact, for an *end user* it makes increasing sense to use application installers that automatically install a correct-version interpreter and all dependencies in a stand-alone manner (i.e. explicitly *not* sharing anything with any other installed application. This makes uninstall much easier, as the lack of external dependencies eases version lock-step problems. It would pain me, as a computer scientist, to do this, but I honestly believe it may be the way forward -- just think, it wouldn't even matter whether an application (and all its extension modules) had been built with VS2003, VS2008 or Mingw. People misunderstood when Mike Driscoll started to provide pure-Python modules as Windows installers, but increasingly your naive Windows programmer is going to be happier doing that. I'm not sure whether that provides easy_f**king_uninstall (Zed Shaw will live on in my memory for that particular PyCon moment), but it (ought to be) relatively easy to do so. Extension modules for programmers still offer more of a challenge, but a build-farm for extension module writers could help there. > Of course, setuptools is not the only piece of software that doesn't > play well, so Windows users collect all kinds of cruft over time. > Eventually, C: will run out of disk space, and they either get a > new machine, or reinstall from scratch. > As someone who just gave away a Windows laptop because I'd become sick of the sight of it I sing amen to that. >> I wonder if a GUI for managing the add-ons would fit the bill, as an >> alternative to packaging them as though they were standalone programs? > > On Windows, it is fairly easy to have an uninstaller registered. There > are wrappers to managing that (such as MSI), but it's really only a > set of registry keys that needs to get written on installation time, > one of them being the command to run on uninstallation. > > Assuming that you uninstall the package before uninstalling Python, that > uninstall program could be a Python script (although using a cmd.exe > batch file would probably be more resilient). > > The concern with "you just need to delete the folder" is "how am I > supposed to know that? and can I be really sure?". If you run the > official uninstall procedure, and it messes things up, you can complain > to setuptools, or the package author that uninstallation "doesn't work". > I quite agree. I believe it might help if we clearly distinguished between "developer install", to add required functionality to a specific Python installation (and here distutils is almost good enough, setuptools begins to seem like overkill), and "application install" to add a bundle of executable functionality to a computer for an end-user. > If you delete stuff manually, and you forgot to remove something in > a remote location you didn't even know it existed, you still think > it's your own fault. So people are hesitant to actually execute the > procedure. > > Of course, once you *do* provide an entry to "Add/Remove Programs", > uninstalling won't be mere deletion, as mere deletion would still > leave these registry keys behind (although Windows got more resilient > over time to provide cleanup in that case: I believe it offers to > remove the ARP entry if the uninstall program has been removed) > We need to stop protesting that our installation tools are easy enough and try to get behind the various platforms, be it with Windows installers, rpms, or other support. We probably aren't doing this because it's work nobody particularly relishes, and has relatively low visibility in the developer world. Non-developer Python programmers and end-users would thank us, though. regards Steve From pje at telecommunity.com Thu Mar 20 20:09:14 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 20 Mar 2008 15:09:14 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <79990c6b0803201055l2831624q5e0fd1117f1ec338@mail.gmail.com > References: <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E26A20.1090402@palladion.com> <79990c6b0803201055l2831624q5e0fd1117f1ec338@mail.gmail.com> Message-ID: <20080320190933.A03A83A4074@sparrow.telecommunity.com> At 05:55 PM 3/20/2008 +0000, Paul Moore wrote: >It's not that I object to the existence of automatic dependency >management, I object to being given no choice, as if my preference for >handling things manually is unacceptable. Note that easy_install has a --no-deps option, and you can make it the default in your distutils.cfg file. Also, setuptools-based packages *can* build bdist_wininst installers. (In fact, if memory serves, I added that feature at your request.) Personally, I'm not very thrilled with the number of complaints on this thread that could be resolved by RTFMing. There are extensive manuals, and they do contain the information that some people are saying isn't there. In several cases that I've seen here today alone, there are actually *entries in the tables of contents* that name the precise thing people here are characterizing as undocumented or even *impossible*, like: * Making your package available for EasyInstall * Installing on Un-networked Machines * Custom Installation Locations * Restricting Downloads with --allow-hosts It's easy to get the impression that people not only didn't RTFM, they didn't even Read The Friendly Table Of Contents of the said M. Nor, when, they found something in the manual that they didn't understand, write to the distutils-sig to ask anybody to explain, and perhaps suggest ways the FM's could be improved. From barry at python.org Thu Mar 20 20:42:22 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 20 Mar 2008 14:42:22 -0500 Subject: [Python-Dev] Python source code on Bazaar vcs Message-ID: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm happy to announce that we now have available for public consumption, the Python source code for 2.5, 2.6 and 3.0 available under the Bazaar distributed version control system. The current Subversion repository is still the master copy of the source code. We have not made a decision to move to Bazaar officially, nor have we made a decision to even move off of Subversion. We're making these branches available exactly so that you, the Python developer community, can kick the tires and see if it makes sense to move to a different vcs. Nothing will happen until after the Python 2.6/3.0 releases anyway. All the gory details are documented here: http://www.python.org/dev/bazaar These branches are available both for core Python developers with commit privileges, and the wider world of developers without commit privileges. It's this latter group that I think will find the most compelling immediate benefit from using Bazaar, because they will no longer need to maintain their own changes using a mass of patch files. For more information on Bazaar in general, see: http://bazaar-vcs.org You will probably be most interested in the Bazaar mirrors of the Subversion master repository. We have a cron job that updates Bazaar from Subversion every 15 minutes. It is also possible to push changes made in your Bazaar branches into the Subversion master, so you can keep reasonably up-to-date and interact with the Python source code solely via Bazaar. Please let me know if you have any questions or anything in the docs referenced above aren't clear. I know I need to document the Bazaar- >Subversion workflow in more detail. Huge thanks go out especially to Thomas Wouters who sprinted with me yesterday on getting the whole infrastructure up and running. Thanks also to Martin v. Loewis, Sean Reifschneider, and the folks here at Pycon from the Bazaar project, Ian, Andrew, John, and Edwin. Enjoy, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR+K+H3EjvBPtnXfVAQL6bwP/d0XKx0mRPgzdOD6zCPwwRl8y2kxWV9Vl zVwN07cK8IlwMa9M470MsERuXuD8hmeXnPgnrUcrX9HciwldmY8C33t0f8GaOk1J iD4Od1midlIaJJiMd+JcFk9NbX3Rwc4HMj3s8jKjjdlGrFe77ei9DCZ/HMl/iG5K fyyjXo4HLEE= =Gcjq -----END PGP SIGNATURE----- From rowen at cesmail.net Thu Mar 20 20:52:12 2008 From: rowen at cesmail.net (Russell E. Owen) Date: Thu, 20 Mar 2008 12:52:12 -0700 Subject: [Python-Dev] Could someone please review patch 799428: fix Tkinter tk_focusNext? Message-ID: Patch is a trivial fix to a long-standing issue with Tkinter: calls to the widget method tk_focusNext() fail with "unsubscriptable object" error. Also issue 1068881 can be closed. This is a case where procrastination paid off. Regards, -- Russell From martin at v.loewis.de Thu Mar 20 21:33:22 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 20 Mar 2008 15:33:22 -0500 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <43aa6ff70803201117y46e09c5frd6e5be6e4902b23c@mail.gmail.com> References: <47E0D883.9030402@taupro.com> <47E11659.9000307@v.loewis.de> <47E1BB1D.4060303@v.loewis.de> <43aa6ff70803201117y46e09c5frd6e5be6e4902b23c@mail.gmail.com> Message-ID: <47E2CA12.2000506@v.loewis.de> >>>> You seem to be implying that some projects may release separate >> >> source distributions. I cannot imagine why somebody would want to >> >> do that. >> > >> > That's odd. I can't imagine why anybody would *not* want to do that. >> > Given the number of issues 2to3 can't fix (because it would be too >> > dangerous to guess) >> >> Like which one specifically? > > Anything having to do with the str->bytes/unicode->str move is so far > off-limits to 2to3. Sure - but does that mean you need to separate code bases? Why does this move prevent people from running the same code in 2.x and 3.x? In 2.x, they should use Unicode objects for text and regular strings for binary data, and such code will run fine after converted by 2to3. Regards, Martin From p.f.moore at gmail.com Thu Mar 20 21:34:18 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 20 Mar 2008 20:34:18 +0000 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <20080320190933.A03A83A4074@sparrow.telecommunity.com> References: <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E26A20.1090402@palladion.com> <79990c6b0803201055l2831624q5e0fd1117f1ec338@mail.gmail.com> <20080320190933.A03A83A4074@sparrow.telecommunity.com> Message-ID: <79990c6b0803201334t4cf8531y2af40edc3e469fe8@mail.gmail.com> On 20/03/2008, Phillip J. Eby wrote: > At 05:55 PM 3/20/2008 +0000, Paul Moore wrote: > >It's not that I object to the existence of automatic dependency > >management, I object to being given no choice, as if my preference for > >handling things manually is unacceptable. > > Note that easy_install has a --no-deps option, and you can make it > the default in your distutils.cfg file. Sorry, I wasn't clear. There's context that helps, but even with it I wasn't clear. Tres told me how to download all the dependencies for use offline (which I actually knew, but that's not the point here). I clarified that I didn't want to use dependency management but preferred to manage dependencies manually. I then went on to say that putting dependency information in setup.exe and expecting users to use automatic dependency resolution encourages developers to omit dependency details from documentation (to an extent I can't quantify, but I believe is non-zero). That lack of documentation "forces" me to rely on the automatic process. THAT is the thing that removes my choice, not easy_install's ability to skip dependency checking. I'm sorry I wasn't clearer - but in my defence, my posting was pretty long already and I was trying to be brief... > Also, setuptools-based packages *can* build bdist_wininst > installers. (In fact, if memory serves, I added that feature at your request.) I know. python setup.py bdist_wininst. And thank you for adding it. But again you miss my point. People are starting to omit distributing bdist_wininst installers in favour of eggs only. And you cannot (to my knowledge) convert an egg into a bdist_wininst installer, and if you can't compile from source (a C extension with complex dependencies, for example) you're stuck (in the sense that you're forced to use eggs without add/remove programs support). > Personally, I'm not very thrilled with the number of complaints on > this thread that could be resolved by RTFMing. There are extensive > manuals, and they do contain the information that some people are > saying isn't there. In several cases that I've seen here today > alone, there are actually *entries in the tables of contents* that > name the precise thing people here are characterizing as undocumented > or even *impossible*, like: > > * Making your package available for EasyInstall > * Installing on Un-networked Machines > * Custom Installation Locations > * Restricting Downloads with --allow-hosts > > It's easy to get the impression that people not only didn't RTFM, > they didn't even Read The Friendly Table Of Contents of the said > M. Nor, when, they found something in the manual that they didn't > understand, write to the distutils-sig to ask anybody to explain, and > perhaps suggest ways the FM's could be improved. As I said, I know there is documentation, but (a) it's very long, and (b) it's full of jargon that you need to understand before you can follow it. Believe me, I've tried to read it. But ultimately, all I'm trying to do is work out how to do something that is as simple as "download exe, run it" (on a PC with browser-only access to the internet) in the bdist_wininst world. That's all. I'm equally not very thrilled at having to read many pages of dense documentation to find out how to do this. Heck, I read the documentation twice, and asked on the distutils-sig, before I worked out how to do easy_install -zmax (and the only reason I can remember that now without looking it up is that "zmax" is memorable - z plus the word max - I have no idea without going back to the manual what the individual options do). I'd say that the documentation needs to be better. (And I said how - a tutorial-style summary. What more should I do short of writing it myself?) Honestly, I'm trying to help improve (by my measure of improvement, certainly) setuptools. I've done as much (more!) homework as I feel is appropriate (no, I haven't studied the whole manual all the way through). Being treated as if it's my fault, and I haven't done enough, is both discouraging and to be honest, somewhat offensive. I'm going to quit this thread for a while now, as I don't want to turn things into a flamewar. I believe Tres took my points on board. I hope others have, too. I certainly don't expect everything I say to be taken as gospel, so that's enough for now. Paul. From bkline at rksystems.com Thu Mar 20 21:50:22 2008 From: bkline at rksystems.com (Bob Kline) Date: Thu, 20 Mar 2008 16:50:22 -0400 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <47E2CA12.2000506@v.loewis.de> References: <47E0D883.9030402@taupro.com> <47E11659.9000307@v.loewis.de> <47E1BB1D.4060303@v.loewis.de> <43aa6ff70803201117y46e09c5frd6e5be6e4902b23c@mail.gmail.com> <47E2CA12.2000506@v.loewis.de> Message-ID: <47E2CE0E.6010108@rksystems.com> Martin v. L?wis wrote: >> Anything having to do with the str->bytes/unicode->str move is so far >> off-limits to 2to3. >> > > Sure - but does that mean you need to separate code bases? > > Why does this move prevent people from running the same > code in 2.x and 3.x? In 2.x, they should use Unicode objects > for text and regular strings for binary data, and such code > will run fine after converted by 2to3. > Not if it includes code that looks like this: if type(response) in (str, unicode): ..... and it's really true that "[a]nything having to do with the str->bytes/unicode->str move is so far off-limits" to the upgrade tool. -- Bob Kline http://www.rksystems.com mailto:bkline at rksystems.com From lists at cheimes.de Thu Mar 20 21:58:33 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 20 Mar 2008 21:58:33 +0100 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> Message-ID: <47E2CFF9.6010400@cheimes.de> Barry Warsaw schrieb: > I'm happy to announce that we now have available for public > consumption, the Python source code for 2.5, 2.6 and 3.0 available > under the Bazaar distributed version control system. Thank you very much to Barry and the rest of team! Great work! Ubuntu users have to install a newer version of bzr before they can check out the sources: sudo vi /etc/apt/sources.list.d/bzr.list --- add --- deb http://ppa.launchpad.net/bzr/ubuntu gutsy main deb-src http://ppa.launchpad.net/bzr/ubuntu gutsy main --- eof --- sudo apt-get update # --force-yes because the packages aren't signed yet sudo apt-get --force-yes -y install bzr bzr-gtk bzrtools Also read https://launchpad.net/~bzr/+archive and http://bazaar-vcs.org/DistroDownloads Christian From p.f.moore at gmail.com Thu Mar 20 22:04:36 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 20 Mar 2008 21:04:36 +0000 Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317151637.532D23A409D@sparrow.telecommunity.com> <20080317161305.22B183A409D@sparrow.telecommunity.com> <47DEC0EB.3000404@palladion.com> <440CD260-F6F6-4484-8326-0EA1D7B83132@zooko.com> <79990c6b0803191134had5def2uff0742cc5188b125@mail.gmail.com> Message-ID: <79990c6b0803201404ub931349k1211d33dffbba0e2@mail.gmail.com> On 19/03/2008, Guido van Rossum wrote: > I'd only do what __loader__ offers. People can always wrap a StringIO around it. > > > Once I have a patch, I'll post it to the tracker. What's the best > > approach? Code a patch for 3.0 and backport, or code for 2.6 and let > > the merging process do its stuff? > > Code for 2.6, let the merge do its thing. http://bugs.python.org/issue2439 Let me know if you want any changes. If it's OK, I'll think about whether grander things would be of use. Paul. From barry at python.org Thu Mar 20 22:15:03 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 20 Mar 2008 16:15:03 -0500 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: <47E2CFF9.6010400@cheimes.de> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <47E2CFF9.6010400@cheimes.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 20, 2008, at 3:58 PM, Christian Heimes wrote: > Barry Warsaw schrieb: >> I'm happy to announce that we now have available for public >> consumption, the Python source code for 2.5, 2.6 and 3.0 available >> under the Bazaar distributed version control system. > > Thank you very much to Barry and the rest of team! Great work! > > Ubuntu users have to install a newer version of bzr before they can > check out the sources: > > sudo vi /etc/apt/sources.list.d/bzr.list > --- add --- > deb http://ppa.launchpad.net/bzr/ubuntu gutsy main > deb-src http://ppa.launchpad.net/bzr/ubuntu gutsy main > --- eof --- > > sudo apt-get update > > # --force-yes because the packages aren't signed yet > sudo apt-get --force-yes -y install bzr bzr-gtk bzrtools > > > Also read https://launchpad.net/~bzr/+archive and http://bazaar-vcs.org/DistroDownloads Thanks Christian, I'll add this to the page. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR+LT2nEjvBPtnXfVAQJ+aAP+JNqjnGdgfqszSGDn8dtBppaxf4d486DD 5GLdPUn696nHw0Q2+OzqFbTk8s/qNDyNmVoLuG80TsyrhqMJTidIwyupjxFXvdfI gk/7Ghl1/ky5QEBXfmE0xrql+uoEmoVD+OVxlrzYy8Z9rm22y0EAUN2BOyM9oLYq TsSj2XJSdVM= =XJlg -----END PGP SIGNATURE----- From martin at v.loewis.de Thu Mar 20 22:17:21 2008 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 20 Mar 2008 16:17:21 -0500 Subject: [Python-Dev] svnmerge and added files Message-ID: <47E2D461.30309@v.loewis.de> It seems that recently, a number of merges broke in the sense that files added to the trunk were not merged into the 3k branch. Is that a general problem with svnmerge? Should that be fixed to automatically do a "svn add" when merging changes that included file additions and removals? Regards, Martin From schmir at gmail.com Thu Mar 20 22:26:22 2008 From: schmir at gmail.com (Ralf Schmitt) Date: Thu, 20 Mar 2008 22:26:22 +0100 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> Message-ID: <932f8baf0803201426ua0ae2edu3f7b62eaf22ef787@mail.gmail.com> On Thu, Mar 20, 2008 at 8:42 PM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I'm happy to announce that we now have available for public > consumption, the Python source code for 2.5, 2.6 and 3.0 available > under the Bazaar distributed version control system. > I have also setup a mirror using mercurial: http://hgpy.de/py/ It contains the 2.4, 2.5, trunk and py3k branches (in case anyone wants to compare this to bzr). -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080320/b1a383a9/attachment.htm From lists at cheimes.de Thu Mar 20 22:32:21 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 20 Mar 2008 22:32:21 +0100 Subject: [Python-Dev] svnmerge and added files In-Reply-To: <47E2D461.30309@v.loewis.de> References: <47E2D461.30309@v.loewis.de> Message-ID: <47E2D7E5.9080407@cheimes.de> Martin v. L?wis schrieb: > It seems that recently, a number of merges broke in the sense > that files added to the trunk were not merged into the > 3k branch. > > Is that a general problem with svnmerge? Should that be > fixed to automatically do a "svn add" when merging changes > that included file additions and removals? It sometimes happens when I do a svnmerge, revert the merge with svn revert -R and do a second svnmerge. Files added by the first svnmerge aren't added to the commit list for the second merge. Unfortunately svnmerge.py doesn't warn me when the file already exists. Christian From martin at v.loewis.de Thu Mar 20 22:28:54 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 20 Mar 2008 16:28:54 -0500 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <47E2CE0E.6010108@rksystems.com> References: <47E0D883.9030402@taupro.com> <47E11659.9000307@v.loewis.de> <47E1BB1D.4060303@v.loewis.de> <43aa6ff70803201117y46e09c5frd6e5be6e4902b23c@mail.gmail.com> <47E2CA12.2000506@v.loewis.de> <47E2CE0E.6010108@rksystems.com> Message-ID: <47E2D716.4030109@v.loewis.de> > Not if it includes code that looks like this: > > if type(response) in (str, unicode): ..... > > and it's really true that "[a]nything having to do with the > str->bytes/unicode->str move is so far off-limits" to the upgrade tool. Depends on what the purpose of the test is. If it tests for "is response text", then 2to3 will work just fine on it, converting it to if type(response) in (str, str): This is redundant, but correct. Regards, Martin From jeff at taupro.com Thu Mar 20 22:36:40 2008 From: jeff at taupro.com (Jeff Rush) Date: Thu, 20 Mar 2008 16:36:40 -0500 Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module) In-Reply-To: <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> Message-ID: <47E2D8E8.8090209@taupro.com> Paul Moore wrote: > On 20/03/2008, zooko wrote: > I'll chime in here, too. I really want to like > setuptools/easy_install, but I don't. I'll try to be specific in my > reasons, in the hope that they can be addressed. I know some of these > are "known about", but one of my meta-dislikes of setuptools is that > known issues never seem to get addressed (I know, patches accepted, > but I haven't got the time either...) Clearly explained problems with the existing arrangement is valuable as well as patches. Thanks for taking the time to help us see your viewpoint. > 1. No integration with the system packager (Windows, in my case). If I > do easy_install nose, then nose does not show up in add/remove > programs. That significantly affects the way I manage my PC. Part of this stems from stretching of the original mission of setuptools, to install modules for Python, into a general-purpose application installation tool. The buildout tool is more suited for application installation, although not ideal yet. In your scenario, what happens when one egg pulls in another and another, until you have a hundred entries in your add/remove menu? And you don't understand the inter-relationship of those so you cannot do a clean uninstall? Similarly, or what do you want to appear in that add/remove menu when you are using independent sandboxes with various applications in them, some of which are accessible only to specific users who are not admins on that box? > 3. The pkg_resources documentation (in particular, that's the one I've > tried to follow) is extremely hard to read. Partly this is just style, > but it's partly because it is couched in very unfamiliar terms > (distributions, working sets, interfaces, providers, etc). It's also > *huge*. A tutorial style overview, supported by API detail, would be > far better. We'll get better docs. Of course, that module contains roughly five different sets of functionality, some of which can be used unrelated to the others so it's not just one API. > 4. Hard to use with limited connectivity. At work, I *only* have > access to the internet via Internet Explorer (MS based proxy). There > are workarounds, but ultimately "download an installer, then run it" > is a far simpler approach for me. This is hard to address using independent eggs re setuptools but fits buildout which provides for deployment of a set of related eggs as a single entity. I'll add it as a use case and see what we can do though. > 5. Auto-discovery doesn't always work. I'm sorry, I really can't > recall the example at the moment, but sometimes easy_install says it > can't find a package I *know* is available. I've hit a few of these myself, although it wasn't an issue with the auto-discovery mechanism but rather quality problems with PyPI itself. Some of the eggs only had binary distributions provided, and they were not for my platform so couldn't be used. Better error messages in this area would help, with encouragement to nag the original author to provide better data on PyPI. > 6. Splitting the community. Windows users rely heavily on binary > installers (at least, I do). We're starting to get a situation where > some projects provide .egg files, and some provide traditional > (bdist_wininst/bdist_msi) installers. This is bad. One way to do it, > and all that :-) Reporting and author nagging facilities built into PyPI could help encourage more consistent behavior. -Jeff From lists at cheimes.de Thu Mar 20 22:49:35 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 20 Mar 2008 22:49:35 +0100 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> Message-ID: <47E2DBEF.8010809@cheimes.de> Barry Warsaw schrieb: > I'm happy to announce that we now have available for public > consumption, the Python source code for 2.5, 2.6 and 3.0 available > under the Bazaar distributed version control system. Somebody has to fix the subversion related code in Python/sysmodule.c: heimes at hamiller:~/dev/python/bzr/trunk$ ./python Fatal Python error: subversion keywords missing Aborted (core dumped) Christian From bkline at rksystems.com Thu Mar 20 22:46:08 2008 From: bkline at rksystems.com (Bob Kline) Date: Thu, 20 Mar 2008 17:46:08 -0400 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <47E2D716.4030109@v.loewis.de> References: <47E0D883.9030402@taupro.com> <47E11659.9000307@v.loewis.de> <47E1BB1D.4060303@v.loewis.de> <43aa6ff70803201117y46e09c5frd6e5be6e4902b23c@mail.gmail.com> <47E2CA12.2000506@v.loewis.de> <47E2CE0E.6010108@rksystems.com> <47E2D716.4030109@v.loewis.de> Message-ID: <47E2DB20.5020909@rksystems.com> Martin v. L?wis wrote: > >> Not if it includes code that looks like this: >> >> if type(response) in (str, unicode): ..... >> >> and it's really true that "[a]nything having to do with the >> str->bytes/unicode->str move is so far off-limits" to the upgrade tool. > > Depends on what the purpose of the test is. If it tests for > "is response text", then 2to3 will work just fine on it, converting > it to > > if type(response) in (str, str): Then I'm taking him too literally, when he writes that the tool won't touch *anything* that has to do with the str->bytes/unicode->str move (I assumed that meant it wouldn't touch "unicode" in the snippet I gave above), right? Will the tool also make the following work correctly? if type(s) is str: s = unicode(s, 'utf-8') -- Bob Kline http://www.rksystems.com mailto:bkline at rksystems.com From pje at telecommunity.com Thu Mar 20 22:51:59 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 20 Mar 2008 17:51:59 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <79990c6b0803201334t4cf8531y2af40edc3e469fe8@mail.gmail.com > References: <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E26A20.1090402@palladion.com> <79990c6b0803201055l2831624q5e0fd1117f1ec338@mail.gmail.com> <20080320190933.A03A83A4074@sparrow.telecommunity.com> <79990c6b0803201334t4cf8531y2af40edc3e469fe8@mail.gmail.com> Message-ID: <20080320215202.618173A4074@sparrow.telecommunity.com> At 08:34 PM 3/20/2008 +0000, Paul Moore wrote: >I then went on to say that putting dependency information in setup.exe >and expecting users to use automatic dependency resolution encourages >developers to omit dependency details from documentation (to an extent >I can't quantify, but I believe is non-zero). That lack of >documentation "forces" me to rely on the automatic process. THAT is >the thing that removes my choice, not easy_install's ability to skip >dependency checking. Ah. Fair enough. So, if we get PyPI to display that information, that should fix this problem for you? >People are starting to omit distributing >bdist_wininst installers in favour of eggs only. You mean, they're shipping a .win32.egg, but not an .exe? > And you cannot (to my >knowledge) convert an egg into a bdist_wininst installer, Not at the moment, no. It seems like it ought to be *possible*, though, since the reverse translation can be done. Eggs are more restrictive in what they can include, so the reverse step actually ought to be relatively easy. Indeed, I would think that it could be done by a standalone tool without even using setuptools. All that really needs to happen (I believe) is that the zipfile directory needs all its names prepended with PURELIB or PLATLIB, and then add the appropriate prefix .exe and bdist_wininst extra data on the front of the restructured zip file. In fact, it should probably be possible to write such a tool by subclassing the distutils bdist_wininst command and overriding the run() and get_inidata() methods, using the existing create_exe() method to do that part of the magic. The other tool that would be handy to have, would be one that unpacks eggs into standard distutils-style installation. > > Personally, I'm not very thrilled with the number of complaints on > > this thread that could be resolved by RTFMing. >... >Honestly, I'm trying to help improve (by my measure of improvement, >certainly) setuptools. I've done as much (more!) homework as I feel is >appropriate (no, I haven't studied the whole manual all the way >through). Being treated as if it's my fault, and I haven't done >enough, is both discouraging and to be honest, somewhat offensive. My comment wasn't aimed specifically at you; you're only one of many people today who have appeared to state that something or other wasn't possible or documented, described optional behavior as required, etc. Addressing each and every one point by point looks petty, but then lumping them together like that makes it look like I'm picking on you specifically. Sorry about that. In any event, I'm not saying that anyone hasn't done enough or that it's their fault. The fact that I'm not thrilled about some of the things said in the thread doesn't somehow magically invalidate other people's frustrations, nor was it my intent to accuse you (or anyone) of making up their problems. I'm just expressing *my* frustration. From p.f.moore at gmail.com Thu Mar 20 22:56:15 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 20 Mar 2008 21:56:15 +0000 Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module) In-Reply-To: <47E2D8E8.8090209@taupro.com> References: <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E2D8E8.8090209@taupro.com> Message-ID: <79990c6b0803201456g30f766b1jfd2af474306b7afc@mail.gmail.com> On 20/03/2008, Jeff Rush wrote: > Paul Moore wrote: > > On 20/03/2008, zooko wrote: > > 1. No integration with the system packager (Windows, in my case). If I > > do easy_install nose, then nose does not show up in add/remove > > programs. That significantly affects the way I manage my PC. > > > Part of this stems from stretching of the original mission of setuptools, to > install modules for Python, into a general-purpose application installation > tool. The buildout tool is more suited for application installation, although > not ideal yet. > > In your scenario, what happens when one egg pulls in another and another, > until you have a hundred entries in your add/remove menu? And you don't > understand the inter-relationship of those so you cannot do a clean uninstall? I don't let it. As I've said elsewhere, I prefer to manage dependencies myself, manually. Anything with that many dependencies shouldn't be using the system Python, in my view. It should be packaged as a standalone application (py2exe style) and as such have a single add/remove entry (and no effect on the system Python). > Similarly, or what do you want to appear in that add/remove menu when you are > using independent sandboxes with various applications in them, some of which > are accessible only to specific users who are not admins on that box? Independent sandboxes isn't a concept I can relate to under Windows. That doesn't mean it's not possible (I don't know if it is) I just don't have any useful comment to make, beyond saying that I personally don't care what happens in that situation. Paul. From martin at v.loewis.de Thu Mar 20 23:07:43 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 20 Mar 2008 17:07:43 -0500 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <47E2DB20.5020909@rksystems.com> References: <47E0D883.9030402@taupro.com> <47E11659.9000307@v.loewis.de> <47E1BB1D.4060303@v.loewis.de> <43aa6ff70803201117y46e09c5frd6e5be6e4902b23c@mail.gmail.com> <47E2CA12.2000506@v.loewis.de> <47E2CE0E.6010108@rksystems.com> <47E2D716.4030109@v.loewis.de> <47E2DB20.5020909@rksystems.com> Message-ID: <47E2E02F.1020007@v.loewis.de> >> if type(response) in (str, str): > > Then I'm taking him too literally, when he writes that the tool won't > touch *anything* that has to do with the str->bytes/unicode->str move (I > assumed that meant it wouldn't touch "unicode" in the snippet I gave > above), right? It definitely replaces unicode->str in this fragment. > Will the tool also make the following work correctly? > > if type(s) is str: s = unicode(s, 'utf-8') It outputs if type(s) is str: s = str(s, 'utf-8') This is probably not correct. However, in 2.6, you could write that as if type(s) is bytes: s = unicode(s, 'utf-8') as bytes is str in 2.6. If you want to support even older versions, do try: bytes except NameError: bytes = str somewhere, then write the code as I proposed above. Regards, Martin From mal at egenix.com Thu Mar 20 23:46:04 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 20 Mar 2008 23:46:04 +0100 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <79990c6b0803201334t4cf8531y2af40edc3e469fe8@mail.gmail.com> References: <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E26A20.1090402@palladion.com> <79990c6b0803201055l2831624q5e0fd1117f1ec338@mail.gmail.com> <20080320190933.A03A83A4074@sparrow.telecommunity.com> <79990c6b0803201334t4cf8531y2af40edc3e469fe8@mail.gmail.com> Message-ID: <47E2E92C.3040501@egenix.com> On 2008-03-20 21:34, Paul Moore wrote: >> Also, setuptools-based packages *can* build bdist_wininst >> installers. (In fact, if memory serves, I added that feature at your request.) > > I know. python setup.py bdist_wininst. And thank you for adding it. > But again you miss my point. People are starting to omit distributing > bdist_wininst installers in favour of eggs only. And you cannot (to my > knowledge) convert an egg into a bdist_wininst installer, and if you > can't compile from source (a C extension with complex dependencies, > for example) you're stuck (in the sense that you're forced to use eggs > without add/remove programs support). You might want to look at the eGenix pre-built packages as an alternative: they include all the information necessary to let standard distutils continue its works *after* the build step. It's basically a distribution of the package as it looks after the build step has run, but before the package is wrapped up using a packager like bdist_wininst or bdist_msi or installed into the system. You can download the pre-built package and create e.g. an MSI installer or a wininst EXE without needing a compiler - in addition to providing all the options of the standard distutils "install" command (which makes repackaging them as part of larger applications easy as well). All the logic for this is included in mxSetup.py which ships with the pre-built packages. http://www.egenix.com/products/python/mxBase/#Download http://www.egenix.com/products/python/mxBase/#Installation The current version we have is not yet perfect. The next iteration will also play nice with distutils extensions. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 20 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From eric+python-dev at trueblade.com Thu Mar 20 23:55:01 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Thu, 20 Mar 2008 18:55:01 -0400 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals Message-ID: <47E2EB45.5080202@trueblade.com> Following up on a python-3000 discussion about making porting from 2.6 to 3.0 easier. Martin suggested making this its own thread. This proposal is to add "from __future__ import unicode_string_literals", which would make all string literals in the importing module into unicode objects in 2.6. This is similar to the -U flag, but would only affect a single module at a time. I think history has shown that -U isn't really usable when using any number of modules, including many in the standard library. There was another proposal from Christian Heimes to add "from __future__ import py3k_literals", which would: 1) '' creates an unicode object instead of a str object 2) b'' creates a str object (aka bytes in Python 3.0) 3) 1 creates a long instead of an int 4) 1L and u'' are invalid 2) is already taken care of in 2.6, since: type(b'') == str. I don't think 3) is necessary. It's an implementation detail. 4) is really two issues. It's my understanding that there's a 2to3 fixer for both of these issues. But I'm open to debate on this. I'm willing to implement this if there's consensus on it. Eric. From pje at telecommunity.com Thu Mar 20 23:42:20 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 20 Mar 2008 18:42:20 -0400 Subject: [Python-Dev] Wow, I think I actually *get* it now! In-Reply-To: <20080320220844.21558.1047009855.divmod.xquotient.8501@joul e.divmod.com> References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <20080320103823.21558.491968946.divmod.xquotient.8438@joule.divmod.com> <20080320150640.379643A40A5@sparrow.telecommunity.com> <20080320220844.21558.1047009855.divmod.xquotient.8501@joule.divmod.com> Message-ID: <20080320224227.601543A4074@sparrow.telecommunity.com> At 10:08 PM 3/20/2008 +0000, glyph at divmod.com wrote (off-list): >No, but in no situation, except one (where I was extremely pressed >for time) was I actually attempting to use setuptools to use any of >its features. My experience of it is: "If a project uses distutils >or apt, installation probably works. If it uses setuptools, it >probably throws a traceback or a wall of text explaining why my >environment is inadequate to perform the installation." Other >people chose to use it and in so doing broke my setup. Manually >copying a few files in these cases was a _lot_ easier than >attempting to diagnose and repair software that I didn't even want to use. > >I am not interesting in packaging or distribution. Far from it: I >run all of my software out of an SVN checkout and I _detest_ being >involved in discussions of deployment or installation. ... >However, the general message of the negative subjective experience I >have had while using setuptools is not FUD. It's an accurate >portrayal of a great deal of frustration. setuptools has, to this >date, not solved a single problem for *me*, personally or >professionally, but it has caused many. distutils, despite its many >flaws, has actually solved quite a few. Actually, this information is VERY helpful. It makes it blindingly obvious to me now that the difference between loving and hating setuptools is whether you're *intentionally* using it, or whether it shows up in your ecosystem uninvited. It also makes the difference in whether you get involved: with no investment in the tool itself, you have minimal motivation to RTFM, ask questions, or fix bugs. And when people in this scenario *do* communicate to me or the distutils-sig, they are much more likely to be impatient and hostile, and more likely to view the system as "fundamentally broken". This makes total sense to me now. I don't have any *solutions* to the problem, mind you, but at least now I understand what before seemed like some sort of bizarre anomaly where literally thousands of people use setuptools and many dozens actually express their happiness with or even love for the system, and then others hate it like they hate Microsoft, or worse. ;-) Meanwhile, from the "outsiders" point of view, setuptools looks like the Matrix or the Borg, happily assimilating the masses, who then start coming to you and say, "But you'll be so much happier once you join us..." ...and off in the distance, you hear a quiet rumbling of zombies chanting "eeeeggs.... eeeeggggs.... mussst havve eggggssss!" :) Hm. So it seems to me that maybe one thing that would help is a "Setuptools Haters' Guide To Setuptools" -- that is, *short* documentation specifically written for people who don't want to use setuptools and want to minimize its impact on their systems. I could probably write something like that fairly easily, now that I have some idea of what to go in it, more than, "the existing documentation sucks". :) Can I count on some non-assimilated persons' help in critiquing such a document and suggesting any topics I miss? From mike.klaas at gmail.com Fri Mar 21 00:02:34 2008 From: mike.klaas at gmail.com (Mike Klaas) Date: Thu, 20 Mar 2008 16:02:34 -0700 Subject: [Python-Dev] svnmerge and added files In-Reply-To: <47E2D7E5.9080407@cheimes.de> References: <47E2D461.30309@v.loewis.de> <47E2D7E5.9080407@cheimes.de> Message-ID: <821DE445-E370-4EBC-B7A8-F4850F1DD9C3@gmail.com> On 20-Mar-08, at 2:32 PM, Christian Heimes wrote: > Martin v. L?wis schrieb: >> It seems that recently, a number of merges broke in the sense >> that files added to the trunk were not merged into the >> 3k branch. >> >> Is that a general problem with svnmerge? Should that be >> fixed to automatically do a "svn add" when merging changes >> that included file additions and removals? > > It sometimes happens when I do a svnmerge, revert the merge with svn > revert -R and do a second svnmerge. Files added by the first svnmerge > aren't added to the commit list for the second merge. Unfortunately > svnmerge.py doesn't warn me when the file already exists. It may not warn explicitly about that, but it certainly does warn: M ... Skipped path/to/missing/file... M ... M ... As someone who deals with svnmerge.py a lot, I find that it is appropriate to treat "Skipped" as critical as a conflict. I too wish that it was more explicit in this respect. -Mike From skip at pobox.com Thu Mar 20 22:15:00 2008 From: skip at pobox.com (skip at pobox.com) Date: Thu, 20 Mar 2008 16:15:00 -0500 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <871w65gt4i.fsf@uwakimon.sk.tsukuba.ac.jp> References: <47E0D883.9030402@taupro.com> <47E11659.9000307@v.loewis.de> <47E19ABE.2040800@taupro.com> <47E1BD14.2030805@v.loewis.de> <47E1EAE4.90802@taupro.com> <47E24C7C.6080209@v.loewis.de> <871w65gt4i.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <18402.54228.621089.563344@montanaro-dyndns-org.local> >> I don't think any of the core committers is qualified to write such a >> document. Instead, it would have to be written by people who >> *actually* ported a project from 2 to 3 in some form. Stephen> Note that this is precisely the kind of experience Skip Stephen> Montanaro is talking about trying to generate in the context of Stephen> SpamBayes ... Correctamundo. Give that man a cigar. Skip From p.f.moore at gmail.com Fri Mar 21 00:12:25 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 20 Mar 2008 23:12:25 +0000 Subject: [Python-Dev] Wow, I think I actually *get* it now! In-Reply-To: <20080320224227.601543A4074@sparrow.telecommunity.com> References: <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <20080320103823.21558.491968946.divmod.xquotient.8438@joule.divmod.com> <20080320150640.379643A40A5@sparrow.telecommunity.com> <20080320220844.21558.1047009855.divmod.xquotient.8501@joule.divmod.com> <20080320224227.601543A4074@sparrow.telecommunity.com> Message-ID: <79990c6b0803201612h5a6d57bt82faa4e71c451f07@mail.gmail.com> On 20/03/2008, Phillip J. Eby wrote: > Actually, this information is VERY helpful. It makes it blindingly > obvious to me now that the difference between loving and hating > setuptools is whether you're *intentionally* using it, or whether it > shows up in your ecosystem uninvited. Exactly. I never thought to express that, but it precisely explains my situation as well. > Hm. So it seems to me that maybe one thing that would help is a > "Setuptools Haters' Guide To Setuptools" -- that is, *short* > documentation specifically written for people who don't want to use > setuptools and want to minimize its impact on their systems. I could > probably write something like that fairly easily, now that I have > some idea of what to go in it, more than, "the existing documentation > sucks". :) > > Can I count on some non-assimilated persons' help in critiquing such > a document and suggesting any topics I miss? I would do so. (From a Windows user's perspective). Paul. From tjreedy at udel.edu Fri Mar 21 00:17:41 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 20 Mar 2008 19:17:41 -0400 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? References: <47E0D883.9030402@taupro.com><47E11659.9000307@v.loewis.de> <47E1BB1D.4060303@v.loewis.de> <43aa6ff70803201117y46e09c5frd6e5be6e4902b23c@mail.gmail.com><47E2CA12.2000506@v.loewis.de> <47E2CE0E.6010108@rksystems.com><47E2D716.4030109@v.loewis.de> <47E2DB20.5020909@rksystems.com> <47E2E02F.1020007@v.loewis.de> Message-ID: ""Martin v. L?wis"" wrote in message news:47E2E02F.1020007 at v.loewis.de... | This is probably not correct. However, in 2.6, you could write | that as | | if type(s) is bytes: s = unicode(s, 'utf-8') | | as bytes is str in 2.6. If you want to support even older | versions, do | | try: | bytes | except NameError: | bytes = str | | somewhere, then write the code as I proposed above. This is exactly the sort of thing that should be and I expect will be in the conversion docs. From greg.ewing at canterbury.ac.nz Fri Mar 21 00:51:00 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 21 Mar 2008 11:51:00 +1200 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <20080320150754.9F6503A40A5@sparrow.telecommunity.com> References: <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <47E1EEE6.5050608@palladion.com> <20080320150754.9F6503A40A5@sparrow.telecommunity.com> Message-ID: <47E2F864.8020900@canterbury.ac.nz> At 12:58 AM 3/20/2008 -0400, Tres Seaver wrote: > A lot of setuptools warts are driven by related design problems in the > distutils, such as the choice to use imperative / procedural code for > everything If a distutils replacement is ever written, I'd like to see it structured as a dependency graph, like a makefile, with each node in the graph knowing how to transform its inputs into its outputs. That would make it a lot easier to extend to accommodate new things like Pyrex. You'd just have to write a new node class that knows how to turn .pyx files into .c files, and the existing machinery would take it from there. -- Greg From fumanchu at aminus.org Fri Mar 21 01:10:03 2008 From: fumanchu at aminus.org (Robert Brewer) Date: Thu, 20 Mar 2008 17:10:03 -0700 Subject: [Python-Dev] Wow, I think I actually *get* it now! In-Reply-To: <20080320224227.601543A4074@sparrow.telecommunity.com> References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com><87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp><20080318223700.C64123A4074@sparrow.telecommunity.com><20080319195438.A28DE3A40A5@sparrow.telecommunity.com><79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com><20080320103823.21558.491968946.divmod.xquotient.8438@joule.divmod.com><20080320150640.379643A40A5@sparrow.telecommunity.com><20080320220844.21558.1047009855.divmod.xquotient.8501@joule.divmod.com> <20080320224227.601543A4074@sparrow.telecommunity.com> Message-ID: Phillip J. Eby wrote: > Hm. So it seems to me that maybe one thing that would help is a > "Setuptools Haters' Guide To Setuptools" -- that is, *short* > documentation specifically written for people who don't want to use > setuptools and want to minimize its impact on their systems. I could > probably write something like that fairly easily, now that I have > some idea of what to go in it, more than, "the existing documentation > sucks". :) > > Can I count on some non-assimilated persons' help in critiquing such > a document and suggesting any topics I miss? I'd be glad to help critique such a doc. Robert Brewer fumanchu at aminus.org From fumanchu at aminus.org Fri Mar 21 01:22:10 2008 From: fumanchu at aminus.org (Robert Brewer) Date: Thu, 20 Mar 2008 17:22:10 -0700 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <20080320215202.618173A4074@sparrow.telecommunity.com> References: <20080318223700.C64123A4074@sparrow.telecommunity.com><20080319195438.A28DE3A40A5@sparrow.telecommunity.com><79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com><47E26A20.1090402@palladion.com><79990c6b0803201055l2831624q5e0fd1117f1ec338@mail.gmail.com><20080320190933.A03A83A4074@sparrow.telecommunity.com><79990c6b0803201334t4cf8531y2af40edc3e469fe8@mail.gmail.com> <20080320215202.618173A4074@sparrow.telecommunity.com> Message-ID: Phillip J. Eby wrote: > The other tool that would be handy to have, would be one that unpacks > eggs into standard distutils-style installation. Hear, hear. I'm an author of a couple libraries that need to interoperate with others. Of the many eggs I've downloaded over the past year, I'd say 80%+ are never installed or even built--I just want to grep the source code, and using my preferred tools, not some lame Find command in a ZIP browser menu. Robert Brewer fumanchu at aminus.org From tjreedy at udel.edu Fri Mar 21 01:31:56 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 20 Mar 2008 20:31:56 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E26A20.1090402@palladion.com> Message-ID: "Tres Seaver" wrote in message news:47E26A20.1090402 at palladion.com... | -----BEGIN PGP SIGNED MESSAGE----- | Hash: SHA1 | | Paul Moore wrote: | | > 1. No integration with the system packager (Windows, in my case). If I | > do easy_install nose, then nose does not show up in add/remove | > programs. That significantly affects the way I manage my PC. | | Point taken. Of course, it isn't really a "program" at that point: it | is an installed "add-on" to Python. However, if Windows users expect | such add-ons to show up in the "system" list, that is good to know. However, this Windows user, and I expect most, do NOT expect add-ons (things under the /Pythonx.y tree) to show up in the add/remove list. Please do not do that. On my system, it already takes several seconds to 'populate the list'. It is bad enough that Windows Update occasionally adds entries like 'Windows Update KB284798324' instead of adding something to the separate 'Manage Windows components' subsystem with readable names and explanations. The standard (and to me, preferable) way of dealing with such things is to have an 'installation manager' that can reinstall as well as delete and that has a check box for various things to delete. This is what Python needs. Terry Jan Reedy From tjreedy at udel.edu Fri Mar 21 01:37:51 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 20 Mar 2008 20:37:51 -0400 Subject: [Python-Dev] [Distutils] PEP 365 (Adding thepkg_resources module) References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E2D8E8.8090209@taupro.com> Message-ID: "Jeff Rush" wrote in message news:47E2D8E8.8090209 at taupro.com... | In your scenario, what happens when one egg pulls in another and another, | until you have a hundred entries in your add/remove menu? As I said in another response, I don't think such things belong in add/remove. From zooko at zooko.com Fri Mar 21 01:48:24 2008 From: zooko at zooko.com (zooko) Date: Thu, 20 Mar 2008 18:48:24 -0600 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <47E26A20.1090402@palladion.com> References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E26A20.1090402@palladion.com> Message-ID: On Mar 20, 2008, at 7:44 AM, Tres Seaver wrote: > Paul Moore wrote: >> 4. Hard to use with limited connectivity. At work, I *only* have >> access to the internet via Internet Explorer (MS based proxy). There >> are workarounds, but ultimately "download an installer, then run it" >> is a far simpler approach for me. > > I don't know how to make this requirement compatible with using shared > dependencies, We've done something like this. The http://allmydata.org project bundles its easy_installable dependencies. If you get the current trunk from our darcs repository [1], or get a release tarball or a snapshot tarball from [2], then it comes with a directory named "misc/dependencies" which has the source tarballs of our easy_installable dependencies. You can browse this directory on the web: [3]. Therefore, if you manually satisfy the non-easy_installable dependencies, you can download an allmydata.org tarball, disconnect from the Internet (which we call "moving to a Desert Island"), and install it. This is, as you say, "compatible with using shared dependencies" because setuptools will detect if you already have sufficiently new versions of some of these dependencies installed (for example, if they are installed in Debian packages), and then skip the step of installing that dependency from its source tarball. The remaining dependencies that cannot be satisfied automatically by our setup.py are listed in the install.html [4]. They are: 1. g++ >= v3.3 -- the Cygwin version of gcc/g++ works for Cygwin and for Windows 2. GNU make 3. Python >= v2.4.2 including development headers i.e. "Python.h" 4. Twisted >= v2.4.0 -- from the Twisted "sumo" source tarball 5. OpenSSL >= v0.9.7, including development headers 6. PyOpenSSL == v0.6 7. Crypto++ >= v5.2.1, including development headers I am hoping that in the future Twisted (see twisted #1286 [5]) and pyOpenSSL will be easy_installable, and that our use of setuptools plugins will eventually replace our GNUmakefile and thus remove our dependency on GNUmake. That will leave only g++, Python, OpenSSL, and Crypto++ as dependencies that a user has to manually deal with in order to build allmydata.org from source. Regards, Zooko [1] http://allmydata.org/source/tahoe/trunk/ [2] http://allmydata.org/source/tahoe/tarballs/ [3] http://allmydata.org/trac/tahoe/browser/misc/dependencies [4] http://allmydata.org/source/tahoe/trunk/docs/install.html [5] http://twistedmatrix.com/trac/ticket/1286 From janzert at janzert.com Fri Mar 21 04:50:23 2008 From: janzert at janzert.com (Janzert) Date: Thu, 20 Mar 2008 23:50:23 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> Message-ID: Guido van Rossum wrote: > On Wed, Mar 19, 2008 at 12:54 PM, Phillip J. Eby wrote: > [a long message] > > I'm back at Google and *really* busy for another week or so, so I'll > have to postpone the rest of this discussion for a while. If other > people want to chime in please do so; if this is just a dialog between > Phillip and me I might incorrectly assume that nobody besides Phillip > really cares. > Since there seems to be a fair number of negative responses to setuptools, I just wanted to add a bit of positive counterbalance. I'm just a random python user that happens to track python-dev a bit, so take all this with the realization that I probably shouldn't have much input into anything. ;) I've been using python for somewhere around 10 years to write various random small scripts, gui applications and recently web applications. For me setuptools is the best thing to happen to python since I've been using it. I develop and deploy on a seemingly constantly changing mix of various flavors of windows and linux. Unlike for others, I love that once I get setuptools installed I can just use the same commands to get the things I need. I guess the contrast for me is that python is the common base that I tend to work from not the underlying OS. So I don't know if I'm part of a large number of quiet users or just happen to be an odd case that works really well with setuptools. I was disappointed when setuptools didn't make it into 2.5 and I really hope it or something very much like it can make it into a release in the near future. Because while setuptools certainly isn't perfect, for me at least, it is much, much better than nothing at all. Brian Haskin From fumanchu at aminus.org Fri Mar 21 05:10:09 2008 From: fumanchu at aminus.org (Robert Brewer) Date: Thu, 20 Mar 2008 21:10:09 -0700 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080317184553.913413A409D@sparrow.telecommunity.com> <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> Message-ID: Janzert wrote: > Since there seems to be a fair number of negative responses to > setuptools, I just wanted to add a bit of positive counterbalance. I'm > just a random python user that happens to track python-dev a bit, so > take all this with the realization that I probably shouldn't have much > input into anything. ;) > > I've been using python for somewhere around 10 years to write various > random small scripts, gui applications and recently web applications. > For me setuptools is the best thing to happen to python since I've been > using it. I develop and deploy on a seemingly constantly changing mix > of > various flavors of windows and linux. Unlike for others, I love that > once I get setuptools installed I can just use the same commands to get > the things I need. I guess the contrast for me is that python is the > common base that I tend to work from not the underlying OS. > > So I don't know if I'm part of a large number of quiet users or just > happen to be an odd case that works really well with setuptools. I was > disappointed when setuptools didn't make it into 2.5 and I really hope > it or something very much like it can make it into a release in the > near future. Because while setuptools certainly isn't perfect, for me > at least, it is much, much better than nothing at all. My interpretation of this is that setuptools suffers from the same malaise all flexible apps do (but especially CLI apps it seems): frequent users love the power and high volume of options, infrequent users despise it. If you're installing apps all day, you probably use it a lot more often than library devs like me who use it once every other month (if we're forced to). Robert Brewer fumanchu at aminus.org From guido at python.org Fri Mar 21 07:31:49 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 20 Mar 2008 23:31:49 -0700 Subject: [Python-Dev] Not backporting PEP 3115 (metaclass __prepare__) In-Reply-To: <20080319051433.GA16598@performancedrivers.com> References: <20080319051433.GA16598@performancedrivers.com> Message-ID: On Tue, Mar 18, 2008 at 10:14 PM, Jack Diederich wrote: > We can't backport the __prepare__ syntax without requiring metaclass > definition on the 'class' line. Because the __metaclass__ definition > can be at the end of the class in 2.6 we can't find it until after we > execute the class and that is too late to use a custom dictionary. That's fine. We need some carrots to encourage people to upgrade too. :-) > I wish I had thought of that yesterday, Me too... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From asmodai at in-nomine.org Fri Mar 21 09:42:49 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Fri, 21 Mar 2008 09:42:49 +0100 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> Message-ID: <20080321084249.GA83712@nexus.in-nomine.org> (To provide counterweight.) -On [20080320 20:44], Barry Warsaw (barry at python.org) wrote: >We have not made a decision to move to Bazaar officially, nor have we made >a decision to even move off of Subversion. Good, because between this now and pytz the other 63 projects I follow use Subversion or Mercurial. Bazaar seems to be mostly limited to Ubuntu users and stuff Canonical does, so the choice for a Bazaar setup next to Subversion strikes me a bit as odd. Mercurial is also Python, distributed and with a, as far as I (can) track things, bigger market share than Bazaar. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ The fragrance always stays in the hand that gives the rose... From matthieu.brucher at gmail.com Fri Mar 21 10:04:10 2008 From: matthieu.brucher at gmail.com (Matthieu Brucher) Date: Fri, 21 Mar 2008 10:04:10 +0100 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <20080321084249.GA83712@nexus.in-nomine.org> Message-ID: Sorry for the double post, Jeroen :| ---------- Forwarded message ---------- From: Matthieu Brucher Date: 21 mars 2008 10:03 Subject: Re: [Python-Dev] Python source code on Bazaar vcs To: Jeroen Ruigrok van der Werven 2008/3/21, Jeroen Ruigrok van der Werven : > > (To provide counterweight.) > > > -On [20080320 20:44], Barry Warsaw (barry at python.org) wrote: > >We have not made a decision to move to Bazaar officially, nor have we > made > >a decision to even move off of Subversion. > > > Good, because between this now and pytz the other 63 projects I follow use > Subversion or Mercurial. > Bazaar seems to be mostly limited to Ubuntu users and stuff Canonical > does, > so the choice for a Bazaar setup next to Subversion strikes me a bit as > odd. > Mercurial is also Python, distributed and with a, as far as I (can) track > things, bigger market share than Bazaar. > Hi, This is not quite true. One of the main bzr developers is a Windows guy, so the user base is/will be large, far larger than only Ubuntu. A lot of projects switched to bzr because of this. Honestly, I think that Hg and bzr fill the same need, but bzr developement at the moment is fantastic. For a still young product it is a good think (Hg is older). One additional good think is that there is a Sourceforge-like site that uses bzr ; launchpad.net. I don't know of something equivalent for Hg. Just my two (euro) cents ;) Matthieu -- French PhD student Website : http://matthieu-brucher.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher -- French PhD student Website : http://matthieu-brucher.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080321/6d6da535/attachment-0001.htm From eric+python-dev at trueblade.com Fri Mar 21 11:57:07 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Fri, 21 Mar 2008 06:57:07 -0400 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <47E2EB45.5080202@trueblade.com> References: <47E2EB45.5080202@trueblade.com> Message-ID: <47E39483.5050504@trueblade.com> Eric Smith wrote: > This proposal is to add "from __future__ import > unicode_string_literals", which would make all string literals in the > importing module into unicode objects in 2.6. I'm going to withdraw this, for 2 reasons. 1) The more I think about it, the less sense it makes. 2) Without some extreme measures, it's not implementable. It's not implementable because the work has to occur in ast.c (see Py_UnicodeFlag). It can't occur later, because you need to skip the encoding being done in parsestr(). But the __future__ import can only be interpreted after the AST is built, at which time the encoding has already been applied. There are some radical things you could do to work around this, but it would be a gigantic change. As for it not making sense, this is really in the realm of 2to3. I'm beginning to really believe this statement in PEP 3000: "There is no requirement that Python 2.6 code will run unmodified on Python 3.0. Not even a subset. (Of course there will be a tiny subset, but it will be missing major functionality.)" For this particular issue, just use u'' in 2.6 and let 2to3 deal with it. If you have some 2.6 code that you want to run in 3.0 (by way of 2to3), I think all of your string literals should either be b'' or u''. Don't use plain ''. Eric. From p.f.moore at gmail.com Fri Mar 21 13:33:31 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 21 Mar 2008 12:33:31 +0000 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E26A20.1090402@palladion.com> Message-ID: <79990c6b0803210533q3f9310c8tf119300a77030ccf@mail.gmail.com> On 21/03/2008, Terry Reedy wrote: > However, this Windows user, and I expect most, do NOT expect add-ons > (things under the /Pythonx.y tree) to show up in the add/remove list. That's an interesting counterpoint to my comments. I presume from this that you dislike (and/or never use) bdist_msi and bdist_wininst precisely because they do add such items to the add/remove programs list? My argument is essentially that bdist_wininst set a de facto standard for this. Then, bdist_msi followed it. Now setuptools is trying to change that standard by ignoring it, rather than by discussion of the pros and cons. Personally, I like the current approach, but that's less relevant. > The standard (and to me, preferable) way of dealing with such things is to > have an 'installation manager' that can reinstall as well as delete and > that has a check box for various things to delete. This is what Python > needs. I'd dispute strongly that this is a "standard". It may be preferable, but I'm not sure where you see evidence of it being a standard. Could I also point out that *if* such a standard is set up for Python, bdist_wininst and bdist_msi should be modified to follow it. Otherwise, it's not a standard, more of competing approach. As you can see, my main concern is for consistency :-) Paul. From pje at telecommunity.com Fri Mar 21 13:49:42 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 21 Mar 2008 08:49:42 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: <79990c6b0803210533q3f9310c8tf119300a77030ccf@mail.gmail.co m> References: <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E26A20.1090402@palladion.com> <79990c6b0803210533q3f9310c8tf119300a77030ccf@mail.gmail.com> Message-ID: <20080321124944.ABDF03A4074@sparrow.telecommunity.com> At 12:33 PM 3/21/2008 +0000, Paul Moore wrote: >On 21/03/2008, Terry Reedy wrote: > > The standard (and to me, preferable) way of dealing with such > things is to > > have an 'installation manager' that can reinstall as well as delete and > > that has a check box for various things to delete. This is what Python > > needs. > >I'd dispute strongly that this is a "standard". It may be preferable, >but I'm not sure where you see evidence of it being a standard. I presume he means that there are a lot of entries in his Add/Remove Programs that work like that, and that it's an emerging standard for Windows. (Certainly I've seen quite a few entries like that in mine, although more often than not they only have one checkbox!) >Could I also point out that *if* such a standard is set up for Python, >bdist_wininst and bdist_msi should be modified to follow it. >Otherwise, it's not a standard, more of competing approach. The best thing to do would be to get a standard (ala PEP 262, but modified by the benefit of experience now) for tracking installed Python package distributions. Then we can standardize on platform tools for managing this data, and include them in the relevant platform distributions. (And that would include making bdist_wininst and bdist_msi follow this installation DB standard.) From solipsis at pitrou.net Fri Mar 21 13:52:16 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 21 Mar 2008 12:52:16 +0000 (UTC) Subject: [Python-Dev] Python source code on Mercurial References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <932f8baf0803201426ua0ae2edu3f7b62eaf22ef787@mail.gmail.com> Message-ID: Ralf Schmitt gmail.com> writes: > > I have also setup a mirror using mercurial: http://hgpy.de/py/It contains the 2.4, 2.5, trunk and py3k branches (in case anyone wants to compare this to bzr). I see your trunk history is stripped. For those who want the complete trunk history (back to 17 years ago!), I have my own mirror here: http://dev.pitrou.net:8000/cpython/trunk/ For Mercurial beginners, you just have to install Mercurial and type: hg clone http://dev.pitrou.net:8000/cpython/trunk/ This gives you a local repository in the "trunk" directory which includes all the pulled history from the said mirror. (however, please note the server is not mine, and I do not guarantee the above URL will function forever :-)) Antoine. From asmodai at in-nomine.org Fri Mar 21 13:54:02 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Fri, 21 Mar 2008 13:54:02 +0100 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E26A20.1090402@palladion.com> <47E29176.9050109@v.loewis.de> Message-ID: <20080321125402.GB83712@nexus.in-nomine.org> -On [20080320 19:24], Steve Holden (steve at holdenweb.com) wrote: >We need to stop protesting that our installation tools are easy enough >and try to get behind the various platforms, be it with Windows >installers, rpms, or other support. We probably aren't doing this >because it's work nobody particularly relishes, and has relatively low >visibility in the developer world. Non-developer Python programmers and >end-users would thank us, though. FreeBSD offers through install of Perl through its ports system a Perl module called 'bsdpan' which registers every module as a package under FreeBSD's package system. Normally ports installs modules as p5-ModuleName, but now it becomes: /var/db/pkg/bsdpan-B-Lint-1.09 And from that point on I can use the pkg* tools. Quite elegant in my opinion. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ To regard the fundamental as the essence, to regard things as coarse, to regard accumulation as deficiency, and to dwell quietly alone with the spiritual and the intelligent -- herein lie the techniques of Tao of the ancients. From pje at telecommunity.com Fri Mar 21 14:47:46 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 21 Mar 2008 09:47:46 -0400 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond Message-ID: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> So, after having some time to absorb the Python-Dev threads about setuptools, bootstrap, and all the rest, I think I see an opportunity to let people route around the "damage" of eggs, while still making it possible for the people who want to use easy_install or to put dependencies in their setup.py to get what they want, too. (And without them needing to install eggs, either.) At the same time, we can address the issues that remain around uninstalling packages, system vs. user packages, PYTHONPATH and site.py woes, and really pretty much every other complaint I've heard in the last few days about setuptools stomping on other people's stuff. (Even Paul's Windows issues, hopefully.) Now, you might be asking, "Okay, so why are you telling me about this? Why not just go fix setuptools?" Well, I *can't*. Not without some help from Python-Dev and the Distutils-SIG, to create an updated standard for installed package metadata, by updating PEP 262 ("A Database of Installed Python Packages") to include input from the system packaging folks, support for namespace packages, and support for setuptools-compatible dependency information. What's that got to do with anything? Well, without it, setuptools can't support uninstall or conflict management without using eggs to compartmentalize the installed files. And because it has to use eggs to do *that*, it has to munge .pth files and install its own site.py when installing to PYTHONPATH. All of this ugliness follows directly from the absence of a PEP 262-style installation database. Sure, setuptools could create its own version of this, and I almost did that four years ago. (If you look at the "open issues" part of PEP 262, you'll see my comments from back then.) I decided not to for two reasons: first, the distutils didn't support it yet, so it didn't help for conflict detection and avoidance in the real world at that point. Second, there were no uninstall tools for it, so I'd have had to write one myself. (Zed's "easy_f'ing_uninstall" to the contrary, it ain't easy, and I have an aversion to deleting stuff on people's systems without knowing what will break. There's a big difference between them typing 'rm -rf' themselves, and me doing it.) However, if tools exist and are distributed for such a "database", and *everybody* agrees to use it as an officially-blessed standard, then it should be possible for setuptools to co-exist with that framework, and we're all happy campers. In particular, the "installing eggs sucks" camp should be happy, because it'll be possible for me (or anyone else) to write a version of easy_install that doesn't install eggs any more, and setuptools-based packages can go back to having "setup.py install" install things the old way by default. So, to accomplish this, we (for some value of "we") need to: 1. Hash out consensus around what changes or enhancements are needed to PEP 262, to resolve the previously-listed open issues, those that have come up since (namespace packages, dependency specifications, canonical name/version forms), and anything else that comes up. 2. Update or replace the implementation as appropriate, and modify the distutils to support it in Python 2.6 and beyond. And "support it" means, "ensure that 'install' and *all* bdist commands update the database". The bdist_rpm, bdist_wininst, and bdist_msi commands, even bdist_dumb. (This should probably also include the add/remove programs stuff in the Windows case.) 3. Create a document for system packagers referencing the PEP and introducing them to what/why/how of the standard, in case they weren't one of the original participants in creating this. It will probably take some non-trivial work to do all this for Python 2.6, but it's probably possible, if we start now. I don't think it's critical to have an uninstall tool distributed with 2.6, as long as there's a reasonable way to bootstrap its installation later. Questions, comments... volunteers? :) From him at online.de Fri Mar 21 16:15:43 2008 From: him at online.de (=?ISO-8859-1?Q?Joachim_K=F6nig?=) Date: Fri, 21 Mar 2008 16:15:43 +0100 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> Message-ID: <47E3D11F.4030503@online.de> Phillip J. Eby wrote: > Second, there were no uninstall tools for it, so I'd have had to > write one myself. (Zed's "easy_f'ing_uninstall" to the contrary, it > ain't easy, and I have an aversion to deleting stuff on people's > systems without knowing what will break. There's a big difference > between them typing 'rm -rf' themselves, and me doing it.) > I think, the uninstall should _not_ 'rm -rf' but only 'rm' the files (and 'rmdir' directories, but not recursively) that it created, and that have not been modified in the meantime (after the installation). This can be easily achieved by recording a checksum (eg. md5 or sha) upon installation and only deleting a file if the checksum is correct and only deleting directories when they are empty (after the installed files in them have been deleted). Otherwise, the uninstall should complain and leave the modified files installed. Joachim From ziade.tarek at gmail.com Fri Mar 21 16:34:32 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 21 Mar 2008 16:34:32 +0100 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> Message-ID: <94bdd2610803210834g454cc1f3o5e3c4454b93e5030@mail.gmail.com> On Fri, Mar 21, 2008 at 2:47 PM, Phillip J. Eby wrote: > Second, there were no uninstall tools for it, so I'd have had to > write one myself. (Zed's "easy_f'ing_uninstall" to the contrary, it > ain't easy, and I have an aversion to deleting stuff on people's > systems without knowing what will break. There's a big difference > between them typing 'rm -rf' themselves, and me doing it.) > Agree. I tried a while ago to write a "easy_uninstall" but this is not possible from an application-point of view, to do clean things. zc.buildout resolves this by installing eggs locally, to an isolated environment, so my main Python installation doesn't hold any extensions at all. So if a database of installed package is created, I would be in favor of an application-oriented system where it is possible to create, update, install, a set of packages dedicated to an environment (default would be Python). Maybe by having a namespace tied to a list of versions. In other words; having the same feature virtualenv provides, in Python itself, and define somehow how to switch to it. $ easy_install the.package --environment MyApp -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080321/8eb07989/attachment.htm From zooko at zooko.com Fri Mar 21 16:50:26 2008 From: zooko at zooko.com (zooko) Date: Fri, 21 Mar 2008 09:50:26 -0600 Subject: [Python-Dev] Wow, I think I actually *get* it now! In-Reply-To: References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com><87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp><20080318223700.C64123A4074@sparrow.telecommunity.com><20080319195438.A28DE3A40A5@sparrow.telecommunity.com><79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com><20080320103823.21558.491968946.divmod.xquotient.8438@joule.divmod.com><20080320150640.379643A40A5@sparrow.telecommunity.com><20080320220844.21558.1047009855.divmod.xquotient.8501@joule.divmod.com> <20080320224227.601543A4074@sparrow.telecommunity.com> Message-ID: <48704E70-8EBC-441A-A4EF-42C049D726EA@zooko.com> Phillip J. Eby wrote: > Hm. So it seems to me that maybe one thing that would help is a > "Setuptools Haters' Guide To Setuptools" -- that is, *short* > documentation specifically written for people who don't want to use > setuptools and want to minimize its impact on their systems. Perhaps relevant are my blog entries on how to use setuptools with GNU stow: https://zooko.com/log-2007.html#d2007-04-27- distutils_or_setuptools_with_GNU_stow https://zooko.com/log-2007.html#d2007-06-02- distutils_or_setuptools_with_GNU_stow Regards, Zooko From zooko at zooko.com Fri Mar 21 16:53:13 2008 From: zooko at zooko.com (zooko) Date: Fri, 21 Mar 2008 09:53:13 -0600 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080318223700.C64123A4074@sparrow.telecommunity.com><20080319195438.A28DE3A40A5@sparrow.telecommunity.com><79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com><47E26A20.1090402@palladion.com><79990c6b0803201055l2831624q5e0fd1117f1ec338@mail.gmail.com><20080320190933.A03A83A4074@sparrow.telecommunity.com><79990c6b0803201334t4cf8531y2af40edc3e469fe8@mail.gmail.com> <20080320215202.618173A4074@sparrow.telecommunity.com> Message-ID: On Mar 20, 2008, at 6:22 PM, Robert Brewer wrote: > Phillip J. Eby wrote: >> The other tool that would be handy to have, would be one that unpacks >> eggs into standard distutils-style installation. > > Hear, hear. I'm an author of a couple libraries that need to > interoperate with others. Of the many eggs I've downloaded over the > past > year, I'd say 80%+ are never installed or even built--I just want to > grep the source code, and using my preferred tools, not some lame Find > command in a ZIP browser menu. Um, isn't this tool called "unzip"? I have done this -- accessed the source code -- many times, and unzip suffices. I don't know what else would be required in order to make an egg into "a standard distutils-style installation". Until PJE's comment above, I thought that unzip already accomplished exactly that. Regards, Zooko From skip at pobox.com Fri Mar 21 17:21:49 2008 From: skip at pobox.com (skip at pobox.com) Date: Fri, 21 Mar 2008 11:21:49 -0500 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E3D11F.4030503@online.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E3D11F.4030503@online.de> Message-ID: <18403.57501.22865.578079@montanaro-dyndns-org.local> Joachim> I think, the uninstall should _not_ 'rm -rf' but only 'rm' the Joachim> files (and 'rmdir' directories, but not recursively) that it Joachim> created, and that have not been modified in the meantime (after Joachim> the installation). That's not sufficient. Suppose file C (e.g. /usr/local/etc/mime.types) is in both packages A and B. Install A - this will create C Install B - this might overwrite C, saving a copy, or it might retain A's copy. Uninstall B - this has to know that C is used by A and not touch it Skip From pje at telecommunity.com Fri Mar 21 17:38:44 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 21 Mar 2008 12:38:44 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E26A20.1090402@palladion.com> <79990c6b0803201055l2831624q5e0fd1117f1ec338@mail.gmail.com> <20080320190933.A03A83A4074@sparrow.telecommunity.com> <79990c6b0803201334t4cf8531y2af40edc3e469fe8@mail.gmail.com> <20080320215202.618173A4074@sparrow.telecommunity.com> Message-ID: <20080321163848.A3B503A4074@sparrow.telecommunity.com> At 09:53 AM 3/21/2008 -0600, zooko wrote: >Um, isn't this tool called "unzip"? I have done this -- accessed the >source code -- many times, and unzip suffices. > >I don't know what else would be required in order to make an egg into >"a standard distutils-style installation". You also have to rename the EGG-INFO directory to a .egg-info file of the same basename as the original .egg; otherwise, pkg_resources and other runtime access to the egg won't know it's installed. From ronaldoussoren at mac.com Fri Mar 21 17:44:31 2008 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Fri, 21 Mar 2008 17:44:31 +0100 Subject: [Python-Dev] magic in setuptools (Was: setuptools in the stdlib) In-Reply-To: <4447E9EA.10106@v.loewis.de> References: <20060418005956.156301E400A@bag.python.org> <5.1.1.6.0.20060418113808.0217a4a0@mail.telecommunity.com> <5.1.1.6.0.20060418133427.0211a268@mail.telecommunity.com> <444537BC.7030702@egenix.com> <5.1.1.6.0.20060418151010.01e06940@mail.telecommunity.com> <5.1.1.6.0.20060418190025.037db340@mail.telecommunity.com> <44473B5C.3000409@v.loewis.de> <4447E9EA.10106@v.loewis.de> Message-ID: <72220A83-BB11-4B4A-9861-47FEC79351BF@mac.com> On 20 Apr, 2006, at 22:07, Martin v. L?wis wrote: > Guido van Rossum wrote: >> This is another area where API standardization is >> important; as soon as modules are loaded out of a zip file, the >> traditional approach of looking relative to os.path.dirname(__file__) >> no longer works. > > > 2. standardize on file names, not API. If I want to deploy > read-only data files, I put them into /usr/share. If I need > read-write files, I put them into /var. I did not have such > a problem yet on other systems, but I would also try to follow > the conventions of these systems. I don't think anyone mentioned this, but part of pkg_resources is an API for loading package-related data. That currently looks for data near the code, which makes Linux packagers unhappy but it should be possible to enhance pkg_resources to look in other locations (such as a directory below /usr/share/python) as well. The data loading API's is one feature of pkg_resources that would make it easier to locate data while at the same time storing data in locations that the various platforms proscribe without hardcoding such knowledge into every program. BTW. Please keep in mind that one of the major platforms for python doesn't have a proper package manager at all. OSX does have an installer application, but that's just that: a GUI for dropping files on you disk. There is no uninstaller, and no real dependency management. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20080321/682e2404/attachment.bin From lists at cheimes.de Fri Mar 21 17:59:45 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 21 Mar 2008 17:59:45 +0100 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> Message-ID: <47E3E981.5010503@cheimes.de> Phillip J. Eby schrieb: > Questions, comments... volunteers? :) I've yet to read the monster package utils thread so I can't comment on it. However I like to draw some attention to my PEP 370 http://python.org/dev/peps/pep-0370/. It's about a site packages directory in the users home directory. I think it quite related to the discussion. Christian From lists at cheimes.de Fri Mar 21 18:24:12 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 21 Mar 2008 18:24:12 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <47E39483.5050504@trueblade.com> References: <47E2EB45.5080202@trueblade.com> <47E39483.5050504@trueblade.com> Message-ID: <47E3EF3C.9090105@cheimes.de> Eric Smith schrieb: > It's not implementable because the work has to occur in ast.c (see > Py_UnicodeFlag). It can't occur later, because you need to skip the > encoding being done in parsestr(). But the __future__ import can only > be interpreted after the AST is built, at which time the encoding has > already been applied. There are some radical things you could do to > work around this, but it would be a gigantic change. So this basically comes down to "Either spend lots of time (and money) to rewrite the tokenizer and AST generator or keep the current behavior"? :/ > For this particular issue, just use u'' in 2.6 and let 2to3 deal with > it. If you have some 2.6 code that you want to run in 3.0 (by way of > 2to3), I think all of your string literals should either be b'' or u''. > Don't use plain ''. For this particular issue one could probably and easily come up with a fast fixer. A simple regexp should be cover 99% of all occurrences of u'' and u"". Christian From eric+python-dev at trueblade.com Fri Mar 21 19:06:23 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Fri, 21 Mar 2008 14:06:23 -0400 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <47E3EF3C.9090105@cheimes.de> References: <47E2EB45.5080202@trueblade.com> <47E39483.5050504@trueblade.com> <47E3EF3C.9090105@cheimes.de> Message-ID: <47E3F91F.3080306@trueblade.com> Christian Heimes wrote: > Eric Smith schrieb: > > It's not implementable because the work has to occur in ast.c (see >> Py_UnicodeFlag). It can't occur later, because you need to skip the >> encoding being done in parsestr(). But the __future__ import can only >> be interpreted after the AST is built, at which time the encoding has >> already been applied. There are some radical things you could do to >> work around this, but it would be a gigantic change. > > So this basically comes down to "Either spend lots of time (and money) > to rewrite the tokenizer and AST generator or keep the current behavior"? :/ Pretty much. And even if it were possible, I don't see the point in doing it. >> For this particular issue, just use u'' in 2.6 and let 2to3 deal with >> it. If you have some 2.6 code that you want to run in 3.0 (by way of >> 2to3), I think all of your string literals should either be b'' or u''. >> Don't use plain ''. > > For this particular issue one could probably and easily come up with a > fast fixer. A simple regexp should be cover 99% of all occurrences of > u'' and u"". 2to3 already does this. My current thinking is that only b'' and u'' strings should be in 2.6 code that you want to move to 3.0. Maybe -3 should warn about regular string literals? From stephen at xemacs.org Fri Mar 21 19:49:26 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 22 Mar 2008 03:49:26 +0900 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <18403.57501.22865.578079@montanaro-dyndns-org.local> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E3D11F.4030503@online.de> <18403.57501.22865.578079@montanaro-dyndns-org.local> Message-ID: <87prtngbex.fsf@uwakimon.sk.tsukuba.ac.jp> skip at pobox.com writes: > > Joachim> I think, the uninstall should _not_ 'rm -rf' but only 'rm' the > Joachim> files (and 'rmdir' directories, but not recursively) that it > Joachim> created, and that have not been modified in the meantime (after > Joachim> the installation). > > That's not sufficient. Suppose file C (e.g. /usr/local/etc/mime.types) is > in both packages A and B. > > Install A - this will create C > Install B - this might overwrite C, saving a copy, or it might retain > A's copy. > Uninstall B - this has to know that C is used by A and not touch it MacPorts has an expensive but interesting approach. When it finds a file used by another package, it backs it up to sth like $file.`date`. From pje at telecommunity.com Fri Mar 21 20:02:25 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 21 Mar 2008 15:02:25 -0400 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <18403.57501.22865.578079@montanaro-dyndns-org.local> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E3D11F.4030503@online.de> <18403.57501.22865.578079@montanaro-dyndns-org.local> Message-ID: <20080321190226.4198D3A4074@sparrow.telecommunity.com> At 11:21 AM 3/21/2008 -0500, skip at pobox.com wrote: > Joachim> I think, the uninstall should _not_ 'rm -rf' but only 'rm' the > Joachim> files (and 'rmdir' directories, but not recursively) that it > Joachim> created, and that have not been modified in the meantime (after > Joachim> the installation). > >That's not sufficient. Suppose file C (e.g. /usr/local/etc/mime.types) is >in both packages A and B. > > Install A - this will create C > Install B - this might overwrite C, saving a copy, or it might retain > A's copy. > Uninstall B - this has to know that C is used by A and not touch it Correct. However, in practice, B should not touch C, unless the file is shared between them. This is a key issue for support of namespace packages, at least if we want to avoid using .pth files. (Which is what setuptools-built system packages do for namespace packages currently.) Of course, one possible solution is for both A and B to depend on a "virtual package" that contains C, such that both A and B can install it if it's not there, and list it in their dependencies. But this is one of the handful of open issues that needs to be resolved with Real Life Package Management people, such as Debian, Fedora, etc. Neither overwriting, refusing to install, nor backups will properly address this issue. However, this is properly a topic for the Distutils-SIG or whatever SIG the actual spec goes to. On Python-Dev I'm only looking for a go/no-go on the overall approach. From mal at egenix.com Fri Mar 21 20:06:14 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 21 Mar 2008 20:06:14 +0100 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> Message-ID: <47E40726.1090209@egenix.com> On 2008-03-21 14:47, Phillip J. Eby wrote: > So, to accomplish this, we (for some value of "we") need to: > > 1. Hash out consensus around what changes or enhancements are needed > to PEP 262, to resolve the previously-listed open issues, those that > have come up since (namespace packages, dependency specifications, > canonical name/version forms), and anything else that comes up. > > 2. Update or replace the implementation as appropriate, and modify > the distutils to support it in Python 2.6 and beyond. And "support > it" means, "ensure that 'install' and *all* bdist commands update the > database". The bdist_rpm, bdist_wininst, and bdist_msi commands, > even bdist_dumb. (This should probably also include the add/remove > programs stuff in the Windows case.) The bdist commands don't need to touch that database in any way, since they don't install anything, nor do they upload things anywhere. They simply package code and put the result into the dist/ subdir. That's all. What you probably mean is that the installers, pre/post-scripts, etc. run when installing one of those packages should update the database of installed packages. Note that there are several package formats which do not execute any code when installing them - the user simply unzips them in some directory. These packages won't be able to register themselves with a database. I guess the only way to support all of these variants is to use a filesystem based approach, e.g. by placing a file with a special extension into some dir on sys.path. The "database" logic could then scan sys.path for these files, read the data and provide an interface to it. All bdist formats would then have to include these files. distutils already writes .egg-info files when running python setup.py install, so perhaps that's a start (though I'd prefer a three letter extension such as ".pkg"). .egg-info files currently only include the package meta-data (the PKG-INFO section from PEP 262). We'd have to add a list of files making up the package (FILES section in PEP 262) and also some extra information about any extra files the package creates that can safely be removed in the uninstall process (e.g. .pyo and .pyc files, temporary files, database files, configuration data, registry entries, etc.) - this is currently not covered in PEP 262. I don't think the REQUIRES and PROVIDES sections from the PEP 262 are needed. That info can easily go into the PKG-INFO section. A separate FILES section also doesn't seem to be necessary - we could just add one or more entries or the format: CreatesDir abc/ CreatesFile abc/xyz1.py CreatesDir abc/def/ CreatesFile abc/def/xyz2.py CreatesFile abc/def/xyz3.py CreatesFile abc/def/xyz4.ini (BTW: wininst writes such a file for the uninstall process) So to keep things simple, the rfc822 approach defined in PEP 241 would easily cover everything needed and we could trim down the PEP 262 format to a simple rfc822 header list. In other words: the .egg-info files already provide the basis and only need to be extended with a list of created files, directories (and possibly other resources) as well as a list of resources which may be removed even if not installed explicitly such as byte-code files, etc. > 3. Create a document for system packagers referencing the PEP and > introducing them to what/why/how of the standard, in case they > weren't one of the original participants in creating this. This should probably be a new PEP defining all the bits and pieces making up the installation "database". > It will probably take some non-trivial work to do all this for Python > 2.6, but it's probably possible, if we start now. I don't think it's > critical to have an uninstall tool distributed with 2.6, as long as > there's a reasonable way to bootstrap its installation later. BTW: There's a simple uninstall command in mxSetup.py that we could contribute to distutils. It works much in the same way as the install command... except that it removes all the files it would have installed. Using pre-built packages, this works without having to rebuild the package just to be able to determine the list of things that need to be removed. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 21 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From tjreedy at udel.edu Fri Mar 21 20:16:45 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 21 Mar 2008 15:16:45 -0400 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) References: <20080318223700.C64123A4074@sparrow.telecommunity.com><20080319195438.A28DE3A40A5@sparrow.telecommunity.com><79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com><47E26A20.1090402@palladion.com> <79990c6b0803210533q3f9310c8tf119300a77030ccf@mail.gmail.com> <79990c6b0803210533q3f9310c8tf119300a77030ccf@mail.gmail.co m> <20080321124944.ABDF03A4074@sparrow.telecommunity.com> Message-ID: "Phillip J. Eby" wrote in message news:20080321124944.ABDF03A4074 at sparrow.telecommunity.com... To Paul's question: I have only installed a couple of things (and not recently) that added their own add/remove entry. But I am not sure I would have called them add-ons as opposed to independent applications written in Python. | I presume he means that there are a lot of entries in his Add/Remove | Programs that work like that, and that it's an emerging standard for | Windows. (Certainly I've seen quite a few entries like that in mine, | although more often than not they only have one checkbox!) Yes. At least half my experience with uninstalls is removing games. Recent games typically have separate boxes for various things such as games files, save files, mods, game directory and any user added content, and icons and registry entries. Most Python add-ons I have downloaded are unziped to site-packages and only a few megabytes in size (versus gigabytes for some games). Hence there is little need to uninstall them (unless dumping everything connected with pyx.y, which is easy). Hence no desire to have add/remove slowed down and cluttered with dozens of entries for such things. I admit that my wish for a better installation manager is something I can only help with on the surface by expressing desires and testing results as a practice user. tjr From optilude at gmx.net Fri Mar 21 15:38:27 2008 From: optilude at gmx.net (Martin Aspeli) Date: Fri, 21 Mar 2008 14:38:27 +0000 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> Message-ID: Phillip J. Eby wrote: > Questions, comments... volunteers? :) This makes a lot of sense. I don't really have anything to add in terms of implementation, but I wonder if we can learn something from how apt or rpms or ports work, and how other programming languages (Ruby gems?) solve this. It's not an easy problem space since you're dealing with disparate and inconsistent target platforms, but it's one that has been solved before by others. Cheers, Martin From p.f.moore at gmail.com Fri Mar 21 20:44:55 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 21 Mar 2008 19:44:55 +0000 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> Message-ID: <79990c6b0803211244g38626197oc454f18dc208e7eb@mail.gmail.com> On 21/03/2008, Phillip J. Eby wrote: > Questions, comments... volunteers? :) Sounds good. I won't volunteer as I have neither time nor expertise to contribute much. But I'd like to see this happen, as it sounds like it would address all my issues with setuptools (and just to reiterate, I'm happy if it is implemented *in place of* add/remove programs - I have no vested interest in that facility as such). For uninstall support, you might look at how bdist_wininst installers handle it. In my experience the uninstall is robust, clean, and reliable, and the supporting metadata is pretty simple (just a text file). Paul. From gnewsg at gmail.com Fri Mar 21 21:15:41 2008 From: gnewsg at gmail.com (Giampaolo Rodola') Date: Fri, 21 Mar 2008 13:15:41 -0700 (PDT) Subject: [Python-Dev] SVN FAQ for Windows users Message-ID: <703b70ea-d057-403c-8fd4-fce68179f699@s37g2000prg.googlegroups.com> I noticed that http://www.python.org/dev/faq/#how-do-i-apply-a-patch ...does not contain instruction for Windows user willing to apply a patch starting from a .diff file. Since Tortoise SVN is the most common choice I tried to describe the steps to apply a patch by using it. This could be added in the same section (5.2) righ after the UNIX instructions. --- snippet --- If you're using Windows make sure you have TortoiseSVN installed. Right-click on the folder containing the trunk source code, expand the TortoiseSVN submenu and select 'Apply Patch...'. Browse to the patch file and select it, then right click on the file appearing in the 'File Patches' window and click on 'Patch selected'. After you're done, close the TortoiseMerge window. --- /snippet --- Hope this helps. From lists at cheimes.de Fri Mar 21 21:31:47 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 21 Mar 2008 21:31:47 +0100 Subject: [Python-Dev] SVN FAQ for Windows users In-Reply-To: <703b70ea-d057-403c-8fd4-fce68179f699@s37g2000prg.googlegroups.com> References: <703b70ea-d057-403c-8fd4-fce68179f699@s37g2000prg.googlegroups.com> Message-ID: Thanks Giampaolo! Totally unrelated to your posting: I wonder why http://www.python.org/dev/faq/#how-to-make-a-patch uses tee instead of > file ? Christian From brett at python.org Fri Mar 21 21:50:47 2008 From: brett at python.org (Brett Cannon) Date: Fri, 21 Mar 2008 13:50:47 -0700 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: <20080321084249.GA83712@nexus.in-nomine.org> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <20080321084249.GA83712@nexus.in-nomine.org> Message-ID: On Fri, Mar 21, 2008 at 1:42 AM, Jeroen Ruigrok van der Werven wrote: > (To provide counterweight.) > > > -On [20080320 20:44], Barry Warsaw (barry at python.org) wrote: > >We have not made a decision to move to Bazaar officially, nor have we made > >a decision to even move off of Subversion. > > Good, because between this now and pytz the other 63 projects I follow use > Subversion or Mercurial. > Bazaar seems to be mostly limited to Ubuntu users and stuff Canonical does, > so the choice for a Bazaar setup next to Subversion strikes me a bit as odd. > Mercurial is also Python, distributed and with a, as far as I (can) track > things, bigger market share than Bazaar. > Just to head this off, this is not a specific vote of confidence for Bazaar. The Bazaar developers were at PyCon and both Barry and Thomas were willing to put the time and effort to get the mirror up and going while the Bazaar team was nearby to answer questions. At this point this is more of an experiment to see if people like the development style of distributed VCS. If people do and someone steps forward to put the time and effort to add Mercurial support, we can then consider doing something similar to what was done for the issue tracker. But for right now this is just to try out distributed VCS, nothing more. -Brett From musiccomposition at gmail.com Fri Mar 21 21:54:04 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 21 Mar 2008 15:54:04 -0500 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> Message-ID: <1afaf6160803211354v615ea33l528327e9a0b6dad9@mail.gmail.com> On Thu, Mar 20, 2008 at 2:42 PM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I'm happy to announce that we now have available for public > consumption, the Python source code for 2.5, 2.6 and 3.0 available > under the Bazaar distributed version control system. > > The current Subversion repository is still the master copy of the > source code. We have not made a decision to move to Bazaar > officially, nor have we made a decision to even move off of > Subversion. We're making these branches available exactly so that > you, the Python developer community, can kick the tires and see if it > makes sense to move to a different vcs. Nothing will happen until > after the Python 2.6/3.0 releases anyway. > > All the gory details are documented here: > > http://www.python.org/dev/bazaar > > These branches are available both for core Python developers with > commit privileges, and the wider world of developers without commit > privileges. It's this latter group that I think will find the most > compelling immediate benefit from using Bazaar, because they will no > longer need to maintain their own changes using a mass of patch files. > > For more information on Bazaar in general, see: > > http://bazaar-vcs.org > > You will probably be most interested in the Bazaar mirrors of the > Subversion master repository. We have a cron job that updates Bazaar > from Subversion every 15 minutes. It is also possible to push changes > made in your Bazaar branches into the Subversion master, so you can > keep reasonably up-to-date and interact with the Python source code > solely via Bazaar. Great work! I love Bazaar. It's a lot more flexible and user-friendlier than anything else out there. I hope we can switch to Bazaar entirely sometime! :) > > > Please let me know if you have any questions or anything in the docs > referenced above aren't clear. I know I need to document the Bazaar- > >Subversion workflow in more detail. > > Huge thanks go out especially to Thomas Wouters who sprinted with me > yesterday on getting the whole infrastructure up and running. Thanks > also to Martin v. Loewis, Sean Reifschneider, and the folks here at > Pycon from the Bazaar project, Ian, Andrew, John, and Edwin. > > Enjoy, > - -Barry > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.8 (Darwin) > > iQCVAwUBR+K+H3EjvBPtnXfVAQL6bwP/d0XKx0mRPgzdOD6zCPwwRl8y2kxWV9Vl > zVwN07cK8IlwMa9M470MsERuXuD8hmeXnPgnrUcrX9HciwldmY8C33t0f8GaOk1J > iD4Od1midlIaJJiMd+JcFk9NbX3Rwc4HMj3s8jKjjdlGrFe77ei9DCZ/HMl/iG5K > fyyjXo4HLEE= > =Gcjq > -----END PGP SIGNATURE----- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080321/4d24b30e/attachment-0001.htm From brett at python.org Fri Mar 21 21:54:52 2008 From: brett at python.org (Brett Cannon) Date: Fri, 21 Mar 2008 13:54:52 -0700 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <47E3F91F.3080306@trueblade.com> References: <47E2EB45.5080202@trueblade.com> <47E39483.5050504@trueblade.com> <47E3EF3C.9090105@cheimes.de> <47E3F91F.3080306@trueblade.com> Message-ID: On Fri, Mar 21, 2008 at 11:06 AM, Eric Smith wrote: > Christian Heimes wrote: > > Eric Smith schrieb: > > > It's not implementable because the work has to occur in ast.c (see > >> Py_UnicodeFlag). It can't occur later, because you need to skip the > >> encoding being done in parsestr(). But the __future__ import can only > >> be interpreted after the AST is built, at which time the encoding has > >> already been applied. There are some radical things you could do to > >> work around this, but it would be a gigantic change. > > > > So this basically comes down to "Either spend lots of time (and money) > > to rewrite the tokenizer and AST generator or keep the current behavior"? :/ > > Pretty much. And even if it were possible, I don't see the point in > doing it. > > > >> For this particular issue, just use u'' in 2.6 and let 2to3 deal with > >> it. If you have some 2.6 code that you want to run in 3.0 (by way of > >> 2to3), I think all of your string literals should either be b'' or u''. > >> Don't use plain ''. > > > > For this particular issue one could probably and easily come up with a > > fast fixer. A simple regexp should be cover 99% of all occurrences of > > u'' and u"". > > 2to3 already does this. > > My current thinking is that only b'' and u'' strings should be in 2.6 > code that you want to move to 3.0. Maybe -3 should warn about regular > string literals? That's a possibility. It might also help to have a 3to2 fixer that goes through a module and adds the needed prefixes so one doesn't have to go through manually to tack them on. -Brett From brett at python.org Fri Mar 21 22:01:19 2008 From: brett at python.org (Brett Cannon) Date: Fri, 21 Mar 2008 14:01:19 -0700 Subject: [Python-Dev] [Python-3000] Python source code on Bazaar vcs In-Reply-To: <47E2DBEF.8010809@cheimes.de> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <47E2DBEF.8010809@cheimes.de> Message-ID: On Thu, Mar 20, 2008 at 2:49 PM, Christian Heimes wrote: > Barry Warsaw schrieb: > > > I'm happy to announce that we now have available for public > > consumption, the Python source code for 2.5, 2.6 and 3.0 available > > under the Bazaar distributed version control system. > > Somebody has to fix the subversion related code in Python/sysmodule.c: > > heimes at hamiller:~/dev/python/bzr/trunk$ ./python > Fatal Python error: subversion keywords missing > Aborted (core dumped) The ironic thing is that Barry is the one that added that code. =) -Brett From martin at v.loewis.de Fri Mar 21 22:20:56 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 21 Mar 2008 22:20:56 +0100 Subject: [Python-Dev] Request for another build slave In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E16844C8A@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E16844C8A@EXMBX04.exchhosting.com> Message-ID: <47E426B8.5080002@v.loewis.de> > [Suggestion: perhaps we could set up a python-buildbots at python.org > list for discussing buildbot administrative minutiae, rather than > polluting python-dev?] I think polluting python-dev is fine. Regards, Martin From pje at telecommunity.com Fri Mar 21 22:21:20 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 21 Mar 2008 17:21:20 -0400 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E40726.1090209@egenix.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> Message-ID: <20080321212127.791B63A4074@sparrow.telecommunity.com> At 08:06 PM 3/21/2008 +0100, M.-A. Lemburg wrote: >I guess the only way to support all of these variants is >to use a filesystem based approach, e.g. by placing a file >with a special extension into some dir on sys.path. >The "database" logic could then scan sys.path for these >files, read the data and provide an interface to it. > >All bdist formats would then have to include these files. That's the idea behind the current version of PEP 262, yes, and I think it should be kept. >A separate FILES section also doesn't seem to be necessary - >we could just add one or more entries or the format: > >CreatesDir abc/ >CreatesFile abc/xyz1.py >CreatesDir abc/def/ >CreatesFile abc/def/xyz2.py >CreatesFile abc/def/xyz3.py >CreatesFile abc/def/xyz4.ini I actually think the size and hash information is good, in order to be able to tell if you're looking at an original file. I'm not sure how useful the permissions and uid/gid info is. I'm hoping we'll hear from anybody who has a use case for that. And of course, there are still some issues to be resolved regarding requirements, package name/version stuff, etc. But we can hash those out once we reach a quorum on the Distutils-SIG. From pje at telecommunity.com Fri Mar 21 22:22:53 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 21 Mar 2008 17:22:53 -0400 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E3E981.5010503@cheimes.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E3E981.5010503@cheimes.de> Message-ID: <20080321212255.32DB03A40A5@sparrow.telecommunity.com> At 05:59 PM 3/21/2008 +0100, Christian Heimes wrote: >Phillip J. Eby schrieb: > > Questions, comments... volunteers? :) > >I've yet to read the monster package utils thread so I can't comment on >it. However I like to draw some attention to my PEP 370 >http://python.org/dev/peps/pep-0370/. It's about a site packages >directory in the users home directory. I think it quite related to the >discussion. Actually, it's 100% orthogonal, if done correctly. If anything, this slightly reduces the need for per-user site-packages, since in the new world, .pth files would generally only be needed for "develop" installs. (Assuming we find a way to support namespace packages without using .pth files, that is.) From p.f.moore at gmail.com Fri Mar 21 22:28:07 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 21 Mar 2008 21:28:07 +0000 Subject: [Python-Dev] [Python-3000] Python source code on Bazaar vcs In-Reply-To: References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <20080321084249.GA83712@nexus.in-nomine.org> Message-ID: <79990c6b0803211428h2eef9fa0s1032570cdf3fe162@mail.gmail.com> On 21/03/2008, Brett Cannon wrote: > Just to head this off, this is not a specific vote of confidence for > Bazaar. The Bazaar developers were at PyCon and both Barry and Thomas > were willing to put the time and effort to get the mirror up and going > while the Bazaar team was nearby to answer questions. At this point > this is more of an experiment to see if people like the development > style of distributed VCS. If people do and someone steps forward to > put the time and effort to add Mercurial support, we can then consider > doing something similar to what was done for the issue tracker. > > But for right now this is just to try out distributed VCS, nothing more. One thing I really like the idea of with Mercurial for my situation (non-committer) is the mq extension, which lets me manage my changes as a "stack of patches" - so I'm completely working with patches, which is what I have to post to the tracker, etc. Is there a similar workflow in Bazaar? I know there's the loom extension (although I haven't used it much yet) but I'm not sure how I'd use that. Basically, can some Bazaar expert offer a suggestion as to how a non-developer with read-only access would best use the Bazaar repositories to maintain a number of patches to be posted to the tracker? Thanks, Paul. From martin at v.loewis.de Fri Mar 21 22:32:19 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 21 Mar 2008 22:32:19 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <47E39483.5050504@trueblade.com> References: <47E2EB45.5080202@trueblade.com> <47E39483.5050504@trueblade.com> Message-ID: <47E42963.1030500@v.loewis.de> > It's not implementable because the work has to occur in ast.c (see > Py_UnicodeFlag). It can't occur later, because you need to skip the > encoding being done in parsestr(). But the __future__ import can only > be interpreted after the AST is built, at which time the encoding has > already been applied. I think it would be possible to check for future statements on the basis of nodes already. Take a look at how Python 2.3 implemented future statements (why was that rewritten to use the AST, anyway?). > As for it not making sense, this is really in the realm of 2to3. I'm > beginning to really believe this statement in PEP 3000: There is still the original use case of people who don't want to run 2to3 (for whatever reasons - mostly probably subjective ones), and who would rather run a single code base unmodified. They don't care that documentation tells them this is impossible, when they feel they are so close to making it possible. Regards, Martin From mal at egenix.com Fri Mar 21 22:35:47 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 21 Mar 2008 22:35:47 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <47E42963.1030500@v.loewis.de> References: <47E2EB45.5080202@trueblade.com> <47E39483.5050504@trueblade.com> <47E42963.1030500@v.loewis.de> Message-ID: <47E42A33.8020407@egenix.com> On 2008-03-21 22:32, Martin v. L?wis wrote: >> It's not implementable because the work has to occur in ast.c (see >> Py_UnicodeFlag). It can't occur later, because you need to skip the >> encoding being done in parsestr(). But the __future__ import can only >> be interpreted after the AST is built, at which time the encoding has >> already been applied. > > I think it would be possible to check for future statements on the > basis of nodes already. Take a look at how Python 2.3 implemented > future statements (why was that rewritten to use the AST, anyway?). > >> As for it not making sense, this is really in the realm of 2to3. I'm >> beginning to really believe this statement in PEP 3000: > > There is still the original use case of people who don't want to run > 2to3 (for whatever reasons - mostly probably subjective ones), and > who would rather run a single code base unmodified. They don't care > that documentation tells them this is impossible, when they feel they > are so close to making it possible. Could we point them to a special byte-code compiler such as Andrew Dalke's python4ply: http://dalkescientific.com/Python/python4ply.html That approach appears to be a lot easier to implement than trying to tweak the C implementation of the Python parser. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 21 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From p.f.moore at gmail.com Fri Mar 21 22:38:18 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 21 Mar 2008 21:38:18 +0000 Subject: [Python-Dev] Python source code on Mercurial In-Reply-To: References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <932f8baf0803201426ua0ae2edu3f7b62eaf22ef787@mail.gmail.com> Message-ID: <79990c6b0803211438w347d052fre2a97ebde72b44b4@mail.gmail.com> On 21/03/2008, Antoine Pitrou wrote: > I see your trunk history is stripped. For those who want the complete trunk > history (back to 17 years ago!), I have my own mirror here: > http://dev.pitrou.net:8000/cpython/trunk/ Excellent! For what it's worth, hg clone took 5 minutes on my PC (with broadband access). That's faster than simply downloading the Bazaar shared repository tarball (which took 13 minutes)! Are you keeping the mirror updated with respect to Subversion? I don't want to start a comparison at this point, but as neither system offers downloading of truncated history at the moment, the speed to grab a new clone of the full repository is likely to be pretty important. Paul. From martin at v.loewis.de Fri Mar 21 22:40:11 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 21 Mar 2008 22:40:11 +0100 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: References: <47E0D883.9030402@taupro.com><47E11659.9000307@v.loewis.de> <47E1BB1D.4060303@v.loewis.de> <43aa6ff70803201117y46e09c5frd6e5be6e4902b23c@mail.gmail.com><47E2CA12.2000506@v.loewis.de> <47E2CE0E.6010108@rksystems.com><47E2D716.4030109@v.loewis.de> <47E2DB20.5020909@rksystems.com> <47E2E02F.1020007@v.loewis.de> Message-ID: <47E42B3B.9000908@v.loewis.de> > | try: > | bytes > | except NameError: > | bytes = str > | > | somewhere, then write the code as I proposed above. > > This is exactly the sort of thing that should be and I expect will be in > the conversion docs. That expectation might be unfounded (in the sense that it might not be written at all). I only happened to learn about that approach as I was converting Django to 3k. I'll happily put my experience in doing so into an email message, but I won't sit down and write a text document. Regards, Martin From martin at v.loewis.de Fri Mar 21 22:46:01 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 21 Mar 2008 22:46:01 +0100 Subject: [Python-Dev] SVN FAQ for Windows users In-Reply-To: <703b70ea-d057-403c-8fd4-fce68179f699@s37g2000prg.googlegroups.com> References: <703b70ea-d057-403c-8fd4-fce68179f699@s37g2000prg.googlegroups.com> Message-ID: <47E42C99.4020202@v.loewis.de> > --- snippet --- > If you're using Windows make sure you have TortoiseSVN installed. > Right-click on the folder containing the trunk source code, expand the > TortoiseSVN submenu and select 'Apply Patch...'. Browse to the patch > file and select it, then right click on the file appearing in the > 'File Patches' window and click on 'Patch selected'. After you're > done, close the TortoiseMerge window. > --- /snippet --- That never worked for me. TortoiseSVN then insists on fetching the revisions mentioned in the patch, runs some math, and tells me that I can't apply the patch because it is out of date (assuming it really is, which is normally the case when I get to look at a patch). I then try cygwin patch, which applies the patch nicely, but messes up the line endings while doing so. So in the end, I conclude that it just isn't possible to apply patches on Windows, and log into a Linux machine to apply the patch. Regards, Martin From martin at v.loewis.de Fri Mar 21 22:47:03 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 21 Mar 2008 22:47:03 +0100 Subject: [Python-Dev] SVN FAQ for Windows users In-Reply-To: References: <703b70ea-d057-403c-8fd4-fce68179f699@s37g2000prg.googlegroups.com> Message-ID: <47E42CD7.2040009@v.loewis.de> > Totally unrelated to your posting: > > I wonder why http://www.python.org/dev/faq/#how-to-make-a-patch uses tee > instead of > file ? So you can see what the patch looks like? Regards, Martin From solipsis at pitrou.net Fri Mar 21 22:53:51 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 21 Mar 2008 21:53:51 +0000 (UTC) Subject: [Python-Dev] Python source code on Mercurial References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <932f8baf0803201426ua0ae2edu3f7b62eaf22ef787@mail.gmail.com> <79990c6b0803211438w347d052fre2a97ebde72b44b4@mail.gmail.com> Message-ID: Hi, Paul Moore gmail.com> writes: > > Excellent! For what it's worth, hg clone took 5 minutes on my PC (with > broadband access). That's faster than simply downloading the Bazaar > shared repository tarball (which took 13 minutes)! > > Are you keeping the mirror updated with respect to Subversion? It is synchronized once an hour. cheers Antoine. From musiccomposition at gmail.com Fri Mar 21 22:57:34 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 21 Mar 2008 16:57:34 -0500 Subject: [Python-Dev] [Python-3000] Python source code on Bazaar vcs In-Reply-To: <79990c6b0803211428h2eef9fa0s1032570cdf3fe162@mail.gmail.com> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <20080321084249.GA83712@nexus.in-nomine.org> <79990c6b0803211428h2eef9fa0s1032570cdf3fe162@mail.gmail.com> Message-ID: <1afaf6160803211457y29cb0ce8s58898876d2f96a5b@mail.gmail.com> On Fri, Mar 21, 2008 at 4:28 PM, Paul Moore wrote: > On 21/03/2008, Brett Cannon wrote: > > Just to head this off, this is not a specific vote of confidence for > > Bazaar. The Bazaar developers were at PyCon and both Barry and Thomas > > were willing to put the time and effort to get the mirror up and going > > while the Bazaar team was nearby to answer questions. At this point > > this is more of an experiment to see if people like the development > > style of distributed VCS. If people do and someone steps forward to > > put the time and effort to add Mercurial support, we can then consider > > doing something similar to what was done for the issue tracker. > > > > But for right now this is just to try out distributed VCS, nothing > more. > > One thing I really like the idea of with Mercurial for my situation > (non-committer) is the mq extension, which lets me manage my changes > as a "stack of patches" - so I'm completely working with patches, > which is what I have to post to the tracker, etc. > > Is there a similar workflow in Bazaar? I know there's the loom > extension (although I haven't used it much yet) but I'm not sure how > I'd use that. > bzr-loom is what you want. > > > Basically, can some Bazaar expert offer a suggestion as to how a > non-developer with read-only access would best use the Bazaar > repositories to maintain a number of patches to be posted to the > tracker? I tend to make a repository and make a working copy for each patch in it. The history is saved in the repository so it's efficient. > > > Thanks, > Paul. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080321/8d9cabd0/attachment.htm From p.f.moore at gmail.com Fri Mar 21 22:58:27 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 21 Mar 2008 21:58:27 +0000 Subject: [Python-Dev] SVN FAQ for Windows users In-Reply-To: <47E42C99.4020202@v.loewis.de> References: <703b70ea-d057-403c-8fd4-fce68179f699@s37g2000prg.googlegroups.com> <47E42C99.4020202@v.loewis.de> Message-ID: <79990c6b0803211458j1030e65cx75619038605de921@mail.gmail.com> On 21/03/2008, "Martin v. L?wis" wrote: [...] > I then try cygwin patch, which applies the patch nicely, but messes > up the line endings while doing so. > > So in the end, I conclude that it just isn't possible to apply patches > on Windows, and log into a Linux machine to apply the patch. Generally, I use gnuwin32 patch (http://gnuwin32.sf.net), with the --binary flag. That seems to work most of the time for me. (The cygwin version may also have --binary). Paul. From p.f.moore at gmail.com Fri Mar 21 23:04:30 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 21 Mar 2008 22:04:30 +0000 Subject: [Python-Dev] [Python-3000] Python source code on Bazaar vcs In-Reply-To: <1afaf6160803211457y29cb0ce8s58898876d2f96a5b@mail.gmail.com> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <20080321084249.GA83712@nexus.in-nomine.org> <79990c6b0803211428h2eef9fa0s1032570cdf3fe162@mail.gmail.com> <1afaf6160803211457y29cb0ce8s58898876d2f96a5b@mail.gmail.com> Message-ID: <79990c6b0803211504o78032f66vbe55064bdf7a9f3@mail.gmail.com> On 21/03/2008, Benjamin Peterson wrote: > I tend to make a repository and make a working copy for each patch in it. > The history is saved in the repository so it's efficient. OK, so just lots of copies, fair enough. Presumably just use bzr diff to create patches? Much like Subversion, in practice, but with local commits of partial work. Paul. From musiccomposition at gmail.com Fri Mar 21 23:10:41 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 21 Mar 2008 17:10:41 -0500 Subject: [Python-Dev] [Python-3000] Python source code on Bazaar vcs In-Reply-To: <79990c6b0803211504o78032f66vbe55064bdf7a9f3@mail.gmail.com> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <20080321084249.GA83712@nexus.in-nomine.org> <79990c6b0803211428h2eef9fa0s1032570cdf3fe162@mail.gmail.com> <1afaf6160803211457y29cb0ce8s58898876d2f96a5b@mail.gmail.com> <79990c6b0803211504o78032f66vbe55064bdf7a9f3@mail.gmail.com> Message-ID: <1afaf6160803211510i3026e2f0n2ee534ff514be459@mail.gmail.com> On Fri, Mar 21, 2008 at 5:04 PM, Paul Moore wrote: > On 21/03/2008, Benjamin Peterson wrote: > > I tend to make a repository and make a working copy for each patch in > it. > > The history is saved in the repository so it's efficient. > > OK, so just lots of copies, fair enough. Presumably just use bzr diff > to create patches? Much like Subversion, in practice, but with local > commits of partial work. Yes, bzr diff should do the trick, although if you have local commits in it, you'll have to give the revision number manually. Bazaar also has this cool feature called merge directives. They let you bundle your branch (with all the commits) in a patch-like file, which can be easily merged into the mainline by core devs. Of course, Python hasn't moved to Bazaar, so widespread use of those is unlikely soon. > > > Paul. > Cheers, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080321/354dce99/attachment.htm From mal at egenix.com Fri Mar 21 23:13:32 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 21 Mar 2008 23:13:32 +0100 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080321212127.791B63A4074@sparrow.telecommunity.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> Message-ID: <47E4330C.5070601@egenix.com> On 2008-03-21 22:21, Phillip J. Eby wrote: > At 08:06 PM 3/21/2008 +0100, M.-A. Lemburg wrote: >> I guess the only way to support all of these variants is >> to use a filesystem based approach, e.g. by placing a file >> with a special extension into some dir on sys.path. >> The "database" logic could then scan sys.path for these >> files, read the data and provide an interface to it. >> >> All bdist formats would then have to include these files. > > That's the idea behind the current version of PEP 262, yes, and I think > it should be kept. > >> A separate FILES section also doesn't seem to be necessary - >> we could just add one or more entries or the format: >> >> CreatesDir abc/ >> CreatesFile abc/xyz1.py >> CreatesDir abc/def/ >> CreatesFile abc/def/xyz2.py >> CreatesFile abc/def/xyz3.py >> CreatesFile abc/def/xyz4.ini > > I actually think the size and hash information is good, in order to be > able to tell if you're looking at an original file. I'm not sure how > useful the permissions and uid/gid info is. I'm hoping we'll hear from > anybody who has a use case for that. You're heading off in the wrong direction: we should not be trying to rewrite RPM or InnoSetup in Python. Anything more complicated should be left to tools which are specifically written to manage complex software setups. I honestly believe that most people would be happy if we just provide these two things (and no more): * install a package from a local archive, a URL or PyPI * uninstall a package in way that doesn't break other installed packages and whatever the mechanism, avoid making any undercover changes to the Python installation such as adding .pth files, overriding site.py, etc. - these are not needed if the tool keeps to the simple task of installing and uninstalling Python packages. Examples: python pypi.py install mypkg-1.0.tgz python pypi.py install http://www.example.com/mypkg-1.0.tgz python pypi.py install mypkg-1.0 python pypi.py uninstall mypkg If there's a dependency problem, the tool should print the list of other packages it needs. It should not try to install things automagically. If a package needs other modules as well, the package docs can point the user to use e.g. python pypi.py install mydep1-1.3 mydep2-2.3 mydep4-0.3 mypkg-1.0 instead. Anything more complicated should be left to specialized tools such as RPM, apt, MSI or the other such tools out there - after all the tool should be about Python *package* installation, not application installation. We *don't* need the tool to: * support multiple versions of a package (that's just bound to cause problems with pickle, isinstance() etc.) * provide namespace hacking (is a completely separate issue and can be handled by the packages rather than the install tool) * support all kinds of funky version numbers (if a package wants to participate in the system, the author better make sure that the version string fits the standard format) * provide some form of intra-package bus interface (ie. "entry points" as you call them) * provide support for keeping whole packages in ZIP files (doesn't play well with C extensions, clutters up the sys.path, is read-only, needs special importers, etc. etc. ) * try automatic version matching for required packages * download things from SourceForge or other sites with special download mechanisms * scan websites for links * make coffee, clean the house, send the kids to school :-) > And of course, there are still some issues to be resolved regarding > requirements, package name/version stuff, etc. But we can hash those > out once we reach a quorum on the Distutils-SIG. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 21 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From guido at python.org Fri Mar 21 23:15:55 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 21 Mar 2008 15:15:55 -0700 Subject: [Python-Dev] PEP 3127 (Integer Literal Support and Syntax): %o and %b In-Reply-To: <47E09255.8090907@trueblade.com> References: <47E09255.8090907@trueblade.com> Message-ID: On Tue, Mar 18, 2008 at 9:11 PM, Eric Smith wrote: > I've been double checking the PEP 3127 implementation in py3k and the > backport I did to 2.6. The PEP says this about the % operator: > > "The string (and unicode in 2.6) % operator will have 'b' format > specifier added for binary, and the alternate syntax of the 'o' option > will need to be updated to add '0o' in front, instead of '0'." > > The %b operator was not added to 3.0, so I'll look into doing that in > both 2.6 and 3.0 (which I opened as issue 2416). > > What should be done for '%#o' formatting in 2.6? The above sentence > from the PEP implies it should be modified to add '0o' instead of '0', > even in 2.6. But that seems like a bad idea to me. Maybe it should > stay as-is, but add a -3 warning? Unfortunately, there'd be no way to > change your code to get rid of the warning, short of switching to > str.format() or adding a __future__ import (shudder). In 3.0, '%#o' > already adds the leading '0o'. I think this is such a tiny detail we shouldn't bother with a -3 warning. I agree that in 2.6, %#o should continue to do what it does in 2.5. Can you update the PEP? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at pobox.com Fri Mar 21 23:17:00 2008 From: skip at pobox.com (skip at pobox.com) Date: Fri, 21 Mar 2008 17:17:00 -0500 Subject: [Python-Dev] Primer on distributed revision control? Message-ID: <18404.13276.592156.81914@montanaro-dyndns-org.local> With all these distributed revision control systems now available (bzr, hg, darcs, svk, many more), I find I need an introduction to the concepts and advantages of repository distribution. It seems to me that it has the potential for leading to anarchy, though I can see how some things would be improved (working offline, maintaining local patches). It's not obvious how I push changes back upstream. Can someone point me to some useful content (web pages or books) which will help me wrap my brain around the ideas? Maybe a compare/contrast of the major players? Thanks, Skip From nyamatongwe at gmail.com Fri Mar 21 23:24:14 2008 From: nyamatongwe at gmail.com (Neil Hodgson) Date: Sat, 22 Mar 2008 09:24:14 +1100 Subject: [Python-Dev] PEP 365 (Adding the pkg_resources module) In-Reply-To: References: <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E26A20.1090402@palladion.com> <79990c6b0803201055l2831624q5e0fd1117f1ec338@mail.gmail.com> <20080320190933.A03A83A4074@sparrow.telecommunity.com> <79990c6b0803201334t4cf8531y2af40edc3e469fe8@mail.gmail.com> <20080320215202.618173A4074@sparrow.telecommunity.com> Message-ID: <50862ebd0803211524o46d68eeaq1120528c2bce758c@mail.gmail.com> zooko: > Um, isn't this tool called "unzip"? I have done this -- accessed the > source code -- many times, and unzip suffices. The type of issue I ran into with eggs is when you get an exception with a trace that includes an egg, you can't use the normal means to look at the code. Instead you have to understand that its an egg, unzip the code, manually translate the path, open the file and go to the line number. Similarly, you can't easily grep the code in its egg state. If there was a global flag where I could say 'install eggs as directories of source' then I'd be much happier. Just reread the EasyInstall documentation and '--always-unzip' is portrayed as a 'don't do this' option. As it is I just avoid eggs. They may make sense for installing applications but for development they get in the way. Neil From musiccomposition at gmail.com Fri Mar 21 23:24:45 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 21 Mar 2008 17:24:45 -0500 Subject: [Python-Dev] Primer on distributed revision control? In-Reply-To: <18404.13276.592156.81914@montanaro-dyndns-org.local> References: <18404.13276.592156.81914@montanaro-dyndns-org.local> Message-ID: <1afaf6160803211524w4f6852cfg265c513755430008@mail.gmail.com> On Fri, Mar 21, 2008 at 5:17 PM, wrote: > With all these distributed revision control systems now available (bzr, > hg, > darcs, svk, many more), I find I need an introduction to the concepts and > advantages of repository distribution. It seems to me that it has the > potential for leading to anarchy, though I can see how some things would > be > improved (working offline, maintaining local patches). It's not obvious > how > I push changes back upstream. Can someone point me to some useful content > (web pages or books) which will help me wrap my brain around the ideas? > Maybe a compare/contrast of the major players? Unfortunately, Wikipedia's article doesn't seem up to snuff. http://www.dwheeler.com/essays/scm.html seems like a good overview of various players. http://ianclatworthy.files.wordpress.com/2007/10/dvcs-why-and-how3.pdf tells you why you should use it. I have found Bazaar's user guide to be a very gently gentle introduction to the topic: http://doc.bazaar-vcs.org/bzr.dev/en/user-guide/index.html When you're ready for laughs, you can watch Linus give a talk on git: http://youtube.com/watch?v=4XpnKHJAok8 For anything else, Google it! > > > Thanks, > > Skip Cheers, Benjamin Peterson > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080321/368d8465/attachment.htm From qgallet at gmail.com Fri Mar 21 23:31:59 2008 From: qgallet at gmail.com (Quentin Gallet-Gilles) Date: Fri, 21 Mar 2008 23:31:59 +0100 Subject: [Python-Dev] Primer on distributed revision control? In-Reply-To: <18404.13276.592156.81914@montanaro-dyndns-org.local> References: <18404.13276.592156.81914@montanaro-dyndns-org.local> Message-ID: <8b943f2b0803211531m7c8e6d3fkfac82528c8f342d2@mail.gmail.com> Eric Raymond started a study for this specific matter recently (announced here : http://article.gmane.org/gmane.emacs.devel/85893). Everything is under source control here : http://thyrsus.com/hg/uvc/ HTH, Quentin On Fri, Mar 21, 2008 at 11:17 PM, wrote: > With all these distributed revision control systems now available (bzr, > hg, > darcs, svk, many more), I find I need an introduction to the concepts and > advantages of repository distribution. It seems to me that it has the > potential for leading to anarchy, though I can see how some things would > be > improved (working offline, maintaining local patches). It's not obvious > how > I push changes back upstream. Can someone point me to some useful content > (web pages or books) which will help me wrap my brain around the ideas? > Maybe a compare/contrast of the major players? > > Thanks, > > Skip > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/qgallet%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080321/f868c1a4/attachment.htm From phd at phd.pp.ru Fri Mar 21 23:24:52 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Sat, 22 Mar 2008 01:24:52 +0300 Subject: [Python-Dev] Primer on distributed revision control? In-Reply-To: <18404.13276.592156.81914@montanaro-dyndns-org.local> References: <18404.13276.592156.81914@montanaro-dyndns-org.local> Message-ID: <20080321222452.GA17274@phd.pp.ru> On Fri, Mar 21, 2008 at 05:17:00PM -0500, skip at pobox.com wrote: > With all these distributed revision control systems now available (bzr, hg, > darcs, svk, many more), I find I need an introduction to the concepts and > advantages of repository distribution. It seems to me that it has the > potential for leading to anarchy, though I can see how some things would be > improved (working offline, maintaining local patches). It's not obvious how > I push changes back upstream. Can someone point me to some useful content > (web pages or books) which will help me wrap my brain around the ideas? http://en.wikipedia.org/wiki/Revision_control#Distributed_revision_control http://www.selenic.com/mercurial/wiki/index.cgi/UnderstandingMercurial http://www.selenic.com/mercurial/wiki/index.cgi/CommunicatingChanges http://doc.bazaar-vcs.org/latest/en/user-guide/index.html#sharing-with-peers Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From p.f.moore at gmail.com Fri Mar 21 23:40:47 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 21 Mar 2008 22:40:47 +0000 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E4330C.5070601@egenix.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> Message-ID: <79990c6b0803211540h25242236kcf3b9a5d6364eef0@mail.gmail.com> On 21/03/2008, M.-A. Lemburg wrote: > You're heading off in the wrong direction: we should not be trying > to rewrite RPM or InnoSetup in Python. > > Anything more complicated should be left to tools which are > specifically written to manage complex software setups. > > I honestly believe that most people would be happy if we just > provide these two things (and no more): > > * install a package from a local archive, a URL or PyPI > > * uninstall a package in way that doesn't break other > installed packages > > and whatever the mechanism, avoid making any undercover > changes to the Python installation such as adding > .pth files, overriding site.py, etc. - these are > not needed if the tool keeps to the simple task of > installing and uninstalling Python packages. My immediate impression is that I completely agree with this. I'd like to add one capability - to be able to list all installed packages. Anything beyond this, I'd need to be persuaded about. That's not to say that it's not valuable, just that it should be separate from the installer. That's where setuptools falls down, in my view - it tries to do too much all in one package. > We *don't* need the tool to: [...] > * make coffee, clean the house, send the kids to school :-) Well, these would be useful :-) Paul. From pje at telecommunity.com Fri Mar 21 23:41:00 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 21 Mar 2008 18:41:00 -0400 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E4330C.5070601@egenix.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> Message-ID: <20080321224110.657C63A40A5@sparrow.telecommunity.com> At 11:13 PM 3/21/2008 +0100, M.-A. Lemburg wrote: >On 2008-03-21 22:21, Phillip J. Eby wrote: > > At 08:06 PM 3/21/2008 +0100, M.-A. Lemburg wrote: > >> I guess the only way to support all of these variants is > >> to use a filesystem based approach, e.g. by placing a file > >> with a special extension into some dir on sys.path. > >> The "database" logic could then scan sys.path for these > >> files, read the data and provide an interface to it. > >> > >> All bdist formats would then have to include these files. > > > > That's the idea behind the current version of PEP 262, yes, and I think > > it should be kept. > > > >> A separate FILES section also doesn't seem to be necessary - > >> we could just add one or more entries or the format: > >> > >> CreatesDir abc/ > >> CreatesFile abc/xyz1.py > >> CreatesDir abc/def/ > >> CreatesFile abc/def/xyz2.py > >> CreatesFile abc/def/xyz3.py > >> CreatesFile abc/def/xyz4.ini > > > > I actually think the size and hash information is good, in order to be > > able to tell if you're looking at an original file. I'm not sure how > > useful the permissions and uid/gid info is. I'm hoping we'll hear from > > anybody who has a use case for that. > >You're heading off in the wrong direction: we should not be trying >to rewrite RPM or InnoSetup in Python. I'm making the assumption that the author(s) of PEP 262 had good reason for including what they did, rather than assuming that we should start the entire process over from scratch. >Anything more complicated should be left to tools which are >specifically written to manage complex software setups. Tools which will need this data, in order to do their work. Hence, the reason for standardizing the data, instead of the tool(s). >[snip long list of features, both desired and undesired] Actually, *all* of these features are out of scope for stdlib development, because I'm not proposing including *any* tools for this in the stdlib, apart from distutils install and bdist_* support. I'm proposing, rather, that we finish the vision of PEP 262, of having a standard specification that *all* tools will abide by -- including rpm, dpkg, and what-have-you. Since *all* of these tools need to abide by that specification, their requirements will need to be considered in the formulation of the spec. And as I said, I'll be happy if all we do is get the distutils to abide by the spec for 2.6, even if it means we don't get an uninstall tool. That can always be installed later using Guido's bootstrap tool. :) From p.f.moore at gmail.com Fri Mar 21 23:47:33 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 21 Mar 2008 22:47:33 +0000 Subject: [Python-Dev] Primer on distributed revision control? In-Reply-To: <18404.13276.592156.81914@montanaro-dyndns-org.local> References: <18404.13276.592156.81914@montanaro-dyndns-org.local> Message-ID: <79990c6b0803211547g2f94aa46vf5334e6662fa32e8@mail.gmail.com> On 21/03/2008, skip at pobox.com wrote: > With all these distributed revision control systems now available (bzr, hg, > darcs, svk, many more), I find I need an introduction to the concepts and > advantages of repository distribution. It seems to me that it has the > potential for leading to anarchy, though I can see how some things would be > improved (working offline, maintaining local patches). It's not obvious how > I push changes back upstream. Can someone point me to some useful content > (web pages or books) which will help me wrap my brain around the ideas? > Maybe a compare/contrast of the major players? The Mercurial book is good (http://hgbook.red-bean.com/) although it's Mercurial-specific (unsurprisingly). The Bazaar user guide (http://doc.bazaar-vcs.org/bzr.dev/en/user-guide/index.html) isn't bad, either, although I personally prefer the Mercurial book. I've yet to see a really good side-by-side comparison of Mercurial and Bazaar, which is a shame, as these 2 are to my mind, the hardest to differentiate in terms of features, etc. (Mercurial is generally considered faster, although Bazaar has caught up a lot recently. An up to date performance comparison would be interesting). Paul. From lists at cheimes.de Sat Mar 22 00:09:51 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 22 Mar 2008 00:09:51 +0100 Subject: [Python-Dev] SVN FAQ for Windows users In-Reply-To: <47E42C99.4020202@v.loewis.de> References: <703b70ea-d057-403c-8fd4-fce68179f699@s37g2000prg.googlegroups.com> <47E42C99.4020202@v.loewis.de> Message-ID: Martin v. L?wis schrieb: > That never worked for me. TortoiseSVN then insists on fetching the > revisions mentioned in the patch, runs some math, and tells me that > I can't apply the patch because it is out of date (assuming it really > is, which is normally the case when I get to look at a patch). Oh yeah, Tortoise tries to be "clever". Stupid turtle :( TortoiseSVN is great tool and IMHO the best GUI app for svn but its patching system it's more than annoying. Christian From brett at python.org Sat Mar 22 00:12:00 2008 From: brett at python.org (Brett Cannon) Date: Fri, 21 Mar 2008 16:12:00 -0700 Subject: [Python-Dev] [Python-3000] Python source code on Bazaar vcs In-Reply-To: <1afaf6160803211510i3026e2f0n2ee534ff514be459@mail.gmail.com> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <20080321084249.GA83712@nexus.in-nomine.org> <79990c6b0803211428h2eef9fa0s1032570cdf3fe162@mail.gmail.com> <1afaf6160803211457y29cb0ce8s58898876d2f96a5b@mail.gmail.com> <79990c6b0803211504o78032f66vbe55064bdf7a9f3@mail.gmail.com> <1afaf6160803211510i3026e2f0n2ee534ff514be459@mail.gmail.com> Message-ID: On Fri, Mar 21, 2008 at 3:10 PM, Benjamin Peterson wrote: > > > > > On Fri, Mar 21, 2008 at 5:04 PM, Paul Moore wrote: > > > > On 21/03/2008, Benjamin Peterson wrote: > > > I tend to make a repository and make a working copy for each patch in > it. > > > The history is saved in the repository so it's efficient. > > > > OK, so just lots of copies, fair enough. Presumably just use bzr diff > > to create patches? Much like Subversion, in practice, but with local > > commits of partial work. > Yes, bzr diff should do the trick, although if you have local commits in it, > you'll have to give the revision number manually. That's not really true. Let's say you have a trunk branch that you keep which is pristine. You branch off of it and create a xxx branch. You can diff between xxx and trunk by running ``bzr diff xxx --old trunk``. You can also run this from within xxx with ``bzr diff --old ../trunk``. -Brett From musiccomposition at gmail.com Sat Mar 22 00:12:55 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 21 Mar 2008 18:12:55 -0500 Subject: [Python-Dev] [Python-3000] Python source code on Bazaar vcs In-Reply-To: References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <20080321084249.GA83712@nexus.in-nomine.org> <79990c6b0803211428h2eef9fa0s1032570cdf3fe162@mail.gmail.com> <1afaf6160803211457y29cb0ce8s58898876d2f96a5b@mail.gmail.com> <79990c6b0803211504o78032f66vbe55064bdf7a9f3@mail.gmail.com> <1afaf6160803211510i3026e2f0n2ee534ff514be459@mail.gmail.com> Message-ID: <1afaf6160803211612v124fb7ft4b2455c6bda5d71f@mail.gmail.com> On Fri, Mar 21, 2008 at 6:12 PM, Brett Cannon wrote: > On Fri, Mar 21, 2008 at 3:10 PM, Benjamin Peterson > wrote: > > > > > > > > > > On Fri, Mar 21, 2008 at 5:04 PM, Paul Moore wrote: > > > > > > On 21/03/2008, Benjamin Peterson wrote: > > > > I tend to make a repository and make a working copy for each patch > in > > it. > > > > The history is saved in the repository so it's efficient. > > > > > > OK, so just lots of copies, fair enough. Presumably just use bzr diff > > > to create patches? Much like Subversion, in practice, but with local > > > commits of partial work. > > Yes, bzr diff should do the trick, although if you have local commits in > it, > > you'll have to give the revision number manually. > > That's not really true. Let's say you have a trunk branch that you > keep which is pristine. You branch off of it and create a xxx branch. > You can diff between xxx and trunk by running ``bzr diff xxx --old > trunk``. You can also run this from within xxx with ``bzr diff --old > ../trunk``. Well, I just learned something. ;) > > > -Brett > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080321/d15d263d/attachment.htm From brett at python.org Sat Mar 22 00:15:12 2008 From: brett at python.org (Brett Cannon) Date: Fri, 21 Mar 2008 16:15:12 -0700 Subject: [Python-Dev] Python source code on Mercurial In-Reply-To: <79990c6b0803211438w347d052fre2a97ebde72b44b4@mail.gmail.com> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <932f8baf0803201426ua0ae2edu3f7b62eaf22ef787@mail.gmail.com> <79990c6b0803211438w347d052fre2a97ebde72b44b4@mail.gmail.com> Message-ID: On Fri, Mar 21, 2008 at 2:38 PM, Paul Moore wrote: > On 21/03/2008, Antoine Pitrou wrote: > > I see your trunk history is stripped. For those who want the complete trunk > > history (back to 17 years ago!), I have my own mirror here: > > http://dev.pitrou.net:8000/cpython/trunk/ > > Excellent! For what it's worth, hg clone took 5 minutes on my PC (with > broadband access). That's faster than simply downloading the Bazaar > shared repository tarball (which took 13 minutes)! > > Are you keeping the mirror updated with respect to Subversion? > > I don't want to start a comparison at this point, but as neither > system offers downloading of truncated history at the moment, Bazaar supports lightweight checkouts which act like svn checkouts. They are also actively working on allowing for partial checkouts. That way you can either specify an initial revision to pull the history down to or start with an initial lightweight checkout and have it save history as it pulls it over the network as you use it. -Brett From collinw at gmail.com Sat Mar 22 00:33:20 2008 From: collinw at gmail.com (Collin Winter) Date: Fri, 21 Mar 2008 16:33:20 -0700 Subject: [Python-Dev] The Breaking of distutils and PyPI for Python 3000? In-Reply-To: <47E42B3B.9000908@v.loewis.de> References: <47E0D883.9030402@taupro.com> <47E1BB1D.4060303@v.loewis.de> <43aa6ff70803201117y46e09c5frd6e5be6e4902b23c@mail.gmail.com> <47E2CA12.2000506@v.loewis.de> <47E2CE0E.6010108@rksystems.com> <47E2D716.4030109@v.loewis.de> <47E2DB20.5020909@rksystems.com> <47E2E02F.1020007@v.loewis.de> <47E42B3B.9000908@v.loewis.de> Message-ID: <43aa6ff70803211633q72833ad0sa55b783b93bb2a0e@mail.gmail.com> On Fri, Mar 21, 2008 at 2:40 PM, "Martin v. L?wis" wrote: > > | try: > > | bytes > > | except NameError: > > | bytes = str > > | > > | somewhere, then write the code as I proposed above. > > > > This is exactly the sort of thing that should be and I expect will be in > > the conversion docs. > > That expectation might be unfounded (in the sense that it might not be > written at all). I only happened to learn about that approach as I > was converting Django to 3k. > > I'll happily put my experience in doing so into an email message, > but I won't sit down and write a text document. Please do. I'll happily convert it into something more formal. Collin Winter From lists at cheimes.de Sat Mar 22 00:41:21 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 22 Mar 2008 00:41:21 +0100 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: References: <47DD545D.6070300@cheimes.de> <47DD613E.6090007@cheimes.de> Message-ID: <47E447A1.6070208@cheimes.de> Guido van Rossum schrieb: > Even though the more popular PyInt_ APIs are still available (even if > only as macros). THe PyInt_* macros are only available when Include/intobject.h is included explicitly. It's not in Python.h any more. > I disagree. Bad merges are already a frequent cause of instability in > 3.0. This could easily double the problems. And while I want to reduce > the instability (I really wish you would no commit until all unittests > pass), I also don't want the merges to cost more of your time! I'm trying my best but sometimes I don't spot the cause of a failing unit test. I got a slightly faster laptop so I'm now able to run the full unit test suite every time I do a svnmerge.py. > It doesn't have to be so black and white. Using the macro approach you > propose we can fix the situation at any time later. > > We could also make the changes in 2.6, so that the 2.6 code looks the > same as 3.0. (However for binary compatibility I think it would be > better if in 2.6 the linker sees PyString where in 3.0 it sees > PyBytes.) Let me get this straight. You propose that we replace PyString_ with PyBytes_ in both Python 2.6 and 3.0 core code. In Python 2.6 some macros replace the PyBytes_* functions with PyString_ so the linker sees PyString_*? Mmh, it sounds like an interesting idea :] Christian From jml at mumak.net Sat Mar 22 01:13:13 2008 From: jml at mumak.net (Jonathan Lange) Date: Sat, 22 Mar 2008 11:13:13 +1100 Subject: [Python-Dev] [Python-3000] Python source code on Bazaar vcs In-Reply-To: <79990c6b0803211428h2eef9fa0s1032570cdf3fe162@mail.gmail.com> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <20080321084249.GA83712@nexus.in-nomine.org> <79990c6b0803211428h2eef9fa0s1032570cdf3fe162@mail.gmail.com> Message-ID: On Sat, Mar 22, 2008 at 8:28 AM, Paul Moore wrote: > Basically, can some Bazaar expert offer a suggestion as to how a > non-developer with read-only access would best use the Bazaar > repositories to maintain a number of patches to be posted to the > tracker? > Here's what I'd do: get Python, make a branch for my patch, hack on the patch in the branch, then use merge --preview to generate a diff. If you want to maintain a long-lived branch of Python with patches, say you're maintaining a distro package, then looms are your best bet. For those inclined to specifics, here's a near-exact example of what I would do: # Get Python bzr init-repo ~/src/python cd ~/src/python bzr branch # Make a patch bzr branch trunk docstring-fix-bug-123456 cd docstring-fix-bug-123456 bzr ci -m 'Correct some spelling' bzr ci -m 'Oops. Make sure we use US English' # Move the unrelated work to another branch. cd .. bzr branch trunk testcase-add-cleanup cd testcase-add-cleanup bzr merge --uncommitted ../docstring-fix-bug-123456 bzr ci -m "Add addCleanup method to TestCase. Schedules a nullary callable to be run before tearDown" # Prepare the patches cd ../trunk bzr merge --preview ../docstring-fix-bug-123456 > ~/Desktop/docstring.patch bzr merge --preview ../testcase-add-cleanup > ~/Desktop/add-cleanup.patch Hope this helps, jml From guido at python.org Sat Mar 22 01:18:04 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 21 Mar 2008 17:18:04 -0700 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: <47E447A1.6070208@cheimes.de> References: <47DD545D.6070300@cheimes.de> <47DD613E.6090007@cheimes.de> <47E447A1.6070208@cheimes.de> Message-ID: On Fri, Mar 21, 2008 at 4:41 PM, Christian Heimes wrote: > Guido van Rossum schrieb: > > > Even though the more popular PyInt_ APIs are still available (even if > > only as macros). > > THe PyInt_* macros are only available when Include/intobject.h is > included explicitly. It's not in Python.h any more. > > > > I disagree. Bad merges are already a frequent cause of instability in > > 3.0. This could easily double the problems. And while I want to reduce > > the instability (I really wish you would no commit until all unittests > > pass), I also don't want the merges to cost more of your time! > > I'm trying my best but sometimes I don't spot the cause of a failing > unit test. I got a slightly faster laptop so I'm now able to run the > full unit test suite every time I do a svnmerge.py. :-) > > It doesn't have to be so black and white. Using the macro approach you > > propose we can fix the situation at any time later. > > > > We could also make the changes in 2.6, so that the 2.6 code looks the > > same as 3.0. (However for binary compatibility I think it would be > > better if in 2.6 the linker sees PyString where in 3.0 it sees > > PyBytes.) > > Let me get this straight. You propose that we replace PyString_ with > PyBytes_ in both Python 2.6 and 3.0 core code. In Python 2.6 some macros > replace the PyBytes_* functions with PyString_ so the linker sees > PyString_*? Mmh, it sounds like an interesting idea :] Right. We've done this 2-stage rename before, during the great (sometimes known as grand) renaming, in the 1.x days. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Mar 22 01:23:25 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 21 Mar 2008 17:23:25 -0700 Subject: [Python-Dev] Trove classifiers In-Reply-To: References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <20080318223700.C64123A4074@sparrow.telecommunity.com> <47E1B858.1090107@v.loewis.de> <47E27482.9090908@v.loewis.de> <20080320154355.GI60713@nexus.in-nomine.org> <20080320155533.GB12099@phd.pp.ru> Message-ID: On Thu, Mar 20, 2008 at 9:43 AM, Fred Drake wrote: > On Mar 20, 2008, at 11:55 AM, Oleg Broytmann wrote: > > Yes, exactly. Eric Raymond claims to be the inventor, but there are > > different voices against him: > > http://damagestudios.net/blog/2005/08/15/sourceforge-founders > > > That contests that Raymond was an "architectural granddaddy of > SourceForge", not that he invented Trove. My understanding is that he > did start the efforts to define the Trove classifiers as part of a > larger effort that never panned out, but that defining the classifiers > was not a solo effort. That's my recollection too. And yes, Eric would like to have invented open source. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Sat Mar 22 02:31:34 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 22 Mar 2008 02:31:34 +0100 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080321224110.657C63A40A5@sparrow.telecommunity.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> Message-ID: <47E46176.4090205@v.loewis.de> > I'm making the assumption that the author(s) of PEP 262 had good > reason for including what they did, rather than assuming that we > should start the entire process over from scratch. The objections to the PEP remain the same as they were then, though: In the requirements, it says "we need", without saying why we need. It then goes on saying "we want" (rephrased) "to duplicate APT and RPM", without saying why we want that, or why that brings us closer to what we need. IOW, the PEP is lacking a rationale. >> Anything more complicated should be left to tools which are >> specifically written to manage complex software setups. > > Tools which will need this data, in order to do their work. Hence, > the reason for standardizing the data, instead of the tool(s). If there was a chance that the infrastructure being developed actually helps these tools, *that* would be a reasonable goal, IMO. However, I'm extremely skeptical that this can ever succeed to the degree that whoever provides RPMs, .debs, or MSI files will actually use such data, as they will find that the data are incomplete, and they have to redo all of it, anyway. > I'm proposing, rather, that we finish the vision of PEP 262, of > having a standard specification that *all* tools will abide by -- > including rpm, dpkg, and what-have-you. Do you also envision the objective of PEP 262, then? I.e. to provide a database of installed packages, in .../install-db? > And as I said, I'll be happy if all we do is get the distutils to > abide by the spec for 2.6, even if it means we don't get an uninstall > tool. That can always be installed later using Guido's bootstrap tool. :) I'm even more skeptical here. If the assumption is that the package database allows for uninstallation, I'm -1. IOW, RPM, deb, MSI should then *not* write to that package database, as they also write to a different database, out of the scope of the PEP, and this is what uninstallation should use. Regards, Martin From amk at amk.ca Sat Mar 22 02:44:42 2008 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 21 Mar 2008 21:44:42 -0400 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080321224110.657C63A40A5@sparrow.telecommunity.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> Message-ID: <20080322014442.GA18652@amk.local> On Fri, Mar 21, 2008 at 06:41:00PM -0400, Phillip J. Eby wrote: > I'm making the assumption that the author(s) of PEP 262 had good > reason for including what they did, rather than assuming that we > should start the entire process over from scratch. The goal *was* originally to provide for RPM-like verification of file content, but I don't know that the verification feature really matters that much; OSes with packaging systems already support such a feature, probably, and it probably isn't particularly useful for systems without a packaging system. --amk From greg at krypto.org Sat Mar 22 02:53:13 2008 From: greg at krypto.org (Gregory P. Smith) Date: Fri, 21 Mar 2008 18:53:13 -0700 Subject: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows In-Reply-To: <083301c863d2$12dc2b40$389481c0$@com.au> References: <009f01c79b51$b1da52c0$1f0a0a0a@enfoldsystems.local> <46512B09.1080304@v.loewis.de> <022401c85cde$678bcf10$36a36d30$@com.au> <083301c863d2$12dc2b40$389481c0$@com.au> Message-ID: <52dc1c820803211853h42876800ma9104e04a33efd0d@mail.gmail.com> I'm following up on this thread without checking if there were other following negating a need to respond... If so, ignore as needed. +1 from me. Always build on windows into an architecture specific PCBuild/XXX directory. A bonus if the directory name matches the return value of platform.machine() but I don't care too much about that part. (that'd mean 'i686' and 'x86_64' and who knows what for Itanic users) On Wed, Jan 30, 2008 at 11:25 PM, Mark Hammond wrote: > I'm wondering if anyone has any comment on this issue? Its currently > impossible to use distutils as documented to build x64 extensions, either > natively or cross-compiled using the trunk and while I'm keen to help fix > this, I'd like agreement on the direction. > > Can I assume silence means general agreement on the compromise proposal I > outlined? > > Thanks, > > Mark > > > -----Original Message----- > > From: python-dev-bounces+mhammond=keypoint.com.au at python.org > > [mailto:python-dev-bounces+mhammond=keypoint.com.au at python.org] On > > Behalf Of Mark Hammond > > Sent: Tuesday, 22 January 2008 9:06 PM > > To: '"Martin v. L?wis"' > > Cc: distutils-sig at python.org; python-dev at python.org > > Subject: Re: [Python-Dev] Adventures with x64, VS7 and VS8 on Windows > > > > Hi Martin, > > > > Way back on Monday, 21 May 2007, you wrote: > > > > > This is an issue to be discussed for Python 2.6. I'm personally > > > hesitant to have the "official" build infrastructure deviate > > > from the layout that has been in-use for so many years, as a lot > > > of things depend on it. > > > > > > I don't find the need to have separate object directories convincing: > > > For building the Win32/Win64 binaries, I have separate checkouts > > > *anyway*, since all the add-on libraries would have to support > > > multi-arch builds, but I think they don't. > > ... > > > > * Re the x64 directory used by the PCBuild8 process. IMO, it makes > > > > sense to generate x64 binaries to their own directory - my > > expectation > > > > is that cross-compiling between platforms is a reasonable use-case, > > > > and we should support multiple achitectures for the same compiler > > > > version. > > > > > > See above; I disagree. > > > > [For additional context; the question was regarding where the output > > binaries were created for each platform, and specifically if x64 build > > should use something like 'PCBuild/x64'] > > > > I'm bringing this topic up again as I noticed the AMD64 builds for > > Python > > 2.6 create their output in the PCBuild/x64 directory, not directly into > > the > > PCBuild directory. When I raised this with Christian, he also > > expressed a > > desire to be able to cross-compile in the same directory structure and > > was > > unaware of this discussion (but I made him aware of it :) > > > > You made excellent points about how tools currently recognize the > > PCBuild > > directory, and we can't break them, and I agree. I'd like to propose a > > compromise that might be able to keep everyone happy. > > > > The main problem I see with all platforms using 'PCBuild' is that in a > > cross-compilation scenario, it is impossible for the distutils to > > determine > > the location of the correct Python libraries to link with (stuff in its > > own > > PYTHONHOME is no good; it's for the wrong platform). Obviously, we can > > change the distutils so that the location of the correct libraries can > > be > > specified, but that implies all other build systems that currently > > assume > > PCBuild must also change to work in a cross-compilation scenario. > > While > > those external tools *will* work today in a native build on x64, > > eventually > > they may need to be upgraded to deal with cross-compilation - and this > > means > > that all such tools will also be unable to automatically determine the > > location of the libraries. They will all need to take the library dir > > as a > > user-specified option, which seems a pain (eg, buildbots would seem to > > need > > site-specific configuration information to make this work). If > > eventually > > all such tools are likely to upgrade, it would be ideal if we can offer > > them > > a way to upgrade so the correct libraries can be determined > > automatically, > > as they can now for native builds. > > > > My compromise proposal is this: Each platform builds in its own > > directory > > (ie, PCBuild/x86_64, PCBuild/x86). Every single build configuration > > has a > > post-build step that copies the result of the build to the PCBuild > > directory. We then add an option to the distutils so that a cross- > > compile > > platform can be specified and when it is, to use 'PCBuild/{platform}" > > instead of PCBuild, and we encourage other tools to use the same > > directory > > should they want to support "configure-less" cross-compilation (if > > distutils > > on the trunk is not still trying to maintain b/w compat, it should just > > *always* look in PCBuild/{platform}, and I'd suggest py3k not bother > > with > > PCBuild at all; build systems will be touched to upgrade to py3k, so > > that > > change isn't painful. But I digress :) > > > > The main advantage of the compromise is that it allows for the status > > quo to > > remain in place - just continue to use separate source trees for each > > platform, and only ever build a single platform in each tree, and tools > > are > > free to have ways of specifying which directory should be used. The > > official build process need not change. However, it also supports a > > way to > > do cross-compilation in a way that build tools can *automatically* > > locate > > the correct platform's libraries, and this will be particularly useful > > for > > extension authors. > > > > Would something like that be acceptable? > > > > Thanks, > > > > Mark > > > > > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: http://mail.python.org/mailman/options/python- > > dev/mhammond%40keypoint.com.au > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/greg%40krypto.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080321/ce86f0d0/attachment.htm From pje at telecommunity.com Sat Mar 22 03:04:45 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 21 Mar 2008 22:04:45 -0400 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E46176.4090205@v.loewis.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> Message-ID: <20080322020447.3B1033A4074@sparrow.telecommunity.com> At 02:31 AM 3/22/2008 +0100, Martin v. L?wis wrote: >>I'm making the assumption that the author(s) of PEP 262 had good >>reason for including what they did, rather than assuming that we >>should start the entire process over from scratch. > >The objections to the PEP remain the same as they were then, >though: In the requirements, it says "we need", without saying >why we need. It then goes on saying "we want" (rephrased) >"to duplicate APT and RPM", without saying why we want that, >or why that brings us closer to what we need. > >IOW, the PEP is lacking a rationale. Ok, well, I have a rationale for it: make it possible to get rid of eggs as a mechanism for supporting easy_install. Many people (yourself included) have criticized eggs as an installation mechanism, and this is an alternative that gets rid of .egg files and directories in that case, and most of the need for .pth file usage as well. >If there was a chance that the infrastructure being developed >actually helps these tools, *that* would be a reasonable goal, >IMO. Yes, I'm of course primarily interested in Python-specific tools such as virtualenv, easy_install, buildout, and the as-yet-unwritten uninstallers, package listers, etc., that can usefully read or write such data. >However, I'm extremely skeptical that this can ever succeed >to the degree that whoever provides RPMs, .debs, or MSI >files will actually use such data, as they will find that >the data are incomplete, and they have to redo all of it, >anyway. The data isn't for them to use to meet their use cases, it's for them to *provide* so that Python tools don't stomp on, uninstall, or otherwise interfere with files installed by the system. In other words, for system packagers, it's a communication from the system to Python, rather than the other way around. Even though the distutils will build the file in the bdist, the system packaging tools would be free to generate their own file listing and signatures and such. >Do you also envision the objective of PEP 262, then? I.e. >to provide a database of installed packages, in .../install-db? In each directory relative to a given sys.path directory, yes. That is, installing a distutils distribution to any directory would result in a file being added to an install-db within that directory. (Assuming we use the proposed implementation model of PEP 262, which at the moment I don't see any substantial obstacle to.) >>And as I said, I'll be happy if all we do is get the distutils to >>abide by the spec for 2.6, even if it means we don't get an >>uninstall tool. That can always be installed later using Guido's >>bootstrap tool. :) > >I'm even more skeptical here. If the assumption is that the package >database allows for uninstallation, I'm -1. IOW, RPM, deb, MSI >should then *not* write to that package database, as they also write >to a different database, out of the scope of the PEP, and this is >what uninstallation should use. I probably should have brought this up, in fact, I think I mentioned it in a previous thread, but I would like to see PEP 262 add a way to say "this is a system-installed package, *don't touch*". The idea again is not to do the job of the native packaging system, but rather to ensure that Python-specific tools (e.g. easy_install and friends) do not interfere or conflict with it. A big problem in the early development of easy_install, even using eggs, was that there was no way to tell whether it was safe to delete or overwrite an existing file or directory that was already installed on the system. A mechanism like this would allow tools like easy_install to say, "oh, your system packager has a conflicting package here, you need to use that tool to sort this out if you really want to do something here. I'm not going to touch that." Without something like this, there is no way to tell the difference on many systems between a system package and something the user has put there with "sudo python setup.py install". From pje at telecommunity.com Sat Mar 22 03:08:37 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 21 Mar 2008 22:08:37 -0400 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080322014442.GA18652@amk.local> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <20080322014442.GA18652@amk.local> Message-ID: <20080322020841.1CF063A4074@sparrow.telecommunity.com> At 09:44 PM 3/21/2008 -0400, A.M. Kuchling wrote: >On Fri, Mar 21, 2008 at 06:41:00PM -0400, Phillip J. Eby wrote: > > I'm making the assumption that the author(s) of PEP 262 had good > > reason for including what they did, rather than assuming that we > > should start the entire process over from scratch. > >The goal *was* originally to provide for RPM-like verification of file >content, but I don't know that the verification feature really matters >that much; OSes with packaging systems already support such a feature, >probably, and it probably isn't particularly useful for systems >without a packaging system. Actually, it's the places where there's no packaging system that it's most useful. For example, an application that installs plugins to itself. A development environment with multiple virtual pythons. Users installing stuff to their PYTHONPATH, etc. In these cases, having the Python-specific tools be able to verify content signatures is useful, to make sure that you know what you're updating or removing. This is particularly important if one installs anything just by unpacking it into the target directory; you could overwrite something and then have only size/signature info to sort out whose version of the file is actually there. I more question the permissions and uid/gid stuff; I'm not really clear on what I'd use that stuff for in easy_install/uninstall/etc. From jjl at pobox.com Sat Mar 22 03:40:00 2008 From: jjl at pobox.com (John J Lee) Date: Sat, 22 Mar 2008 02:40:00 +0000 (GMT) Subject: [Python-Dev] httplib &c. timeouts and global state Message-ID: http://python.org/sf/2451 """ The new timeout support in 2.6 makes use of new function socket.create_connection(). socket.create_connection() provides no way to disable timeouts, other than by relying on socket.getdefaulttimeout() returning None. This is unfortunate, because it was the purpose of the new timeout support to allow control of timeouts without reliance on global state. setdefaultsocket.create_connection() should always call sock.settimeout() with the timeout passed to create_connection(), unless a special non-None value is passed indicating that the global default is to be used. Specific modules may then use that special non-None value where required, to preserve backwards-compatibility. """ Facundo doesn't seem to agree. If I understand him correctly, he objects to the use of a special value other than None as an argument value. I think avoiding this minor complication is secondary to avoiding gratuitously imposing global state on users where it's not needed. Much existing code uses .setdefaulttimeout(), because legacy code does not expose the socket timeout. Of course, not all of that code can be changed immediately (otherwise, why did we wait so long before Facundo did his valuable work on this?). Of course, very often, code that calls socket.setdefaulttimeout(some_non_none_value) wants a default timeout for everything. But that isn't *always* the case, and standard library modules that expose the timeout feature should not make that assumption. For example, an application might only want to set a timeout for some non-essential network operation. As another example, useful but poorly-written library code might call .setdefaulttimeout(). Also, ISTM there's a big difference between nobody having contributed the time to implement the feature on the one hand, and deliberately imposing the global socket timeout state on users on the other! Facundo indicates in the tracker discussion for 2451 that a decision was made here about this, but doesn't recall exactly which post, and directed me here. What I found in the archive is this thread (sorry for the non-python.org URL): http://www.gossamer-threads.com/lists/python/dev/555039?do=post_view_threaded#555039 In that discussion, this issue is raised, but I don't see any resolution that answers the objection made in issue 2451. Anyway, apologies in advance if there was a decision here that takes account of the above objection. I do want to thank Facundo for adding the timeout support to modules such as httplib. But I also want to loudly complain about this detail :-) John From stephen at xemacs.org Sat Mar 22 03:55:57 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 22 Mar 2008 11:55:57 +0900 Subject: [Python-Dev] Primer on distributed revision control? In-Reply-To: <18404.13276.592156.81914@montanaro-dyndns-org.local> References: <18404.13276.592156.81914@montanaro-dyndns-org.local> Message-ID: <87iqzffow2.fsf@uwakimon.sk.tsukuba.ac.jp> skip at pobox.com writes: > With all these distributed revision control systems now available (bzr, hg, > darcs, svk, many more), I find I need an introduction to the concepts and > advantages of repository distribution. > Can someone point me to some useful content (web pages or books) > which will help me wrap my brain around the ideas? Maybe a > compare/contrast of the major players? Others have mentioned a bunch of resources. They're all OK, and should reassure you that dVCS is not all that different from what you're used to. Here I'll post some comments that as far as I know are not in any of the existing resources. One caveat that you should be very careful about while reading them: any command that involves receiving changes from a remote repo and updating your workspace. Terms like "get", "fetch", "pull", "update", "clone", and even "checkout" are commonly used to describe these operations, but the actual semantics of the commands named by those terms differs wildly. For example, in git, "fetch" receives the metadata and new content, while "pull" does a fetch, then attempts to update the workspace and performs 3-way merges. In Mercurial, "pull" and "fetch" have the opposite semantics. Also, note that while CVS and Subversion automatically do a merge when updating, the dVCSes all make this distinction between fetching new content and merging it into your workspace. As already described, some of them normally combine the operations, others tend to ask the user to do them "by hand", but all provide both ways to do it. (That should send a shiver down your Pythonic spine!) Another one is that in git and Mercurial, "clone" replicates a repo, while "checkout" switches between branches in a single workspace. In bzr, "clone" is an alias for "checkout", and normally creates a new workspace. > It's not obvious how I push changes back upstream. It is very important to remember that in centralized systems like CVS or Subversion, committing *is* pushing, while in a dVCS these operations are separate. In many projects the workflow is that you *record* your changes in a local repo, then you *push* them to a shared repo. AFAIK all of the contenders do call the record operation "commit", and the publish operation "push".[1] (Other workflows you will hear about are based on mailing patches -- eg, Linux, while yet others are based on gatekeepers pulling from your repo -- Arch developers tend to favor this. Don't worry about them, these are not going to be relevant to Python for a while.) >From a participating developer's point of view, Eric Raymond's in-progress survey captures this aspect pretty well as a distinction between "update before record" and "merge after record" (my terms, not his). The point is that in CVS or Subversion, you *cannot* commit to mainline if you haven't merged in all concurrent changes to mainline. In a distributed VCS, on the other hand, you record your changes locally at will, then merge revisions at your convenience, finally pushing them upstream. It sound like a small difference, but it's actually amazingly liberating. Otherwise, it doesn't need to be any different from your current workflow, and I doubt that it will be until the most active Python developers start to feel a need for it. > It seems to me that it has the potential for leading to anarchy, git is *designed* around Linus's ability to manage chaos, but because of Linus, there is no anarchy. The same will be true for Python (although the personalities of the leading developers are different, so the details of why no anarchy will differ). A more optimistic way to put your point is that dVCSes have a great potential for encouraging experimentation. In general, you'll always have a pretty good idea where you want to pull from, and the gatekeepers will tell if you may and where to push. Footnotes: [1] Darcs uses "record" for the record operation, but Darcs is highly unlikely to become the Python dVCS of choice. From talin at acm.org Sat Mar 22 06:20:43 2008 From: talin at acm.org (Talin) Date: Fri, 21 Mar 2008 22:20:43 -0700 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080321190226.4198D3A4074@sparrow.telecommunity.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E3D11F.4030503@online.de> <18403.57501.22865.578079@montanaro-dyndns-org.local> <20080321190226.4198D3A4074@sparrow.telecommunity.com> Message-ID: <47E4972B.30008@acm.org> Phillip J. Eby wrote: > At 11:21 AM 3/21/2008 -0500, skip at pobox.com wrote: >> Joachim> I think, the uninstall should _not_ 'rm -rf' but only 'rm' the >> Joachim> files (and 'rmdir' directories, but not recursively) that it >> Joachim> created, and that have not been modified in the meantime (after >> Joachim> the installation). >> >> That's not sufficient. Suppose file C (e.g. /usr/local/etc/mime.types) is >> in both packages A and B. >> >> Install A - this will create C >> Install B - this might overwrite C, saving a copy, or it might retain >> A's copy. >> Uninstall B - this has to know that C is used by A and not touch it > > Correct. However, in practice, B should not touch C, unless the file > is shared between them. > > This is a key issue for support of namespace packages, at least if we > want to avoid using .pth files. (Which is what setuptools-built > system packages do for namespace packages currently.) > > Of course, one possible solution is for both A and B to depend on a > "virtual package" that contains C, such that both A and B can install > it if it's not there, and list it in their dependencies. But this is > one of the handful of open issues that needs to be resolved with Real > Life Package Management people, such as Debian, Fedora, etc. I've always thought that the right way to handle the dependency DAG is to treat it as a garbage collection problem. Assume that for each package there is a way to derive the following two pieces of information: (a) whether this package was installed explicitly by the user, or implicitly as the result of a dependency, and (b) the set of dependencies for this package. Then, starting with the list of 'explicit' packages as the root set, do a standard mark & sweep; Any package not marked is a candidate for removal. -- Talin From ronaldoussoren at mac.com Sat Mar 22 08:53:24 2008 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sat, 22 Mar 2008 08:53:24 +0100 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080322014442.GA18652@amk.local> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <20080322014442.GA18652@amk.local> Message-ID: On 22 Mar, 2008, at 2:44, A.M. Kuchling wrote: > On Fri, Mar 21, 2008 at 06:41:00PM -0400, Phillip J. Eby wrote: >> I'm making the assumption that the author(s) of PEP 262 had good >> reason for including what they did, rather than assuming that we >> should start the entire process over from scratch. > > The goal *was* originally to provide for RPM-like verification of file > content, but I don't know that the verification feature really matters > that much; OSes with packaging systems already support such a feature, > probably, and it probably isn't particularly useful for systems > without a packaging system. Why not? Checksums would allow you to build a small packaging system on top the python system. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20080322/64742661/attachment.bin From g.brandl at gmx.net Sat Mar 22 10:50:51 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 22 Mar 2008 10:50:51 +0100 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <20080321084249.GA83712@nexus.in-nomine.org> Message-ID: Matthieu Brucher schrieb: > Good, because between this now and pytz the other 63 projects I > follow use > Subversion or Mercurial. > Bazaar seems to be mostly limited to Ubuntu users and stuff > Canonical does, > so the choice for a Bazaar setup next to Subversion strikes me a bit > as odd. > Mercurial is also Python, distributed and with a, as far as I (can) > track > things, bigger market share than Bazaar. > This is not quite true. One of the main bzr developers is a Windows guy, > so the user base is/will be large, far larger than only Ubuntu. A lot of > projects switched to bzr because of this. Honestly, I think that Hg and > bzr fill the same need, but bzr developement at the moment is fantastic. > For a still young product it is a good think (Hg is older). > One additional good think is that there is a Sourceforge-like site that > uses bzr ; launchpad.net . I don't know of > something equivalent for Hg. In any case, the reason for using a VCS should never be the market share, but its advantages for development, e.g. speed, usability, community support. cheers, Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Sat Mar 22 10:50:50 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 22 Mar 2008 10:50:50 +0100 Subject: [Python-Dev] Python source code on Mercurial In-Reply-To: References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <932f8baf0803201426ua0ae2edu3f7b62eaf22ef787@mail.gmail.com> Message-ID: Antoine Pitrou schrieb: > Ralf Schmitt gmail.com> writes: >> >> I have also setup a mirror using mercurial: http://hgpy.de/py/It contains the > 2.4, 2.5, trunk and py3k branches (in case anyone wants to compare this to bzr). > > I see your trunk history is stripped. For those who want the complete trunk > history (back to 17 years ago!), I have my own mirror here: > http://dev.pitrou.net:8000/cpython/trunk/ > > For Mercurial beginners, you just have to install Mercurial and type: > hg clone http://dev.pitrou.net:8000/cpython/trunk/ > This gives you a local repository in the "trunk" directory which includes all > the pulled history from the said mirror. > > (however, please note the server is not mine, and I do not guarantee the above > URL will function forever :-)) Thanks, nice work! That is certain to make my Python contributions a bit better organized... cheers, Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ndbecker2 at gmail.com Sat Mar 22 11:15:59 2008 From: ndbecker2 at gmail.com (Neal Becker) Date: Sat, 22 Mar 2008 06:15:59 -0400 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <20080322014442.GA18652@amk.local> <20080322020841.1CF063A4074@sparrow.telecommunity.com> Message-ID: Another use case, which I find in my world, is that there are always packages that interest me (found at pypi), that my vendor hasn't packaged as rpms yet. With finite resources, this will always be true. From steve at holdenweb.com Sat Mar 22 11:25:18 2008 From: steve at holdenweb.com (Steve Holden) Date: Sat, 22 Mar 2008 06:25:18 -0400 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E4330C.5070601@egenix.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> Message-ID: <47E4DE8E.6010809@holdenweb.com> M.-A. Lemburg wrote: > On 2008-03-21 22:21, Phillip J. Eby wrote: >> At 08:06 PM 3/21/2008 +0100, M.-A. Lemburg wrote: >>> I guess the only way to support all of these variants is >>> to use a filesystem based approach, e.g. by placing a file >>> with a special extension into some dir on sys.path. >>> The "database" logic could then scan sys.path for these >>> files, read the data and provide an interface to it. >>> >>> All bdist formats would then have to include these files. >> That's the idea behind the current version of PEP 262, yes, and I think >> it should be kept. >> >>> A separate FILES section also doesn't seem to be necessary - >>> we could just add one or more entries or the format: >>> >>> CreatesDir abc/ >>> CreatesFile abc/xyz1.py >>> CreatesDir abc/def/ >>> CreatesFile abc/def/xyz2.py >>> CreatesFile abc/def/xyz3.py >>> CreatesFile abc/def/xyz4.ini >> I actually think the size and hash information is good, in order to be >> able to tell if you're looking at an original file. I'm not sure how >> useful the permissions and uid/gid info is. I'm hoping we'll hear from >> anybody who has a use case for that. > > You're heading off in the wrong direction: we should not be trying > to rewrite RPM or InnoSetup in Python. > > Anything more complicated should be left to tools which are > specifically written to manage complex software setups. > We *do* need a way to play nice with all the various platform installers. This would surely be welcomed by the many third parties who now have to package Python for their platforms. > I honestly believe that most people would be happy if we just > provide these two things (and no more): > > * install a package from a local archive, a URL or PyPI > > * uninstall a package in way that doesn't break other > installed packages > > and whatever the mechanism, avoid making any undercover > changes to the Python installation such as adding > .pth files, overriding site.py, etc. - these are > not needed if the tool keeps to the simple task of > installing and uninstalling Python packages. > > Examples: > > python pypi.py install mypkg-1.0.tgz > python pypi.py install http://www.example.com/mypkg-1.0.tgz > python pypi.py install mypkg-1.0 > > python pypi.py uninstall mypkg > > If there's a dependency problem, the tool should print the > list of other packages it needs. It should not try to install > things automagically. > ... unless explicitly asked to do so? It seems silly to omit this capability when it's essentially just omitting a recursive call. > If a package needs other modules as well, the package docs > can point the user to use e.g. > > python pypi.py install mydep1-1.3 mydep2-2.3 mydep4-0.3 mypkg-1.0 > > instead. > Why would this be better than using --load-dependencies? > Anything more complicated should be left to specialized > tools such as RPM, apt, MSI or the other such tools out > there - after all the tool should be about Python *package* > installation, not application installation. > > We *don't* need the tool to: > > * support multiple versions of a package (that's just bound > to cause problems with pickle, isinstance() etc.) > Another argument for installing apps as separate components with all dependencies. I really don't believe enough consideration has been given as to the essential difference between these two activities. > * provide namespace hacking (is a completely separate issue > and can be handled by the packages rather than the install > tool) > > * support all kinds of funky version numbers (if a package > wants to participate in the system, the author better > make sure that the version string fits the standard format) > Agreed. > * provide some form of intra-package bus interface (ie. > "entry points" as you call them) > > * provide support for keeping whole packages in ZIP files > (doesn't play well with C extensions, clutters up the > sys.path, is read-only, needs special importers, etc. etc. ) > It shouldn't require special importers, though. And if the zip file contains compiled code the read-only nature doesn't matter if the zips are version-specific. > * try automatic version matching for required packages > > * download things from SourceForge or other sites with special > download mechanisms > > * scan websites for links > > * make coffee, clean the house, send the kids to school :-) > But a package that *would* do this could be immensely popular. >> And of course, there are still some issues to be resolved regarding >> requirements, package name/version stuff, etc. But we can hash those >> out once we reach a quorum on the Distutils-SIG. > Well, I've probably been killfiled into non-existence on this list by now, but it seems to me that we are in danger of answering the wrong problem yet again. But that's all I have to say on this topic, so you can all heave a sigh a relief and get on with messing it up ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Sat Mar 22 11:25:18 2008 From: steve at holdenweb.com (Steve Holden) Date: Sat, 22 Mar 2008 06:25:18 -0400 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E4330C.5070601@egenix.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> Message-ID: <47E4DE8E.6010809@holdenweb.com> M.-A. Lemburg wrote: > On 2008-03-21 22:21, Phillip J. Eby wrote: >> At 08:06 PM 3/21/2008 +0100, M.-A. Lemburg wrote: >>> I guess the only way to support all of these variants is >>> to use a filesystem based approach, e.g. by placing a file >>> with a special extension into some dir on sys.path. >>> The "database" logic could then scan sys.path for these >>> files, read the data and provide an interface to it. >>> >>> All bdist formats would then have to include these files. >> That's the idea behind the current version of PEP 262, yes, and I think >> it should be kept. >> >>> A separate FILES section also doesn't seem to be necessary - >>> we could just add one or more entries or the format: >>> >>> CreatesDir abc/ >>> CreatesFile abc/xyz1.py >>> CreatesDir abc/def/ >>> CreatesFile abc/def/xyz2.py >>> CreatesFile abc/def/xyz3.py >>> CreatesFile abc/def/xyz4.ini >> I actually think the size and hash information is good, in order to be >> able to tell if you're looking at an original file. I'm not sure how >> useful the permissions and uid/gid info is. I'm hoping we'll hear from >> anybody who has a use case for that. > > You're heading off in the wrong direction: we should not be trying > to rewrite RPM or InnoSetup in Python. > > Anything more complicated should be left to tools which are > specifically written to manage complex software setups. > We *do* need a way to play nice with all the various platform installers. This would surely be welcomed by the many third parties who now have to package Python for their platforms. > I honestly believe that most people would be happy if we just > provide these two things (and no more): > > * install a package from a local archive, a URL or PyPI > > * uninstall a package in way that doesn't break other > installed packages > > and whatever the mechanism, avoid making any undercover > changes to the Python installation such as adding > .pth files, overriding site.py, etc. - these are > not needed if the tool keeps to the simple task of > installing and uninstalling Python packages. > > Examples: > > python pypi.py install mypkg-1.0.tgz > python pypi.py install http://www.example.com/mypkg-1.0.tgz > python pypi.py install mypkg-1.0 > > python pypi.py uninstall mypkg > > If there's a dependency problem, the tool should print the > list of other packages it needs. It should not try to install > things automagically. > ... unless explicitly asked to do so? It seems silly to omit this capability when it's essentially just omitting a recursive call. > If a package needs other modules as well, the package docs > can point the user to use e.g. > > python pypi.py install mydep1-1.3 mydep2-2.3 mydep4-0.3 mypkg-1.0 > > instead. > Why would this be better than using --load-dependencies? > Anything more complicated should be left to specialized > tools such as RPM, apt, MSI or the other such tools out > there - after all the tool should be about Python *package* > installation, not application installation. > > We *don't* need the tool to: > > * support multiple versions of a package (that's just bound > to cause problems with pickle, isinstance() etc.) > Another argument for installing apps as separate components with all dependencies. I really don't believe enough consideration has been given as to the essential difference between these two activities. > * provide namespace hacking (is a completely separate issue > and can be handled by the packages rather than the install > tool) > > * support all kinds of funky version numbers (if a package > wants to participate in the system, the author better > make sure that the version string fits the standard format) > Agreed. > * provide some form of intra-package bus interface (ie. > "entry points" as you call them) > > * provide support for keeping whole packages in ZIP files > (doesn't play well with C extensions, clutters up the > sys.path, is read-only, needs special importers, etc. etc. ) > It shouldn't require special importers, though. And if the zip file contains compiled code the read-only nature doesn't matter if the zips are version-specific. > * try automatic version matching for required packages > > * download things from SourceForge or other sites with special > download mechanisms > > * scan websites for links > > * make coffee, clean the house, send the kids to school :-) > But a package that *would* do this could be immensely popular. >> And of course, there are still some issues to be resolved regarding >> requirements, package name/version stuff, etc. But we can hash those >> out once we reach a quorum on the Distutils-SIG. > Well, I've probably been killfiled into non-existence on this list by now, but it seems to me that we are in danger of answering the wrong problem yet again. But that's all I have to say on this topic, so you can all heave a sigh a relief and get on with messing it up ;-) regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From martin at v.loewis.de Sat Mar 22 12:33:49 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 22 Mar 2008 12:33:49 +0100 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080322020447.3B1033A4074@sparrow.telecommunity.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> Message-ID: <47E4EE9D.2090605@v.loewis.de> > Ok, well, I have a rationale for it: make it possible to get rid of eggs > as a mechanism for supporting easy_install. Many people (yourself > included) have criticized eggs as an installation mechanism, and this is > an alternative that gets rid of .egg files and directories in that case, > and most of the need for .pth file usage as well. How so? I cannot see the need for .egg files or .pth files in the first place. If they exist so that you can import stuff: just install to site-packages, and be done. > The data isn't for them to use to meet their use cases, it's for them to > *provide* so that Python tools don't stomp on, uninstall, or otherwise > interfere with files installed by the system. In other words, for > system packagers, it's a communication from the system to Python, rather > than the other way around. Even though the distutils will build the > file in the bdist, the system packaging tools would be free to generate > their own file listing and signatures and such. Ok, that's a reasonable requirement. It will be difficult to implement, though, as it will require Linux distributors (in particular) to include the database snippets in their packages. Essentially, one would have to contribute patches to all the distributions (we care about, at least), and then nag the respective maintainers to include these patches. > I probably should have brought this up, in fact, I think I mentioned it > in a previous thread, but I would like to see PEP 262 add a way to say > "this is a system-installed package, *don't touch*". The idea again is > not to do the job of the native packaging system, but rather to ensure > that Python-specific tools (e.g. easy_install and friends) do not > interfere or conflict with it. Something like a read-only flag? For those without the read-only flag, the specification should explicitly say what manipulation is allowed. Regards, Martin From armin.ronacher at active-4.com Sat Mar 22 12:53:14 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Sat, 22 Mar 2008 11:53:14 +0000 (UTC) Subject: [Python-Dev] Proposal: Slightly Changed Semantics for from-Import Message-ID: Hi all, I propose a small change on the "from"-import behavior. Currently there are two ways to import a module name foo from bar. Namely "import foo.bar as bar" and "from foo import bar". The main problem with the second is that it will not work in every situation. Modules are registered with their names in sys.modules before the import but become and attribute of their parent after. This leads to the surprising behavior that "from foo.bar import baz" works but "from foo import bar" not until the foo.bar module finished setting up. This behavior is especially surprising if you have circular dependencies and you notice that your imports work properly if you import a specific module before another one and will fail if you import it afterwards although it doesn't seem like that should make any difference. My proposal is that "from foo import bar" returns sys.modules["foo.bar"] if this one exists and the foo.bar attribute lookup failed. If a test case is needed I can attach one later on. Regards, Armin From g.brandl at gmx.net Sat Mar 22 13:08:54 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 22 Mar 2008 13:08:54 +0100 Subject: [Python-Dev] Testing documentation snippets Message-ID: Hi, the newest version of Sphinx supports testing doctest (and other) snippets in the documentation. Since we have many examples in the docs that may get out of date, I think this is a valuable thing to have. I've started making the doctests runnable with Sphinx in three documents; the functional howto, the RE and the Decimal docs. If you want to know how to run the doctests, it should be as easy as $ cd Doc $ make doctest PYTHON=../python on UNIX. The PYTHON=../python is needed to really test against the tree version of Python, not the system-wide installed one. If you want to run only tests in one source file, run $ make doctest PYTHON=../python SOURCES=library/decimal.rst If there is further interest in this, I'll explain the details how to "activate" examples to work with "make doctest". Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From p.f.moore at gmail.com Sat Mar 22 13:39:24 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 22 Mar 2008 12:39:24 +0000 Subject: [Python-Dev] Python source code on Mercurial In-Reply-To: References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <932f8baf0803201426ua0ae2edu3f7b62eaf22ef787@mail.gmail.com> <79990c6b0803211438w347d052fre2a97ebde72b44b4@mail.gmail.com> Message-ID: <79990c6b0803220539n78a04e51o3df672ac391918c2@mail.gmail.com> On 21/03/2008, Brett Cannon wrote: > Bazaar supports lightweight checkouts which act like svn checkouts. > They are also actively working on allowing for partial checkouts. That > way you can either specify an initial revision to pull the history > down to or start with an initial lightweight checkout and have it save > history as it pulls it over the network as you use it. I know, but the details aren't 100% clear yet. Mercurial is in much the same situation (although perhaps less far down that route - Bazaar development seems far faster at the moment, for better or worse). One point, which I assume you know but others may not - a bazaar "checkout" is not like a local branch. In a checkout, all commits go straight back to the parent branch, meaning that local commits aren't possible (OK, that's an oversimplification, but let's keep things simple) and the workflow is much more like Subversion. The biggest problem with DVCSs is the vast range of subtly different terminology, for related but distinct concepts. It makes having a common terminology for comparisons really hard, resulting in all sorts of confusion :-( Paul. From musiccomposition at gmail.com Sat Mar 22 14:24:33 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sat, 22 Mar 2008 08:24:33 -0500 Subject: [Python-Dev] Python source code on Mercurial In-Reply-To: <79990c6b0803220539n78a04e51o3df672ac391918c2@mail.gmail.com> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <932f8baf0803201426ua0ae2edu3f7b62eaf22ef787@mail.gmail.com> <79990c6b0803211438w347d052fre2a97ebde72b44b4@mail.gmail.com> <79990c6b0803220539n78a04e51o3df672ac391918c2@mail.gmail.com> Message-ID: <1afaf6160803220624v75f6aafbve7a99f201c4e3442@mail.gmail.com> On Sat, Mar 22, 2008 at 7:39 AM, Paul Moore wrote: > On 21/03/2008, Brett Cannon wrote: > > Bazaar supports lightweight checkouts which act like svn checkouts. > > They are also actively working on allowing for partial checkouts. That > > way you can either specify an initial revision to pull the history > > down to or start with an initial lightweight checkout and have it save > > history as it pulls it over the network as you use it. > > I know, but the details aren't 100% clear yet. Mercurial is in much > the same situation (although perhaps less far down that route - Bazaar > development seems far faster at the moment, for better or worse). > > One point, which I assume you know but others may not - a bazaar > "checkout" is not like a local branch. In a checkout, all commits go > straight back to the parent branch, meaning that local commits aren't > possible (OK, that's an oversimplification, but let's keep things > simple) and the workflow is much more like Subversion. You can commit locally on a Bazaar checkout by adding the --local option to commit. This feature is supposed to add the flexibility to have team branches, where a bunch of people can work on a branch. > > > The biggest problem with DVCSs is the vast range of subtly different > terminology, for related but distinct concepts. It makes having a > common terminology for comparisons really hard, resulting in all sorts > of confusion :-( > > Paul. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080322/d9d5d22a/attachment.htm From p.f.moore at gmail.com Sat Mar 22 14:46:42 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 22 Mar 2008 13:46:42 +0000 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E4DE8E.6010809@holdenweb.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <47E4DE8E.6010809@holdenweb.com> Message-ID: <79990c6b0803220646r50e9b455y5bd9b955814f334a@mail.gmail.com> On 22/03/2008, Steve Holden wrote: > Well, I've probably been killfiled into non-existence on this list by > now, but it seems to me that we are in danger of answering the wrong > problem yet again. But that's all I have to say on this topic, so you > can all heave a sigh a relief and get on with messing it up ;-) You probably have my company in the killfile, but I have a nagging feeling in the same direction. My biggest problem is that I can't express what I believe is the *right* problem, beyond the over-general statement that it seems crucial to me that there should be a single, unified way of managing *all* packages installed in a given Python installation. Whether that's a Python-only solution, or the system packager, doesn't matter. There should be only one way to do it, to reuse a well-known phrase :-) If you know how to state nature of the right problem, that would be useful. All this talk of "playing nicely with the system packager" seems to imply that people are designing a second solution, and trying to manage the interaction, rather than deciding on *one* solution (which by definition has no interaction to worry about). It's reasonable to have multiple solutions for multiple Python installations (system packager for the system python, python packager for a local install, for example) but that's a different matter. Oh, and application installation is (should be) completely different. On Windows, applications should probably be bundled with their own Python interpreter, a la py2exe. On Unix/Linux, I don't know what the standard is, so I'd have to defer to others. Paul. From martin at v.loewis.de Sat Mar 22 15:02:33 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 22 Mar 2008 15:02:33 +0100 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <525f23e80803220621u4e826023u975048a45b76a079@mail.gmail.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <525f23e80803220621u4e826023u975048a45b76a079@mail.gmail.com> Message-ID: <47E51179.5040601@v.loewis.de> > It seems to me that this discussion is being undermined by not > acknowledging the many use cases up front. There is no rationale > because there are too many tacit rationales. I honestly, really, cannot imagine what those are. Explicit is better than implicit. > Nevertheless, the many > use cases are related at some level and would benefit from some form > of lowest-common-denominator support in the standard library. As an > example, IF I wanted to use a library to support multi-version > packages or one to ensure I had all the dependencies, I would need to > know what versions of things were currently installed. How would you install multiple versions in the first place? Python supports no such thing, at least not without setting PYTHONPATH, or otherwise changing sys.path. > My personal use case is for multi-version packages. I deploy many > small scripts (not applications) that use an evolving set of base > libraries. I don't want the older scripts to hold me back from pushing > forward with the base libraries, so I peg the scripts to their > respective major versions as I release them. I could not have imagined that as a use case. I never install multiple versions of the same thing on the same system, except for Python itself. I also try to avoid using evolving libraries as much as possible, and quickly abandon libraries if they change in an incompatible manner across releases, at least if porting becomes too much of a burden. Notice that this use case isn't listed in the PEP, and I cannot see how the PEP can help providing that functionality. Regards, Martin From p.f.moore at gmail.com Sat Mar 22 15:14:01 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 22 Mar 2008 14:14:01 +0000 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <525f23e80803220621u4e826023u975048a45b76a079@mail.gmail.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <525f23e80803220621u4e826023u975048a45b76a079@mail.gmail.com> Message-ID: <79990c6b0803220714q7e2f2593wa46694ede30f594b@mail.gmail.com> On 22/03/2008, Alexander Michael wrote: > > IOW, the PEP is lacking a rationale. > > It seems to me that this discussion is being undermined by not > acknowledging the many use cases up front. There is no rationale > because there are too many tacit rationales. Absolutely! It feels like people are trying to design a solution without having written up the problem spec. Maybe that's because there are too many problems for a single solution to work in all cases? > My personal use case is for multi-version packages. I deploy many > small scripts (not applications) that use an evolving set of base > libraries. I don't want the older scripts to hold me back from pushing > forward with the base libraries, so I peg the scripts to their > respective major versions as I release them. My personal use case is for a Windows system with a number of adhoc scripts, which may depend on 3rd party libraries, and a number of "applications" (3rd party or otherwise) written in Python. I want the applications to use a bundled Python and library, via py2exe. There should be no dependency on the "system" Python. Adhoc scripts can depend on libraries installed into the system Python, but generally there will be a "base" set of libraries that I will always have installed and which can be assumed to be present (pywin32, cx_Oracle, py2exe, ...). Scripts can use these without restriction but should be prepared to work with whatever version is present. It should (almost) never be a problem to a script if a library gets upgraded. Additional libraries may be installed in the system Python, for one-off jobs, or for testing and development. I will install and remove these at will, and nothing critical should break ifn I do. There is generally no other Python present on the system. On my development machine, I may have alternative versions installed, and I may have trunk checkouts with a python.exe built, but these will never be used for anything other than specific development tasks, and are treated as "throw-away". It's very rare that any 3rd party library will be installed in these, and special handling when it does happen is entirely acceptable. For the system Python, I need: - a single way to list what's installed (including version) - a single way to uninstall items as needed - a way (or more than one) to install 3rd party software *which ties into the above* The key maintenance task I do on the system Python is to list everything installed, uninstall them all, upgrade the system Python, then reinstall the ones I had installed previously. (Don't bother telling me there are other ways I could do this - that's not the point here, this is how I actually work). The reason setuptools/easy_install breaks this is because it fails the "single way" test. I have to use bdist_wininst for some packages, so setuptools *has* to integrate with that. Also, setuptools doesn't have the list and uninstall features. So that's my problem/requirements. Anything that solves this gets +1 from me. Anything else gets -1 or at best -0. Anything that adds extra features while not solving my problem gets a strong -1, as the extra features will encourage take-up of that solution, exacerbating my current problem. Paul. From martin at v.loewis.de Sat Mar 22 15:14:05 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 22 Mar 2008 15:14:05 +0100 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080322132418.GB8513@laurie.devork> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> <47E4EE9D.2090605@v.loewis.de> <20080322132418.GB8513@laurie.devork> Message-ID: <47E5142D.7010902@v.loewis.de> >> Essentially, one would have to contribute patches to all the >> distributions (we care about, at least), and then nag the respective >> maintainers to include these patches. > > Not true. You just need to make sure that "setup.py install" creates > that database. With the proposed format of the database this is just > a file in the correct location - exactly for this reason. Next time > the distribution will build the package that database file will be in > place. How so? Are you /sure/ the packaging process even *runs* setup.py? And if they do, why do you think they will pick up the database file? Regards, Martin From p.f.moore at gmail.com Sat Mar 22 15:17:20 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 22 Mar 2008 14:17:20 +0000 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E51179.5040601@v.loewis.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <525f23e80803220621u4e826023u975048a45b76a079@mail.gmail.com> <47E51179.5040601@v.loewis.de> Message-ID: <79990c6b0803220717u7e0208a9na15a1531b48fd545@mail.gmail.com> On 22/03/2008, "Martin v. L?wis" wrote: > How would you install multiple versions in the first place? Python > supports no such thing, at least not without setting PYTHONPATH, > or otherwise changing sys.path. That's an unrelated feature of setuptools, providing a way to "install" multiple versions of a package and select which version you want at runtime. It does this by sys.path manipulation, I believe. It's a good example of the sort of orthogonal feature which makes setuptools such an issue. It's a genuinely useful feature to some people, but it's unrelated to packaging and installation, so people buy into a packaging solution (eggs) for the non-packaging benefits they provide. Paul. From p.f.moore at gmail.com Sat Mar 22 15:21:33 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 22 Mar 2008 14:21:33 +0000 Subject: [Python-Dev] Python source code on Mercurial In-Reply-To: <1afaf6160803220624v75f6aafbve7a99f201c4e3442@mail.gmail.com> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <932f8baf0803201426ua0ae2edu3f7b62eaf22ef787@mail.gmail.com> <79990c6b0803211438w347d052fre2a97ebde72b44b4@mail.gmail.com> <79990c6b0803220539n78a04e51o3df672ac391918c2@mail.gmail.com> <1afaf6160803220624v75f6aafbve7a99f201c4e3442@mail.gmail.com> Message-ID: <79990c6b0803220721o64872e07wf0d8d93da9b0112e@mail.gmail.com> On 22/03/2008, Benjamin Peterson wrote: > > One point, which I assume you know but others may not - a bazaar > > "checkout" is not like a local branch. In a checkout, all commits go > > straight back to the parent branch, meaning that local commits aren't > > possible (OK, that's an oversimplification, but let's keep things > > simple) and the workflow is much more like Subversion. > You can commit locally on a Bazaar checkout by adding the --local option to > commit. This feature is supposed to add the flexibility to have team > branches, where a bunch of people can work on a branch. Sigh. I said I know it's a simplification. IIRC, you can't manage a mixture of local and remote commits. You have to push your local commits before doing a remote commit. But that's more detail than is needed at the moment. Please can we keep things at a high level for the people who haven't experimented with DVCS tools yet. (I know, I'm as much to blame as anyone. I'm going to stop now). Paul. From ndbecker2 at gmail.com Sat Mar 22 15:45:35 2008 From: ndbecker2 at gmail.com (Neal Becker) Date: Sat, 22 Mar 2008 10:45:35 -0400 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> <47E4EE9D.2090605@v.loewis.de> <20080322132418.GB8513@laurie.devork> <47E5142D.7010902@v.loewis.de> Message-ID: "Martin v. L?wis" wrote: >>> Essentially, one would have to contribute patches to all the >>> distributions (we care about, at least), and then nag the respective >>> maintainers to include these patches. >> >> Not true. You just need to make sure that "setup.py install" creates >> that database. With the proposed format of the database this is just >> a file in the correct location - exactly for this reason. Next time >> the distribution will build the package that database file will be in >> place. > > How so? Are you /sure/ the packaging process even *runs* setup.py? > And if they do, why do you think they will pick up the database > file? > In the case of Fedora rpms, the usual install uses setup.py. From barry at python.org Sat Mar 22 15:54:04 2008 From: barry at python.org (Barry Warsaw) Date: Sat, 22 Mar 2008 10:54:04 -0400 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: <47E2DBEF.8010809@cheimes.de> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <47E2DBEF.8010809@cheimes.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 20, 2008, at 5:49 PM, Christian Heimes wrote: > Barry Warsaw schrieb: >> I'm happy to announce that we now have available for public >> consumption, the Python source code for 2.5, 2.6 and 3.0 available >> under the Bazaar distributed version control system. > > Somebody has to fix the subversion related code in Python/sysmodule.c: > > heimes at hamiller:~/dev/python/bzr/trunk$ ./python > Fatal Python error: subversion keywords missing > Aborted (core dumped) Yep. I've opened issue 2456 for this in the bug tracker. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR+UdjnEjvBPtnXfVAQKQ8gQAuh8ynIbNXFaFEViQAvJ84M/D6Db4q008 X3XgoEq0wJWpjO2pA3lzDY0uuowkXGUMuhgMOQ6qeTz+1nDeazQPDdHwKr2wdNff yBBG+3jstDdiwr9uGjRA39gpu/29JVE0Kxd9aMyOorW48RYX6wGcfbg1nGYkOO0u haY0pWG4g+0= =q6Yg -----END PGP SIGNATURE----- From barry at python.org Sat Mar 22 16:02:07 2008 From: barry at python.org (Barry Warsaw) Date: Sat, 22 Mar 2008 11:02:07 -0400 Subject: [Python-Dev] [Python-3000] Python source code on Bazaar vcs In-Reply-To: <79990c6b0803211428h2eef9fa0s1032570cdf3fe162@mail.gmail.com> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <20080321084249.GA83712@nexus.in-nomine.org> <79990c6b0803211428h2eef9fa0s1032570cdf3fe162@mail.gmail.com> Message-ID: <3EE240A4-8265-45A2-AEB8-2D53B21AEBEE@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 21, 2008, at 5:28 PM, Paul Moore wrote: > > One thing I really like the idea of with Mercurial for my situation > (non-committer) is the mq extension, which lets me manage my changes > as a "stack of patches" - so I'm completely working with patches, > which is what I have to post to the tracker, etc. > > Is there a similar workflow in Bazaar? I know there's the loom > extension (although I haven't used it much yet) but I'm not sure how > I'd use that. Yes, looms are awesome, I highly recommend you install the plugin and take a look. There's a link to the loom plugin on . I don't know much about Mercurial queues, but I've been told the feature is similar. I do have a lot of experience using Bazaar looms, and I'm a huge fan. > Basically, can some Bazaar expert offer a suggestion as to how a > non-developer with read-only access would best use the Bazaar > repositories to maintain a number of patches to be posted to the > tracker? I used it in Mailman work all the time, and I've been using it on the train home from Pycon for working on the email package in Python 3.0. Quick tutorial: bzr branch http://code.python.org/python/3.0 my30feature cd my30feature bzr nickname upstream bzr loomify bzr create-thread firststuff bzr commit bzr create-thread secondstuff bzr commit bzr create thread thirdstuff bzr commit bzr down-thread secondstuff # see what's different between secondstuff and firststuff bzr diff -r thread: rinse-and-repeat-ly y'rs, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR+Ufb3EjvBPtnXfVAQI2ywP9GLK07pVEKmfb7K3s/I7oUnw3MA6uQPML 41rUi8fJQIIejPkmsKrrchSSkp3ZeSot4btxxXYD6G+HX3yzNgK3ydijZtpofIm1 dTIreCOYE9uGS6xk2Frj58rCrDwrsZ0eADyCZ4V18pBNEwpTsDw0wLFktqx2yzhH BZBLqUy//NM= =cuNj -----END PGP SIGNATURE----- From pje at telecommunity.com Sat Mar 22 16:12:12 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 22 Mar 2008 11:12:12 -0400 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080322110010.GA8513@laurie.devork> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> <20080322110010.GA8513@laurie.devork> Message-ID: <20080322151218.7EDB73A40A5@sparrow.telecommunity.com> At 11:00 AM 3/22/2008 +0000, Floris Bruynooghe wrote: >As long as systems (dpkg, rpm, ...) install the .egg-info files they >do communicate which modules/distributions are installed. The >installdb would just duplicate this information (according to the >current PEP). .egg-info/PKG-INFO don't list the specific files, though. >There is a way of telling if you have to keep you hands off a package >(sorry to bring this up again): installation paths. /usr/lib is the >system path, the local admin (and hence setuptools) should keep their >hands off it at all times (unless requested with a --prefix or so for >building the debs or rpms but an uninstall in those cases won't be >required from setuptools). As I mentioned previously, if the spec says anything about specific paths, it will be full of fail. The spec MUST be able to work with *any* local policy about where Python packages are to be installed. Otherwise, any tool that wants to work with install-dbs will end up accumulating a long list of paths to be handled specially for each OS vendor and version... and still not handle everything. No can do. This has to be a mechanism, not a policy. Vendors and admins should be able to enforce reasonable policies, without requiring that every tool have those policies built in. For one thing, it's an entry barrier to tools. Basically, what I'm proposing here is like WSGI for package management tools -- and building anything about paths into the spec would be like WSGI spelling out what pages should be at what URLs! From pje at telecommunity.com Sat Mar 22 16:19:17 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 22 Mar 2008 11:19:17 -0400 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E4EE9D.2090605@v.loewis.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> <47E4EE9D.2090605@v.loewis.de> Message-ID: <20080322151918.D9EBC3A40B1@sparrow.telecommunity.com> At 12:33 PM 3/22/2008 +0100, Martin v. L?wis wrote: >>I probably should have brought this up, in fact, I think I >>mentioned it in a previous thread, but I would like to see PEP 262 >>add a way to say "this is a system-installed package, *don't >>touch*". The idea again is not to do the job of the native >>packaging system, but rather to ensure that Python-specific tools >>(e.g. easy_install and friends) do not interfere or conflict with it. > >Something like a read-only flag? Not exactly. More like, "package management tool X claims exclusive rights to this package". Python tools would always defer this right to the system packager, i.e. a system packager is not obliged to respect a Python tool's claim to a file, but not the other way around. That way, system packaging tools don't need to do anything but mark the installed files as belonging to them. Since most vendors at least *begin* with a "setup.py install", we could provide a way to indicate that the installation is being done on behalf of a system packaging tool, so that it can provide that indication. >For those without the read-only flag, the specification should >explicitly say what manipulation is allowed. Since a distribution isn't really "mutable", I would think that uninstallation and reinstallation would be the only manipulation available. (As distinct from inspection, verification, and other read-only activities.) It's possible, though, that there might also be actions such as restoring or relocating scripts or data in shared locations outside of the sys.path directory. That will get clearer as the spec gets defined. From barry at python.org Sat Mar 22 16:21:46 2008 From: barry at python.org (Barry Warsaw) Date: Sat, 22 Mar 2008 11:21:46 -0400 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: <47DDF318.4080303@v.loewis.de> References: <2C2D7EC7-51F2-4F4E-8127-87ACCC6F8413@python.org> <5E86F9D7-4421-4C57-8294-B0EC6A4EE03C@python.org> <47DDF318.4080303@v.loewis.de> Message-ID: <96720E99-D5A3-46B8-BBAA-88C021F24653@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 17, 2008, at 12:27 AM, Martin v. L?wis wrote: >> 'critical' is fine (or 'immediate'). My problem before was that I >> couldn't do one query that gave me all the critical issues for >> both 2.6 and 3.0. That certainly could have been pebkac though. >> Neal mentioned that that kind of query should be possible, if it's >> not already there. > > I just created a "Showstoppers" public query (go to Your Queries/ > *edit*, > and set it to "leave in"). > > This *just* searches for critical open issues, all versions. Given > that there are currently really no critical issues, that semantically > is the same as further restricting it to 2.6 and 3.0. > > Creating a query that searches for multiple versions is possible > through URL editing, but not the form (currently). I'm not sure > whether that would search for issues which are marked both 2.6 > and 3.0, or for issues that are either marked 2.6 *or* 3.0. Thanks Martin, I think this will work for now. Is there any way you can allow me to edit this query too? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR+UkDnEjvBPtnXfVAQItWgP9FF++/A19BvM4+wjf8xyV2oCbMa0KH1+D ssTdA+pnB70c06CuGNe7Wf7OEGNNmJmMtGmTcHqa5irJc/BPVlOvZ4Uj9iC9qJKe BW0P4BoCmQ1Dtap7tPa9ACb2+b0zwPLgXG3O/0WMwkG3sdeLYm6oscWYmXcYzF2V do1rmkIAbmw= =ziz3 -----END PGP SIGNATURE----- From pje at telecommunity.com Sat Mar 22 16:25:45 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 22 Mar 2008 11:25:45 -0400 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <79990c6b0803220714q7e2f2593wa46694ede30f594b@mail.gmail.co m> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <525f23e80803220621u4e826023u975048a45b76a079@mail.gmail.com> <79990c6b0803220714q7e2f2593wa46694ede30f594b@mail.gmail.com> Message-ID: <20080322152549.E4C943A40A5@sparrow.telecommunity.com> At 02:14 PM 3/22/2008 +0000, Paul Moore wrote: >For the system Python, I need: >- a single way to list what's installed (including version) >- a single way to uninstall items as needed >- a way (or more than one) to install 3rd party software *which ties >into the above* Right, and the PEP effort is devoted to having one way to store the information required, to tie these things together. If there is a standard way to store that info on your system, then it doesn't matter how many tools there are, you still have your "one way" to list what's installed or to uninstall things, because you just pick the one lister and/or uninstaller whose UI you prefer. From martin at v.loewis.de Sat Mar 22 16:29:57 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 22 Mar 2008 16:29:57 +0100 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080322151918.D9EBC3A40B1@sparrow.telecommunity.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> <47E4EE9D.2090605@v.loewis.de> <20080322151918.D9EBC3A40B1@sparrow.telecommunity.com> Message-ID: <47E525F5.40705@v.loewis.de> >> For those without the read-only flag, the specification should >> explicitly say what manipulation is allowed. > > Since a distribution isn't really "mutable", I would think that > uninstallation and reinstallation would be the only manipulation > available. (As distinct from inspection, verification, and other > read-only activities.) Sure, but what is precisely the semantics of uninstallation, in terms of changes to the system state? I think any model where uninstallation is merely the removal of files is too limited to be practical. Regards, Martin From martin at v.loewis.de Sat Mar 22 16:31:15 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 22 Mar 2008 16:31:15 +0100 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: <96720E99-D5A3-46B8-BBAA-88C021F24653@python.org> References: <2C2D7EC7-51F2-4F4E-8127-87ACCC6F8413@python.org> <5E86F9D7-4421-4C57-8294-B0EC6A4EE03C@python.org> <47DDF318.4080303@v.loewis.de> <96720E99-D5A3-46B8-BBAA-88C021F24653@python.org> Message-ID: <47E52643.1080804@v.loewis.de> > Thanks Martin, I think this will work for now. Is there any way you can > allow me to edit this query too? Not easily. I could just remove it, and allow you to create a new one (or you create one yourself, anyway, and I remove mine later). Regards, Martin From pje at telecommunity.com Sat Mar 22 16:33:02 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 22 Mar 2008 11:33:02 -0400 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080322151918.D9EBC3A40B1@sparrow.telecommunity.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> <47E4EE9D.2090605@v.loewis.de> <20080322151918.D9EBC3A40B1@sparrow.telecommunity.com> Message-ID: <20080322153304.B7CAD3A40B1@sparrow.telecommunity.com> At 11:19 AM 3/22/2008 -0400, Phillip J. Eby wrote: >Not exactly. More like, "package management tool X claims exclusive >rights to this package". Python tools would always defer this right >to the system packager, i.e. a system packager is not obliged to >respect a Python tool's claim to a file, but not the other way around. > >That way, system packaging tools don't need to do anything but mark >the installed files as belonging to them. This probably needs to be refined a little. Exclusive right is too strong, and it goes against Paul Moore's desire for using a single tool. Perhaps instead what it should be is an "uninstall warning" field that must be displayed to a user if an interactive program is doing uninstallation, and that a non-interactive program must refuse to uninstall unless explicitly requested to go ahead. Unfortunately, a warning message might then need to be localized. So this idea still needs some work. From martin at v.loewis.de Sat Mar 22 16:42:36 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 22 Mar 2008 16:42:36 +0100 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080322152150.GA16277@laurie.devork> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> <47E4EE9D.2090605@v.loewis.de> <20080322132418.GB8513@laurie.devork> <47E5142D.7010902@v.loewis.de> <20080322152150.GA16277@laurie.devork> Message-ID: <47E528EC.7010605@v.loewis.de> > I speak for Debian, so for Debian: yes. The setup.py would have to be > pretty bad for a packager to not use it. There is no reason to > re-write upstream's installation procedure as you would have to figure > out which files need to be installed where and this would create many > bugs. > > The canonical way is something like this: > > $ pythonX.Y setup.py --root=$(pwd)/debian/tmp > $ # Fixup anything done wrong/badly (for debian) by setup.py > $ # Make a tarball of $(pwd)/debian/tmp > > In reality it's slightly more complicated of course. More than slightly so, with pycentral, pysupport, and all that. I still doubt that the database would show up in a directory on sys.path. IIUC, things get install into /usr/share/pycentral/package, and then get symlinked into /usr/lib/pythonX.Y/site-packages at installation time. It's not all that clear to me how that deals with "non-regular" files. > At [1] there are > many packages, paste and parallelpython are good examples if you're > interested (look in the debian/rules file). I started with nevow. It uses cdbs, so its debian/rules is nearly empty, and does not include a call to setup.py at all. Instead, the distutils support is in /usr/share/cdbs/1/class/python-distutils.mk where setup.py is invoked as ifeq (,$(DEB_PYTHON_REAL_LIB_PACKAGES)) common-install-arch common-install-indep:: common-install-impl common-install-impl:: cd $(DEB_SRCDIR) && python$(DEB_PYTHON_COMPILE_VERSION) $(DEB_PYTHON_SETUP_CMD) install --root=$(DEB_DESTDIR) $(DEB_PYTHON_INSTALL_ARGS_ALL) $(DEB_PYTHON_INSTALL_ARGS_$(cdbs_curpkg)) else $(patsubst %,install/%,$(DEB_PYTHON_REAL_LIB_PACKAGES)) :: install/% : cd $(DEB_SRCDIR) && python$(cdbs_python_ver) $(DEB_PYTHON_SETUP_CMD) install --root=$(CURDIR)/debian/$(cdbs_curpkg) $(DEB_PYTHON_INSTALL_ARGS_ALL) $(DEB_PYTHON_INSTALL_ARGS_$(cdbs_curpkg)) endif $(patsubst %,install/%,$(DEB_PYTHON_SIMPLE_PACKAGES)) :: install/% : cd $(DEB_SRCDIR) && python $(DEB_PYTHON_SETUP_CMD) install --root=$(CURDIR)/debian/$(cdbs_curpkg) $(DEB_PYTHON_INSTALL_ARGS_ALL) $(DEB_PYTHON_INSTALL_ARGS_$(cdbs_curpkg)) I cannot infer from that whether supplying a package database snippet in setup.py would actually work. Regards, Martin From p.f.moore at gmail.com Sat Mar 22 16:44:32 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 22 Mar 2008 15:44:32 +0000 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080322153304.B7CAD3A40B1@sparrow.telecommunity.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> <47E4EE9D.2090605@v.loewis.de> <20080322151918.D9EBC3A40B1@sparrow.telecommunity.com> <20080322153304.B7CAD3A40B1@sparrow.telecommunity.com> Message-ID: <79990c6b0803220844u3499e83cwae684867d6d1e02d@mail.gmail.com> On 22/03/2008, Phillip J. Eby wrote: > This probably needs to be refined a little. Exclusive right is too > strong, and it goes against Paul Moore's desire for using a single > tool. Huh? How's that? Don't forget that I'm on Windows, and on Windows there is no "system tool" - just bdist_wininst, bdist_msi and easy_install. The fact that bdist_wininst and bdist_msi link into the system UI for listing and uninstallation doesn't make packages using them "system packages". If easy_install put add/remove program info in place, my "single tool" would be the add/remove list. If something else (for example, the proposed index of installed package, with a suitable UI) is deemed the "single tool", then bdist_wininst and bdist_msi have to be changed to respect that. The only fly in this ointment is bdist_msi, which uses MSI format, which is a lot nearer to a Windows "system packager" than anything else. Whether that means bdist_msi can't be changed to work with a package index rather than (or as well as, I don't care) add/remove, I don't know. > Unfortunately, a warning message might then need to be localized. So > this idea still needs some work. The "one way to do it" uninstaller should be able to uninstall everything. If it needs to call the system uninstaller for a specific package, there's nothing wrong with doing that. But why tell me to run a different command? Just do it for me. I only want one UI, the rest is implementation detail. (Others may have different preferences, so a choice may need to be made. If so, and if it goes against me, fair enough, I have to decide what to do about that for myself. But I'd rather force people to tell me "no", than leave people thinking they satisfied me and wondering why I'm still complaining...) Paul. From pje at telecommunity.com Sat Mar 22 16:44:48 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 22 Mar 2008 11:44:48 -0400 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E525F5.40705@v.loewis.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> <47E4EE9D.2090605@v.loewis.de> <20080322151918.D9EBC3A40B1@sparrow.telecommunity.com> <47E525F5.40705@v.loewis.de> Message-ID: <20080322154449.BDA423A40A5@sparrow.telecommunity.com> At 04:29 PM 3/22/2008 +0100, Martin v. L?wis wrote: >>>For those without the read-only flag, the specification should >>>explicitly say what manipulation is allowed. >>Since a distribution isn't really "mutable", I would think that >>uninstallation and reinstallation would be the only manipulation >>available. (As distinct from inspection, verification, and other >>read-only activities.) > >Sure, but what is precisely the semantics of uninstallation, in >terms of changes to the system state? > >I think any model where uninstallation is merely the removal >of files is too limited to be practical. The distutils only support the *addition* of files, so I'm not sure how only removing files is a limit here. Could you explain? From martin at v.loewis.de Sat Mar 22 16:45:08 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 22 Mar 2008 16:45:08 +0100 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond In-Reply-To: References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> <47E4EE9D.2090605@v.loewis.de> <20080322132418.GB8513@laurie.devork> <47E5142D.7010902@v.loewis.de> Message-ID: <47E52984.3070407@v.loewis.de> > In the case of Fedora rpms, the usual install uses setup.py. Ok. Does it then also package all files that get installed into the RPM file? If it produces multiple RPMs from a single source package, how does it know which files go into what RPM? Regards, Martin From barry at python.org Sat Mar 22 16:46:06 2008 From: barry at python.org (Barry Warsaw) Date: Sat, 22 Mar 2008 11:46:06 -0400 Subject: [Python-Dev] 2.6 and 3.0 project management In-Reply-To: <47E52643.1080804@v.loewis.de> References: <2C2D7EC7-51F2-4F4E-8127-87ACCC6F8413@python.org> <5E86F9D7-4421-4C57-8294-B0EC6A4EE03C@python.org> <47DDF318.4080303@v.loewis.de> <96720E99-D5A3-46B8-BBAA-88C021F24653@python.org> <47E52643.1080804@v.loewis.de> Message-ID: <3ECA8175-85BF-4350-8856-BBAB40AC0784@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 22, 2008, at 11:31 AM, Martin v. L?wis wrote: >> Thanks Martin, I think this will work for now. Is there any way >> you can allow me to edit this query too? > > Not easily. > > I could just remove it, and allow you to create a new one (or you > create > one yourself, anyway, and I remove mine later). Naw, no need for the extra work. Thanks! - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR+UpvnEjvBPtnXfVAQLHDAP/WVC9IJpGbe0RgoG/5AkWki0AioEvvrPL 2i9+wSPJYXmNz1960tpH+iehjEtkxdCHFNSbSP9BAB71ANRQXJtpxcibNvdHnF55 yWI7lM/Qt1NMYyUD4vn8HDNSMLFSQqztvkCm4OmkzhO/ZdODp4kHdUUfhn7ggC72 jzSq9Qt9eJY= =4xE2 -----END PGP SIGNATURE----- From martin at v.loewis.de Sat Mar 22 16:51:09 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 22 Mar 2008 16:51:09 +0100 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080322020841.1CF063A4074@sparrow.telecommunity.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <20080322014442.GA18652@amk.local> <20080322020841.1CF063A4074@sparrow.telecommunity.com> Message-ID: <47E52AED.5070802@v.loewis.de> > I more question the permissions and uid/gid stuff; I'm not really > clear on what I'd use that stuff for in easy_install/uninstall/etc. I think this was all captured in amk's statement "RPM-like verification". RPM not only verifies file content, but also whether the permissions are all correct. So if the administrator mistakenly does a chown -R on a subtree, he can get back to what it was with the package manager, without having to reinstall everything, or can atleast find out what packages he broke. IIUC, the Solaris package manager provides the same feature. MSI provides a "repair installation", which doesn't really check anything, but reruns the entire installation (and then skipping those files who passed the checksum test, where checksums had been recorded in the MSI). Regards, Martin From martin at v.loewis.de Sat Mar 22 16:52:33 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 22 Mar 2008 16:52:33 +0100 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <20080322014442.GA18652@amk.local> <20080322020841.1CF063A4074@sparrow.telecommunity.com> Message-ID: <47E52B41.7030802@v.loewis.de> Neal Becker wrote: > Another use case, which I find in my world, is that there are always > packages that interest me (found at pypi), that my vendor hasn't packaged > as rpms yet. > > With finite resources, this will always be true. Why do you need a package database for that? Can't you just run "setup.py install", or perhaps "setup.py bdist_rpm", and then install the RPM? Regards, Martin From martin at v.loewis.de Sat Mar 22 17:06:05 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 22 Mar 2008 17:06:05 +0100 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <79990c6b0803220844u3499e83cwae684867d6d1e02d@mail.gmail.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> <47E4EE9D.2090605@v.loewis.de> <20080322151918.D9EBC3A40B1@sparrow.telecommunity.com> <20080322153304.B7CAD3A40B1@sparrow.telecommunity.com> <79990c6b0803220844u3499e83cwae684867d6d1e02d@mail.gmail.com> Message-ID: <47E52E6D.5090303@v.loewis.de> > Huh? How's that? Don't forget that I'm on Windows, and on Windows > there is no "system tool" - just bdist_wininst, bdist_msi and > easy_install. The fact that bdist_wininst and bdist_msi link into the > system UI for listing and uninstallation doesn't make packages using > them "system packages". In pje's terminology, they do. He uses "system package" as a shorthand for "package in a format defined by the system vendor", not as "package supplied by the system vendor" (IIUC). So .msi files and self-extracting .exe files are all "system packages", as opposed to .eggs (which are in a format that wasn't defined by the system vendor). > The only fly in this ointment is bdist_msi, which uses MSI format, > which is a lot nearer to a Windows "system packager" than anything > else. Whether that means bdist_msi can't be changed to work with a > package index rather than (or as well as, I don't care) add/remove, I > don't know. If the package database is merely a directory with additional files in it, one file per package, then most likely both bdist_wininst and bdist_msi could support that. If the file needs to contain file names specific to the target system, then supporting it in bdist_msi is a bit tricky, as one would have to generate the file at installation time. That's a "custom action"; I'd probably generate a VB script at packaging time which is then run at installation time to edit the package database. > The "one way to do it" uninstaller should be able to uninstall > everything. If it needs to call the system uninstaller for a specific > package, there's nothing wrong with doing that. But why tell me to run > a different command? Just do it for me. I only want one UI, the rest > is implementation detail. The uninstallation procedure of the system installer probably has a separate UI which can't really be suppressed. For example, uninstallation may be rejected as additional applications rely on the package, or uninstallation could cause automatic removal of prerequisite packages that are then no longer used, all requiring user confirmation. Also, on some systems, it's difficult to know what specific tool to run to interact with the system packaging. On some systems, you have a choice of multiple command-line, text mode, and GUI tools, some of which may not be installed, or may fail to work (e.g. when you don't have the windowing system running, and the tool is a windowed one), or may not be the user's preference. Regards, Martin From martin at v.loewis.de Sat Mar 22 17:10:24 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 22 Mar 2008 17:10:24 +0100 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080322154449.BDA423A40A5@sparrow.telecommunity.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> <47E4EE9D.2090605@v.loewis.de> <20080322151918.D9EBC3A40B1@sparrow.telecommunity.com> <47E525F5.40705@v.loewis.de> <20080322154449.BDA423A40A5@sparrow.telecommunity.com> Message-ID: <47E52F70.1010806@v.loewis.de> >> Sure, but what is precisely the semantics of uninstallation, in >> terms of changes to the system state? >> >> I think any model where uninstallation is merely the removal >> of files is too limited to be practical. > > The distutils only support the *addition* of files, so I'm not sure > how only removing files is a limit here. Could you explain? For files, yes, it only supports addition. But it supports arbitrary other actions, such as: - addition of registry keys - addition of user accounts - creation of database tables in a relational database - updating the shared library loader path - creation and start of a system service - integration of documentation into info - registration of DTDs with the system catalog - ... It's turing-complete, and it has full interface to the operating system, so installation of a distutils package can do *much* more than merely installing files. Uninstallation needs to revert anything installation did, so it is often more than mere removal of files. HTH, Martin From martin at v.loewis.de Sat Mar 22 17:30:45 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 22 Mar 2008 17:30:45 +0100 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <79990c6b0803220646r50e9b455y5bd9b955814f334a@mail.gmail.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <47E4DE8E.6010809@holdenweb.com> <79990c6b0803220646r50e9b455y5bd9b955814f334a@mail.gmail.com> Message-ID: <47E53435.4090104@v.loewis.de> > Oh, and application installation is (should be) completely different. > On Windows, applications should probably be bundled with their own > Python interpreter, a la py2exe. On Unix/Linux, I don't know what the > standard is, so I'd have to defer to others. This I disagree with. I think it's an overall bad thing to have all kinds of applications ship their own copy of Python; see also Aza Raskin's PyCon keynote. On Linux, python typically comes with the system pre-installed; it is not even an option not to have it, except for minimalist installations. So if you write python scripts, you typically expect that #!/usr/bin/env python works; you might put python2.5 there if you don't trust that system one is "good enough". For installing the application, you typically want the choice between a "system installation" (in /usr/bin, or perhaps /usr/local/bin), and a "user installation". As distutils supports both cases, it works fairly well for applications as well. Regards, Martin From martin at v.loewis.de Sat Mar 22 17:33:20 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 22 Mar 2008 17:33:20 +0100 Subject: [Python-Dev] Proposal: Slightly Changed Semantics for from-Import In-Reply-To: References: Message-ID: <47E534D0.9090900@v.loewis.de> > If a test case is needed I can attach one later on. You should put all this into bugs.python.org, preferably with a patch implementing it also. Regards, Martin From lists at cheimes.de Sat Mar 22 19:20:15 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 22 Mar 2008 19:20:15 +0100 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: References: <47DD545D.6070300@cheimes.de> <47DD613E.6090007@cheimes.de> <47E447A1.6070208@cheimes.de> Message-ID: <47E54DDF.50208@cheimes.de> Guido van Rossum schrieb: > Right. We've done this 2-stage rename before, during the great > (sometimes known as grand) renaming, in the 1.x days. I haven't been around during the 1.x -> 2.x days. I was still in the dark ages (aka PHP user). How do you want me to tackle down the PyString / PyBytes problem? Christian From stephen at xemacs.org Sat Mar 22 19:51:33 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 23 Mar 2008 03:51:33 +0900 Subject: [Python-Dev] Editing "public" queries in tracker [was: ... project management] In-Reply-To: <96720E99-D5A3-46B8-BBAA-88C021F24653@python.org> References: <2C2D7EC7-51F2-4F4E-8127-87ACCC6F8413@python.org> <5E86F9D7-4421-4C57-8294-B0EC6A4EE03C@python.org> <47DDF318.4080303@v.loewis.de> <96720E99-D5A3-46B8-BBAA-88C021F24653@python.org> Message-ID: <87abkqfv7u.fsf@uwakimon.sk.tsukuba.ac.jp> I think this howto is of general interest to the community, but I'm crossposting to Tracker-Discuss and redirecting discussion there. Reply-To set. Barry Warsaw writes: > Thanks Martin, I think this will work for now. Is there any way you > can allow me to edit this query too? While as Martin says only the author can edit a query, you can (in Roundup 1.4.2, which bugs.python.org may not have yet?) edit a copy of that query. (Since a query is an object, you can keep the same name for multiple queries. **But:** This is a doubleplusungood idea unless you coordinate with the first author to remove one of the versions, because there is no way to distinguish multiple queries by the same name except whether they are editable or not.) What I see in the edit interface at first is Query Include ... Edit Private ... YourQuery_ [not yours to edit] (where trailing _ indicates a link and <> a GUI widget). After setting YourQuery "leave in: yes", I see Query Include ... Edit Private ... YourQuery_ edit_ [delete] YourQuery_ [not yours to edit] ie, it looks like Roundup automatically makes a copy for you to edit. Urk ... once I've edited it, I now see Query Include ... Edit Private ... YourQuery_ edit_ [delete] YourQuery_ edit_ [delete] YourQuery_ [not yours to edit] where one of the editables is my version, and the other a copy of the original query You wrote. So even the editor is likely to find it confusing unless he renames his new version. Also, in 1.4.2 there seems to be a bug, such that my ordinary User was unable retire or remove the query. For now, you may not want to do this a lot as your sidebar will get cluttered. I wonder if it might be a good idea to have a convention to distinguish public queries that may be used as templates? From p.f.moore at gmail.com Sat Mar 22 20:56:45 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 22 Mar 2008 19:56:45 +0000 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E53435.4090104@v.loewis.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <47E4DE8E.6010809@holdenweb.com> <79990c6b0803220646r50e9b455y5bd9b955814f334a@mail.gmail.com> <47E53435.4090104@v.loewis.de> Message-ID: <79990c6b0803221256jf4e6ea1qce463ab711dcdb51@mail.gmail.com> On 22/03/2008, "Martin v. L?wis" wrote: > > Oh, and application installation is (should be) completely different. > > On Windows, applications should probably be bundled with their own > > Python interpreter, a la py2exe. On Unix/Linux, I don't know what the > > standard is, so I'd have to defer to others. > > > This I disagree with. I think it's an overall bad thing to have all > kinds of applications ship their own copy of Python; see also Aza > Raskin's PyCon keynote. Is this on Windows? It's fairly common practice. Can you give me a pointer to Aza Raskin's keynote? Is it online anywhere? I'd be interested in his point of view. Paul. From martin at v.loewis.de Sat Mar 22 21:13:24 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 22 Mar 2008 21:13:24 +0100 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <79990c6b0803221256jf4e6ea1qce463ab711dcdb51@mail.gmail.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <47E4DE8E.6010809@holdenweb.com> <79990c6b0803220646r50e9b455y5bd9b955814f334a@mail.gmail.com> <47E53435.4090104@v.loewis.de> <79990c6b0803221256jf4e6ea1qce463ab711dcdb51@mail.gmail.com> Message-ID: <47E56864.4080201@v.loewis.de> >>> Oh, and application installation is (should be) completely different. >> > On Windows, applications should probably be bundled with their own >> > Python interpreter, a la py2exe. On Unix/Linux, I don't know what the >> > standard is, so I'd have to defer to others. >> >> >> This I disagree with. I think it's an overall bad thing to have all >> kinds of applications ship their own copy of Python; see also Aza >> Raskin's PyCon keynote. > > Is this on Windows? It's fairly common practice. Unfortunately so, yes. This can be viewed a burden to the adoption of Python: for a small application, you get this huge download to bundle. > Can you give me a > pointer to Aza Raskin's keynote? Is it online anywhere? I'd be > interested in his point of view. Unfortunately no. I was looking for it, but couldn't find it. He mentioned a website with a "call for action", but I couldn't find that, either :-( Regards, Martin From guido at python.org Sat Mar 22 21:23:12 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 22 Mar 2008 13:23:12 -0700 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: <47E54DDF.50208@cheimes.de> References: <47DD545D.6070300@cheimes.de> <47DD613E.6090007@cheimes.de> <47E447A1.6070208@cheimes.de> <47E54DDF.50208@cheimes.de> Message-ID: > I haven't been around during the 1.x -> 2.x days. I was still in the > dark ages (aka PHP user). :-) > How do you want me to tackle down the PyString / PyBytes problem? I think it can actually be simplified. I think maintaining binary compatibility between 2.6 and earlier versions is hopeless anyway, so we might as well just rename PyString to PyBytes in 2.6 and 3.0, and have an extra set of macros so that code using PyString needs to be recompiled but not otherwise touched. E.g. typedef { ... } PyBytesObject; #define PyStringObject PyBytesObject ... PyString_Type; #define PyBytes_Type PyString_Type -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Sat Mar 22 21:34:33 2008 From: steven.bethard at gmail.com (Steven Bethard) Date: Sat, 22 Mar 2008 14:34:33 -0600 Subject: [Python-Dev] Making sys.py3k_warning writable Message-ID: Right now, test_py3kwarn only runs if the test suite is run using the -3 command line flag to Python. So for most regrtest runs (e.g. the buildbots) this test will never be run. It would be nice to be able to turn the flag on from Python (e.g. within test_py3kwarn). Is that possible? Here's what I tried and it didn't seem to work:: $ python_d.exe Python 2.6a1+ (trunk:61715M, Mar 21 2008, 14:33:00) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import sys; sys.py3k_warning = True; callable(int) True And here's what happens when you specify -3 at the command line: $ python_d.exe -3 Python 2.6a1+ (trunk:61715M, Mar 21 2008, 14:33:00) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> callable(int) __main__:1: DeprecationWarning: callable() not supported in 3.x. Use hasattr(o, '__call__'). True Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From musiccomposition at gmail.com Sat Mar 22 21:38:39 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sat, 22 Mar 2008 15:38:39 -0500 Subject: [Python-Dev] Making sys.py3k_warning writable In-Reply-To: References: Message-ID: <1afaf6160803221338o7cbaae18wfed45c8082de8349@mail.gmail.com> On Sat, Mar 22, 2008 at 3:34 PM, Steven Bethard wrote: > Right now, test_py3kwarn only runs if the test suite is run using the > -3 command line flag to Python. So for most regrtest runs (e.g. the > buildbots) this test will never be run. > > It would be nice to be able to turn the flag on from Python (e.g. > within test_py3kwarn). Is that possible? Here's what I tried and it > didn't seem to work:: That's because sys.py3kwarning is set on startup from Py_Py3kWarningFlag and never checked again. Either we should make it immutable or fix it. > > > $ python_d.exe > Python 2.6a1+ (trunk:61715M, Mar 21 2008, 14:33:00) [MSC v.1500 32 bit > (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import sys; sys.py3k_warning = True; callable(int) > True > > And here's what happens when you specify -3 at the command line: > > $ python_d.exe -3 > Python 2.6a1+ (trunk:61715M, Mar 21 2008, 14:33:00) [MSC v.1500 32 bit > (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> callable(int) > __main__:1: DeprecationWarning: callable() not supported in 3.x. Use > hasattr(o, '__call__'). > True > > Steve > -- > I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a > tiny blip on the distant coast of sanity. > --- Bucky Katt, Get Fuzzy > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -- Cheers, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080322/5c1c9908/attachment.htm From martin at v.loewis.de Sat Mar 22 21:57:21 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 22 Mar 2008 21:57:21 +0100 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: References: <47DD545D.6070300@cheimes.de> <47DD613E.6090007@cheimes.de> <47E447A1.6070208@cheimes.de> <47E54DDF.50208@cheimes.de> Message-ID: <47E572B1.5030202@v.loewis.de> > I think it can actually be simplified. I think maintaining binary > compatibility between 2.6 and earlier versions is hopeless anyway ABI-wise or API-wise? I would surely hope that the 2.6 API is "mostly" compatible with the 2.5 API. Regards, Martin From martin at v.loewis.de Sat Mar 22 21:58:18 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 22 Mar 2008 21:58:18 +0100 Subject: [Python-Dev] Making sys.py3k_warning writable In-Reply-To: References: Message-ID: <47E572EA.1080607@v.loewis.de> Steven Bethard wrote: > Right now, test_py3kwarn only runs if the test suite is run using the > -3 command line flag to Python. So for most regrtest runs (e.g. the > buildbots) this test will never be run. For the buildbots, it would be easy to turn -3 on, though. Regards, Martin From lists at cheimes.de Sat Mar 22 22:09:05 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 22 Mar 2008 22:09:05 +0100 Subject: [Python-Dev] 2.6 and 3.0 tasks In-Reply-To: References: Message-ID: <47E57571.5050900@cheimes.de> Guido van Rossum schrieb: >> When the new buffer protocol is available in 2.6 we can start back >> porting bytesarray and the new IO framework. > > Are these really so closely tied that they have to wait before they > can be backported? I've started with the backport of the bytearray type in a new branch svn+ssh://pythondev at svn.python.org/python/branches/trunk-bytearray Any help is appreciated. I already solved most issues. Open tasks: * fix bytearray.extend() * add PyString support to bytearray.fromhex * Add old style char buffer interface to bytearray (for codecs) * Add old style read write buffer interface to bytearray (for file.readinto) * Fix pickle and copy support for bytearray Christian From musiccomposition at gmail.com Sat Mar 22 22:04:38 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sat, 22 Mar 2008 16:04:38 -0500 Subject: [Python-Dev] Making sys.py3k_warning writable In-Reply-To: <47E572EA.1080607@v.loewis.de> References: <47E572EA.1080607@v.loewis.de> Message-ID: <1afaf6160803221404h47b710cbw7a118591f95ffbda@mail.gmail.com> On Sat, Mar 22, 2008 at 3:58 PM, "Martin v. L?wis" wrote: > Steven Bethard wrote: > > Right now, test_py3kwarn only runs if the test suite is run using the > > -3 command line flag to Python. So for most regrtest runs (e.g. the > > buildbots) this test will never be run. > > For the buildbots, it would be easy to turn -3 on, though. Should I work a patch to allow Python code to disable/enable the flag? > > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -- Cheers, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080322/cc759721/attachment.htm From martin at v.loewis.de Sat Mar 22 22:08:27 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 22 Mar 2008 22:08:27 +0100 Subject: [Python-Dev] Making sys.py3k_warning writable In-Reply-To: <1afaf6160803221404h47b710cbw7a118591f95ffbda@mail.gmail.com> References: <47E572EA.1080607@v.loewis.de> <1afaf6160803221404h47b710cbw7a118591f95ffbda@mail.gmail.com> Message-ID: <47E5754B.9060204@v.loewis.de> > For the buildbots, it would be easy to turn -3 on, though. > > Should I work a patch to allow Python code to disable/enable the flag? +0. Regards, Martin From musiccomposition at gmail.com Sat Mar 22 22:29:48 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sat, 22 Mar 2008 16:29:48 -0500 Subject: [Python-Dev] Making sys.py3k_warning writable In-Reply-To: <47E5754B.9060204@v.loewis.de> References: <47E572EA.1080607@v.loewis.de> <1afaf6160803221404h47b710cbw7a118591f95ffbda@mail.gmail.com> <47E5754B.9060204@v.loewis.de> Message-ID: <1afaf6160803221429h9a66656ucac3bb5d9a8ca033@mail.gmail.com> On Sat, Mar 22, 2008 at 4:08 PM, "Martin v. L?wis" wrote: > > For the buildbots, it would be easy to turn -3 on, though. > > > > Should I work a patch to allow Python code to disable/enable the flag? > > +0. See issue 2458. > > > Regards, > Martin > -- Cheers, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080322/17db5114/attachment.htm From msully4321 at gmail.com Sat Mar 22 22:50:42 2008 From: msully4321 at gmail.com (Mike Sullivan) Date: Sat, 22 Mar 2008 17:50:42 -0400 Subject: [Python-Dev] State of N-dimensional array interface Message-ID: What is the current state of the N-D Array Interface proposal (http://numpy.scipy.org/array_interface.shtml). Some work was done on this in an earlier Summer of Code, but it seems to never have been included. Does anybody know what state that work is in and what prevented it's inclusion? (I am interested in completing this as a Summer of Code project. I posted about this on the SOC list, and received a suggestion to try asking python-dev.) -- Mike Sullivan From solipsis at pitrou.net Sat Mar 22 23:43:31 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 22 Mar 2008 22:43:31 +0000 (UTC) Subject: [Python-Dev] Two questions about jump opcodes Message-ID: Hi, I'm attempting some bytecode tweaks (see http://bugs.python.org/issue2459) and I've got two questions about jump instructions: - Why are there both relative and absolute jump instructions? The traditional rationale for relative jumps (apart from position-independent code) is to allow for shorter operand sizes; but Python opcodes all have the same operand size (and 16 bits is more than enough to address most bytecode arrays). - Why are relative jumps unsigned? This means they can only jump forward, and as soon as you want to jump backward you have to switch to an absolute jump... (in that regard, I don't understand what JUMP_FORWARD can possibly bring over JUMP_ABSOLUTE) Thanks for your kind answers Antoine. From skip at pobox.com Sat Mar 22 23:58:29 2008 From: skip at pobox.com (skip at pobox.com) Date: Sat, 22 Mar 2008 17:58:29 -0500 Subject: [Python-Dev] Two questions about jump opcodes In-Reply-To: References: Message-ID: <18405.36629.870819.376043@montanaro-dyndns-org.local> Antoine> - Why are there both relative and absolute jump instructions? The best place to search for the reasons behind this is Python/compile.c. (JUMP_ABSOLUTE can jump backwards.) There have been lots and lots of changes to the Python virtual machine the past few years. When trying to understand the basic concepts it might be best to check out a very old version of the code from Subversion (1.5.2, 2.0, 2.1, etc). Those versions have many fewer optimizations, so it's likely that compile.c and ceval.c will be more readable. (My full understanding of the virtual machine probably ended with 1.5.2.) That should give you a basic understanding of how things work without the obfuscation added by the many optimizations. You can then move to more recent versions and more easily see what's going on, even in the face of all the optimizations. Antoine> (in that regard, I don't understand what JUMP_FORWARD can Antoine> possibly bring over JUMP_ABSOLUTE) Well, you can't chain jumps together with the latter. If, for some reason, you needed to jump forward more than 16kbytes you could accomplish that with multiple JUMP_FORWARD opcodes. Skip From pje at telecommunity.com Sat Mar 22 23:57:23 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 22 Mar 2008 18:57:23 -0400 Subject: [Python-Dev] Two questions about jump opcodes In-Reply-To: References: Message-ID: <20080322225842.1E5C63A40D9@sparrow.telecommunity.com> At 10:43 PM 3/22/2008 +0000, Antoine Pitrou wrote: >- Why are there both relative and absolute jump instructions? The traditional >rationale for relative jumps (apart from position-independent code) >is to allow >for shorter operand sizes; but Python opcodes all have the same operand size Actually they don't. They can have 32-bit arguments, with the EXTENDED_ARG opcode. EXTENDED_ARG loads the high 16 bits of the argument in the opcode that immediately follows. >(and 16 bits is more than enough to address most bytecode arrays). Ah, but not *all* bytecode arrays. Apparently some (automatically generated) code at LucasFilm (if memory serves) exceeded some of the 16-bit limits for bytecode, so the EXTENDED_ARG opcode was added to fix this. >- Why are relative jumps unsigned? This means they can only jump >forward, and as >soon as you want to jump backward you have to switch to an absolute jump... With a backward jump, you already know the exact offset, so you know if you need a 16-bit or 32-bit operand. >(in that regard, I don't understand what JUMP_FORWARD can possibly bring over >JUMP_ABSOLUTE) It means you don't have to guess whether your jump target is going to cross the 64K boundary, thereby requiring you to have used a 32-bit operand. Of course, it does limit your forward jumping to skipping no more than a 64K block, but apparently nobody has exceeded that limit yet. :) Merely having 64K of total bytecode is presumably an easier limit to reach than *jumping over* 64K worth of bytecode. :) In truth, I don't know if that's really the reason why things were originally set up this way, but these are certainly among the reasons thing will probably stay this way. :) From solipsis at pitrou.net Sun Mar 23 00:07:25 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 22 Mar 2008 23:07:25 +0000 (UTC) Subject: [Python-Dev] Two questions about jump opcodes References: <20080322225842.1E5C63A40D9@sparrow.telecommunity.com> Message-ID: Wow, thanks to both of you (Phillip & Skip) for the fast answers. Just in case anyone has time to spare, I have more pesky questions (and a working patch :-)) in the aforementioned bug entry (http://bugs.python.org/issue2459). Regards Antoine. From regebro at gmail.com Sun Mar 23 08:37:10 2008 From: regebro at gmail.com (Lennart Regebro) Date: Sun, 23 Mar 2008 08:37:10 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals Message-ID: <319e029f0803230037s76af8233hbefbfacccf77b930@mail.gmail.com> Eric Smith wrote: > It's not implementable because the work has to occur in ast.c (see > Py_UnicodeFlag). It can't occur later, because you need to skip the > encoding being done in parsestr(). But the __future__ import can only > be interpreted after the AST is built, at which time the encoding has > already been applied. There are some radical things you could do to > work around this, but it would be a gigantic change. Oh, ! > Pretty much. And even if it were possible, I don't see the point in > doing it. The point is this: http://regebro.wordpress.com/2008/03/22/why-python-26-and-30-compatibility-would-be-a-very-good-thing/ Basically, the 2to3 strategy on large codebases supported by a wide set of developers isn't likely to be maintanable, and without a gradual path forward, 3.0 isn't likely to be adopted in some parts of the Python community. Worst case this splits the efforts of the community into two groups, which would be damaging to Python as a whole. Howver: If 2.6 doesn't support this it's not the end of the world. Because if it turn out to be necessary, a 2.7 that does could always be released. I don't know about other large codebases like Twisted, but the Zope/Plone world isn't likely to even try moving to 3.0 any time soon after it's final release, so since it's complicated you can probably wait with this support until it turns out to be needed. M.-A. Lemburg wrote: > Could we point them to a special byte-code compiler such as Andrew > Dalke's python4ply: ??? I'm not sure what this means... :) -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From ziade.tarek at gmail.com Sun Mar 23 12:28:03 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 23 Mar 2008 12:28:03 +0100 Subject: [Python-Dev] Distutils patches review request Message-ID: <94bdd2610803230428i4702255fpf6f3f50030dcd06a@mail.gmail.com> Hi, I would like to make request for a review of three patches: #2385: fixes a run_setup bug, and adds a test_core module, that can be used later to improve core.py coverage (http://bugs.python.org/issue2385) #2461: adds a test_util.py to distutils, to cover util.py. It does not cover byte_compile yet, but others are covered. (http://bugs.python.org/issue2461) #1858 (http://bugs.python.org/issue1858): - makes .pypirc support multiple servers (see http://wiki.python.org/moin/EnhancedPyPI) - superseeds #1741 (see http://bugs.python.org/issue1741) - superseeds #2166 ( see http://bugs.python.org/issue2166) - adds unit tests for upload and register (test_upload, test_register) Best regards Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080323/b57b49cb/attachment.htm From thomas at python.org Sun Mar 23 14:13:41 2008 From: thomas at python.org (Thomas Wouters) Date: Sun, 23 Mar 2008 14:13:41 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <319e029f0803230037s76af8233hbefbfacccf77b930@mail.gmail.com> References: <319e029f0803230037s76af8233hbefbfacccf77b930@mail.gmail.com> Message-ID: <9e804ac0803230613k38683ee6m41e7a0aa18275d7d@mail.gmail.com> On Sun, Mar 23, 2008 at 8:37 AM, Lennart Regebro wrote: > Eric Smith wrote: > > It's not implementable because the work has to occur in ast.c (see > > Py_UnicodeFlag). It can't occur later, because you need to skip the > > encoding being done in parsestr(). But the __future__ import can only > > be interpreted after the AST is built, at which time the encoding has > > already been applied. There are some radical things you could do to > > work around this, but it would be a gigantic change. > > Oh, ! > I don't believe this to be true (we can simply store the source encoding information (which it might already be) and translate the *use* of string literals into unicode(literal, encoding).) But I still think this is a bad idea. Using the same codebase for 3.0 and 2.6 will leave you with inefficient and fragile code in one of the two codebases -- quite likely both. I know, I've tried. I don't see how it improves maintainability to leave your source full of forward- and backward-compatibility hacks. 3.0 was meant to be a clean break. Please treat it as such. Yes, that means you can't run the same source tree in two different Python versions -- too bad. It means adding a compilation-style step to one of the two Python versions -- too bad. It's quite easily automated, as proven by the vast majority of programming languages out there, which need such a step anyway. > > Howver: If 2.6 doesn't support this it's not the end of the world. > Because if it turn out to be necessary, a 2.7 that does could always > be released. I don't know about other large codebases like Twisted, > but the Zope/Plone world isn't likely to even try moving to 3.0 any > time soon after it's final release, so since it's complicated you can > probably wait with this support until it turns out to be needed. > That was always the assumption, and also the idea for 2.6 and 2.7. I would much rather 3.0 isn't widely accepted for a few years longer than that it be cluttered with backward compatibility crap, and even rather than that widely used code be riddled with such. But it's up to Guido in the end. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080323/46de93c9/attachment.htm From ncoghlan at gmail.com Sun Mar 23 15:56:03 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 24 Mar 2008 00:56:03 +1000 Subject: [Python-Dev] [Python-checkins] r61796 - in python/trunk: Doc/library/random.rst Lib/random.py Lib/test/test_random.py Misc/NEWS In-Reply-To: <20080323133232.D7E9C1E401D@bag.python.org> References: <20080323133232.D7E9C1E401D@bag.python.org> Message-ID: <47E66F83.8000904@gmail.com> > +## -------------------- triangular -------------------- > + > + def triangular(self, low, high, mode): > + """Triangular distribution. > + > + Continuous distribution bounded by given lower and upper limits, > + and having a given mode value in-between. > + > + http://en.wikipedia.org/wiki/Triangular_distribution > + > + """ > + u = self.random() > + c = (mode - low) / (high - low) > + if u > c: > + u = 1 - u > + c = 1 - c > + low, high = high, low > + return low + (high - low) * (u * c) ** 0.5 > + Would it be worth having default values on the parameters for this? def triangular(self, low=0, high=1, mode=None): u = self.random() if mode is None: c = 0.5 else: c = (mode - low) / (high - low) if u > c: u = 1 - u c = 1 - c low, high = high, low return low + (high - low) * (u * c) ** 0.5 At the very least, supporting mode=None would be convenient when you want a symmetric triangular distribution - having to calculate the median, pass it in as the mode, then have the function reverse that to get 0.5 seems a little wasteful. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From amk at amk.ca Sun Mar 23 18:08:49 2008 From: amk at amk.ca (A.M. Kuchling) Date: Sun, 23 Mar 2008 13:08:49 -0400 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E56864.4080201@v.loewis.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <47E4DE8E.6010809@holdenweb.com> <79990c6b0803220646r50e9b455y5bd9b955814f334a@mail.gmail.com> <47E53435.4090104@v.loewis.de> <79990c6b0803221256jf4e6ea1qce463ab711dcdb51@mail.gmail.com> <47E56864.4080201@v.loewis.de> Message-ID: <20080323170849.GA938@amk.dc.dc.cox.net> On Sat, Mar 22, 2008 at 09:13:24PM +0100, "Martin v. L?wis" wrote: > Unfortunately no. I was looking for it, but couldn't find it. He > mentioned a website with a "call for action", but I couldn't find > that, either :-( We're working on the video, but I think it'll be a few weeks before things start to get posted. --amk From nnorwitz at gmail.com Sun Mar 23 19:16:42 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 23 Mar 2008 11:16:42 -0700 Subject: [Python-Dev] State of N-dimensional array interface In-Reply-To: References: Message-ID: Hi Mike. Travis is the best person to discuss the status of the buffer APIs. Cheers, n On Sat, Mar 22, 2008 at 2:50 PM, Mike Sullivan wrote: > What is the current state of the N-D Array Interface proposal > (http://numpy.scipy.org/array_interface.shtml). Some work was done on > this in an earlier Summer of Code, but it seems to never have been > included. Does anybody know what state that work is in and what > prevented it's inclusion? > > (I am interested in completing this as a Summer of Code project. I > posted about this on the SOC list, and received a suggestion to try > asking python-dev.) > > -- > Mike Sullivan > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > From ajaksu at gmail.com Sun Mar 23 20:29:53 2008 From: ajaksu at gmail.com (ajaksu) Date: Sun, 23 Mar 2008 12:29:53 -0700 (PDT) Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E56864.4080201@v.loewis.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <47E4DE8E.6010809@holdenweb.com> <79990c6b0803220646r50e9b455y5bd9b955814f334a@mail.gmail.com> <47E53435.4090104@v.loewis.de> <79990c6b0803221256jf4e6ea1qce463ab711dcdb51@mail.gmail.com> <47E56864.4080201@v.loewis.de> Message-ID: <77a20399-7ec7-4b8e-b230-2dce8f91c597@t54g2000hsg.googlegroups.com> On Mar 22, 5:13?pm, "Martin v. L?wis" wrote: > Unfortunately no. I was looking for it, but couldn't find it. He > mentioned a website with a "call for action", but I couldn't find > that, either :-( As I tried to reply (in a message that is waiting for moderation), the site seems to be http://www.toolness.com/wp/?p=23 From martin at v.loewis.de Sun Mar 23 21:10:02 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 23 Mar 2008 21:10:02 +0100 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080323170849.GA938@amk.dc.dc.cox.net> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <47E4DE8E.6010809@holdenweb.com> <79990c6b0803220646r50e9b455y5bd9b955814f334a@mail.gmail.com> <47E53435.4090104@v.loewis.de> <79990c6b0803221256jf4e6ea1qce463ab711dcdb51@mail.gmail.com> <47E56864.4080201@v.loewis.de> <20080323170849.GA938@amk.dc.dc.cox.net> Message-ID: <47E6B91A.5070606@v.loewis.de> A.M. Kuchling wrote: > On Sat, Mar 22, 2008 at 09:13:24PM +0100, "Martin v. L?wis" wrote: >> Unfortunately no. I was looking for it, but couldn't find it. He >> mentioned a website with a "call for action", but I couldn't find >> that, either :-( > > We're working on the video, but I think it'll be a few weeks before > things start to get posted. Could you ask him whether he can put his slides online then, somewhere? Same for the other speakers, I guess. Regards, Martin From regebro at gmail.com Sun Mar 23 21:11:16 2008 From: regebro at gmail.com (Lennart Regebro) Date: Sun, 23 Mar 2008 21:11:16 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <9e804ac0803230613k38683ee6m41e7a0aa18275d7d@mail.gmail.com> References: <319e029f0803230037s76af8233hbefbfacccf77b930@mail.gmail.com> <9e804ac0803230613k38683ee6m41e7a0aa18275d7d@mail.gmail.com> Message-ID: <319e029f0803231311r4bd6031ftfc918ec0a785d4d7@mail.gmail.com> On Sun, Mar 23, 2008 at 2:13 PM, Thomas Wouters wrote: > That was always the assumption, and also the idea for 2.6 and 2.7. I would > much rather 3.0 isn't widely accepted for a few years longer than that it be > cluttered with backward compatibility crap, and even rather than that widely > used code be riddled with such. But it's up to Guido in the end. Sure but this is a bit misleading. The risk isn't that 3.0 is not widely accepted quickly. The risk is that the community splits in two. And the suggestion is not for backwards compatibility in 3.0, but forwards compatibility in 2.6. So the question is rather: Do you want to disk a community split, or add forwards compatibility? But as I noted, if it turns out to be necessary to add that forwards compatibility, it's always possible to do a 2.7 that has it. I have loads of experience in writing code for evolving APIs, this is how things have been done in the Zope community for years. It's not a problem. It does not lead to fragile code. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From martin at v.loewis.de Sun Mar 23 21:11:27 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 23 Mar 2008 21:11:27 +0100 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <77a20399-7ec7-4b8e-b230-2dce8f91c597@t54g2000hsg.googlegroups.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <47E4DE8E.6010809@holdenweb.com> <79990c6b0803220646r50e9b455y5bd9b955814f334a@mail.gmail.com> <47E53435.4090104@v.loewis.de> <79990c6b0803221256jf4e6ea1qce463ab711dcdb51@mail.gmail.com> <47E56864.4080201@v.loewis.de> <77a20399-7ec7-4b8e-b230-2dce8f91c597@t54g2000hsg.googlegroups.com> Message-ID: <47E6B96F.9000403@v.loewis.de> ajaksu wrote: > On Mar 22, 5:13 pm, "Martin v. L?wis" wrote: >> Unfortunately no. I was looking for it, but couldn't find it. He >> mentioned a website with a "call for action", but I couldn't find >> that, either :-( > > As I tried to reply (in a message that is waiting for moderation), the > site seems to be http://www.toolness.com/wp/?p=23 Thanks! That's the one. (hopefully, this one gets through the moderation) Regards, Martin From steven.bethard at gmail.com Sun Mar 23 21:26:23 2008 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 23 Mar 2008 14:26:23 -0600 Subject: [Python-Dev] Making sys.py3k_warning writable In-Reply-To: <47E572EA.1080607@v.loewis.de> References: <47E572EA.1080607@v.loewis.de> Message-ID: On Sat, Mar 22, 2008 at 2:58 PM, "Martin v. L?wis" wrote: > Steven Bethard wrote: > > Right now, test_py3kwarn only runs if the test suite is run using the > > -3 command line flag to Python. So for most regrtest runs (e.g. the > > buildbots) this test will never be run. > > For the buildbots, it would be easy to turn -3 on, though. Right now, running the test suite with -3 spews a load of warnings since there is a lot of code that hasn't been updated for even simple things like ``dict.has_key``. Would turning the -3 flag on make it too hard to diagnose problems? If people don't think so, I'm happy to have it turned on. I did find a minor bug in the test suite when I turned it on and ran the full regrtest. On Sat, Mar 22, 2008 at 3:29 PM, Benjamin Peterson wrote: [http://bugs.python.org/issue2458] Thanks for putting that together! Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From martin at v.loewis.de Sun Mar 23 22:02:51 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 23 Mar 2008 22:02:51 +0100 Subject: [Python-Dev] Making sys.py3k_warning writable In-Reply-To: References: <47E572EA.1080607@v.loewis.de> Message-ID: <47E6C57B.6000206@v.loewis.de> > Would turning the -3 flag on make it too hard to diagnose problems? Yes. I don't think it should be turned on regularly in the tests until they regularly are quiet under -3. Regards, Martin From lists at cheimes.de Sun Mar 23 22:19:02 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 23 Mar 2008 22:19:02 +0100 Subject: [Python-Dev] Backport of bytearray type and io module Message-ID: <47E6C946.5020302@cheimes.de> Hello! I've successfully back ported the bytearray type and the io module from 3.0 to 2.6. The code is available in the branch trunk-bytearray [1]. I'm down to four failing byte tests and one failing io test. The failing byte tests are all related to pickling and subclassing. I like to get the remaining tests fixed for the upcoming release in a week. Please checkout the branch and help me figure out the remaining issues. Christian [1] svn+ssh://pythondev at svn.python.org/python/branches/trunk-bytearray From steven.bethard at gmail.com Sun Mar 23 22:41:24 2008 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 23 Mar 2008 15:41:24 -0600 Subject: [Python-Dev] Fixing code that produces -3 warnings in the Python 2.6 stdlib Message-ID: On Sun, Mar 23, 2008 at 3:02 PM, "Martin v. L?wis" wrote: [talking about running the buildbots using the -3 flag to issue Py3K warnings] > Yes. I don't think it should be turned on regularly in the tests until > they regularly are quiet under -3. So the plan is to silence all Py3K warnings in the Python 2.6 stdlib? I'd like to see this happen, but I vaguely remembered some people being against this idea, so I'd like to make sure. For what it's worth, I started an issue for this earlier: http://bugs.python.org/issue2402 Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From martin at v.loewis.de Sun Mar 23 22:45:09 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 23 Mar 2008 22:45:09 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <319e029f0803231311r4bd6031ftfc918ec0a785d4d7@mail.gmail.com> References: <319e029f0803230037s76af8233hbefbfacccf77b930@mail.gmail.com> <9e804ac0803230613k38683ee6m41e7a0aa18275d7d@mail.gmail.com> <319e029f0803231311r4bd6031ftfc918ec0a785d4d7@mail.gmail.com> Message-ID: <47E6CF65.1020900@v.loewis.de> > So the question is rather: Do you want to disk a community split, or > add forwards compatibility? I don't think the risk is big. As far as people start saying "I will only support Python 3", or saying "I will not support Python 3" - that's fine. In the former case, people can still continue to use the old versions of the software (assuming we are talking about open source here), and continue to use those with 2.x. They won't get all the new features, and perhaps that is a reason for them to move to 3.x. In the latter case, people relying on the library either have to stay with 2.x until all their dependencies get ported, or they will have to contribute 3.x ports themselves to the developers. In some cases, this may cause a fork of the project, but I guess these cases are rare (and occur only if the maintainer is not cooperative in at least incorporating patches even if its for stuff he doesn't care about). So in short: no, the risk that the community splits is very small. When people contribute code, it's not because of care about the community, but because of their own needs. That's how open-source software works. Regards, Martin From martin at v.loewis.de Sun Mar 23 22:47:49 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 23 Mar 2008 22:47:49 +0100 Subject: [Python-Dev] Fixing code that produces -3 warnings in the Python 2.6 stdlib In-Reply-To: References: Message-ID: <47E6D005.5080107@v.loewis.de> Steven Bethard wrote: > On Sun, Mar 23, 2008 at 3:02 PM, "Martin v. L?wis" wrote: > [talking about running the buildbots using the -3 flag to issue Py3K warnings] >> Yes. I don't think it should be turned on regularly in the tests until >> they regularly are quiet under -3. > > So the plan is to silence all Py3K warnings in the Python 2.6 stdlib? I don't know; I don't have this plan. I'm only saying that it shouldn't be turned on in the test suite if it triggers a lot of noise. Regards, Martin From skip at pobox.com Sun Mar 23 23:01:24 2008 From: skip at pobox.com (skip at pobox.com) Date: Sun, 23 Mar 2008 17:01:24 -0500 Subject: [Python-Dev] Making sys.py3k_warning writable In-Reply-To: <47E6C57B.6000206@v.loewis.de> References: <47E572EA.1080607@v.loewis.de> <47E6C57B.6000206@v.loewis.de> Message-ID: <18406.54068.274352.742302@montanaro-dyndns-org.local> >> Would turning the -3 flag on make it too hard to diagnose problems? Martin> Yes. I don't think it should be turned on regularly in the tests Martin> until they regularly are quiet under -3. Maybe regrtest should gain a -3 flag. All it would have to do is restart itself moving the -3 from after regrtest to before it. Skip From skip at pobox.com Sun Mar 23 23:04:55 2008 From: skip at pobox.com (skip at pobox.com) Date: Sun, 23 Mar 2008 17:04:55 -0500 Subject: [Python-Dev] Making sys.py3k_warning writable In-Reply-To: <18406.54068.274352.742302@montanaro-dyndns-org.local> References: <47E572EA.1080607@v.loewis.de> <47E6C57B.6000206@v.loewis.de> <18406.54068.274352.742302@montanaro-dyndns-org.local> Message-ID: <18406.54279.752968.226187@montanaro-dyndns-org.local> skip> Maybe regrtest should gain a -3 flag. All it would have to do is skip> restart itself moving the -3 from after regrtest to before it. Ehh... That didn't seem like such a great idea about 10 seconds after I hit send. Adding a test3 target to Makefile.pre.in is a lot easier. Skip From regebro at gmail.com Sun Mar 23 23:17:05 2008 From: regebro at gmail.com (Lennart Regebro) Date: Sun, 23 Mar 2008 23:17:05 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <47E6CF65.1020900@v.loewis.de> References: <319e029f0803230037s76af8233hbefbfacccf77b930@mail.gmail.com> <9e804ac0803230613k38683ee6m41e7a0aa18275d7d@mail.gmail.com> <319e029f0803231311r4bd6031ftfc918ec0a785d4d7@mail.gmail.com> <47E6CF65.1020900@v.loewis.de> Message-ID: <319e029f0803231517t259b84d5sa3c2071873eb6399@mail.gmail.com> On Sun, Mar 23, 2008 at 10:45 PM, "Martin v. L?wis" wrote: > > So the question is rather: Do you want to disk a community split, or > > add forwards compatibility? > > I don't think the risk is big. As far as people start saying "I will > only support Python 3", or saying "I will not support Python 3" - that's > fine. No, it isn't. That's the whole thing. It isn't fine. > In the latter case, people relying on the library either have to stay > with 2.x until all their dependencies get ported, or they will have > to contribute 3.x ports themselves to the developers. You are still only seeing this as a case of libraries with a small number of people developing them and making regular well defined releases. That is not how the world I am talking about looks. I repeat: I have no doubt the 2to3 approach works well in that case, if you want to support both 2.6 and 3.0. > So in short: no, the risk that the community splits is very small. No, it is a significant risk. Don't brush it away. We do NOT end up having a 2.x python world and a 3.x python world. The community doesn't have the resources to maintain momentum in a language if the energy is divided in half. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From martin at v.loewis.de Mon Mar 24 00:14:13 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 24 Mar 2008 00:14:13 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <319e029f0803231517t259b84d5sa3c2071873eb6399@mail.gmail.com> References: <319e029f0803230037s76af8233hbefbfacccf77b930@mail.gmail.com> <9e804ac0803230613k38683ee6m41e7a0aa18275d7d@mail.gmail.com> <319e029f0803231311r4bd6031ftfc918ec0a785d4d7@mail.gmail.com> <47E6CF65.1020900@v.loewis.de> <319e029f0803231517t259b84d5sa3c2071873eb6399@mail.gmail.com> Message-ID: <47E6E445.60305@v.loewis.de> > You are still only seeing this as a case of libraries with a small > number of people developing them and making regular well defined > releases. That is not how the world I am talking about looks. Can you give me examples of such software? Are you perhaps talking about closed source software? Regards, Martin From skip at pobox.com Mon Mar 24 00:21:07 2008 From: skip at pobox.com (skip at pobox.com) Date: Sun, 23 Mar 2008 18:21:07 -0500 Subject: [Python-Dev] Making sys.py3k_warning writable In-Reply-To: <18406.54279.752968.226187@montanaro-dyndns-org.local> References: <47E572EA.1080607@v.loewis.de> <47E6C57B.6000206@v.loewis.de> <18406.54068.274352.742302@montanaro-dyndns-org.local> <18406.54279.752968.226187@montanaro-dyndns-org.local> Message-ID: <18406.58851.394949.727362@montanaro-dyndns-org.local> So, regarding -3 warnings in the 2.6 test code, should they be fixed or not? Skip From musiccomposition at gmail.com Mon Mar 24 00:35:57 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sun, 23 Mar 2008 18:35:57 -0500 Subject: [Python-Dev] Making sys.py3k_warning writable In-Reply-To: <18406.58851.394949.727362@montanaro-dyndns-org.local> References: <47E572EA.1080607@v.loewis.de> <47E6C57B.6000206@v.loewis.de> <18406.54068.274352.742302@montanaro-dyndns-org.local> <18406.54279.752968.226187@montanaro-dyndns-org.local> <18406.58851.394949.727362@montanaro-dyndns-org.local> Message-ID: <1afaf6160803231635v28458814q1d91c13ee83bf432@mail.gmail.com> On Sun, Mar 23, 2008 at 6:21 PM, wrote: > > So, regarding -3 warnings in the 2.6 test code, should they be fixed or > not? I think we should introduce a decorator and/or a context manager in test_support like this: def silence_py3k(func): def decorator(*args, **kwargs): sys.disablepy3kwarning() func(*args, **kwargs) sys.enablepy3kwarning() return decorator Of course, this depends on my patch in issue 2458. I think we could also do it with warnings.simplefilter. > > > Skip > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -- Cheers, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080323/e3f331a4/attachment.htm From jimjjewett at gmail.com Mon Mar 24 01:52:46 2008 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 23 Mar 2008 20:52:46 -0400 Subject: [Python-Dev] [Python-checkins] r61709 - python/trunk/Doc/library/functions.rst python/trunk/Doc/library/future_builtins.rst python/trunk/Doc/library/python.rst In-Reply-To: <20080321193757.B2DD71E400B@bag.python.org> References: <20080321193757.B2DD71E400B@bag.python.org> Message-ID: What is the precise specification of the builtin "print" function. Does it call "str", or does it just behave as if the builtin str had been called? In 2.5, the print statement ignores any overrides of the str builtin, but I'm not sure whether a _function_ should -- and I do think it should be specified. -jJ On 3/21/08, georg.brandl wrote: > New Revision: 61709 ============================================================================== > +++ python/trunk/Doc/library/functions.rst Fri Mar 21 20:37:57 2008 > @@ -817,6 +817,33 @@ ... > +.. function:: print([object, ...][, sep=' '][, end='\n'][, file=sys.stdout]) ... > + All non-keyword arguments are converted to strings like :func:`str` does From exarkun at divmod.com Mon Mar 24 02:30:51 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Sun, 23 Mar 2008 20:30:51 -0500 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <47E6E445.60305@v.loewis.de> Message-ID: <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> On Mon, 24 Mar 2008 00:14:13 +0100, "\"Martin v. L?wis\"" wrote: >> You are still only seeing this as a case of libraries with a small >> number of people developing them and making regular well defined >> releases. That is not how the world I am talking about looks. > >Can you give me examples of such software? Are you perhaps talking >about closed source software? I'm not sure what software he was talking about. I can say that for the work I do on both Twisted and Divmod software, I'd be quite happy to see this feature. As either part of a migration path towards 3k _or_ as a feature entirely on its own merits, this would be very useful to me. I'm a bit curious about why Thomas said this sort of thing results in fragile code. Twisted has been using __future__ imports for years and they've never been a problem. Twisted currently supports Python 2.3 through Python 2.5, and the only thing that's really difficult about that is subtle changes in library behavior, not syntax. I'm also curious about why Lennart thinks that this would only be relevant for large projects with lots of developers making regular releases. Sure, I'd use it in that scenario, but that's because it's a subset of "all development". ;) Jean-Paul From jimjjewett at gmail.com Mon Mar 24 03:16:55 2008 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 23 Mar 2008 22:16:55 -0400 Subject: [Python-Dev] windows standard [was: PEP 365 (Adding the pkg_resources module)] Message-ID: Terry Reedy > The standard (and to me, preferable) way of dealing > with such things is to have an 'installation manager' > that can reinstall as well as delete and that has a > check box for various things to delete. This is what > Python needs. Paul Moore: > I'd dispute strongly that this is a "standard". > It may be preferable, but I'm not sure where you > see evidence of it being a standard. When I install a large program (such as developer tools, or python itself) on Windows, I expect a choice of "default" or "custom". When I choose custom, I expect a list of components, which can be chosen, not chosen, or mixed (meaning that it has subcomponents, only some of which are chosen). The whole thing only shows up once in Add/Remove programs. If I select it, I do get options to Change or Repair. These let me change my mind on which subcomponents are installed. If I install python and then separately install Zope, it may or may not make sense for Zope to be listed separately as a "program" to Add or Remove. It does not make sense (to me anyhow) have several individual packages within Zope each listed independently at the Windows level. (Though, to be fair, many (non-python) applications *do* make more than one entry.) -jJ From tjreedy at udel.edu Mon Mar 24 08:06:59 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 24 Mar 2008 03:06:59 -0400 Subject: [Python-Dev] windows standard [was: PEP 365 (Adding thepkg_resources module)] References: Message-ID: "Jim Jewett" wrote in message news:fb6fbf560803231916p1c03cf24yc05104f1cf46c762 at mail.gmail.com... | Terry Reedy | > The standard (and to me, preferable) way of dealing | > with such things is to have an 'installation manager' | > that can reinstall as well as delete and that has a | > check box for various things to delete. This is what | > Python needs. 'such things' referred to 'add-ons' as discussed in snipped text both mine and Paul's. | Paul Moore: | > I'd dispute strongly that this is a "standard". | > It may be preferable, but I'm not sure where you | > see evidence of it being a standard. | | When I install a large program (such as developer tools, or python | itself) on Windows, I expect a choice of "default" or "custom". When | I choose custom, I expect a list of components, which can be chosen, | not chosen, or mixed (meaning that it has subcomponents, only some of | which are chosen). | | The whole thing only shows up once in Add/Remove programs. If I | select it, I do get options to Change or Repair. These let me change | my mind on which subcomponents are installed. This is what I am requesting for Python. | If I install python and then separately install Zope, it may or may | not make sense for Zope to be listed separately as a "program" to Add | or Remove. Neither Paul nor I defined 'add-on', but I would be willing to call Zope/Plone something more than that, preferably with its own multi-option entry. tjr From regebro at gmail.com Mon Mar 24 09:10:21 2008 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 24 Mar 2008 09:10:21 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <47E6E445.60305@v.loewis.de> References: <319e029f0803230037s76af8233hbefbfacccf77b930@mail.gmail.com> <9e804ac0803230613k38683ee6m41e7a0aa18275d7d@mail.gmail.com> <319e029f0803231311r4bd6031ftfc918ec0a785d4d7@mail.gmail.com> <47E6CF65.1020900@v.loewis.de> <319e029f0803231517t259b84d5sa3c2071873eb6399@mail.gmail.com> <47E6E445.60305@v.loewis.de> Message-ID: <319e029f0803240110mf3a72d3hc8eb5ea3982ca7ed@mail.gmail.com> On Mon, Mar 24, 2008 at 12:14 AM, "Martin v. L?wis" wrote: > > You are still only seeing this as a case of libraries with a small > > number of people developing them and making regular well defined > > releases. That is not how the world I am talking about looks. > > Can you give me examples of such software? I'll repeat the link where I explained my point on this: http://regebro.wordpress.com/2008/03/22/why-python-26-and-30-compatibility-would-be-a-very-good-thing/ -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From regebro at gmail.com Mon Mar 24 09:22:47 2008 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 24 Mar 2008 09:22:47 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> Message-ID: <319e029f0803240122n1c40a47cr4e10e0c74fdaccca@mail.gmail.com> On Mon, Mar 24, 2008 at 2:30 AM, Jean-Paul Calderone wrote: > I'm also curious about why Lennart thinks that this would only be relevant > for large projects with lots of developers making regular releases. No, you misunderstood, or I miswrote. I think 2to3 is a procedure that will work well for library type projects with a reasonably small set of developers that make regular releases. There you can release both a python 2 and a python 3 version of the module, for example. I think more future compatibility is relevant for everybody, but I think it's really *necessary* for large projects with lots of developers that do NOT make regular releases, but where releases are done when the project as a whole is done. I.e, the developers themselves work on a project, and use trunk of most modules. Many modules are updated in parallell, and developed on in parallell, and (if the project is reasonably well-managed) releases are made when new releases are sent to the customer. Nuxeo CPS worked like this, but we can ignore them as that project is all but dead will never move to Python 3 in any case. Zope/CMF/Plone works like this. The Plone collective works like this, and it is *not* reasonably well managed, so there software quite often doens't get released, but people run against trunk. I know Django people both strive to support multiple versions of Python and regularily run their production sites on trunk. Insane, perhaps, but that's how things are. :) This is often how big in-house projects work as well, although I don't know of any of those in Python, reasonably because I'm an independent contractor in open source. :) So, in short: Large projects with interconnected modules where the developers and users of module are the same people will have big difficulties with the 2to3 approach and would be the people who are most likely to not be able to in practice go forward to Python 3 unless they have some sort of smooth path forward. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From martin at v.loewis.de Mon Mar 24 11:27:50 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 24 Mar 2008 11:27:50 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <319e029f0803240122n1c40a47cr4e10e0c74fdaccca@mail.gmail.com> References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> <319e029f0803240122n1c40a47cr4e10e0c74fdaccca@mail.gmail.com> Message-ID: <47E78226.1070607@v.loewis.de> > Nuxeo CPS worked like this, but we can ignore them as that project is > all but dead will never move to Python 3 in any case. Zope/CMF/Plone > works like this. I don't understand. AFAICT, Zope *is* a library, i.e. you have to run setup.py for lots of packages. Do you not have to run setup.py, for, say, zope.interface, or zope.psycopgda? It's fine that all people develop on the same subversion repository. Doing so integrates nicely with 2to3. > The Plone collective works like this, and it is *not* > reasonably well managed, so there software quite often doens't get > released, but people run against trunk. And that's fine. You still can integrate 2to3 with that transparently. > I know Django people both > strive to support multiple versions of Python and regularily run their > production sites on trunk. Insane, perhaps, but that's how things are. And no need to change that, see http://wiki.python.org/moin/PortingDjangoTo3k Just to repeat myself: With that patch to Django, you can a) support all versions of Python simultaneously, from 2.x to 3.0 b) work from the subversion trunk all the time c) transparently invoke 2to3 on the trunk; you won't even notice that you do (except for all the diffs it currently prints to stdout also :-) FWIW, your approach (of adding __future__ imports to 2.6) wouldn't help Django at all, as they would need to give up 2.5 and earlier. > So, in short: Large projects with interconnected modules where the > developers and users of module are the same people will have big > difficulties with the 2to3 approach and would be the people who are > most likely to not be able to in practice go forward to Python 3 > unless they have some sort of smooth path forward. I still don't see why that is. In the examples you gave, no such difficulties are apparent. Regards, Martin From thomas at python.org Mon Mar 24 11:29:33 2008 From: thomas at python.org (Thomas Wouters) Date: Mon, 24 Mar 2008 11:29:33 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> Message-ID: <9e804ac0803240329p2c813805o617e4e38dd240f81@mail.gmail.com> On Mon, Mar 24, 2008 at 2:30 AM, Jean-Paul Calderone wrote: > On Mon, 24 Mar 2008 00:14:13 +0100, "\"Martin v. L?wis\"" < > martin at v.loewis.de> wrote: > >> You are still only seeing this as a case of libraries with a small > >> number of people developing them and making regular well defined > >> releases. That is not how the world I am talking about looks. > > > >Can you give me examples of such software? Are you perhaps talking > >about closed source software? > > I'm not sure what software he was talking about. I can say that for > the work I do on both Twisted and Divmod software, I'd be quite happy > to see this feature. As either part of a migration path towards 3k > _or_ as a feature entirely on its own merits, this would be very useful > to me. > > I'm a bit curious about why Thomas said this sort of thing results in > fragile code. Twisted has been using __future__ imports for years and > they've never been a problem. Twisted currently supports Python 2.3 > through Python 2.5, and the only thing that's really difficult about > that is subtle changes in library behavior, not syntax. > I wasn't talking about future imports. I was talking about writing Python 2.6 code and expecting it to work *the same way* in 3.0 without modification. It requires all programmers working on the project to have knowledge of how 3.0 and 2.6 differ. *I* can't even look at code and tell you how 2.6 and 3.0 will differ. Since Lennart was talking specifically about projects with a large number of developers, I do not believe this is a safe assumption to make. A simple preprocessor step involving 2to3 requires no such knowledge. Or, alternatively, a preprocessor step involving 3to2, which I think will result in better code. Unfortunately I don't have time to work on either anytime soon. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080324/982885b2/attachment-0001.htm From regebro at gmail.com Mon Mar 24 11:39:05 2008 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 24 Mar 2008 11:39:05 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <47E78226.1070607@v.loewis.de> References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> <319e029f0803240122n1c40a47cr4e10e0c74fdaccca@mail.gmail.com> <47E78226.1070607@v.loewis.de> Message-ID: <319e029f0803240339t75c4556at23816ec1ed69b064@mail.gmail.com> On Mon, Mar 24, 2008 at 11:27 AM, "Martin v. L?wis" wrote: > > Nuxeo CPS worked like this, but we can ignore them as that project is > > all but dead will never move to Python 3 in any case. Zope/CMF/Plone > > works like this. > > I don't understand. AFAICT, Zope *is* a library, i.e. you have to run > setup.py for lots of packages. Do you not have to run setup.py, for, > say, zope.interface, or zope.psycopgda? No, Zope is not a library, it's an application. No, you typically do not setup packages, although most (but not all) parts of Zope 3 is setup if you run Zope in a buildout configuration. Zope 2 does not. > > The Plone collective works like this, and it is *not* > > reasonably well managed, so there software quite often doens't get > > released, but people run against trunk. > > And that's fine. You still can integrate 2to3 with that transparently. I can not see how that would work. > I still don't see why that is. In the examples you gave, no such > difficulties are apparent. Maybe it's not apparent to people that hasn't developed in that kind of environment, and I'm sorry I'm not able to make this clearer. But that's just the way it is. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From regebro at gmail.com Mon Mar 24 11:41:08 2008 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 24 Mar 2008 11:41:08 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <9e804ac0803240329p2c813805o617e4e38dd240f81@mail.gmail.com> References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> <9e804ac0803240329p2c813805o617e4e38dd240f81@mail.gmail.com> Message-ID: <319e029f0803240341x4da992f8i9d0526ca033f5b5d@mail.gmail.com> On Mon, Mar 24, 2008 at 11:29 AM, Thomas Wouters wrote: > safe assumption to make. A simple preprocessor step involving 2to3 requires > no such knowledge. As I understood it nobody has claimed 2to3 to be perfect either, but that using 2to3 will also require you to test the code in both environments, so I don't see how that is a difference. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From eric+python-dev at trueblade.com Mon Mar 24 12:04:11 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Mon, 24 Mar 2008 07:04:11 -0400 Subject: [Python-Dev] PEP 3127 (Integer Literal Support and Syntax): %o and %b In-Reply-To: References: <47E09255.8090907@trueblade.com> Message-ID: <47E78AAB.3030401@trueblade.com> Guido van Rossum wrote: > On Tue, Mar 18, 2008 at 9:11 PM, Eric Smith > wrote: >> I've been double checking the PEP 3127 implementation in py3k and the >> backport I did to 2.6. The PEP says this about the % operator: >> >> "The string (and unicode in 2.6) % operator will have 'b' format >> specifier added for binary, and the alternate syntax of the 'o' option >> will need to be updated to add '0o' in front, instead of '0'." >> >> The %b operator was not added to 3.0, so I'll look into doing that in >> both 2.6 and 3.0 (which I opened as issue 2416). >> >> What should be done for '%#o' formatting in 2.6? The above sentence >> from the PEP implies it should be modified to add '0o' instead of '0', >> even in 2.6. But that seems like a bad idea to me. Maybe it should >> stay as-is, but add a -3 warning? Unfortunately, there'd be no way to >> change your code to get rid of the warning, short of switching to >> str.format() or adding a __future__ import (shudder). In 3.0, '%#o' >> already adds the leading '0o'. > > I think this is such a tiny detail we shouldn't bother with a -3 warning. > > I agree that in 2.6, %#o should continue to do what it does in 2.5. > Can you update the PEP? Done in r61845. Eric. From martin at v.loewis.de Mon Mar 24 12:26:35 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 24 Mar 2008 12:26:35 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <319e029f0803240339t75c4556at23816ec1ed69b064@mail.gmail.com> References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> <319e029f0803240122n1c40a47cr4e10e0c74fdaccca@mail.gmail.com> <47E78226.1070607@v.loewis.de> <319e029f0803240339t75c4556at23816ec1ed69b064@mail.gmail.com> Message-ID: <47E78FEB.9040706@v.loewis.de> >> I don't understand. AFAICT, Zope *is* a library, i.e. you have to run >> setup.py for lots of packages. Do you not have to run setup.py, for, >> say, zope.interface, or zope.psycopgda? > > No, Zope is not a library, it's an application. No, you typically do > not setup packages, although most (but not all) parts of Zope 3 is > setup if you run Zope in a buildout configuration. Zope 2 does not. I was talking about http://svn.zope.org/zope.psycopgda/trunk/ Is that not the right source? In any case, using 2to3 for applications is even easier than using it for libraries, assuming there is an installation procedure for the application. >> > The Plone collective works like this, and it is *not* >> > reasonably well managed, so there software quite often doens't get >> > released, but people run against trunk. >> >> And that's fine. You still can integrate 2to3 with that transparently. > > I can not see how that would work. To me "run against trunk" means that I check out some source, then run "setup.py install" (or buildout, or make). This will invoke 2to3 behind the scenes. I don't know exactly how plone works, but I could imagine that it's also possible to run 2to3 when plone "loads" a "component" (and I'm sure I'm using the wrong words here). > Maybe it's not apparent to people that hasn't developed in that kind > of environment, and I'm sorry I'm not able to make this clearer. But > that's just the way it is. I guess I could better understand with a very specific example. You gave Django as a very specific example, and I looked at Django, and it works just fine with 2to3. "The Plone collective" is not a specific example, as it doesn't allow me to reproduce your problems. What is the specific thing I would want to do with it, and what specific source repository do I need to check out to do that? Regards, Martin From regebro at gmail.com Mon Mar 24 12:38:55 2008 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 24 Mar 2008 12:38:55 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <47E78FEB.9040706@v.loewis.de> References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> <319e029f0803240122n1c40a47cr4e10e0c74fdaccca@mail.gmail.com> <47E78226.1070607@v.loewis.de> <319e029f0803240339t75c4556at23816ec1ed69b064@mail.gmail.com> <47E78FEB.9040706@v.loewis.de> Message-ID: <319e029f0803240438n3ec18980k50f7a825c6e85bf@mail.gmail.com> On Mon, Mar 24, 2008 at 12:26 PM, "Martin v. L?wis" wrote: > http://svn.zope.org/zope.psycopgda/trunk/ > > Is that not the right source? No, and yes. Many of the zope3 modules are installable as separate modules. Zope 3 in general has been "eggified". This is not however how everybody installs Zope3. Zope2 is not installed this way. Also, most Zope2 products are not modules installed by setup.py. > To me "run against trunk" means that I check out some source, then > run "setup.py install" (or buildout, or make). This will invoke > 2to3 behind the scenes. I repeat: There is no setup.py install to run. > I guess I could better understand with a very specific example. > You gave Django as a very specific example, and I looked at Django, > and it works just fine with 2to3. "The Plone collective" is not > a specific example It is a specific example. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From musiccomposition at gmail.com Mon Mar 24 12:56:54 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 24 Mar 2008 06:56:54 -0500 Subject: [Python-Dev] Commit access request Message-ID: <1afaf6160803240456u7ce0aa53t19514771e3434b30@mail.gmail.com> Hi Python devs, I have been contributing to since December. (See me first issue on the tracker, #1828; it was a major learning experience.) :P In that time, I have contributed many patches and actively participated on this list. This will enable me to help triage bugs on the tracker, something I've been wanting to help with. I'm willing to field questions. -- Thanks for your consideration, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080324/f2ce5c55/attachment.htm From p.f.moore at gmail.com Mon Mar 24 12:59:16 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 24 Mar 2008 11:59:16 +0000 Subject: [Python-Dev] windows standard [was: PEP 365 (Adding thepkg_resources module)] In-Reply-To: References: Message-ID: <79990c6b0803240459x60c1006w1de3c6c6b0220015@mail.gmail.com> On 24/03/2008, Terry Reedy wrote: > | If I install python and then separately install Zope, it may or may > | not make sense for Zope to be listed separately as a "program" to Add > | or Remove. > > Neither Paul nor I defined 'add-on', but I would be willing to call > Zope/Plone something more than that, preferably with its own multi-option > entry. Fair comment. I'd agree that Zope/Plone are probably more than an add-on. Actually, we agree - *if* you accept that my definition of "add-on" excludes anything you download separately, which has its own installer - which is what a bdist_wininst exe is. Of course, conversely, if all Python packages are add-ons (and so have their own "internal" means of installing - easy_install, for argument's sake - and listing/uninstalling - the mythical "new" package manager) then they shouldn't have add/remove program entries (any more than each MS Office option should). The purist in me says you can't have it both ways. But practicality says it's not a major issue (as long as there's *one* option that covers everything, no matter how it's installed). Personally, my only concern is having a single tool that can manage *all* of my non-core Python packages. (One ring to rule them all, and all that :-)) I have that at the moment, by refusing to use easy_install and eggs. But it's getting harder to do that, so this thread, for me, is about finding a better option. Paul. From p.f.moore at gmail.com Mon Mar 24 13:03:47 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 24 Mar 2008 12:03:47 +0000 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <319e029f0803240341x4da992f8i9d0526ca033f5b5d@mail.gmail.com> References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> <9e804ac0803240329p2c813805o617e4e38dd240f81@mail.gmail.com> <319e029f0803240341x4da992f8i9d0526ca033f5b5d@mail.gmail.com> Message-ID: <79990c6b0803240503k43428ff3s66714a874988a23c@mail.gmail.com> On 24/03/2008, Lennart Regebro wrote: > On Mon, Mar 24, 2008 at 11:29 AM, Thomas Wouters wrote: > > safe assumption to make. A simple preprocessor step involving 2to3 requires > > no such knowledge. > > As I understood it nobody has claimed 2to3 to be perfect either, but > that using 2to3 will also require you to test the code in both > environments, so I don't see how that is a difference. Do you not test the code you distribute under each version of Python that you (claim to) support, in any case? If not, what does "supported under Python 2.4" mean to you? Paul. From regebro at gmail.com Mon Mar 24 13:08:13 2008 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 24 Mar 2008 13:08:13 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <79990c6b0803240503k43428ff3s66714a874988a23c@mail.gmail.com> References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> <9e804ac0803240329p2c813805o617e4e38dd240f81@mail.gmail.com> <319e029f0803240341x4da992f8i9d0526ca033f5b5d@mail.gmail.com> <79990c6b0803240503k43428ff3s66714a874988a23c@mail.gmail.com> Message-ID: <319e029f0803240508i186fd0fen804f5d05eaa6f3b3@mail.gmail.com> On Mon, Mar 24, 2008 at 1:03 PM, Paul Moore wrote: > On 24/03/2008, Lennart Regebro wrote: > > On Mon, Mar 24, 2008 at 11:29 AM, Thomas Wouters wrote: > > > safe assumption to make. A simple preprocessor step involving 2to3 requires > > > no such knowledge. > > > > As I understood it nobody has claimed 2to3 to be perfect either, but > > that using 2to3 will also require you to test the code in both > > environments, so I don't see how that is a difference. > > Do you not test the code you distribute under each version of Python > that you (claim to) support, in any case? Yes I do. I don't think my text above was unclear on the issue or could be misunderstood in this way. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From p.f.moore at gmail.com Mon Mar 24 13:26:14 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 24 Mar 2008 12:26:14 +0000 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <319e029f0803240508i186fd0fen804f5d05eaa6f3b3@mail.gmail.com> References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> <9e804ac0803240329p2c813805o617e4e38dd240f81@mail.gmail.com> <319e029f0803240341x4da992f8i9d0526ca033f5b5d@mail.gmail.com> <79990c6b0803240503k43428ff3s66714a874988a23c@mail.gmail.com> <319e029f0803240508i186fd0fen804f5d05eaa6f3b3@mail.gmail.com> Message-ID: <79990c6b0803240526v3b8f4c1dq2546c97feaf798a3@mail.gmail.com> On 24/03/2008, Lennart Regebro wrote: > On Mon, Mar 24, 2008 at 1:03 PM, Paul Moore wrote: > > On 24/03/2008, Lennart Regebro wrote: > > > As I understood it nobody has claimed 2to3 to be perfect either, but > > > that using 2to3 will also require you to test the code in both > > > environments, so I don't see how that is a difference. > > > > Do you not test the code you distribute under each version of Python > > that you (claim to) support, in any case? > > Yes I do. I don't think my text above was unclear on the issue or > could be misunderstood in this way. Your statement "using 2to3 will also require you to test the code in both environments" seemed to me to say that *not* having to use 2to3 would save you from doing this (as if this were either desirable, or your current practice). My apologies if I misread your comment. What *are* you trying to say, then? It seems that you're saying that using 2to3 doesn't make things any more complex, but that contradicts your previous argument. Paul. From regebro at gmail.com Mon Mar 24 13:31:02 2008 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 24 Mar 2008 13:31:02 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <79990c6b0803240526v3b8f4c1dq2546c97feaf798a3@mail.gmail.com> References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> <9e804ac0803240329p2c813805o617e4e38dd240f81@mail.gmail.com> <319e029f0803240341x4da992f8i9d0526ca033f5b5d@mail.gmail.com> <79990c6b0803240503k43428ff3s66714a874988a23c@mail.gmail.com> <319e029f0803240508i186fd0fen804f5d05eaa6f3b3@mail.gmail.com> <79990c6b0803240526v3b8f4c1dq2546c97feaf798a3@mail.gmail.com> Message-ID: <319e029f0803240531h7e62e075k66122ee6214d29ea@mail.gmail.com> On Mon, Mar 24, 2008 at 1:26 PM, Paul Moore wrote: > Your statement "using 2to3 will also require you to test the code in > both environments" seemed to me to say that *not* having to use 2to3 > would save you from doing this (as if this were either desirable, or > your current practice). I think maybe you missed the statement I responded to, claiming that 2to3 would require no knowledge about the differences between Python 2.6 and 3.0, implying that you could just run it, and it would always work, which I don't believe. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From barry at python.org Mon Mar 24 14:50:12 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 24 Mar 2008 09:50:12 -0400 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <319e029f0803240122n1c40a47cr4e10e0c74fdaccca@mail.gmail.com> References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> <319e029f0803240122n1c40a47cr4e10e0c74fdaccca@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I've missed most of this thread, but let me just put my two cents in. I'd still like a future import to turn on unicode string literals (or more accurately, treat unadorned string literals as unicodes). As someone who is striving mightily to get various libraries and applications unicode clean, it's simply a matter of training my brain to correctly think, "this is a bytes" or "this is a string", and training my fingers to type the right thing. I'd like to be able to start retraining the muscle memory so that by the time 3.0 comes around, it'll will be a much smoother transition, for me and other coders. Now, if it's not feasible to add such a future import, that's one thing. If it's feasible then I think we should do it. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR+exl3EjvBPtnXfVAQLa0QQAl07BSwokgspNoIT0s2nn3kcWDn//PBmM ARgUCwd2fwZhHkiFsx5YgfzHJaBOuQjPNM4jOwUVy8vZpwUEVZNWmWE7rh+AHxQD FFLyier6/O1PkIe4US1RwuE3/53viP2qWo2Fr0z4zwbJbI6QOQvRVZeZ6OhU02jn GsNFuhuBz58= =1hYN -----END PGP SIGNATURE----- From ncoghlan at gmail.com Mon Mar 24 16:34:19 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 25 Mar 2008 01:34:19 +1000 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <319e029f0803240438n3ec18980k50f7a825c6e85bf@mail.gmail.com> References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> <319e029f0803240122n1c40a47cr4e10e0c74fdaccca@mail.gmail.com> <47E78226.1070607@v.loewis.de> <319e029f0803240339t75c4556at23816ec1ed69b064@mail.gmail.com> <47E78FEB.9040706@v.loewis.de> <319e029f0803240438n3ec18980k50f7a825c6e85bf@mail.gmail.com> Message-ID: <47E7C9FB.6020109@gmail.com> Lennart Regebro wrote: > On Mon, Mar 24, 2008 at 12:26 PM, "Martin v. L?wis" wrote: >> I guess I could better understand with a very specific example. >> You gave Django as a very specific example, and I looked at Django, >> and it works just fine with 2to3. "The Plone collective" is not >> a specific example > > It is a specific example. No it isn't. A specific example would be "I have environment X setup on a machine. I go to website/SVN repository Y and retrieve source code Z and start using it". Without the step by step process, it is impossible to identify if there is any point in the procedure where an invocation of 2to3 could be inserted relatively painlessly. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From regebro at gmail.com Mon Mar 24 16:43:25 2008 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 24 Mar 2008 16:43:25 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <47E7C9FB.6020109@gmail.com> References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> <319e029f0803240122n1c40a47cr4e10e0c74fdaccca@mail.gmail.com> <47E78226.1070607@v.loewis.de> <319e029f0803240339t75c4556at23816ec1ed69b064@mail.gmail.com> <47E78FEB.9040706@v.loewis.de> <319e029f0803240438n3ec18980k50f7a825c6e85bf@mail.gmail.com> <47E7C9FB.6020109@gmail.com> Message-ID: <319e029f0803240843y4e3d120em1228112f593dd088@mail.gmail.com> On Mon, Mar 24, 2008 at 4:34 PM, Nick Coghlan wrote: > No it isn't. A specific example would be "I have environment X setup on > a machine. I go to website/SVN repository Y and retrieve source code Z > and start using it". "I have environment Plone setup on a machine. I also have several products from the Plone collective which I use from the custom product that contains the custom site code". Thats it. It is a specific example. I can't get more specific than that without you learning Plone. > Without the step by step process, it is impossible to identify if there > is any point in the procedure where an invocation of 2to3 could be > inserted relatively painlessly. It can't. That's the whole point. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From martin at v.loewis.de Mon Mar 24 17:45:13 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 24 Mar 2008 17:45:13 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: References: Message-ID: <47E7DA99.4090809@v.loewis.de> > For example, if I'm using the (real source) py2.6 code, and I create a > patch that works for me, it is ready for testing and submission. If > I'm using the (generated) py3 code, then I first have to get a copy of > the (source) 2.6, figure out how I *would* have written it there, then > keep tweaking it so that the generator eventually puts out ... what I > had originally written by hand. Yes, that's tedious. In that case, it is easier to edit the original source, and then rerun 2to3, rather than editing the compiler output. Regards, Martin From jimjjewett at gmail.com Mon Mar 24 17:12:25 2008 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 24 Mar 2008 12:12:25 -0400 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals Message-ID: > Maybe it's not apparent to people that hasn't > developed in that kind of environment, and > I'm sorry I'm not able to make this clearer. I think I understand the issue. Some contributors will be running under 2.6, others will be running under 3.0. Either the code forks, or one of them is working with (and developing patches against) the result of a compilation step, instead of against the original source code. For example, if I'm using the (real source) py2.6 code, and I create a patch that works for me, it is ready for testing and submission. If I'm using the (generated) py3 code, then I first have to get a copy of the (source) 2.6, figure out how I *would* have written it there, then keep tweaking it so that the generator eventually puts out ... what I had originally written by hand. My (working in 3.0) task would be easier if there is also a 3to2 (so that I can treat my own code as the source), but then entire files will do flip-flops on a regular basis (depening on which version was generated), unless 2to3 and 3to2 somehow create a perfect round-trip. And that compile step -- it can be automated, but I suspect most python projects don't yet have a good place to put the hooks, simply because they haven't needed to in the past. The end result is that the barrier to contributions becomes much higher for people working in at least one of the versions. -jJ From lists at cheimes.de Mon Mar 24 17:58:26 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 24 Mar 2008 17:58:26 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> <319e029f0803240122n1c40a47cr4e10e0c74fdaccca@mail.gmail.com> Message-ID: <47E7DDB2.8020603@cheimes.de> Barry Warsaw schrieb: > I've missed most of this thread, but let me just put my two cents in. > I'd still like a future import to turn on unicode string literals (or > more accurately, treat unadorned string literals as unicodes). As > someone who is striving mightily to get various libraries and > applications unicode clean, it's simply a matter of training my brain > to correctly think, "this is a bytes" or "this is a string", and > training my fingers to type the right thing. > > I'd like to be able to start retraining the muscle memory so that by > the time 3.0 comes around, it'll will be a much smoother transition, > for me and other coders. > > Now, if it's not feasible to add such a future import, that's one > thing. If it's feasible then I think we should do it. I'm working on a node based __future__ parser based on the Python 2.3 code as MvL suggested. I'm making good progress. If I get it working it should be trivial to implement a __future__ unicode_string_literals feature. Christian From martin at v.loewis.de Mon Mar 24 20:30:48 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 24 Mar 2008 20:30:48 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <319e029f0803240843y4e3d120em1228112f593dd088@mail.gmail.com> References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> <319e029f0803240122n1c40a47cr4e10e0c74fdaccca@mail.gmail.com> <47E78226.1070607@v.loewis.de> <319e029f0803240339t75c4556at23816ec1ed69b064@mail.gmail.com> <47E78FEB.9040706@v.loewis.de> <319e029f0803240438n3ec18980k50f7a825c6e85bf@mail.gmail.com> <47E7C9FB.6020109@gmail.com> <319e029f0803240843y4e3d120em1228112f593dd088@mail.gmail.com> Message-ID: <47E80168.6050602@v.loewis.de> > "I have environment Plone setup on a machine. I also have several > products from the Plone collective which I use from the custom product > that contains the custom site code". > > Thats it. It is a specific example. I can't get more specific than > that without you learning Plone. Ok, let me try to guess, then. You use https://svn.plone.org/svn/collective/LatexTool/ and you perform an svn checkout into /var/lib/zope2.10/instance/plone-site/Products You start zope, edit the source, or perform a svn update, and then restart or refresh the product. Correct? If so, there is a fairly simple way to integrate 2to3: In OFS.Application.import_product, run 2to3 first before importing the product, if a file "run2to3.txt" exists in the product's root directory. This will perform a copy of the product into /var/lib/zope2.10/instance/plone-site/Products.2to3 Voila, transparent invocation of 2to3. Now, if you want pdb to display the original source rather than the one being converted, subclass pdb and strip out .2to3 from the source filename. Regards, Martin From gnewsg at gmail.com Mon Mar 24 20:43:33 2008 From: gnewsg at gmail.com (Giampaolo Rodola') Date: Mon, 24 Mar 2008 12:43:33 -0700 (PDT) Subject: [Python-Dev] Py3k and asyncore/asynchat In-Reply-To: References: <20080215142414.GA6805@amk-desktop.matrixgroup.net> <20080215142832.GC13291@snowy.squish.net> <47B5FE8E.2060901@cornell.edu> Message-ID: <2fa8db6c-bdec-4f66-8d4d-c6f857f051e0@d45g2000hsc.googlegroups.com> I'm going to refresh this discussion since it seems no decisions are still taken. Any chance to see a commit finally done? --- Giampaolo http://code.google.com/p/pyftpdlib From regebro at gmail.com Mon Mar 24 20:51:01 2008 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 24 Mar 2008 20:51:01 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <47E80168.6050602@v.loewis.de> References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> <319e029f0803240122n1c40a47cr4e10e0c74fdaccca@mail.gmail.com> <47E78226.1070607@v.loewis.de> <319e029f0803240339t75c4556at23816ec1ed69b064@mail.gmail.com> <47E78FEB.9040706@v.loewis.de> <319e029f0803240438n3ec18980k50f7a825c6e85bf@mail.gmail.com> <47E7C9FB.6020109@gmail.com> <319e029f0803240843y4e3d120em1228112f593dd088@mail.gmail.com> <47E80168.6050602@v.loewis.de> Message-ID: <319e029f0803241251h715fe946m9082f46a5aaefbbc@mail.gmail.com> On Mon, Mar 24, 2008 at 8:30 PM, "Martin v. L?wis" wrote: > .... Ah, I see. Your point was that with enough magic there is some way to run 2to3 automatically. Sure, I never doubted that. I don't see how that changes anything I said really. I still think the forwards compatibility that exists in 2.6 is a good idea, and the more of it the better. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From steve at holdenweb.com Mon Mar 24 21:35:44 2008 From: steve at holdenweb.com (Steve Holden) Date: Mon, 24 Mar 2008 16:35:44 -0400 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <47E7DA99.4090809@v.loewis.de> References: <47E7DA99.4090809@v.loewis.de> Message-ID: Martin v. L?wis wrote: >> For example, if I'm using the (real source) py2.6 code, and I create a >> patch that works for me, it is ready for testing and submission. If >> I'm using the (generated) py3 code, then I first have to get a copy of >> the (source) 2.6, figure out how I *would* have written it there, then >> keep tweaking it so that the generator eventually puts out ... what I >> had originally written by hand. > > Yes, that's tedious. In that case, it is easier to edit the original > source, and then rerun 2to3, rather than editing the compiler output. > This technique has actually been the one recommended by Guido for migration for at least a year now. Clearly you can't have developers tweaking source on both sides of the "great divide", as one may have to re-cast bits of one's 2.6 code in order to get a satisfactory translation into 3.0. Once you start editing 3.0 source you have to either leave the 2.X world behind or accept a dual-source development. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From thomas at python.org Mon Mar 24 22:04:20 2008 From: thomas at python.org (Thomas Wouters) Date: Mon, 24 Mar 2008 22:04:20 +0100 Subject: [Python-Dev] Py3k and asyncore/asynchat In-Reply-To: References: <95d2b6e8-2524-4464-b5a9-c4d56e5f4318@t16g2000hsc.googlegroups.com> <20080215142414.GA6805@amk-desktop.matrixgroup.net> <20080215142832.GC13291@snowy.squish.net> Message-ID: <9e804ac0803241404u5d21afa6oab0c7d1e2e49166f@mail.gmail.com> On Fri, Feb 15, 2008 at 9:11 PM, Josiah Carlson wrote: > Twisted core has been proposed, but I believe the consensus was that > it wasn't desirable, generally. > I remember only a couple of dissenting voices, and only a small number of participants. Of the dissenting voices, I do not recall any actual arguments about undesireability, just misunderstandings of how Twisted actually works. Getting Twisted core (meaning Deferreds, a simple reactor and the Protocol class) into the core is still on my TODO list. I'm also pretty sure that people learn twisted because everyone learns > twisted. It's one of those buzz-words ;). > I think that's quite an unfair assessment, even in jest :) Twisted is well worth learning to actually use it, as it's a very versatile event loop and does it best to integrate nicely with other event systems. And including it in the standard library improves integration with other event loops by creating a single interface. It's not a matter of dropping it in, though; it requires some careful structuring to avoid embarrassing situations like we have with the xml package, but still people to provide their own reactor. In case you're wondering how the twisted reactor in the stdlib is useful to people not using Twisted, take a look at what you currently need to do to combine stdlib modules like urllib and ftplib with event systems like Tkinter and PyGTK. Not to mention that the Twisted implementations of various protocols are really quite, quite good -- in many cases quite a lot better than the stdlib ones. But including those takes yet more time. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080324/9c3fc60a/attachment-0001.htm From rhamph at gmail.com Mon Mar 24 22:21:53 2008 From: rhamph at gmail.com (Adam Olsen) Date: Mon, 24 Mar 2008 15:21:53 -0600 Subject: [Python-Dev] Py3k and asyncore/asynchat In-Reply-To: <9e804ac0803241404u5d21afa6oab0c7d1e2e49166f@mail.gmail.com> References: <20080215142414.GA6805@amk-desktop.matrixgroup.net> <20080215142832.GC13291@snowy.squish.net> <9e804ac0803241404u5d21afa6oab0c7d1e2e49166f@mail.gmail.com> Message-ID: On Mon, Mar 24, 2008 at 3:04 PM, Thomas Wouters wrote: > > On Fri, Feb 15, 2008 at 9:11 PM, Josiah Carlson > wrote: > > Twisted core has been proposed, but I believe the consensus was that > > it wasn't desirable, generally. > > > > I remember only a couple of dissenting voices, and only a small number of > participants. Of the dissenting voices, I do not recall any actual arguments > about undesireability, just misunderstandings of how Twisted actually works. > Getting Twisted core (meaning Deferreds, a simple reactor and the Protocol > class) into the core is still on my TODO list. > > > > I'm also pretty sure that people learn twisted because everyone learns > > twisted. It's one of those buzz-words ;). > > > > I think that's quite an unfair assessment, even in jest :) Twisted is well > worth learning to actually use it, as it's a very versatile event loop and > does it best to integrate nicely with other event systems. And including it > in the standard library improves integration with other event loops by > creating a single interface. It's not a matter of dropping it in, though; it > requires some careful structuring to avoid embarrassing situations like we > have with the xml package, but still people to provide their own reactor. > > In case you're wondering how the twisted reactor in the stdlib is useful to > people not using Twisted, take a look at what you currently need to do to > combine stdlib modules like urllib and ftplib with event systems like > Tkinter and PyGTK. Not to mention that the Twisted implementations of > various protocols are really quite, quite good -- in many cases quite a lot > better than the stdlib ones. But including those takes yet more time. In that sense it'd be competing with safethread for inclusion in Python. Whereas safethread requires little if any API changes, twisted requires an entirely new API that can be event-driven. Worse, we'd likely to be stuck maintaining both APIs for a long time, if not forever. Twisted may be one of the best (if not *the* best) ways of writing concurrent programs today, but it doesn't need to be in the stdlib for that. If safethread is going to solve many of the same problems, with less changes required by the users of the language, then this is the wrong time to add twisted. -- Adam Olsen, aka Rhamphoryncus From thomas at python.org Mon Mar 24 22:37:43 2008 From: thomas at python.org (Thomas Wouters) Date: Mon, 24 Mar 2008 22:37:43 +0100 Subject: [Python-Dev] Py3k and asyncore/asynchat In-Reply-To: References: <20080215142414.GA6805@amk-desktop.matrixgroup.net> <20080215142832.GC13291@snowy.squish.net> <9e804ac0803241404u5d21afa6oab0c7d1e2e49166f@mail.gmail.com> Message-ID: <9e804ac0803241437n20bed852m551309e8ef24b114@mail.gmail.com> On Mon, Mar 24, 2008 at 10:21 PM, Adam Olsen wrote: > On Mon, Mar 24, 2008 at 3:04 PM, Thomas Wouters wrote: > > > > On Fri, Feb 15, 2008 at 9:11 PM, Josiah Carlson < > josiah.carlson at gmail.com> > > wrote: > > > Twisted core has been proposed, but I believe the consensus was that > > > it wasn't desirable, generally. > > > > > > > I remember only a couple of dissenting voices, and only a small number > of > > participants. Of the dissenting voices, I do not recall any actual > arguments > > about undesireability, just misunderstandings of how Twisted actually > works. > > Getting Twisted core (meaning Deferreds, a simple reactor and the > Protocol > > class) into the core is still on my TODO list. > > > > > > > I'm also pretty sure that people learn twisted because everyone learns > > > twisted. It's one of those buzz-words ;). > > > > > > > I think that's quite an unfair assessment, even in jest :) Twisted is > well > > worth learning to actually use it, as it's a very versatile event loop > and > > does it best to integrate nicely with other event systems. And including > it > > in the standard library improves integration with other event loops by > > creating a single interface. It's not a matter of dropping it in, > though; it > > requires some careful structuring to avoid embarrassing situations like > we > > have with the xml package, but still people to provide their own > reactor. > > > > In case you're wondering how the twisted reactor in the stdlib is useful > to > > people not using Twisted, take a look at what you currently need to do > to > > combine stdlib modules like urllib and ftplib with event systems like > > Tkinter and PyGTK. Not to mention that the Twisted implementations of > > various protocols are really quite, quite good -- in many cases quite a > lot > > better than the stdlib ones. But including those takes yet more time. > > In that sense it'd be competing with safethread for inclusion in > Python. Whereas safethread requires little if any API changes, > twisted requires an entirely new API that can be event-driven. Worse, > we'd likely to be stuck maintaining both APIs for a long time, if not > forever. > I am not going to worry about this for the time being. Even if safethread goes in and becomes ubiquitous, Twisted is still a very valid approach to the problem. And I'm not just saying that because I don't subscribe to your enthusiasm of safethreads as a concept or as an implementation :) > > Twisted may be one of the best (if not *the* best) ways of writing > concurrent programs today, but it doesn't need to be in the stdlib for > that. If safethread is going to solve many of the same problems, with > less changes required by the users of the language, then this is the > wrong time to add twisted. > You must have missed the part where we already have a large set of event loops, and not having a single interface to them is in fact hurting people. Twisted goes out of its way to interact nicely with event loops, but it can only do that with ones it knows about (and are versatile enough to hook into.) Having a single event system in the standard library is definitely advantageous, even if safethreads were available everywhere and the performance in the common case was satisfactory. It used to be the case that people thought asyncore was this standard event system, but it's long since ceased to be. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080324/12929d8d/attachment.htm From tseaver at palladion.com Mon Mar 24 23:12:59 2008 From: tseaver at palladion.com (Tres Seaver) Date: Mon, 24 Mar 2008 18:12:59 -0400 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E52F70.1010806@v.loewis.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> <47E4EE9D.2090605@v.loewis.de> <20080322151918.D9EBC3A40B1@sparrow.telecommunity.com> <47E525F5.40705@v.loewis.de> <20080322154449.BDA423A40A5@sparrow.telecommunity.com> <47E52F70.1010806@v.loewis.de> Message-ID: <47E8276B.8030800@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin v. L?wis wrote: >>> Sure, but what is precisely the semantics of uninstallation, in >>> terms of changes to the system state? >>> >>> I think any model where uninstallation is merely the removal >>> of files is too limited to be practical. >> The distutils only support the *addition* of files, so I'm not sure >> how only removing files is a limit here. Could you explain? > > For files, yes, it only supports addition. But it supports > arbitrary other actions, such as: > - addition of registry keys > - addition of user accounts > - creation of database tables in a relational database > - updating the shared library loader path > - creation and start of a system service > - integration of documentation into info > - registration of DTDs with the system catalog > - ... > > It's turing-complete, and it has full interface to the operating > system, so installation of a distutils package can do *much* > more than merely installing files. Which is exactly what is wrong with distutils: turing completeness in an installer is an *anti* goal, from the perspective of manageability. I'd be willing to bet that if you asked system packagers to list the dozen or so packages which they *hate* to maintain, the large majority of each list would be packages which acutally use the full power of distutils. (Note: I'm aware that people believe it to be necessary to munge the Windows registry when installing Python packages; I just don't agree with the practice, and don't think we should distort Python's process to coddle it). > Uninstallation needs to revert anything installation did, > so it is often more than mere removal of files. Practically speacking, nobody but the author of the TC-abusing setup.py is *ever* going to be able to do this, and most of them won't get the edge cases right. Maybe we can just punt on such packages, and make a tool which actually works for the huge majority of distributions which don't need to do more than install files. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD4DBQFH6Cdr+gerLs4ltQ4RAlFzAJi3gs8fzb9o8/Dtct1G9P0EJxNSAKCc7V7m uT5MgTzltBDhdrgoNxt8nA== =zgqI -----END PGP SIGNATURE----- From amk at amk.ca Mon Mar 24 23:18:23 2008 From: amk at amk.ca (A.M. Kuchling) Date: Mon, 24 Mar 2008 18:18:23 -0400 Subject: [Python-Dev] Py3k and asyncore/asynchat In-Reply-To: <9e804ac0803241404u5d21afa6oab0c7d1e2e49166f@mail.gmail.com> References: <95d2b6e8-2524-4464-b5a9-c4d56e5f4318@t16g2000hsc.googlegroups.com> <20080215142414.GA6805@amk-desktop.matrixgroup.net> <20080215142832.GC13291@snowy.squish.net> <9e804ac0803241404u5d21afa6oab0c7d1e2e49166f@mail.gmail.com> Message-ID: <20080324221823.GA24766@amk-desktop.matrixgroup.net> On Mon, Mar 24, 2008 at 10:04:20PM +0100, Thomas Wouters wrote: > I remember only a couple of dissenting voices, and only a small number of > participants. Of the dissenting voices, I do not recall any actual arguments Weren't some of those dissenting voices the Twisted developers, though? --amk From rhamph at gmail.com Mon Mar 24 23:25:51 2008 From: rhamph at gmail.com (Adam Olsen) Date: Mon, 24 Mar 2008 16:25:51 -0600 Subject: [Python-Dev] Py3k and asyncore/asynchat In-Reply-To: <9e804ac0803241437n20bed852m551309e8ef24b114@mail.gmail.com> References: <20080215142414.GA6805@amk-desktop.matrixgroup.net> <20080215142832.GC13291@snowy.squish.net> <9e804ac0803241404u5d21afa6oab0c7d1e2e49166f@mail.gmail.com> <9e804ac0803241437n20bed852m551309e8ef24b114@mail.gmail.com> Message-ID: On Mon, Mar 24, 2008 at 3:37 PM, Thomas Wouters wrote: > > On Mon, Mar 24, 2008 at 10:21 PM, Adam Olsen wrote: > > > > Twisted may be one of the best (if not *the* best) ways of writing > > concurrent programs today, but it doesn't need to be in the stdlib for > > that. If safethread is going to solve many of the same problems, with > > less changes required by the users of the language, then this is the > > wrong time to add twisted. > > > > You must have missed the part where we already have a large set of event > loops, and not having a single interface to them is in fact hurting people. > Twisted goes out of its way to interact nicely with event loops, but it can > only do that with ones it knows about (and are versatile enough to hook > into.) Having a single event system in the standard library is definitely > advantageous, even if safethreads were available everywhere and the > performance in the common case was satisfactory. It used to be the case that > people thought asyncore was this standard event system, but it's long since > ceased to be. I'm not opposed to standardizing on twisted as the canonical way to do events in python, or to having an event system in python. My concern is that may be used as an excuse to slowly rewrite the entire language into an event-driven model. However, that was based on the assumption that modules like urllib2 weren't already event-driven. Looking now, it seems it already is, and wraps it in a blocking API for simple use cases. So long as we're only talking about replacing existing event-driven stuff, and so long as we retain the existing blocking API (including calling from threads!), I don't think I have any valid opposition. -- Adam Olsen, aka Rhamphoryncus From tseaver at palladion.com Mon Mar 24 23:26:42 2008 From: tseaver at palladion.com (Tres Seaver) Date: Mon, 24 Mar 2008 18:26:42 -0400 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E53435.4090104@v.loewis.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <47E4DE8E.6010809@holdenweb.com> <79990c6b0803220646r50e9b455y5bd9b955814f334a@mail.gmail.com> <47E53435.4090104@v.loewis.de> Message-ID: <47E82AA2.1020502@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin v. L?wis wrote: >> Oh, and application installation is (should be) completely different. >> On Windows, applications should probably be bundled with their own >> Python interpreter, a la py2exe. On Unix/Linux, I don't know what the >> standard is, so I'd have to defer to others. > > This I disagree with. I think it's an overall bad thing to have all > kinds of applications ship their own copy of Python; see also Aza > Raskin's PyCon keynote. > > On Linux, python typically comes with the system pre-installed; > it is not even an option not to have it, except for minimalist > installations. So if you write python scripts, you typically > expect that #!/usr/bin/env python works; you might put python2.5 > there if you don't trust that system one is "good enough". > > For installing the application, you typically want the choice > betaween a "system installation" (in /usr/bin, or perhaps > /usr/local/bin), and a "user installation". As distutils supports > both cases, it works fairly well for applications as well. Sharing the system python is hugely problematic on a unix box which actually *uses* python for its own tools: the application is not "safe" from additions / updates / removeals of the packages in /usr/lib/python2.x/site-packages done to support those system tools. The problem gets worse as multiple non-system applications install files there: e.g., the 'twisted' package on Debian boxes depends on an ancient version of 'zope.interface', which can't be used with any currently supported version of Zope. virtualenv makes using the system python on such systems somewhat more tolerable, because each virtualenv can be isolated from the site-packages of the "host" environment (and therefore from the other applications). You still have to live with the choices the system packagers make (UCS4, for instance), but the pain is at least tolerable. For a long-running Python application (as opposed to desktop tools or scripts), installing a custom Python is the "safest" choice: it insulates the application not only from unexpected system-driven site-packages changes, but also allows greater control over other Python configuration choices (the UCS2 / UCS4 knob, etc.). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFH6Cqi+gerLs4ltQ4RAlqZAKCxr2lraLSycVsksYAevtf+urALOgCeLzs9 fE2g7IAb+22B+UbSUFPqj4w= =re0h -----END PGP SIGNATURE----- From josiah.carlson at gmail.com Mon Mar 24 23:51:56 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Mon, 24 Mar 2008 15:51:56 -0700 Subject: [Python-Dev] Py3k and asyncore/asynchat In-Reply-To: <20080324221823.GA24766@amk-desktop.matrixgroup.net> References: <95d2b6e8-2524-4464-b5a9-c4d56e5f4318@t16g2000hsc.googlegroups.com> <20080215142414.GA6805@amk-desktop.matrixgroup.net> <20080215142832.GC13291@snowy.squish.net> <9e804ac0803241404u5d21afa6oab0c7d1e2e49166f@mail.gmail.com> <20080324221823.GA24766@amk-desktop.matrixgroup.net> Message-ID: Let us not get side-tracked in this discussion. Whether or not to include any portion of Twisted into Python 2.6 is well past being a reasonable question; 2.6 alpha 1 has been released. It's a question as to whether someone with commit access can or will commit the patch as posted, run the tests to verify that they aren't broken, and perhaps actually look at the code to verify that we (Giampaolo and I) aren't insane. Mind you, I've been using very similar variants of this patch for months; it fixes outstanding bugs, improves performance, makes the handle* interfaces more consistent, and even offers a 'sample' implementation of a basic protocol in the source (for those who are willing to look). Do we want to fix asyncore/asynchat with work that has already been done and tested? If you want a reason as to why twisted shouldn't *replace* asyncore/asynchat, I'll give you a few: As stated previously by Guido and others (please see previous threads about this over the course of the last 4 years), asyncore/asynchat provide a more or less minimal interface for asynchronous sockets in Python. Any module/package that seeks to implement asynchronous sockets will need to, at least, implement that much. Asyncore/asynchat at present will play nicely with any event loop available, given certain caveats*. Further, if someone spends a half hour reading the source of a reasonably written asyncore server/client, they should understand the basics and be able to begin using it directly (see any one of the simple echo variants). As to whether twisted should be added to the standard library at some point in the future as a "this is better than asyncore in every way, use this instead"; that is a different discussion, not related to 2.6 (perhaps not even related to the 2.x series at all, depending on the future of 2.x). - Josiah * If your application strictly responds to socket IO, then implement your application as part of handle_* methods. If your application does more, then call asyncore.poll() often enough for data to be handled sufficiently fast. If neither offer sufficient performance or flexibility (maybe something like bittorrent + wxPython), use one thread for your GUI, one thread for your sockets, and use Queue.Queue() or some other threadsafe message delivery method. On Mon, Mar 24, 2008 at 3:18 PM, A.M. Kuchling wrote: > On Mon, Mar 24, 2008 at 10:04:20PM +0100, Thomas Wouters wrote: > > I remember only a couple of dissenting voices, and only a small number of > > participants. Of the dissenting voices, I do not recall any actual arguments > > Weren't some of those dissenting voices the Twisted developers, though? > > --amk > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/josiah.carlson%40gmail.com > From tjreedy at udel.edu Tue Mar 25 00:39:39 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 24 Mar 2008 19:39:39 -0400 Subject: [Python-Dev] Proposal: from __future__import unicode_string_literals References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm><319e029f0803240122n1c40a47cr4e10e0c74fdaccca@mail.gmail.com> <47E78226.1070607@v.loewis.de> Message-ID: ""Martin v. L?wis"" wrote in message news:47E78226.1070607 at v.loewis.de... I think we all can agree that we would like Plone to migrate to 3.x without too much pain and that their collective situation may be more difficult for migration than some others. | http://wiki.python.org/moin/PortingDjangoTo3k | | Just to repeat myself: With that patch to Django, you can | a) support all versions of Python simultaneously, from 2.x to 3.0 I find this surprising for two reasons. 1. I had the impression from discussions over the past year that fully automatic use of 2to3 would presume use of 2.6 and its backported 3.0 features and __future__ imports. If it really works with ealier 2.x code, great, but please pardon any initial surprise or scepticism. 2. You report has caveats such as * there are certainly large parts of the code base that I haven't touched, so more issues are likely to show up *This port attempts to maintain compatibility with older Python versions from a single code base; the goal is to support all versions that Django supports, plus 3.0. The current patch fails to do so in certain details, due to bugs in the 2to3 tool. *This approach mostly works, and has the following flaws: some of the fixers work incorrectly (bugs 2453, 2446, 2468) *I have worked with sqlite3 only; all the other databases have not been tested. So your unqualified assertion seems more like an anticipated future (certain to you but maybe not to others) than present reality. 3. Also, you said you worked around some 2to3 failings with conditional blocks like, I presume, the following. if sys.version < (3,0,0): else: Do I assume correctly that you, rather than 2to3 had to do such? Will 2to3 remove the wrapper to leave just the 3.0 code? Or would someone have to go thru by hand to get clean 3.0 code? I understand that this is a standard method for multiple release code, but it seems a bit like cheating since the point of 2to3 is to not to have to do this. Or is converting 'types.ClassType' to 'types' a future fixer item? Terry From musiccomposition at gmail.com Tue Mar 25 03:35:41 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 24 Mar 2008 21:35:41 -0500 Subject: [Python-Dev] Commit access request In-Reply-To: <1afaf6160803240456u7ce0aa53t19514771e3434b30@mail.gmail.com> References: <1afaf6160803240456u7ce0aa53t19514771e3434b30@mail.gmail.com> Message-ID: <1afaf6160803241935y2eb27f05t80dd0b4e0f6357cf@mail.gmail.com> I've attached my SSH keys. On Mon, Mar 24, 2008 at 6:56 AM, Benjamin Peterson < musiccomposition at gmail.com> wrote: > Hi Python devs, > I have been contributing to since December. (See me first issue on the > tracker, #1828; it was a major learning experience.) :P In that time, I have > contributed many patches and actively participated on this list. > This will enable me to help triage bugs on the tracker, something I've > been wanting to help with. > > I'm willing to field questions. > > -- > Thanks for your consideration, > Benjamin Peterson -- Cheers, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080324/73ea22f3/attachment.htm -------------- next part -------------- A non-text attachment was scrubbed... Name: id_dsa.pub Type: application/octet-stream Size: 589 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20080324/73ea22f3/attachment.obj -------------- next part -------------- A non-text attachment was scrubbed... Name: id_rsa.pub Type: application/octet-stream Size: 399 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20080324/73ea22f3/attachment-0001.obj From martin at v.loewis.de Tue Mar 25 03:37:15 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 25 Mar 2008 03:37:15 +0100 Subject: [Python-Dev] Proposal: from __future__import unicode_string_literals In-Reply-To: References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm><319e029f0803240122n1c40a47cr4e10e0c74fdaccca@mail.gmail.com> <47E78226.1070607@v.loewis.de> Message-ID: <47E8655B.4050809@v.loewis.de> > | Just to repeat myself: With that patch to Django, you can > | a) support all versions of Python simultaneously, from 2.x to 3.0 > > I find this surprising for two reasons. > > 1. I had the impression from discussions over the past year that fully > automatic use of 2to3 would presume use of 2.6 and its backported 3.0 > features and __future__ imports. If it really works with ealier 2.x code, > great, but please pardon any initial surprise or scepticism. This is precisely why I started this porting experiment. If you are still skeptic, please substantiate your skepticism with facts: run my patch, and tell me why it doesn't work, or couldn't be completed. If you are now merely surprised, but not skeptic anymore: my pleasure. The believe that you must port to 2.6 first is wide-spread. It probably originates from the statement that the official porting strategy involves porting to 2.6 first. That strategy does so, however, to enable you to run the -3 option, so you can find porting problems more easily. If you can find the porting problems the hard way, i.e. by running the software and testing it, you don't need the -3 warnings. When I started a week ago, a few essential 2to3 fixers did not exist (in particular the one David Wolever wrote to make implicit relative imports explicit). That fixer really falls into the 2to2.5 category; it would have been possible to change the code to use relative imports everywhere, thereby breaking 2.3 compatibility. It is possible that other examples like this still exist (i.e. 2to3 doesn't fix it, but doesn't have to if you can assume 2.5), but I'm not aware of any (actually, that's not entirely true - the email module renaming is of the same kind. However, this can be dealt with by ImportError guards. Still, having a fixer for that might be useful) > 2. You report has caveats such as > > * there are certainly large parts of the code base that I haven't touched, > so more issues are likely to show up True. > *This port attempts to maintain compatibility with older Python versions > from a single code base; the goal is to support all versions that Django > supports, plus 3.0. The current patch fails to do so in certain details, > due to bugs in the 2to3 tool. > > *This approach mostly works, and has the following flaws: > some of the fixers work incorrectly (bugs 2453, 2446, 2468) These bugs are really shallow, and some have been fixed already. > *I have worked with sqlite3 only; all the other databases have not been > tested. True. > So your unqualified assertion seems more like an anticipated future > (certain to you but maybe not to others) than present reality. Likewise, the statement that you *can't* possibly use the same code base from 2.1 to 3.0 is unfounded, and, unlike my claim, doesn't have any kind of proof behind it. > 3. Also, you said you worked around some 2to3 failings with conditional > blocks like, I presume, the following. > > if sys.version < (3,0,0): > else: > > Do I assume correctly that you, rather than 2to3 had to do such? Indeed. > Will 2to3 remove the wrapper to leave just the 3.0 code? Currently, it leaves the code unchanged. It could fairly easily remove it, but doing so might shift line numbers, which in turn is undesirable. 2to3 has support for optional fixers, and that might be a use case. > Or would someone have to go thru by hand to get clean 3.0 code? See above. Writing a fixer for it is fairly easy, and somebody will certainly do that, but I would only run it when using a "burn the bridges" run. > I understand that this is a standard method for multiple release code, but > it seems a bit like cheating since the point of 2to3 is to not to have to > do this. Or is converting 'types.ClassType' to 'types' a future fixer > item? No. This is the sort of change that 2to3 can't do right. If Django would require Python 2.5, the conditional could go away, as this appears in a context of creating exception classes on-the-fly; they can be new-style classes in 2.5. However, I consider conditional code blocks not as cheating at all. If you want to provide backwards compatibility, you *always* have to compromise readability for portability. This was even the case within 2.x, where you can't use True and False if you want to support Python versions that didn't have it, or where you can use generators, or iterators, or ..., when older Python versions need to be supported. The main point of 2to3 is not to have the 3.x code "look nice", in fact, in many cases, it won't, since 2to3 must make conservative assumptions in some cases, so to not break the semantics of the program. Instead, the main point of 2to3 is to replace syntax that is going away with the 3.x equivalent syntax. In the cases where I conventionalize on the Python version, it's not syntax that has changed. Regards, Martin From greg.ewing at canterbury.ac.nz Tue Mar 25 03:51:03 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 25 Mar 2008 14:51:03 +1200 Subject: [Python-Dev] [Python-checkins] r61709 - python/trunk/Doc/library/functions.rst python/trunk/Doc/library/future_builtins.rst python/trunk/Doc/library/python.rst In-Reply-To: References: <20080321193757.B2DD71E400B@bag.python.org> Message-ID: <47E86897.5030007@canterbury.ac.nz> Jim Jewett wrote: > In 2.5, the print statement ignores any overrides of the str builtin, > but I'm not sure whether a _function_ should -- and I do think it > should be specified. I expect there are plenty of other things that use str()-like behaviour without going through str(), so making print alone do this probably wouldn't be very useful. -- Greg From skip at pobox.com Tue Mar 25 04:06:23 2008 From: skip at pobox.com (skip at pobox.com) Date: Mon, 24 Mar 2008 22:06:23 -0500 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> Message-ID: <18408.27695.339064.649345@montanaro-dyndns-org.local> Barry> All the gory details are documented here: Barry> http://www.python.org/dev/bazaar Thanks. I checked out, made a branch named test3, changed Makefile.pre.in to have a test3 target, checked it in, then tried to push it: % pwd /Users/skip/src/python-bzr/test3 % bzr push bzr+ssh://pythonbzr at code.python.org/python/users/skip/test3 bzr: ERROR: Parent directory of bzr+ssh://pythonbzr at code.python.org/python/users/skip/test3 does not exist. You may supply --create-prefix to create all leading parent directories. Did I misread the directions or do I really need the --create-prefix arg? Skip From ajaksu at gmail.com Sun Mar 23 03:16:50 2008 From: ajaksu at gmail.com (ajaksu) Date: Sat, 22 Mar 2008 19:16:50 -0700 (PDT) Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E56864.4080201@v.loewis.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <47E4DE8E.6010809@holdenweb.com> <79990c6b0803220646r50e9b455y5bd9b955814f334a@mail.gmail.com> <47E53435.4090104@v.loewis.de> <79990c6b0803221256jf4e6ea1qce463ab711dcdb51@mail.gmail.com> <47E56864.4080201@v.loewis.de> Message-ID: On Mar 22, 5:13 pm, "Martin v. L?wis" wrote: > > Can you give me a > > pointer to Aza Raskin's keynote? Is it online anywhere? I'd be > > interested in his point of view. > > Unfortunately no. I was looking for it, but couldn't find it. He > mentioned a website with a "call for action", but I couldn't find > that, either :-( I guess the website could be http://www.toolness.com/wp/?p=23#more-23 - > "Python as a Platform". Via Ned Batchelder's notes at http://nedbatchelder.com/blog/200803/pycon_2008_notes.html From the post: "Something that recently occurred to me is that the only operating system that doesn't come with Python pre-installed on it is Windows. While Linux and OS X both view Python as essentially a first-class development platform-i.e., as something that shrink-wrap applications can be built on-Windows does not. Instead, it's generally expected that a Python-based Windows application be "frozen": bundled into a self-contained package that includes a copy of the Python interpreter and whatever libraries it uses, which are private to the particular application. While this ensures that the application will function as expected and not run into 'dependency hell', it also results in a relatively large download-distributing a simple 'Hello World' program is at least a megabyte in size, and makes extending the program's functionality more difficult." Regards, Daniel From tseaver at palladion.com Mon Mar 24 23:26:42 2008 From: tseaver at palladion.com (Tres Seaver) Date: Mon, 24 Mar 2008 18:26:42 -0400 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E53435.4090104@v.loewis.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <47E4DE8E.6010809@holdenweb.com> <79990c6b0803220646r50e9b455y5bd9b955814f334a@mail.gmail.com> <47E53435.4090104@v.loewis.de> Message-ID: <47E82AA2.1020502@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin v. L?wis wrote: >> Oh, and application installation is (should be) completely different. >> On Windows, applications should probably be bundled with their own >> Python interpreter, a la py2exe. On Unix/Linux, I don't know what the >> standard is, so I'd have to defer to others. > > This I disagree with. I think it's an overall bad thing to have all > kinds of applications ship their own copy of Python; see also Aza > Raskin's PyCon keynote. > > On Linux, python typically comes with the system pre-installed; > it is not even an option not to have it, except for minimalist > installations. So if you write python scripts, you typically > expect that #!/usr/bin/env python works; you might put python2.5 > there if you don't trust that system one is "good enough". > > For installing the application, you typically want the choice > betaween a "system installation" (in /usr/bin, or perhaps > /usr/local/bin), and a "user installation". As distutils supports > both cases, it works fairly well for applications as well. Sharing the system python is hugely problematic on a unix box which actually *uses* python for its own tools: the application is not "safe" from additions / updates / removeals of the packages in /usr/lib/python2.x/site-packages done to support those system tools. The problem gets worse as multiple non-system applications install files there: e.g., the 'twisted' package on Debian boxes depends on an ancient version of 'zope.interface', which can't be used with any currently supported version of Zope. virtualenv makes using the system python on such systems somewhat more tolerable, because each virtualenv can be isolated from the site-packages of the "host" environment (and therefore from the other applications). You still have to live with the choices the system packagers make (UCS4, for instance), but the pain is at least tolerable. For a long-running Python application (as opposed to desktop tools or scripts), installing a custom Python is the "safest" choice: it insulates the application not only from unexpected system-driven site-packages changes, but also allows greater control over other Python configuration choices (the UCS2 / UCS4 knob, etc.). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFH6Cqi+gerLs4ltQ4RAlqZAKCxr2lraLSycVsksYAevtf+urALOgCeLzs9 fE2g7IAb+22B+UbSUFPqj4w= =re0h -----END PGP SIGNATURE----- From david at wolever.net Thu Mar 20 20:56:30 2008 From: david at wolever.net (David Wolever) Date: Thu, 20 Mar 2008 15:56:30 -0400 Subject: [Python-Dev] 2to3 speedup patch Message-ID: <50F16185-3EB2-47A6-9EB8-83E7C033E136@wolever.net> Under the instruction of Martin, I've made some small changes to 2to3 so keeps track of which fixers act on which level of node. The speedup isn't too shabby: running on the example file, processing time went from 9 to 7 seconds, and the test suite dropped from 400 to 350. I have attached the hacky, ugly, proof-of-concept patch to http:// bugs.python.org/issue2431 If there's no reason not to implement this sort of thing, I'll clean it up and commit it when I get home (or something). -- David Wolever - http://wolever.net/~wolever AIM: davidswolever MSN: david at wolever.net Phone: 416-906-0403 "Without payment you have received; without payment you are to give." (Mat 10:8 ISV) From floris.bruynooghe at gmail.com Sat Mar 22 12:00:10 2008 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Sat, 22 Mar 2008 11:00:10 +0000 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080322020447.3B1033A4074@sparrow.telecommunity.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> Message-ID: <20080322110010.GA8513@laurie.devork> On Fri, Mar 21, 2008 at 10:04:45PM -0400, Phillip J. Eby wrote: > At 02:31 AM 3/22/2008 +0100, Martin v. L?wis wrote: > >However, I'm extremely skeptical that this can ever succeed > >to the degree that whoever provides RPMs, .debs, or MSI > >files will actually use such data, as they will find that > >the data are incomplete, and they have to redo all of it, > >anyway. > > The data isn't for them to use to meet their use cases, it's for them > to *provide* so that Python tools don't stomp on, uninstall, or > otherwise interfere with files installed by the system. In other > words, for system packagers, it's a communication from the system to > Python, rather than the other way around. Even though the distutils > will build the file in the bdist, the system packaging tools would be > free to generate their own file listing and signatures and such. As long as systems (dpkg, rpm, ...) install the .egg-info files they do communicate which modules/distributions are installed. The installdb would just duplicate this information (according to the current PEP). > >>And as I said, I'll be happy if all we do is get the distutils to > >>abide by the spec for 2.6, even if it means we don't get an > >>uninstall tool. That can always be installed later using Guido's > >>bootstrap tool. :) > > > >I'm even more skeptical here. If the assumption is that the package > >database allows for uninstallation, I'm -1. IOW, RPM, deb, MSI > >should then *not* write to that package database, as they also write > >to a different database, out of the scope of the PEP, and this is > >what uninstallation should use. > > I probably should have brought this up, in fact, I think I mentioned > it in a previous thread, but I would like to see PEP 262 add a way to > say "this is a system-installed package, *don't touch*". The idea > again is not to do the job of the native packaging system, but rather > to ensure that Python-specific tools (e.g. easy_install and friends) > do not interfere or conflict with it. > > A big problem in the early development of easy_install, even using > eggs, was that there was no way to tell whether it was safe to delete > or overwrite an existing file or directory that was already installed > on the system. A mechanism like this would allow tools like > easy_install to say, "oh, your system packager has a conflicting > package here, you need to use that tool to sort this out if you > really want to do something here. I'm not going to touch > that." Without something like this, there is no way to tell the > difference on many systems between a system package and something the > user has put there with "sudo python setup.py install". There is a way of telling if you have to keep you hands off a package (sorry to bring this up again): installation paths. /usr/lib is the system path, the local admin (and hence setuptools) should keep their hands off it at all times (unless requested with a --prefix or so for building the debs or rpms but an uninstall in those cases won't be required from setuptools). What an installdb could help with is tell setuptools to keep it's hands off a distribution manually installed (or by another tool) in the local admin directory (/usr/local) or user directory (~/). Your proposal of an installdb for each directory on sys.path would solve this, but the installdb in /usr/lib will be completely usused. But frankly, for that just an extra field in the .egg-info "Installed-By: setuptools" would do no? Possibly followed by a "X-Setuptools-Installed-Files:" section if you need that. -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From floris.bruynooghe at gmail.com Sat Mar 22 14:24:18 2008 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Sat, 22 Mar 2008 13:24:18 +0000 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E4EE9D.2090605@v.loewis.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> <47E4EE9D.2090605@v.loewis.de> Message-ID: <20080322132418.GB8513@laurie.devork> On Sat, Mar 22, 2008 at 12:33:49PM +0100, "Martin v. L?wis" wrote: > > The data isn't for them to use to meet their use cases, it's for them to > > *provide* so that Python tools don't stomp on, uninstall, or otherwise > > interfere with files installed by the system. In other words, for > > system packagers, it's a communication from the system to Python, rather > > than the other way around. Even though the distutils will build the > > file in the bdist, the system packaging tools would be free to generate > > their own file listing and signatures and such. > > Ok, that's a reasonable requirement. It will be difficult to implement, > though, as it will require Linux distributors (in particular) to include > the database snippets in their packages. > > Essentially, one would have to contribute patches to all the > distributions (we care about, at least), and then nag the respective > maintainers to include these patches. Not true. You just need to make sure that "setup.py install" creates that database. With the proposed format of the database this is just a file in the correct location - exactly for this reason. Next time the distribution will build the package that database file will be in place. I still maintain that an installdb for the system packages doesn't bring anything to the party as it would be in a system-only directory. But I've argued that in my other emails... Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From floris.bruynooghe at gmail.com Sat Mar 22 16:21:50 2008 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Sat, 22 Mar 2008 15:21:50 +0000 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E5142D.7010902@v.loewis.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> <47E4EE9D.2090605@v.loewis.de> <20080322132418.GB8513@laurie.devork> <47E5142D.7010902@v.loewis.de> Message-ID: <20080322152150.GA16277@laurie.devork> On Sat, Mar 22, 2008 at 03:14:05PM +0100, "Martin v. L?wis" wrote: >>> Essentially, one would have to contribute patches to all the >>> distributions (we care about, at least), and then nag the respective >>> maintainers to include these patches. >> >> Not true. You just need to make sure that "setup.py install" creates >> that database. With the proposed format of the database this is just >> a file in the correct location - exactly for this reason. Next time >> the distribution will build the package that database file will be in >> place. > > How so? Are you /sure/ the packaging process even *runs* setup.py? > And if they do, why do you think they will pick up the database > file? I speak for Debian, so for Debian: yes. The setup.py would have to be pretty bad for a packager to not use it. There is no reason to re-write upstream's installation procedure as you would have to figure out which files need to be installed where and this would create many bugs. The canonical way is something like this: $ pythonX.Y setup.py --root=$(pwd)/debian/tmp $ # Fixup anything done wrong/badly (for debian) by setup.py $ # Make a tarball of $(pwd)/debian/tmp In reality it's slightly more complicated of course. At [1] there are many packages, paste and parallelpython are good examples if you're interested (look in the debian/rules file). Regards Floris [1] http://svn.debian.org/wsvn/python-modules/packages/?rev=0&sc=0 -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From floris.bruynooghe at gmail.com Sat Mar 22 18:13:12 2008 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Sat, 22 Mar 2008 17:13:12 +0000 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E528EC.7010605@v.loewis.de> References: <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> <47E4EE9D.2090605@v.loewis.de> <20080322132418.GB8513@laurie.devork> <47E5142D.7010902@v.loewis.de> <20080322152150.GA16277@laurie.devork> <47E528EC.7010605@v.loewis.de> Message-ID: <20080322171312.GA19074@laurie.devork> On Sat, Mar 22, 2008 at 04:42:36PM +0100, "Martin v. L?wis" wrote: >> I speak for Debian, so for Debian: yes. The setup.py would have to be >> pretty bad for a packager to not use it. There is no reason to >> re-write upstream's installation procedure as you would have to figure >> out which files need to be installed where and this would create many >> bugs. >> >> The canonical way is something like this: >> >> $ pythonX.Y setup.py --root=$(pwd)/debian/tmp >> $ # Fixup anything done wrong/badly (for debian) by setup.py >> $ # Make a tarball of $(pwd)/debian/tmp >> >> In reality it's slightly more complicated of course. > > More than slightly so, with pycentral, pysupport, and all that. > > I still doubt that the database would show up in a directory on > sys.path. If not it would be a bug in pycentral/pysupport. Only two bugs to file, not that bad I think! >> At [1] there are >> many packages, paste and parallelpython are good examples if you're >> interested (look in the debian/rules file). > > I started with nevow. It uses cdbs, so its debian/rules is nearly > empty, and does not include a call to setup.py at all. > > Instead, the distutils support is in > > /usr/share/cdbs/1/class/python-distutils.mk [...] Again, that would be a bug in CDBS. For the specific snippet you showed, yes that does essentially "pythonX.Y setup.py --root=$some_alternate_root" as I said above. As an aside I also happen to be in the camp that dislikes CDBS... The specifics and complications don't matter for this discussion I think. If setup.py installs the correct file into the installdb then it will work in almost all cases, Neal Becker confirmed this is true for Fedora as well. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From ndbecker2 at gmail.com Sat Mar 22 17:00:18 2008 From: ndbecker2 at gmail.com (Neal Becker) Date: Sat, 22 Mar 2008 12:00:18 -0400 Subject: [Python-Dev] =?utf-8?q?=5BDistutils=5D_How_we_can_get_rid_of_eggs?= =?utf-8?q?_for_2=2E6=09and_beyond?= In-Reply-To: <47E52984.3070407@v.loewis.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E52984.3070407@v.loewis.de> Message-ID: <200803221200.18772.ndbecker2@gmail.com> On Saturday 22 March 2008, Martin v. L?wis wrote: > > In the case of Fedora rpms, the usual install uses setup.py. > > Ok. Does it then also package all files that get installed into > the RPM file? If it produces multiple RPMs from a single source > package, how does it know which files go into what RPM? > > Regards, > Martin Offhand, I can't think of any examples that produce multiple RPMS, but in general, it doesn't _know_ anything. What goes in what rpm is either 1) Automated using python setup.py install -O1 --root $RPM_BUILD_ROOT --prefix %{_prefix} --record=%{name}.files or 2) Manually listing the files that go into a package. From pwang at enthought.com Thu Mar 20 17:46:36 2008 From: pwang at enthought.com (Peter Wang) Date: Thu, 20 Mar 2008 11:46:36 -0500 Subject: [Python-Dev] [Distutils] PEP 365 (Adding the pkg_resources module) In-Reply-To: <47E29176.9050109@v.loewis.de> References: <79990c6b0803171319k336110cckdc7563d6ff9b387a@mail.gmail.com> <87r6e7vjzy.fsf@uwakimon.sk.tsukuba.ac.jp> <20080318223700.C64123A4074@sparrow.telecommunity.com> <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <47E26A20.1090402@palladion.com> <47E29176.9050109@v.loewis.de> Message-ID: On Mar 20, 2008, at 11:31 AM, Martin v. L?wis wrote: >> I'll note that I use easy_install *only* to work in *non-system* >> locations: if I want to install Python packages to /usr/lib/ >> python2.x/, >> I use the standard system installer, e.g. 'apt-get install >> python-frobnatz'. > > This is probably not the Windows way of doing things (at least not how > I use Windows). Windows doesn't really have the notion of "system > location" (or multiple levels of them, where \Windows and > \Windows\system32 is "more system" than \Program Files, say). > > Windows users typically view the entire system as "theirs", and > have no concerns at all about putting things into Program Files, > system32, or, for that matter, \python25. In fact, they don't care > or even know where stuff ends up - they expect that the system will > find it, and they expect that they can remove anything they installed > even without known where it is - because there is a standard place > to look for uninstalling things. While these observations are accurate for most home users, it is worth noting that many IT departments deploy locked-down versions of windows that either have fine-grained group policies to forbid modifications to the system disk (and require the user to write things to a mounted network home directory), or that give write access to the system disk but then re-image it upon reboot. IT departments that deploy this sort of setup usually have the "hostile user" mentality, and that is strongly correlated, in turn, with users that are reluctant to engage IT to allow them install an app. We have run into this a few times, and it would be good to keep this scenario in mind. -Peter From lxander.m at gmail.com Sat Mar 22 14:21:18 2008 From: lxander.m at gmail.com (Alexander Michael) Date: Sat, 22 Mar 2008 09:21:18 -0400 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E46176.4090205@v.loewis.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> Message-ID: <525f23e80803220621u4e826023u975048a45b76a079@mail.gmail.com> On Fri, Mar 21, 2008 at 9:31 PM, "Martin v. L?wis" wrote: > The objections to the PEP remain the same as they were then, > though: In the requirements, it says "we need", without saying > why we need. It then goes on saying "we want" (rephrased) > "to duplicate APT and RPM", without saying why we want that, > or why that brings us closer to what we need. > > IOW, the PEP is lacking a rationale. It seems to me that this discussion is being undermined by not acknowledging the many use cases up front. There is no rationale because there are too many tacit rationales. Nevertheless, the many use cases are related at some level and would benefit from some form of lowest-common-denominator support in the standard library. As an example, IF I wanted to use a library to support multi-version packages or one to ensure I had all the dependencies, I would need to know what versions of things were currently installed. I can't create a library to provided these functionalities and use it downstream of the package creator without some form of standardization to report package versions. (Or at least without running into the assimilation problem that setuptools has). My personal use case is for multi-version packages. I deploy many small scripts (not applications) that use an evolving set of base libraries. I don't want the older scripts to hold me back from pushing forward with the base libraries, so I peg the scripts to their respective major versions as I release them. Regards, Alex From lxander.m at gmail.com Mon Mar 24 21:57:44 2008 From: lxander.m at gmail.com (Alexander Michael) Date: Mon, 24 Mar 2008 16:57:44 -0400 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E51179.5040601@v.loewis.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <525f23e80803220621u4e826023u975048a45b76a079@mail.gmail.com> <47E51179.5040601@v.loewis.de> Message-ID: <525f23e80803241357r63b091c1r5b5f46c9734b13ce@mail.gmail.com> On Sat, Mar 22, 2008 at 10:02 AM, "Martin v. L?wis" wrote: > > It seems to me that this discussion is being undermined by not > > acknowledging the many use cases up front. There is no rationale > > because there are too many tacit rationales. > > I honestly, really, cannot imagine what those are. Explicit is > better than implicit. Exactly. I was attempting to suggest that we be explicit about "the problem" being solved so it was easier for everyone interested to verify that their "needs" (i.e. use case) will be met and that the proposed solution solves the problem. Since the topic of the thread is how to get rid of eggs in 2.6, the question (to me, at least) is how to enable the functionality provided by setuptools to be provided outside the standard library in a way that does not create pressure for the entire python community to be assimilated into egg-ified zombies. Why do we have the current pressure for egg-ification? The main pressure appears to arise from a strong desire by the python community for a simple installation tool like CPAN that downloads and installs the package of interest *as well as* any dependencies you don't already have installed. Of course, it is the desire for the dependencies to be discovered, downloaded, and installed in a manner that honors the current state of your python environment that creates all of the problems. My participation may be unwanted or unwarranted due to my lack of education/understanding, but I like to live dangerously in the hopes of actually being helpful. With that preamble, here's my attempt at an explicit rationale for a database of installed packages (A.K.A. The New PEP 262): """ Rationale ========= It is often necessary during the course of managing a python installation for an administrator to determine the following things: 1. If the current installation state satisfies the requirements of a new package being considered for installation. 2. If it is safe for the administrator to upgrade or remove a package, and if so, how (e.g. use a system-level package management tool). 3. What files to remove in order to uninstall the package (if a system-level package management tool was not used to install it). 4. If the current installation was modified in-place or custom configured in an way, so that such changes can be noted before upgrading. Furthermore, many administrators want to do as much of this as possible with automated tools, without manually inspecting README files, INSTALL files, or the installed code itself to determine the list of dependencies and installed versions, so there is a desire to be able to make the above determinations programmatically. Current efforts to provide these capabilities without standard library support have resulted in many users being forced to use non-standard package management tools because other users desired these capabilities. This proposal is motivated by a desire to provide the minimum required infrastructure so that both segments of the python community can peacefully coexist without getting in each others way (i.e. the ability to "opt in" to python-based non-system-level package managers). Proposal ======== The proposal is to provide in the standard library the following capabilities: 1. List the installed packages, along with the version and dependency list for each package. 2. Query the ownership of a currently installed package (standard library, system-level package management tool, etc.). 3. List the files installed be specific package. 4. Recall the original message digest for each installed file to determine if the file has been modified. ... Alternatives ============ There are at least three alternatives to providing a database of installed packages that could also potentially enable administrators to accomplish the same ultimate goal of installing new packages and their dependencies without breaking or interfering with the "system" installation. Switchable Environments ----------------------- Eliminate the need to be careful by providing a mechanism for the creation of and installation into what amounts to user-selectable site-packages directories that each "inherit" from what is currently called site-packages (something like an eggless virtualenv with a python comannd-line option to choose the environment). Need to upgrade a package provided by the standard library or your system-level package manager? No Problem! Of course, this won't help you manage your sand-boxed environment, but you can have lots of them for very little cost (it's not a whole new python installation) so just create a new one each time you want to change something. Ubiquitous System Packages -------------------------- Eliminate the need to *not* use your OS's system-level package manager by creating the necessary infrastructure so that python packages can easily be distributed in the major system-level packager formats (RPM, DEB, MSI, MPKG, etc.) for all publicly released python packages. A windows-based developer who releases their pure-python package should be able to create RPMs, DEBs, etc. automatically (using distutils, for instance) without access to a computer running the appropriate OS (python setup.py bdist_all upload). Remote Hosted Installation Database ----------------------------------- Eliminate the requirement that your installation be self-describing by maintaining the information in PyPI necessary for one to figure out what versions of what things are installed in your system (either by recipes or a table of message digest values for the directories) in addition the dependencies and list of installed files for each version of each package thats ever been released. """ Regards, Alex From mhammond at skippinet.com.au Tue Mar 25 05:15:35 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Tue, 25 Mar 2008 15:15:35 +1100 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E8276B.8030800@palladion.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> <20080322020447.3B1033A4074@sparrow.telecommunity.com> <47E4EE9D.2090605@v.loewis.de> <20080322151918.D9EBC3A40B1@sparrow.telecommunity.com> <47E525F5.40705@v.loewis.de> <20080322154449.BDA423A40A5@sparrow.telecommunity.com> <47E52F70.1010806@v.loewis.de> <47E8276B.8030800@palladion.com> Message-ID: <014c01c88e2e$e4aad5a0$ae0080e0$@com.au> > (Note: I'm aware that people believe it to be necessary to munge the > Windows registry when installing Python packages; I just don't agree > with the practice, and don't think we should distort Python's process > to coddle it). Whoever thinks it necessary is misguided. Installing a package doesn't require munging the registry and none of the popular installation techniques do. Installing Python itself does, and some packages have special requirements (eg, pywin32 registering COM objects), but in general, Python packages on Windows can ignore the registry. Mark From mnordhoff at mattnordhoff.com Tue Mar 25 05:24:53 2008 From: mnordhoff at mattnordhoff.com (Matt Nordhoff) Date: Tue, 25 Mar 2008 04:24:53 +0000 Subject: [Python-Dev] [Python-3000] Python source code on Bazaar vcs In-Reply-To: <18408.27695.339064.649345@montanaro-dyndns-org.local> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <18408.27695.339064.649345@montanaro-dyndns-org.local> Message-ID: <47E87E95.6090500@mattnordhoff.com> skip at pobox.com wrote: > Barry> All the gory details are documented here: > > Barry> http://www.python.org/dev/bazaar > > Thanks. I checked out, made a branch named test3, changed Makefile.pre.in > to have a test3 target, checked it in, then tried to push it: > > % pwd > /Users/skip/src/python-bzr/test3 > % bzr push bzr+ssh://pythonbzr at code.python.org/python/users/skip/test3 > bzr: ERROR: Parent directory of bzr+ssh://pythonbzr at code.python.org/python/users/skip/test3 does not exist. > You may supply --create-prefix to create all leading parent directories. > > Did I misread the directions or do I really need the --create-prefix arg? > > Skip I don't know if there's a special setup here, but doesn't list a "skip" directory yet. You'll need to use --create-prefix to get bzr to create it. -- From nad at acm.org Tue Mar 25 06:16:18 2008 From: nad at acm.org (Ned Deily) Date: Mon, 24 Mar 2008 22:16:18 -0700 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <47E4DE8E.6010809@holdenweb.com> <79990c6b0803220646r50e9b455y5bd9b955814f334a@mail.gmail.com> <47E53435.4090104@v.loewis.de> <79990c6b0803221256jf4e6ea1qce463ab711dcdb51@mail.gmail.com> <47E56864.4080201@v.loewis.de> Message-ID: In article , ajaksu wrote: > [...] While Linux and OS X both view Python as essentially a first-class > development platform-i.e., as something that shrink-wrap applications > can be built on-Windows does not. Instead, it's generally expected > that a Python-based Windows application be "frozen": bundled into a > self-contained package that includes a copy of the Python interpreter > and whatever libraries it uses, which are private to the particular > application. While this ensures that the application will function as > expected and not run into 'dependency hell', it also results in a > relatively large download-distributing a simple 'Hello World' program > is at least a megabyte in size, and makes extending the program's > functionality more difficult." While it is true that OS X does provide a built-in Python that can be used as a shared component for shrink-wrap applications, it should be noted that py2app, the de facto standard tool for packaging Python apps on OS X, by default includes a private copy of the Python interpreter and library within the application bundle for similar reasons, i.e. avoiding "dependency hell". py2app does, however, go to some trouble to analyze dependencies and include only a minimal set of required packages. -- Ned Deily, nad at acm.org From g.brandl at gmx.net Tue Mar 25 08:58:23 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 25 Mar 2008 08:58:23 +0100 Subject: [Python-Dev] Commit access request In-Reply-To: <1afaf6160803240456u7ce0aa53t19514771e3434b30@mail.gmail.com> References: <1afaf6160803240456u7ce0aa53t19514771e3434b30@mail.gmail.com> Message-ID: Benjamin Peterson schrieb: > Hi Python devs, > I have been contributing to since December. (See me first issue on the > tracker, #1828; it was a major learning experience.) :P In that time, I > have contributed many patches and actively participated on this list. > This will enable me to help triage bugs on the tracker, something I've > been wanting to help with. I'm +1 to Benjamin getting commit privileges, letting commits sign off by a senior developer. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From mal at egenix.com Tue Mar 25 10:43:33 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 25 Mar 2008 10:43:33 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <319e029f0803240122n1c40a47cr4e10e0c74fdaccca@mail.gmail.com> References: <47E6E445.60305@v.loewis.de> <20080324013051.6859.1806027896.divmod.quotient.22131@ohm> <319e029f0803240122n1c40a47cr4e10e0c74fdaccca@mail.gmail.com> Message-ID: <47E8C945.80101@egenix.com> On 2008-03-24 09:22, Lennart Regebro wrote: > I think 2to3 is a procedure that will work well for library type > projects with a reasonably small set of developers that make regular > releases. There you can release both a python 2 and a python 3 version > of the module, for example. > ... > So, in short: Large projects with interconnected modules where the > developers and users of module are the same people will have big > difficulties with the 2to3 approach and would be the people who are > most likely to not be able to in practice go forward to Python 3 > unless they have some sort of smooth path forward. I don't think there's a lot to worry about: Companies using Python for applications typically have a completely different life-cycle of releases and applications compared to the Python release schedule, i.e. they often still run Python 2.3 or 2.4 and wait for major releases to settle before deciding to port to them. Every now and then, they make the decision to port to the next release (for the next version of their software) and this change is then managed accordingly - sometimes skipping a complete major release of Python. In such projects, 2to3 will get applied to the sources once and then all development continues on the Python 3.0 version of the code. In reality, I don't think that 2to3 will get used for continuous porting between a 2.x code base and a 3.0 one all that much. The transition from 2.x to 3.0 will happen during a longer period of time (probably a few years) and depend a lot on the release cycle of the applications using Python, whether or not the 3.0 version provides better features, more performance, etc. and whether the 2.x branches of Python and the used 3rd party modules are still supported or not. New applications will likely choose 3.0 right away - provided that the needed 3rd party modules are available and stable enough. In summary: 2to3 is a very useful tool to have. Whether or not it is used for continuous porting between the two worlds is really secondary. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 25 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From martin at v.loewis.de Mon Mar 24 10:23:15 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 24 Mar 2008 10:23:15 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <319e029f0803240110mf3a72d3hc8eb5ea3982ca7ed@mail.gmail.com> References: <319e029f0803230037s76af8233hbefbfacccf77b930@mail.gmail.com> <9e804ac0803230613k38683ee6m41e7a0aa18275d7d@mail.gmail.com> <319e029f0803231311r4bd6031ftfc918ec0a785d4d7@mail.gmail.com> <47E6CF65.1020900@v.loewis.de> <319e029f0803231517t259b84d5sa3c2071873eb6399@mail.gmail.com> <47E6E445.60305@v.loewis.de> <319e029f0803240110mf3a72d3hc8eb5ea3982ca7ed@mail.gmail.com> Message-ID: <47E77303.1000101@v.loewis.de> Lennart Regebro wrote: > On Mon, Mar 24, 2008 at 12:14 AM, "Martin v. L?wis" wrote: >>> You are still only seeing this as a case of libraries with a small >> > number of people developing them and making regular well defined >> > releases. That is not how the world I am talking about looks. >> >> Can you give me examples of such software? > > I'll repeat the link where I explained my point on this: > http://regebro.wordpress.com/2008/03/22/why-python-26-and-30-compatibility-would-be-a-very-good-thing/ Yet, there you merely claim that such software exists ("within larger organizations", and "within large communities like Zope and Plone"), without giving specific examples. This is not very convincing. Regards, Martin From amk at amk.ca Tue Mar 25 12:00:51 2008 From: amk at amk.ca (A.M. Kuchling) Date: Tue, 25 Mar 2008 07:00:51 -0400 Subject: [Python-Dev] Py3k and asyncore/asynchat In-Reply-To: References: <20080215142414.GA6805@amk-desktop.matrixgroup.net> <20080215142832.GC13291@snowy.squish.net> <9e804ac0803241404u5d21afa6oab0c7d1e2e49166f@mail.gmail.com> <20080324221823.GA24766@amk-desktop.matrixgroup.net> Message-ID: <20080325110051.GA2118@amk.local> On Mon, Mar 24, 2008 at 03:51:56PM -0700, Josiah Carlson wrote: > reasonable question; 2.6 alpha 1 has been released. It's a question > as to whether someone with commit access can or will commit the patch > as posted, run the tests to verify that they aren't broken, and > perhaps actually look at the code to verify that we (Giampaolo and I) > aren't insane. Mind you, I've been using very similar variants of I think we should just give you commit access so that you can commit changes to asyncore/asynchat yourself; it doesn't seem as if any of the committers use asyncore enough to check patches for it. --amk From regebro at gmail.com Tue Mar 25 12:10:25 2008 From: regebro at gmail.com (Lennart Regebro) Date: Tue, 25 Mar 2008 12:10:25 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <47E77303.1000101@v.loewis.de> References: <319e029f0803230037s76af8233hbefbfacccf77b930@mail.gmail.com> <9e804ac0803230613k38683ee6m41e7a0aa18275d7d@mail.gmail.com> <319e029f0803231311r4bd6031ftfc918ec0a785d4d7@mail.gmail.com> <47E6CF65.1020900@v.loewis.de> <319e029f0803231517t259b84d5sa3c2071873eb6399@mail.gmail.com> <47E6E445.60305@v.loewis.de> <319e029f0803240110mf3a72d3hc8eb5ea3982ca7ed@mail.gmail.com> <47E77303.1000101@v.loewis.de> Message-ID: <319e029f0803250410p77590dbfhb6c2db205a649260@mail.gmail.com> On Mon, Mar 24, 2008 at 10:23 AM, "Martin v. L?wis" wrote: > Yet, there you merely claim that such software exists ("within larger > organizations", and "within large communities like Zope and Plone"), > without giving specific examples. No I don't. Here is what I said: "In many other cases, this is not how code is developed. Both within larger organisations and within large communities like Zope and Plone". I don't think chewing through this issue once more is meaningful, so I'll stop now. -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From phd at phd.pp.ru Tue Mar 25 14:46:17 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Tue, 25 Mar 2008 16:46:17 +0300 Subject: [Python-Dev] Decimal(unicode) Message-ID: <20080325134617.GB2851@phd.pp.ru> Hello. In Python 2.5.1 the code import decimal for d in '123', u'123': x = decimal.Decimal(d) print type(x.to_eng_string()) prints In 2.5.2 it prints Why the change? Is it a bug or a feature? Shouldn't .to_eng_string() always return a str? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From barry at python.org Tue Mar 25 15:35:49 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 25 Mar 2008 10:35:49 -0400 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: <18408.27695.339064.649345@montanaro-dyndns-org.local> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <18408.27695.339064.649345@montanaro-dyndns-org.local> Message-ID: <08E8188C-AEA2-4E78-B74C-AF213757DEA0@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 24, 2008, at 11:06 PM, skip at pobox.com wrote: > Barry> All the gory details are documented here: > > Barry> http://www.python.org/dev/bazaar > > Thanks. I checked out, made a branch named test3, changed > Makefile.pre.in > to have a test3 target, checked it in, then tried to push it: > > % pwd > /Users/skip/src/python-bzr/test3 > % bzr push bzr+ssh://pythonbzr at code.python.org/python/users/skip/ > test3 > bzr: ERROR: Parent directory of bzr+ssh:// > pythonbzr at code.python.org/python/users/skip/test3 does not exist. > You may supply --create-prefix to create all leading parent > directories. > > Did I misread the directions or do I really need the --create-prefix > arg? You do, the first time you push a user branch because users/skip doesn't exist yet. It's mentioned in the docs, but it's pretty easy to overlook ;). - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBR+kNxXEjvBPtnXfVAQJMWQP8DJrB6QbXt7O/lT2Y/CzxpuEqa09rOSGq QxE6RDro/k4dvuwRGh3WKjo+AwKcSSTsUAp48o3O1fM7X3u54JxhtbwoEeZj7xnv 9Nw6ZCpN+DDY83QAtNjkWtSkNfRaL3K1Y4gZsnsxsIDGs3y0HV5a2n8vZh18b+gV xfJzxu+hb/o= =BsR5 -----END PGP SIGNATURE----- From dickinsm at gmail.com Tue Mar 25 15:47:42 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 25 Mar 2008 10:47:42 -0400 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <20080325134617.GB2851@phd.pp.ru> References: <20080325134617.GB2851@phd.pp.ru> Message-ID: <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> On Tue, Mar 25, 2008 at 9:46 AM, Oleg Broytmann wrote: > In 2.5.2 it prints > > > > > Why the change? Is it a bug or a feature? Shouldn't .to_eng_string() > always return a str? I'd call this a bug. The change is an accident, a side-effect of the fact that in 2.5.1 the coefficient (mantissa) of a Decimal was stored as a tuple, and in 2.5.2 it's stored as a string (which greatly improves efficiency). Clearly in 2.5.2 the mantissa is being stored as a unicode instance in the second case; it should be explicitly coerced to str in Decimal.__new__. If others agree that it's a bug, I'll fix it. Mark From dickinsm at gmail.com Tue Mar 25 15:48:29 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 25 Mar 2008 10:48:29 -0400 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <20080325134617.GB2851@phd.pp.ru> References: <20080325134617.GB2851@phd.pp.ru> Message-ID: <5c6f2a5d0803250748r43475cbdo501bf7efec84ff15@mail.gmail.com> On Tue, Mar 25, 2008 at 9:46 AM, Oleg Broytmann wrote: > In 2.5.2 it prints > > > > > Why the change? Is it a bug or a feature? Shouldn't .to_eng_string() > always return a str? I'd call this a bug. The change is an accident, a side-effect of the fact that in 2.5.1 the coefficient (mantissa) of a Decimal was stored as a tuple, and in 2.5.2 it's stored as a string (which greatly improves efficiency). Clearly in 2.5.2 the mantissa is being stored as a unicode instance in the second case; it should be explicitly coerced to str in Decimal.__new__. If others agree that it's a bug, I'll fix it. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080325/7d54fcd3/attachment.htm From facundobatista at gmail.com Tue Mar 25 16:00:35 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Tue, 25 Mar 2008 12:00:35 -0300 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> Message-ID: 2008/3/25, Mark Dickinson : > > I'd call this a bug. The change is an accident, a side-effect of the fact > that in 2.5.1 the coefficient (mantissa) of a Decimal was stored as a > tuple, and in 2.5.2 it's stored as a string (which greatly improves efficiency). > Clearly in 2.5.2 the mantissa is being stored as a unicode instance in the > second case; it should be explicitly coerced to str in Decimal.__new__. > > If others agree that it's a bug, I'll fix it. > +1 -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From ncoghlan at gmail.com Tue Mar 25 16:29:51 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 26 Mar 2008 01:29:51 +1000 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> Message-ID: <47E91A6F.7090601@gmail.com> Mark Dickinson wrote: > On Tue, Mar 25, 2008 at 9:46 AM, Oleg Broytmann wrote: >> In 2.5.2 it prints >> >> >> >> >> Why the change? Is it a bug or a feature? Shouldn't .to_eng_string() >> always return a str? > > I'd call this a bug. The change is an accident, a side-effect of the fact > that in 2.5.1 the coefficient (mantissa) of a Decimal was stored as a > tuple, and in 2.5.2 it's stored as a string (which greatly improves efficiency). > Clearly in 2.5.2 the mantissa is being stored as a unicode instance in the > second case; it should be explicitly coerced to str in Decimal.__new__. > > If others agree that it's a bug, I'll fix it. I thought that might be what happened, but I couldn't remember if that optimisation was a 2.6 only change or not (I suspect it was included in 2.5 as a prereq to the spec compliance updates). Anyway, +1 on coercing the mantissa to a str() instance in 2.5. This does raise an interesting point though - currently Decimal in Py3k is storing the mantissa as a Unicode instance instead of a bytes instance. The performance implications of that are horrendous since PyLong_FromUnicode does a malloc, encodes the string into the malloced buffer, then invokes PyLong_FromString on the result. To fix this, decimal probably needs to grow something like the following near the top of the module: try: _bytes = bytes except NameError: # 2.5 or earlier _bytes = str and then use _bytes instead of str as appropriate throughout the rest of the module. The following is also a problem in Py3k: >>> from decimal import Decimal as d >>> d(1) Decimal('1') >>> d('1') Decimal('1') >>> d(b'1') Traceback (most recent call last): File "", line 1, in File "/home/ncoghlan/devel/py3k/Lib/decimal.py", line 659, in __new__ raise TypeError("Cannot convert %r to Decimal" % value) TypeError: Cannot convert b'1' to Decimal The isinstance(value, str) check in Py3k is too restrictive - it needs to accept bytes instances as well. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From phd at phd.pp.ru Tue Mar 25 16:38:25 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Tue, 25 Mar 2008 18:38:25 +0300 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> Message-ID: <20080325153825.GA8488@phd.pp.ru> On Tue, Mar 25, 2008 at 10:47:42AM -0400, Mark Dickinson wrote: > On Tue, Mar 25, 2008 at 9:46 AM, Oleg Broytmann wrote: > > In 2.5.2 it prints > > > > > > > > > > Why the change? Is it a bug or a feature? Shouldn't .to_eng_string() > > always return a str? > > I'd call this a bug. The change is an accident, a side-effect of the fact > that in 2.5.1 the coefficient (mantissa) of a Decimal was stored as a > tuple, and in 2.5.2 it's stored as a string (which greatly improves efficiency). > Clearly in 2.5.2 the mantissa is being stored as a unicode instance in the > second case; it should be explicitly coerced to str in Decimal.__new__. > > If others agree that it's a bug, I'll fix it. http://bugs.python.org/issue2482 Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From dickinsm at gmail.com Tue Mar 25 16:41:06 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 25 Mar 2008 11:41:06 -0400 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47E91A6F.7090601@gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> Message-ID: <5c6f2a5d0803250841o41cd8af6le263c7f6a1fb79cc@mail.gmail.com> On Tue, Mar 25, 2008 at 11:29 AM, Nick Coghlan wrote: > I thought that might be what happened, but I couldn't remember if that > optimisation was a 2.6 only change or not (I suspect it was included in > 2.5 as a prereq to the spec compliance updates). > Exactly. > Anyway, +1 on coercing the mantissa to a str() instance in 2.5. > > This does raise an interesting point though - currently Decimal in Py3k > is storing the mantissa as a Unicode instance instead of a bytes > instance. The performance implications of that are horrendous since > PyLong_FromUnicode does a malloc, encodes the string into the malloced > buffer, then invokes PyLong_FromString on the result. > Urk! Yes, this definitely needs to be looked at. > The following is also a problem in Py3k: > [...] > > The isinstance(value, str) check in Py3k is too restrictive - it needs > to accept bytes instances as well. > I don't understand this. Why does Decimal.__new__ need to accept bytes instances? Isn't it just supposed to be creating a Decimal from a string? Or is the idea that it should accept ASCII strings that are masquerading as bytes instances, for reasons of convenience/efficiency/something else? Unicode/bytes/str and encoding issues frighten me much more than floating-point arithmetic ever did, so I expect I'm missing many things here. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080325/848424ca/attachment-0001.htm From facundobatista at gmail.com Tue Mar 25 16:51:20 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Tue, 25 Mar 2008 12:51:20 -0300 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47E91A6F.7090601@gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> Message-ID: 2008/3/25, Nick Coghlan : > > Anyway, +1 on coercing the mantissa to a str() instance in 2.5. > I don't know about 2.5, I'm sure about 2.6. > To fix this, decimal probably needs to grow something like the following > near the top of the module: > > try: > _bytes = bytes > except NameError: # 2.5 or earlier > _bytes = str > > and then use _bytes instead of str as appropriate throughout the rest of > the module. +1, I updated the bug created by Oleg. > The isinstance(value, str) check in Py3k is too restrictive - it needs > to accept bytes instances as well. Why? The number in a string should be just strings, IMHO, not bytes... do you have a case of use for this? Thanks! -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From dickinsm at gmail.com Tue Mar 25 16:53:03 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 25 Mar 2008 11:53:03 -0400 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47E91A6F.7090601@gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> Message-ID: <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> On Tue, Mar 25, 2008 at 11:29 AM, Nick Coghlan wrote: > The isinstance(value, str) check in Py3k is too restrictive - it needs > to accept bytes instances as well. > Hmm. There's not a lot of consistency here: >>> int(b'1') 1 >>> float(b'1') 1.0 >>> complex(b'1') Traceback (most recent call last): File "", line 1, in TypeError: complex() argument must be a string or a number >>> from fractions import Fraction >>> Fraction(b'1') Traceback (most recent call last): File "", line 1, in File "/Users/dickinsm/python_source/py3k/Lib/fractions.py", line 98, in __new__ numerator = numerator.__index__() AttributeError: 'bytes' object has no attribute '__index__' So int and float accepts bytes, while complex, Decimal and Fraction do not... Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080325/41694f3f/attachment.htm From facundobatista at gmail.com Tue Mar 25 16:57:50 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Tue, 25 Mar 2008 12:57:50 -0300 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> Message-ID: 2008/3/25, Mark Dickinson : > So int and float accepts bytes, while complex, Decimal and Fraction do > not... I'm -1 to accept bytes as input for Decimal, I don't see a case of use, and I think that conceptually there's no reason to do it. Of course, I can be wrong, ;) Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From dickinsm at gmail.com Tue Mar 25 17:28:29 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 25 Mar 2008 12:28:29 -0400 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> Message-ID: <5c6f2a5d0803250928w14b4fd25x5cea0a0e3f6884@mail.gmail.com> On Tue, Mar 25, 2008 at 11:57 AM, Facundo Batista wrote: > 2008/3/25, Mark Dickinson : > > > So int and float accepts bytes, while complex, Decimal and Fraction do > > not... > > I'm -1 to accept bytes as input for Decimal, I don't see a case of > use, and I think that conceptually there's no reason to do it. > I've opened http://bugs.python.org/issue2483 to keep track of this. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080325/091b83bb/attachment.htm From ncoghlan at gmail.com Tue Mar 25 17:43:10 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 26 Mar 2008 02:43:10 +1000 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> Message-ID: <47E92B9E.3000003@gmail.com> Facundo Batista wrote: > 2008/3/25, Mark Dickinson : > >> So int and float accepts bytes, while complex, Decimal and Fraction do >> not... > > I'm -1 to accept bytes as input for Decimal, I don't see a case of > use, and I think that conceptually there's no reason to do it. > > Of course, I can be wrong, ;) I was thinking converting directly from bytes would be significantly quicker than going through Unicode (e.g. for numbers read from a file), but that may not actually be the case (it'll definitely be faster because there is less data copying and movement involved, but the speed difference may be less dramatic than I first thought). So while the internal storage of the mantissa definitely needs to be changed to a bytes object in order to retain Mark's hard-won performance improvements, the case of whether or not to accept bytes is far less clear. The way I see it either complex, Decimal and Fraction all need to be updated to accept bytes objects, or else int and float need to be updated to reject them. It *definitely* needs to be possible to convert bytes objects to integers as if they were ASCII strings - otherwise a lot of wire protocol processing would become a nightmare. Indeed, the proposed change to Decimal to have it store the mantissa as a bytes object in Py3k assumes that it will still be possible to convert that value directly to a long object. Since we have some strong use cases at least for the bytes->int case, consistency then suggests that the other numeric types should all accept bytes as well (interpreting them as ASCII encoded strings). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From aleaxit at gmail.com Tue Mar 25 18:00:07 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Tue, 25 Mar 2008 10:00:07 -0700 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47E92B9E.3000003@gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E92B9E.3000003@gmail.com> Message-ID: On Tue, Mar 25, 2008 at 9:43 AM, Nick Coghlan wrote: ... > Since we have some strong use cases at least for the bytes->int case, > consistency then suggests that the other numeric types should all accept > bytes as well (interpreting them as ASCII encoded strings). +1 -- it seems very practical as well as consistent, and I see no downsides. Alex From g.brandl at gmx.net Tue Mar 25 18:40:13 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 25 Mar 2008 18:40:13 +0100 Subject: [Python-Dev] Commit access request In-Reply-To: References: <1afaf6160803240456u7ce0aa53t19514771e3434b30@mail.gmail.com> Message-ID: Georg Brandl schrieb: > Benjamin Peterson schrieb: >> Hi Python devs, >> I have been contributing to since December. (See me first issue on the >> tracker, #1828; it was a major learning experience.) :P In that time, I >> have contributed many patches and actively participated on this list. >> This will enable me to help triage bugs on the tracker, something I've >> been wanting to help with. > > I'm +1 to Benjamin getting commit privileges, letting commits sign off > by a senior developer. Since there are no objections, I've now added Benjamin's key, under that condition. Welcome on board, Benjamin! :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From greg at krypto.org Tue Mar 25 19:00:21 2008 From: greg at krypto.org (Gregory P. Smith) Date: Tue, 25 Mar 2008 11:00:21 -0700 Subject: [Python-Dev] Two questions about jump opcodes In-Reply-To: References: <20080322225842.1E5C63A40D9@sparrow.telecommunity.com> Message-ID: <52dc1c820803251100w141d271cv22d4dfa0c795e73c@mail.gmail.com> This across the board speedup of the python byte code looks promising! I'm not familiar enough with that part of the code to review it but here's a big +1 to make sure someone else takes a look. On Sat, Mar 22, 2008 at 4:07 PM, Antoine Pitrou wrote: > > Wow, thanks to both of you (Phillip & Skip) for the fast answers. > Just in case anyone has time to spare, I have more pesky questions (and a > working patch :-)) in the aforementioned bug entry > (http://bugs.python.org/issue2459). > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/greg%40krypto.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080325/847b6827/attachment.htm From skip at pobox.com Tue Mar 25 19:05:17 2008 From: skip at pobox.com (skip at pobox.com) Date: Tue, 25 Mar 2008 13:05:17 -0500 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: <08E8188C-AEA2-4E78-B74C-AF213757DEA0@python.org> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <18408.27695.339064.649345@montanaro-dyndns-org.local> <08E8188C-AEA2-4E78-B74C-AF213757DEA0@python.org> Message-ID: <18409.16093.799920.286191@montanaro-dyndns-org.local> >> Did I misread the directions or do I really need the --create-prefix >> arg? Barry> You do, the first time you push a user branch because users/skip Barry> doesn't exist yet. It's mentioned in the docs, but it's pretty Barry> easy to overlook ;). Well, I noticed the mention in .../dev/bazaar, where it reads, "the first time you do this, you might need to add --create-prefix". Perhaps that should read "... you will need to ...". It pushed fine with --create-prefix. Thanks, Skip From tjreedy at udel.edu Tue Mar 25 20:01:21 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 25 Mar 2008 15:01:21 -0400 Subject: [Python-Dev] Decimal(unicode) References: <20080325134617.GB2851@phd.pp.ru><5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com><47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> Message-ID: "Mark Dickinson" wrote in message news:5c6f2a5d0803250853g4814eac6n25b07326f2e7805d at mail.gmail.com... | On Tue, Mar 25, 2008 at 11:29 AM, Nick Coghlan wrote: | | > The isinstance(value, str) check in Py3k is too restrictive - it needs | > to accept bytes instances as well. | > | | Hmm. There's not a lot of consistency here: | | >>> int(b'1') | 1 | >>> float(b'1') | 1.0 | >>> complex(b'1') | Traceback (most recent call last): | File "", line 1, in | TypeError: complex() argument must be a string or a number | >>> from fractions import Fraction | >>> Fraction(b'1') | Traceback (most recent call last): | File "", line 1, in | File "/Users/dickinsm/python_source/py3k/Lib/fractions.py", line 98, in | __new__ | numerator = numerator.__index__() | AttributeError: 'bytes' object has no attribute '__index__' | | So int and float accepts bytes, while complex, Decimal and Fraction do | not... The purpose of type constructors is to construct instances from reasonable inputs. I think all number constructors should accept bytes and so the latter three should be changed. tjr From lists at cheimes.de Tue Mar 25 20:48:34 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 25 Mar 2008 20:48:34 +0100 Subject: [Python-Dev] Proposal: from __future__ import unicode_string_literals In-Reply-To: <47E2EB45.5080202@trueblade.com> References: <47E2EB45.5080202@trueblade.com> Message-ID: <47E95712.700@cheimes.de> Follow up: Neal and I've created a working patch, http://bugs.python.org/issue2477 We had to modify the parser API and add two functions. The two new functions are slightly modified versions of existing functions. We needed the flag argument to be an input/output variable (pointer) instead of a input only variable. The rest of the code is straight forward. I like to get the review of another developer before I commit the code. Christian From musiccomposition at gmail.com Tue Mar 25 21:27:51 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 25 Mar 2008 15:27:51 -0500 Subject: [Python-Dev] Commit access request In-Reply-To: References: <1afaf6160803240456u7ce0aa53t19514771e3434b30@mail.gmail.com> Message-ID: <1afaf6160803251327l26a67491yea3bf852ba5b9d2d@mail.gmail.com> On Tue, Mar 25, 2008 at 12:40 PM, Georg Brandl wrote: > Georg Brandl schrieb: > > Benjamin Peterson schrieb: > >> Hi Python devs, > >> I have been contributing to since December. (See me first issue on the > >> tracker, #1828; it was a major learning experience.) :P In that time, I > >> have contributed many patches and actively participated on this list. > >> This will enable me to help triage bugs on the tracker, something I've > >> been wanting to help with. > > > > I'm +1 to Benjamin getting commit privileges, letting commits sign off > > by a senior developer. > > Since there are no objections, I've now added Benjamin's key, under that > condition. Thanks! > > > Welcome on board, Benjamin! :) Glad to help! > > > Georg > > > -- > Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. > Four shall be the number of spaces thou shalt indent, and the number of > thy > indenting shall be four. Eight shalt thou not indent, nor either indent > thou > two, excepting that thou then proceed to four. Tabs are right out. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -- Cheers, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080325/bd090e73/attachment.htm From martin at v.loewis.de Tue Mar 25 23:16:50 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 25 Mar 2008 23:16:50 +0100 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> Message-ID: <47E979D2.7070407@v.loewis.de> > I'd call this a bug. The change is an accident, a side-effect of the fact > that in 2.5.1 the coefficient (mantissa) of a Decimal was stored as a > tuple, and in 2.5.2 it's stored as a string (which greatly improves efficiency). > Clearly in 2.5.2 the mantissa is being stored as a unicode instance in the > second case; it should be explicitly coerced to str in Decimal.__new__. > > If others agree that it's a bug, I'll fix it. If people agree it's a bug, please do fix it. Regards, Martin From eric+python-dev at trueblade.com Tue Mar 25 23:43:45 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Tue, 25 Mar 2008 18:43:45 -0400 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47E979D2.7070407@v.loewis.de> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E979D2.7070407@v.loewis.de> Message-ID: <47E98021.7060503@trueblade.com> Martin v. L?wis wrote: >> I'd call this a bug. The change is an accident, a side-effect of the fact >> that in 2.5.1 the coefficient (mantissa) of a Decimal was stored as a >> tuple, and in 2.5.2 it's stored as a string (which greatly improves efficiency). >> Clearly in 2.5.2 the mantissa is being stored as a unicode instance in the >> second case; it should be explicitly coerced to str in Decimal.__new__. >> >> If others agree that it's a bug, I'll fix it. > > If people agree it's a bug, please do fix it. It looks like a bug to me. From greg.ewing at canterbury.ac.nz Wed Mar 26 00:17:28 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 26 Mar 2008 11:17:28 +1200 Subject: [Python-Dev] How we can get rid of eggs for 2.6 and beyond In-Reply-To: References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <47E4DE8E.6010809@holdenweb.com> <79990c6b0803220646r50e9b455y5bd9b955814f334a@mail.gmail.com> <47E53435.4090104@v.loewis.de> <79990c6b0803221256jf4e6ea1qce463ab711dcdb51@mail.gmail.com> <47E56864.4080201@v.loewis.de> Message-ID: <47E98808.3020207@canterbury.ac.nz> ajaksu wrote: > While Linux and OS X both view Python as essentially a first-class > development platform-i.e., as something that shrink-wrap applications > can be built on-Windows does not. Even on MacOSX and Linux, you can't rely on the system coming with the particular version of Python you want to use pre-installed. So unless you target a very old version of Python, you have to bundle anyway if you don't want users to have to download and install a bunch of dependencies. If the situation is any better on Linux, it's probably because Linux users tend to be more willing and/or able to track down and install dependencies along with an app. -- Greg From greg.ewing at canterbury.ac.nz Wed Mar 26 00:31:24 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 26 Mar 2008 11:31:24 +1200 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47E92B9E.3000003@gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E92B9E.3000003@gmail.com> Message-ID: <47E98B4C.6070104@canterbury.ac.nz> Nick Coghlan wrote: > Since we have some strong use cases at least for the bytes->int case, > consistency then suggests that the other numeric types should all accept > bytes as well (interpreting them as ASCII encoded strings). How far should this go? Is conversion to numbers really so special, or should bytes be acceptable in any context requiring a string, with an implicit encoding of ascii? -- Greg From lists at cheimes.de Wed Mar 26 01:05:09 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 26 Mar 2008 01:05:09 +0100 Subject: [Python-Dev] Backport of bytearray type and io module In-Reply-To: <47E6C946.5020302@cheimes.de> References: <47E6C946.5020302@cheimes.de> Message-ID: <47E99335.5040306@cheimes.de> Christian Heimes schrieb: > Hello! > > I've successfully back ported the bytearray type and the io module from > 3.0 to 2.6. The code is available in the branch trunk-bytearray [1]. I'm > down to four failing byte tests and one failing io test. The failing > byte tests are all related to pickling and subclassing. > > I like to get the remaining tests fixed for the upcoming release in a > week. Please checkout the branch and help me figure out the remaining > issues. Follow up: All failing bytearray tests were caused by subclasses of bytearray. I've removed the Py_TPFLAGS_BASETYPE flag for now. Are you fine with the inclusion of bytearray although it can't be subclassed? I'm going to fix the last io bug now. I like to merge the backport of bytearray and io before the next alpha gets shipped out. Christian From guido at python.org Wed Mar 26 01:49:09 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 25 Mar 2008 17:49:09 -0700 Subject: [Python-Dev] Backport of bytearray type and io module In-Reply-To: <47E99335.5040306@cheimes.de> References: <47E6C946.5020302@cheimes.de> <47E99335.5040306@cheimes.de> Message-ID: I'm okay with bytearray not being subclassable in 2.6 as a temporary measure. I wouldn't want that to leak into 3.0 though, and I'd rather have it subclassable in 2.6 as well. I wonder why it doesn't work in 2.6 but does work in 3.0? On Tue, Mar 25, 2008 at 5:05 PM, Christian Heimes wrote: > Christian Heimes schrieb: > > > Hello! > > > > I've successfully back ported the bytearray type and the io module from > > 3.0 to 2.6. The code is available in the branch trunk-bytearray [1]. I'm > > down to four failing byte tests and one failing io test. The failing > > byte tests are all related to pickling and subclassing. > > > > I like to get the remaining tests fixed for the upcoming release in a > > week. Please checkout the branch and help me figure out the remaining > > issues. > > Follow up: > > All failing bytearray tests were caused by subclasses of bytearray. I've > removed the Py_TPFLAGS_BASETYPE flag for now. Are you fine with the > inclusion of bytearray although it can't be subclassed? > > I'm going to fix the last io bug now. I like to merge the backport of > bytearray and io before the next alpha gets shipped out. > > Christian > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Wed Mar 26 01:59:47 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 26 Mar 2008 01:59:47 +0100 Subject: [Python-Dev] Backport of bytearray type and io module In-Reply-To: References: <47E6C946.5020302@cheimes.de> <47E99335.5040306@cheimes.de> Message-ID: <47E9A003.6030808@cheimes.de> Guido van Rossum schrieb: > I'm okay with bytearray not being subclassable in 2.6 as a temporary > measure. I wouldn't want that to leak into 3.0 though, and I'd rather > have it subclassable in 2.6 as well. I wonder why it doesn't work in > 2.6 but does work in 3.0? It *seems* like the comparison ops don't work correctly for subclasses. In general subclassing works but comparison of subclasses result in wrong results. It's probably easy to fix but I haven't figured it out yet. Christian From greg.ewing at canterbury.ac.nz Wed Mar 26 01:37:31 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 26 Mar 2008 12:37:31 +1200 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> Message-ID: <47E99ACB.8010802@canterbury.ac.nz> Terry Reedy wrote: > The purpose of type constructors is to construct instances from reasonable > inputs. I think all number constructors should accept bytes What should bytes as input to a number constructor mean, though? People seem to be assuming it should be interpreted as ASCII-encoded characters. But an equally plausible interpretation might be that it's some binary representation of a number. -- Greg From ncoghlan at gmail.com Wed Mar 26 04:02:10 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 26 Mar 2008 13:02:10 +1000 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47E99ACB.8010802@canterbury.ac.nz> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E99ACB.8010802@canterbury.ac.nz> Message-ID: <47E9BCB2.6090008@gmail.com> Greg Ewing wrote: > Terry Reedy wrote: >> The purpose of type constructors is to construct instances from reasonable >> inputs. I think all number constructors should accept bytes > > What should bytes as input to a number constructor > mean, though? > > People seem to be assuming it should be interpreted > as ASCII-encoded characters. > > But an equally plausible interpretation might be > that it's some binary representation of a number. The difference is that there are some hardware control protocols which it makes sense to treat as sequences of bytes, which also contain numbers as ASCII digits which need to be processed. It's also the case that the permitted characters when passing a *string* to a numeric constructor are themselves an ASCII subset. For binary representations, we already have the struct module to handle the parsing, but for byte sequences with embedded ASCII digits it's reasonably common practice to use strings along with the respective type constructors. However, Mark found another problem when he attempted to speed up the Py3k version of decimal by storing the mantissa as a bytes object instead of a unicode string: there is currently no efficient way to serialise a number into a byte sequence. So storing the mantissa as a bytes object is actually currently slower than storing it as a string, as you have to convert the number to a string before you can store it in a bytes object. That still leaves us with the problem that decimal is about 25% slower in 3.0 than it is in 2.6, due to the fact that the unicode->int conversion is much slower than the corresponding 2.x str->int conversion. Ugly problem :P Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From facundobatista at gmail.com Wed Mar 26 04:07:57 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Wed, 26 Mar 2008 00:07:57 -0300 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E92B9E.3000003@gmail.com> Message-ID: 2008/3/25, Alex Martelli : > > Since we have some strong use cases at least for the bytes->int case, > > consistency then suggests that the other numeric types should all accept > > bytes as well (interpreting them as ASCII encoded strings). > > +1 -- it seems very practical as well as consistent, and I see no downsides. Mmm... Py3k-ish speaking.... "2.13" is an unicode string that holds four digits, two point one three, which if converted to Decimal, gives me, well, Decimal("2.13"). b"2.13", as it's not a string of digits anymore, but a stream of 4 bytes, that represents the binary number 0x322e3133... So, what I find difficult to know is how can you undoubtly express a collection of digits (inherent to strings) through bytes (without mixing pre-3k concepts). Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From nnorwitz at gmail.com Wed Mar 26 05:00:18 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 25 Mar 2008 21:00:18 -0700 Subject: [Python-Dev] Py3k and asyncore/asynchat In-Reply-To: References: <95d2b6e8-2524-4464-b5a9-c4d56e5f4318@t16g2000hsc.googlegroups.com> Message-ID: On Thu, Feb 14, 2008 at 10:09 AM, Giampaolo Rodola' wrote: > On 14 Feb, 16:36, "Giampaolo Rodola'" wrote: > > Ok, I'll try to take a look at all asyncore/chat reports and try to > > summarize them by splitting patches which solve bugs and patches which > > add enhancements or functionalities. > > === Patches for existing issues === > > - 1736190 which includes fixes for the following issues among other > improvements: > - 1063924 (asyncore should handle ECONNRESET in send()) > - 1736101 (asyncore should handle ECONNABORTED in recv()) > - 760475 (handle_error() should call handle_close() instead of > close()) > - 1740572 (refill_buffer() should call handle_close() rather than > close()) > - 777588 (wrong "connection refused" behavior on Windows) > - 889153 (wrong connect() behavior) > - 953599 (asyncore misses socket closes when poll is used) > - 1025525 (asyncore.file_dispatcher should not take fd as argument) > > - 1519 (async_chat.__init__() and asyncore.dispatcher.__init__ > parameters inconsistency) > - 1541 (Bad OOB data management when using asyncore with > select.poll()) > - 2073 (asynchat push always sends 512 bytes (ignoring > ac_out_buffer_size)) > > > === Open issues with no patches (need review) === > > - 658749 (asyncore connect() and winsock errors) > - 1161031 (neverending warnings from asyncore) > > > === Enhancements & new features === > > - 1641 (add delayed calls feature) > - 1563 (conversion to py3k and some other changes) That's a lot of patches. My immediate concern is that test_asynchat is very flaky and fails often. Sometimes it causes other tests to fail. Is there a patch that addresses this? If you need examples, just look through the buildbot mails that mention test_asynchat in: http://mail.python.org/pipermail/python-checkins/ Some platforms have more problems than others, but almost all platforms have failed test_asynchat at one point or another. Please break up the patches into 2 sets and prioritize the patches with the set. Set #1: Patches that have a test and doc updates if required Set #2: Patches that don't have a test or doc updates as required For the patches that fall into Set #1, list them in priority order. Top priority would be a problem that fixes failures seen in the buildbots. Next priority would go to the patches that solve more serious problems. Post the results here. I will try to look at them. For every patch you list, make sure that it conforms to the proper style (e.g, PEP 8) and is essentially perfect and ready for inclusion. This means that there is a single file to download that contains all the modifications. The changes are appropriately commented, lines are less than 80 characters, etc. One of the modifications should be an entry in Misc/NEWS. n From greg.ewing at canterbury.ac.nz Wed Mar 26 06:06:49 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 26 Mar 2008 17:06:49 +1200 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47E9BCB2.6090008@gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> Message-ID: <47E9D9E9.8030905@canterbury.ac.nz> I thought Decimal was going to be replaced by a C implementation soon anyway. If so, is it worth going to much trouble over this? -- Greg From martin at v.loewis.de Wed Mar 26 07:11:58 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 26 Mar 2008 07:11:58 +0100 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47E9BCB2.6090008@gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> Message-ID: <47E9E92E.4090401@v.loewis.de> > For binary representations, we already have the struct module to handle > the parsing, but for byte sequences with embedded ASCII digits it's > reasonably common practice to use strings along with the respective type > constructors. Sure, but why can't you write foo = int(bar[start:stop].decode("ascii")) then? Explicit is better than implicit. Regards, Martin From nnorwitz at gmail.com Wed Mar 26 07:21:04 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 25 Mar 2008 23:21:04 -0700 Subject: [Python-Dev] the release gods are angry at python Message-ID: The next releases of 2.6/3.0 are planned for April 2, just over a week from now. There is much work that needs to be done. The buildbots are in a pretty sad state and the gods are seeing too much red. http://www.python.org/dev/buildbot/stable/ http://www.python.org/dev/buildbot/all/ See my other mail that discusses the stable buildbots. The criteria for release is that all the stable buildbots are passing all the tests. So we really gotta get these green before Barry notices. You don't want to see Barry angry. You wouldn't like him when he's angry. I propose a code chill for new features. Changes to doc and tests can continue as usual. However, only submit a new feature *after* you fix a broken test first. If we have to get draconian, we can start breaking fingers when you break a test just like we do at work. :-) Specifically tests that need some TLC are: * test_winsound - http://www.python.org/dev/buildbot/stable/x86%20XP-3%20trunk/builds/1166/step-test/0 * test_threading - test_no_refcycle_through_target - http://www.python.org/dev/buildbot/stable/x86%20XP-3%20trunk/builds/1166/step-test/0 * test_socket deadlocks and times out - http://www.python.org/dev/buildbot/stable/x86%20W2k8%20trunk/builds/210/step-test/0 * test_ssl deadlocks and times out - http://www.python.org/dev/buildbot/all/S-390%20Debian%20trunk/builds/255/step-test/0 * test_xmlrpc transient socket errors - http://www.python.org/dev/buildbot/stable/g4%20osx.4%20trunk/builds/3101/step-test/0 * test_mailbox - http://www.python.org/dev/buildbot/stable/x86%20XP-3%203.0/builds/723/step-test/0 * test_asynchat - http://www.python.org/dev/buildbot/all/alpha%20Tru64%205.1%20trunk/builds/2756/step-test/0 Hopefully test_timeout is fixed, but that might be flaky too. There have been other tests that have also been flaky like test_asynchat, test_smtplib, test_ssl, test_urllib2net, test_urllibnet, test_xmlrpc_net and some of the tests that use networking. These all need to be fixed so the tests are 100% reliable and only fail when there is a real error. There are currently no release blocker issues: http://bugs.python.org/issue?%40columns=title&%40columns=id&%40sort=activity&priority=1&%40group=priority&status=1&%40action=search There are 48 critical issues: http://bugs.python.org/issue?%40columns=title&%40columns=id&%40sort=activity&priority=2&%40group=priority&status=1&%40action=search If you believe any issue should block the release, set the priority to release blocker and assign it to me (nnorwitz). Many of the critical issues are those that require 2to3 fixers. These can be checked in as they are written. Be sure to test them thoroughly and try to think of all the conditions that could possibly cause the fixer to fail or do the wrong thing. Right now, I don't know of any reason to hold up the release other than the failing tests. n From nnorwitz at gmail.com Wed Mar 26 07:21:11 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 25 Mar 2008 23:21:11 -0700 Subject: [Python-Dev] stable buildbots Message-ID: We need to get the tests for Python to be more stable so we can push out solid releases. In order to achieve this result, we need tests that are *100% reliable* and fail _only when there is a problem with Python_. While we aren't nearly as close to that goal as we need to be, we have to work towards it. The buildbots that have been more reliable are separated onto their own page: http://www.python.org/dev/buildbot/stable/ This is the page that should be tracked by most people. This is the page we will use to determine if Python is ready for release. All green means the release is ready to ship. This page should *always* show all green. Any change that causes any buildbot to fail, should be immediately reverted. When you commit code, make sure to refresh the stable buildbot page to ensure you haven't broken anything. The stable buildbots need to represent all major platforms, at least Windows, Mac OSX, and Linux. All of those are currently represented on the page along with several others. Unfortunately some of the tests, particularly on Windows, are not passing. We need to get these in better shape. Please help out. See my other mail for details or use the link above to find the current set of problems. As we fix more tests and more platforms become stable, I will add them to the stable page. This requires restarting the server, so I plan to do this during quiet times when all the buildbot slaves are idle. n From ncoghlan at gmail.com Wed Mar 26 07:57:22 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 26 Mar 2008 16:57:22 +1000 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47E9D9E9.8030905@canterbury.ac.nz> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> <47E9D9E9.8030905@canterbury.ac.nz> Message-ID: <47E9F3D2.6090400@gmail.com> Greg Ewing wrote: > I thought Decimal was going to be replaced by a C > implementation soon anyway. If so, is it worth going > to much trouble over this? > I believe that was found to be more trouble than it was worth. So the optimisations focused on various ways of making the Python implementation more efficient. One of those ways was to store the mantissa as a string in order to gain the benefit of the fast str->int and int->str conversions. The 3.0 version no longer has that benefit, and it shows. It looks like it may be necessary to switch to a custom object for the mantissa storage in order to get those fast conversions back. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Mar 26 08:06:34 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 26 Mar 2008 17:06:34 +1000 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47E9E92E.4090401@v.loewis.de> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> <47E9E92E.4090401@v.loewis.de> Message-ID: <47E9F5FA.3030809@gmail.com> Martin v. L?wis wrote: >> For binary representations, we already have the struct module to handle >> the parsing, but for byte sequences with embedded ASCII digits it's >> reasonably common practice to use strings along with the respective type >> constructors. > > Sure, but why can't you write > > foo = int(bar[start:stop].decode("ascii")) > > then? Explicit is better than implicit. Yeah, this thread has convinced me that it would be better to start rejecting bytes in int() and float() as well rather than implicitly assuming an ASCII encoding. If we decide the fast path for ASCII is still important (e.g. to solve 3.0's current speed problems in decimal), then it would be better to add separate methods to int to expose the old 2.x str->int and int->str optimisations (e.g. an int.from_ascii class method and an int.to_ascii instance method). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From josiah.carlson at gmail.com Wed Mar 26 08:21:18 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Wed, 26 Mar 2008 00:21:18 -0700 Subject: [Python-Dev] Py3k and asyncore/asynchat In-Reply-To: References: <95d2b6e8-2524-4464-b5a9-c4d56e5f4318@t16g2000hsc.googlegroups.com> Message-ID: On Tue, Mar 25, 2008 at 11:26 PM, Neal Norwitz wrote: > Any reason this was sent just to me and not the list? Because gmail only replies to the sender by default. I need to remember to cc python-dev when I reply (I used the same email client for 8 1/2 years, remembering the quirks of gmail may take some time). > On Tue, Mar 25, 2008 at 10:34 PM, Josiah Carlson > wrote: > > > > On Tue, Mar 25, 2008 at 9:00 PM, Neal Norwitz wrote: > > > On Thu, Feb 14, 2008 at 10:09 AM, Giampaolo Rodola' wrote: > > > > On 14 Feb, 16:36, "Giampaolo Rodola'" wrote: > > > > > Ok, I'll try to take a look at all asyncore/chat reports and try to > > > > > summarize them by splitting patches which solve bugs and patches which > > > > > add enhancements or functionalities. > > > > > > > > > > > > > > === Patches for existing issues === > > > > > > > > - 1736190 which includes fixes for the following issues among other > > > > improvements: > > > > - 1063924 (asyncore should handle ECONNRESET in send()) > > > > - 1736101 (asyncore should handle ECONNABORTED in recv()) > > > > - 760475 (handle_error() should call handle_close() instead of > > > > close()) > > > > - 1740572 (refill_buffer() should call handle_close() rather than > > > > close()) > > > > - 777588 (wrong "connection refused" behavior on Windows) > > > > - 889153 (wrong connect() behavior) > > > > - 953599 (asyncore misses socket closes when poll is used) > > > > - 1025525 (asyncore.file_dispatcher should not take fd as argument) > > > > > > > > - 1519 (async_chat.__init__() and asyncore.dispatcher.__init__ > > > > parameters inconsistency) > > > > - 1541 (Bad OOB data management when using asyncore with > > > > select.poll()) > > > > - 2073 (asynchat push always sends 512 bytes (ignoring > > > > ac_out_buffer_size)) > > > > > > > > > > > > === Open issues with no patches (need review) === > > > > > > > > - 658749 (asyncore connect() and winsock errors) > > > > - 1161031 (neverending warnings from asyncore) > > > > > > > > > > > > === Enhancements & new features === > > > > > > > > - 1641 (add delayed calls feature) > > > > - 1563 (conversion to py3k and some other changes) > > > > > > That's a lot of patches. My immediate concern is that test_asynchat > > > is very flaky and fails often. Sometimes it causes other tests to > > > fail. Is there a patch that addresses this? If you need examples, > > > just look through the buildbot mails that mention test_asynchat in: > > > http://mail.python.org/pipermail/python-checkins/ > > > > No, it's one patch. All of the fixes were performed mostly by myself > > last spring, tested and verified in personal servers, then re-used by > > Giampaolo in his async ftp server (which pointed out a few small bugs, > > which have been fixed). > > > > > > > Some platforms have more problems than others, but almost all > > > platforms have failed test_asynchat at one point or another. > > > > Certainly that is the case. And according to my reading of a few > > buildbot failures, aside from someone breaking asyncore itself, the > > failures seem to stem from the test being unable to create a port to > > listen on in order to test the server/client functionality. This is a > > buildbot configuration issue (per host), not an asyncore issue. > > That was the case a long time (~3? months) ago, but hasn't been the > case recently. See my recent message about the release. I'll look for it tomorrow. For reference, searches of 'site:mail.python.org test_asynchat failure buildbot' only seem to produce the socket listen error. If there is a better incantation to get google to produce the proper errors (and/or a link), I would appreciate the help. > > > Please break up the patches into 2 sets and prioritize the patches > > > with the set. > > > > > > Set #1: Patches that have a test and doc updates if required > > > Set #2: Patches that don't have a test or doc updates as required > > > > > > For the patches that fall into Set #1, list them in priority order. > > > Top priority would be a problem that fixes failures seen in the > > > buildbots. Next priority would go to the patches that solve more > > > serious problems. Post the results here. I will try to look at them. > > > > > > For every patch you list, make sure that it conforms to the proper > > > style (e.g, PEP 8) and is essentially perfect and ready for inclusion. > > > This means that there is a single file to download that contains all > > > the modifications. The changes are appropriately commented, lines are > > > less than 80 characters, etc. One of the modifications should be an > > > entry in Misc/NEWS. > > > > I lied earlier; really there are two patches. The first is a patch to > > asyncore.py and asynchat.py . It addresses those bugs that Giampaolo > > has listed, it is tested, and works. The second patch is to update > > the documentation to mention the sample methods in asynchat for use as > > examples, as well as any other changes to the language used in the > > documentation that I had made last spring, but which are out of date > > from my posting of the original patch. I can update the documentation > > in the next week. > > Can you provide a link to the patches? Do the patches include changes > to test_asyncore and test_asynchat? The next release is April 2. I > would like to commit any patches before Monday to ensure they are > stable. Can you get me the patches by this Saturday? See http://bugs.python.org/issue1736190 for an updated patch for the modules. The current test cases pass without issue, though we may want to add tests, which I need to look at the original patch and the original file from which it was created against, then compare it with the most recent changes to the tests from Facundo last June or July. I should have the time to get patches for tests and documentation by Monday. - Josiah From mal at egenix.com Wed Mar 26 10:29:58 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 26 Mar 2008 10:29:58 +0100 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47E9E92E.4090401@v.loewis.de> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> <47E9E92E.4090401@v.loewis.de> Message-ID: <47EA1796.1000504@egenix.com> On 2008-03-26 07:11, Martin v. L?wis wrote: >> For binary representations, we already have the struct module to handle >> the parsing, but for byte sequences with embedded ASCII digits it's >> reasonably common practice to use strings along with the respective type >> constructors. > > Sure, but why can't you write > > foo = int(bar[start:stop].decode("ascii")) > > then? Explicit is better than implicit. Agreed. The whole purpose of Unicode is to store text. Data from a file isn't text per-se. You have to tell Python that a particular set of bytes is to be interpreted as text and that only works by explicitly converting the bytes to text. Numbers or digits aren't any different in this context. b"1234" is just a sequence of bytes and could well represent the binary encoding of an integer, the start of a base64 encoded image, an SSH key or an audio file. Don't get fooled by the looks of b"1234". It's really just a shorter way of writing 0x31 0x32 0x33 0x34. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 26 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From facundobatista at gmail.com Wed Mar 26 13:14:21 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Wed, 26 Mar 2008 09:14:21 -0300 Subject: [Python-Dev] stable buildbots In-Reply-To: References: Message-ID: 2008/3/26, Neal Norwitz : > We need to get the tests for Python to be more stable so we can push > out solid releases. In order to achieve this result, we need tests > that are *100% reliable* and fail _only when there is a problem with +1 > Python_. While we aren't nearly as close to that goal as we need to > be, we have to work towards it. The buildbots that have been more > reliable are separated onto their own page: > > http://www.python.org/dev/buildbot/stable/ This is for trunk or 3k? Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From lists at cheimes.de Wed Mar 26 13:36:53 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 26 Mar 2008 13:36:53 +0100 Subject: [Python-Dev] Backport of bytearray type and io module In-Reply-To: References: <47E6C946.5020302@cheimes.de> <47E99335.5040306@cheimes.de> Message-ID: <47EA4365.7050908@cheimes.de> Guido van Rossum schrieb: > I'm okay with bytearray not being subclassable in 2.6 as a temporary > measure. I wouldn't want that to leak into 3.0 though, and I'd rather > have it subclassable in 2.6 as well. I wonder why it doesn't work in > 2.6 but does work in 3.0? This fix for the issue was easy once I noticed the cause of the problem Modified: python/branches/trunk-bytearray/Objects/typeobject.c ============================================================================== --- python/branches/trunk-bytearray/Objects/typeobject.c (original) +++ python/branches/trunk-bytearray/Objects/typeobject.c Wed Mar 26 13:20:46 2008 @@ -3762,6 +3762,8 @@ COPYBUF(bf_getwritebuffer); COPYBUF(bf_getsegcount); COPYBUF(bf_getcharbuffer); + COPYBUF(bf_getbuffer); + COPYBUF(bf_releasebuffer); } basebase = base->tp_base; Christian From facundobatista at gmail.com Wed Mar 26 13:43:06 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Wed, 26 Mar 2008 09:43:06 -0300 Subject: [Python-Dev] Backport of bytearray type and io module In-Reply-To: <47EA4365.7050908@cheimes.de> References: <47E6C946.5020302@cheimes.de> <47E99335.5040306@cheimes.de> <47EA4365.7050908@cheimes.de> Message-ID: 2008/3/26, Christian Heimes : > > I'm okay with bytearray not being subclassable in 2.6 as a temporary > > measure. I wouldn't want that to leak into 3.0 though, and I'd rather > > have it subclassable in 2.6 as well. I wonder why it doesn't work in > > 2.6 but does work in 3.0? > > This fix for the issue was easy once I noticed the cause of the problem > > Modified: python/branches/trunk-bytearray/Objects/typeobject.c So, now the byte object behaves equal in 2.6 and 3.0, right? Thanks! -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From lists at cheimes.de Wed Mar 26 14:00:00 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 26 Mar 2008 14:00:00 +0100 Subject: [Python-Dev] Backport of bytearray type and io module In-Reply-To: References: <47E6C946.5020302@cheimes.de> <47E99335.5040306@cheimes.de> <47EA4365.7050908@cheimes.de> Message-ID: <47EA48D0.9060002@cheimes.de> Facundo Batista schrieb: > So, now the byte object behaves equal in 2.6 and 3.0, right? > > Thanks! Correct! The bytearray type and the new IO system are now backported to Python 2.6. Christian From facundobatista at gmail.com Wed Mar 26 14:00:54 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Wed, 26 Mar 2008 10:00:54 -0300 Subject: [Python-Dev] Backport of bytearray type and io module In-Reply-To: <47EA48D0.9060002@cheimes.de> References: <47E6C946.5020302@cheimes.de> <47E99335.5040306@cheimes.de> <47EA4365.7050908@cheimes.de> <47EA48D0.9060002@cheimes.de> Message-ID: 2008/3/26, Christian Heimes : > Correct! > > The bytearray type and the new IO system are now backported to Python 2.6. Thank you very much for this effort! Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From dickinsm at gmail.com Wed Mar 26 15:40:16 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Wed, 26 Mar 2008 10:40:16 -0400 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47E9F3D2.6090400@gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> <47E9D9E9.8030905@canterbury.ac.nz> <47E9F3D2.6090400@gmail.com> Message-ID: <5c6f2a5d0803260740n6511be1t34a09d01c864d927@mail.gmail.com> On Wed, Mar 26, 2008 at 2:57 AM, Nick Coghlan wrote: > Greg Ewing wrote: > > I thought Decimal was going to be replaced by a C > > implementation soon anyway. If so, is it worth going > > to much trouble over this? > > > > I believe that was found to be more trouble than it was worth. So the > optimisations focused on various ways of making the Python > implementation more efficient. > I think it's still worth considering a hybrid implementation of Decimal: C code for the basic integer arithmetic (that is, supply a long int replacement whose underlying implementation works in base a power of 10), and Python for all the complicated logic (dealing with flags, special values, etc.). This will speed things up in the usual cases, and also give everything the right asymptotics for those few people using Decimal to do really high precision arithmetic. (Right now, addition of two Decimals takes quadratic time.) The decimal long integer implementation is already in the sandbox, so this probably isn't as much work as it sounds. I won't have time for it until May, though. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080326/fe751adb/attachment.htm From facundobatista at gmail.com Wed Mar 26 16:04:21 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Wed, 26 Mar 2008 12:04:21 -0300 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <5c6f2a5d0803260740n6511be1t34a09d01c864d927@mail.gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> <47E9D9E9.8030905@canterbury.ac.nz> <47E9F3D2.6090400@gmail.com> <5c6f2a5d0803260740n6511be1t34a09d01c864d927@mail.gmail.com> Message-ID: 2008/3/26, Mark Dickinson : > I think it's still worth considering a hybrid implementation of Decimal: > C code for the basic integer arithmetic (that is, supply a long int > replacement whose underlying implementation works in base a > power of 10), and Python for all the complicated logic (dealing I think that this is the way to go, also. -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From ncoghlan at gmail.com Wed Mar 26 16:04:43 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 27 Mar 2008 01:04:43 +1000 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <5c6f2a5d0803260740n6511be1t34a09d01c864d927@mail.gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> <47E9D9E9.8030905@canterbury.ac.nz> <47E9F3D2.6090400@gmail.com> <5c6f2a5d0803260740n6511be1t34a09d01c864d927@mail.gmail.com> Message-ID: <47EA660B.9070908@gmail.com> Mark Dickinson wrote: > On Wed, Mar 26, 2008 at 2:57 AM, Nick Coghlan > wrote: > > Greg Ewing wrote: > > I thought Decimal was going to be replaced by a C > > implementation soon anyway. If so, is it worth going > > to much trouble over this? > > > > I believe that was found to be more trouble than it was worth. So the > optimisations focused on various ways of making the Python > implementation more efficient. > > > I think it's still worth considering a hybrid implementation of Decimal: > C code for the basic integer arithmetic (that is, supply a long int > replacement whose underlying implementation works in base a > power of 10), and Python for all the complicated logic (dealing > with flags, special values, etc.). This will speed things up in the > usual cases, and also give everything the right asymptotics for > those few people using Decimal to do really high precision arithmetic. > (Right now, addition of two Decimals takes quadratic time.) > > The decimal long integer implementation is already in the sandbox, > so this probably isn't as much work as it sounds. Ah, I didn't know that - I guess you're talking about extracting the integer arithmetic section from the decimal-c implementation? In that case, yes, using a custom type for the guts of the mantissa arithmetic instead of trying to get reasonable speed out of a mixture of builtin types would be a very good thing (and would obviously eliminate the current performance problems in the Py3k version of decimal). Do you think it would be feasible to get this done for the first beta at the beginning of June? (I did have a look at _decimal.c in the sandbox to see how much help I could offer, but I have to confess my eyes started to glaze over a bit ;) Or would it be better to pursue a simple C object that just stored a sequence of integers and provided methods for fast conversion to/from a long for 2.6/3.0 and defer the arithmetic-in-C to 3.1? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From barry at python.org Wed Mar 26 16:38:09 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 26 Mar 2008 11:38:09 -0400 Subject: [Python-Dev] the release gods are angry at python In-Reply-To: References: Message-ID: <86F37BB3-1545-4F68-A0B2-A7F5C09821E2@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 26, 2008, at 2:21 AM, Neal Norwitz wrote: > The next releases of 2.6/3.0 are planned for April 2, just over a week > from now. There is much work that needs to be done. The buildbots > are in a pretty sad state and the gods are seeing too much red. > > http://www.python.org/dev/buildbot/stable/ > http://www.python.org/dev/buildbot/all/ > > See my other mail that discusses the stable buildbots. The criteria > for release is that all the stable buildbots are passing all the > tests. So we really gotta get these green before Barry notices. You > don't want to see Barry angry. You wouldn't like him when he's angry. Of course, most people don't like me when I'm /not/ angry either! :) Thanks for being the Bad Cop on this Neal. Please folks, please help make these buildbots go green! I think we should all be disappointed if the releases have to slip because our stable buildbots are red. Neal and I will have free rein to back out changes if that turns them green so if your code changes cause a failure, and you want your changes to stay in, please spend some time fixing them. > I propose a code chill for new features. Changes to doc and tests can > continue as usual. However, only submit a new feature *after* you fix > a broken test first. +1 - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBR+pt4XEjvBPtnXfVAQKHQwP/WXCoUtJY7nq3xFExs5RCCEeWFceSBRJR XBjymIVVODaWyhzh2obCuD/reHiEHneVWBzrS+kuQPEdigR6R4wJIyLQNqol5bfD auDJzUAa4vrMjfJtf2+i3/GRaHPHzgwPfgyj1t5rlyE/3pLbSh+d4+qhtH8Hq0MY ExF6WVIWRDc= =FP2J -----END PGP SIGNATURE----- From lists at cheimes.de Wed Mar 26 16:59:16 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 26 Mar 2008 16:59:16 +0100 Subject: [Python-Dev] the release gods are angry at python In-Reply-To: References: Message-ID: <47EA72D4.8000709@cheimes.de> Georg Brandl schrieb: > Perhaps make it an optional resource? In the py3k branch I've assigned the audio resource to the winsound tests. Only regrtest.py -uall or -uaudio runs the winsound test. Reason: the test sound was freaking out my poor cat. :/ Christian From p.f.moore at gmail.com Wed Mar 26 17:04:01 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 26 Mar 2008 16:04:01 +0000 Subject: [Python-Dev] [Python-3000] the release gods are angry at python In-Reply-To: References: Message-ID: <79990c6b0803260904w7a8df216md8b67cbf160c6cd4@mail.gmail.com> On 26/03/2008, Thomas Heller wrote: > What I would like to see is a way to disable certain tests on certain machines; > maybe by setting environment variables? Could this be done by something like the following (completely untested!!!! no time at the moment) change to regrtest.py? --- Lib\test\regrtest.py.orig 2008-03-26 15:36:24.519590600 +0000 +++ Lib\test\regrtest.py 2008-03-26 16:02:52.513001800 +0000 @@ -341,6 +341,12 @@ stdtests.remove(arg) nottests[:0] = args args = [] + + exclusions = os.getenv('EXCLUDE_TESTS').split() + for excl in exclusions: + if excl in stdtests: + stdtests.remove(excl) + tests = tests or args or findtests(testdir, stdtests, nottests) if single: tests = tests[:1] Paul. From barry at python.org Wed Mar 26 17:04:30 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 26 Mar 2008 12:04:30 -0400 Subject: [Python-Dev] stable buildbots In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 26, 2008, at 8:14 AM, Facundo Batista wrote: > 2008/3/26, Neal Norwitz : > >> We need to get the tests for Python to be more stable so we can push >> out solid releases. In order to achieve this result, we need tests >> that are *100% reliable* and fail _only when there is a problem with > > +1 > > >> Python_. While we aren't nearly as close to that goal as we need to >> be, we have to work towards it. The buildbots that have been more >> reliable are separated onto their own page: >> >> http://www.python.org/dev/buildbot/stable/ > > This is for trunk or 3k? The page has stable buildbots for trunk, 2.5, and 3.0. I'm thinking that we should remove the 2.5 buildbots from the stable page. Neal, if you agree, can you do that? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBR+p0D3EjvBPtnXfVAQIHPgP/XxK3gSSlh4WRhJs3IGdohWvr00wGFkv4 CZOvdjxFtCoQ96kJlOGuBl8QZDpImD7xAROM+mH20IxXbhnWC5CVEXmaxOdVT412 53HZFViPKq7f0j/I7wNOSXOlmEm7lrCvOQ1afNcnkxQ7B3k2aeqBHtLiKAE4eQHZ o04LCvyetnw= =seJD -----END PGP SIGNATURE----- From nnorwitz at gmail.com Wed Mar 26 17:29:10 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 26 Mar 2008 09:29:10 -0700 Subject: [Python-Dev] stable buildbots In-Reply-To: References: Message-ID: On Wed, Mar 26, 2008 at 9:04 AM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > On Mar 26, 2008, at 8:14 AM, Facundo Batista wrote: > > 2008/3/26, Neal Norwitz : > > > >> We need to get the tests for Python to be more stable so we can push > >> out solid releases. In order to achieve this result, we need tests > >> that are *100% reliable* and fail _only when there is a problem with > > > > +1 > > > > > >> Python_. While we aren't nearly as close to that goal as we need to > >> be, we have to work towards it. The buildbots that have been more > >> reliable are separated onto their own page: > >> > >> http://www.python.org/dev/buildbot/stable/ > > > > This is for trunk or 3k? > > The page has stable buildbots for trunk, 2.5, and 3.0. I'm thinking > that we should remove the 2.5 buildbots from the stable page. Neal, > if you agree, can you do that? I'm fine with removing 2.5, but I'm not sure if it's easy. HINT HINT. If everyone else fixes the broken and flaky tests, I'll have more than enough time to fix this. n From guido at python.org Wed Mar 26 17:58:53 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 26 Mar 2008 09:58:53 -0700 Subject: [Python-Dev] Backport of bytearray type and io module In-Reply-To: References: <47E6C946.5020302@cheimes.de> <47E99335.5040306@cheimes.de> <47EA4365.7050908@cheimes.de> <47EA48D0.9060002@cheimes.de> Message-ID: Yay indeed! Of course the new I/O module will undergo changes (Ping is working on it still I believe). Try to keep an eye on it so the improvements can be backported. On Wed, Mar 26, 2008 at 6:00 AM, Facundo Batista wrote: > 2008/3/26, Christian Heimes : > > > > Correct! > > > > The bytearray type and the new IO system are now backported to Python 2.6. > > Thank you very much for this effort! > > Regards, > > > > -- > . Facundo > > Blog: http://www.taniquetil.com.ar/plog/ > PyAr: http://www.python.org/ar/ > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Wed Mar 26 18:10:32 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 26 Mar 2008 18:10:32 +0100 Subject: [Python-Dev] Backport of bytearray type and io module In-Reply-To: References: <47E6C946.5020302@cheimes.de> <47E99335.5040306@cheimes.de> <47EA4365.7050908@cheimes.de> <47EA48D0.9060002@cheimes.de> Message-ID: <47EA8388.6010704@cheimes.de> Guido van Rossum schrieb: > Yay indeed! Of course the new I/O module will undergo changes (Ping is > working on it still I believe). Try to keep an eye on it so the > improvements can be backported. Could somebody (perhaps me *g*) create a 3to2 fixer that removes the function annotations? IIRC I only had to remove the annotations from the io module and add a "from __future__ import print_function" to get it working correctly. Christian From zooko at zooko.com Wed Mar 26 20:18:54 2008 From: zooko at zooko.com (zooko) Date: Wed, 26 Mar 2008 12:18:54 -0700 Subject: [Python-Dev] how to easily consume just the parts of eggs that are good for you Message-ID: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com> Folks: Here is a simple proposal: make the standard Python "import" mechanism notice eggs on the PYTHONPATH and insert them (into the *same* location) on the sys.path. This eliminates the #1 problem with eggs -- that they don't easily work when installing them into places other than your site-packages and that if you allow any of them to be installed on your system then they take precedence over your non-egg packages even you explicitly put those other packages earlier in your PYTHONPATH. (That latter behavior is very disagreeable to more than a few prorgammers.) This also preserves most of the value of eggs for many use cases. This is backward-compatible with most current use cases that rely on eggs. This is very likely forward-compatible with new schemes that are currently being cooked up and will be deployed in the future. Regards, Zooko From regebro at gmail.com Wed Mar 26 21:20:25 2008 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 26 Mar 2008 21:20:25 +0100 Subject: [Python-Dev] how to easily consume just the parts of eggs that are good for you In-Reply-To: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com> References: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com> Message-ID: <319e029f0803261320r60f1e8f5mfe679e8c1e111299@mail.gmail.com> Has somebody made a list of the problems with eggs? Because I use them all the time and hasn't encountered any problems whatsoever, myself... :) So I am a bit surprised at the various discussions about them. From musiccomposition at gmail.com Wed Mar 26 22:10:41 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Wed, 26 Mar 2008 16:10:41 -0500 Subject: [Python-Dev] [Python-3000] the release gods are angry at python In-Reply-To: References: Message-ID: <1afaf6160803261410n4e0067a3waa779d62a4fcc777@mail.gmail.com> On Wed, Mar 26, 2008 at 1:21 AM, Neal Norwitz wrote: > The next releases of 2.6/3.0 are planned for April 2, just over a week > from now. There is much work that needs to be done. The buildbots > are in a pretty sad state and the gods are seeing too much red. > > http://www.python.org/dev/buildbot/stable/ > http://www.python.org/dev/buildbot/all/ > > See my other mail that discusses the stable buildbots. The criteria > for release is that all the stable buildbots are passing all the > tests. So we really gotta get these green before Barry notices. You > don't want to see Barry angry. You wouldn't like him when he's angry. > > I propose a code chill for new features. Changes to doc and tests can > continue as usual. However, only submit a new feature *after* you fix > a broken test first. If we have to get draconian, we can start > breaking fingers when you break a test just like we do at work. :-) > > Specifically tests that need some TLC are: > * test_winsound > - > http://www.python.org/dev/buildbot/stable/x86%20XP-3%20trunk/builds/1166/step-test/0 > * test_threading - test_no_refcycle_through_target > - > http://www.python.org/dev/buildbot/stable/x86%20XP-3%20trunk/builds/1166/step-test/0 > * test_socket deadlocks and times out > - > http://www.python.org/dev/buildbot/stable/x86%20W2k8%20trunk/builds/210/step-test/0 > * test_ssl deadlocks and times out > - > http://www.python.org/dev/buildbot/all/S-390%20Debian%20trunk/builds/255/step-test/0 > * test_xmlrpc transient socket errors > - > http://www.python.org/dev/buildbot/stable/g4%20osx.4%20trunk/builds/3101/step-test/0 > * test_mailbox > - > http://www.python.org/dev/buildbot/stable/x86%20XP-3%203.0/builds/723/step-test/0 > * test_asynchat > - > http://www.python.org/dev/buildbot/all/alpha%20Tru64%205.1%20trunk/builds/2756/step-test/0 > > Hopefully test_timeout is fixed, but that might be flaky too. There > have been other tests that have also been flaky like test_asynchat, > test_smtplib, test_ssl, test_urllib2net, test_urllibnet, > test_xmlrpc_net and some of the tests that use networking. These all > need to be fixed so the tests are 100% reliable and only fail when > there is a real error. > > There are currently no release blocker issues: > > http://bugs.python.org/issue?%40columns=title&%40columns=id&%40sort=activity&priority=1&%40group=priority&status=1&%40action=search > > There are 48 critical issues: > > > http://bugs.python.org/issue?%40columns=title&%40columns=id&%40sort=activity&priority=2&%40group=priority&status=1&%40action=search > > If you believe any issue should block the release, set the priority to > release blocker and assign it to me (nnorwitz). Many of the critical > issues are those that require 2to3 fixers. These can be checked in as > they are written. Be sure to test them thoroughly and try to think of > all the conditions that could possibly cause the fixer to fail or do > the wrong thing. There are also some backporting issues in that pile. Should those hold up betas? (when we get there) > > > Right now, I don't know of any reason to hold up the release other > than the failing tests. > > n > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/musiccomposition%40gmail.com > -- Cheers, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080326/0d8af3d1/attachment-0001.htm From python-dev at xhaus.com Wed Mar 26 22:18:03 2008 From: python-dev at xhaus.com (Alan Kennedy) Date: Wed, 26 Mar 2008 21:18:03 +0000 Subject: [Python-Dev] httplib &c. timeouts and global state Message-ID: <4a951aa00803261418m642b6a92gbde277f2dc6ce0bc@mail.gmail.com> [This message will not be threaded properly since I wasn't subscribed at the time the original was sent] [John] > What I found in the archive is this thread (sorry for the > non-python.org URL): > > http://www.gossamer-threads.com/lists/python/dev/555039?do=post_view_threaded#555039 > > In that discussion, this issue is raised, but I don't see any resolution > that answers the objection made in issue 2451. Anyway, apologies in > advance if there was a decision here that takes account of the above > objection. The solution to the problem John descibes WAS decided back at the time of discussion of the contribution, but for some reason, that solution never made into contribution that was committed. http://www.gossamer-threads.com/lists/python/dev/555108?do=post_view_threaded#555108 http://www.gossamer-threads.com/lists/python/dev/555110?do=post_view_threaded#555110 The solution is simple, as described on the bug that John created for this issue. http://bugs.python.org/issue2451 Alan. From barry at python.org Wed Mar 26 22:26:13 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 26 Mar 2008 17:26:13 -0400 Subject: [Python-Dev] [Python-3000] the release gods are angry at python In-Reply-To: <1afaf6160803261410n4e0067a3waa779d62a4fcc777@mail.gmail.com> References: <1afaf6160803261410n4e0067a3waa779d62a4fcc777@mail.gmail.com> Message-ID: <8E0A8347-66EF-428D-94A1-60C8F9D00388@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 26, 2008, at 5:10 PM, Benjamin Peterson wrote: > There are also some backporting issues in that pile. Should those > hold up > betas? (when we get there) Yes, but I would simply release the monthly alpha and push the beta back a month. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR+q/dXEjvBPtnXfVAQJYJAP9En+87NRtSOKcnEB4Om/BYE+Jw7KZ08JI HY1MGAHleya8MHIgaEkaQf142pzfHKWxWZLNWD5UShHPlarwfvIxHdT07q5ozGda l4J95FU2EaLMGxN+5HvxlY+pdXAiVcNAnO12TO1Gqahf9Mmk1eXkKM0p6vBPJHqZ vtekrWIzeck= =3v/o -----END PGP SIGNATURE----- From greg.ewing at canterbury.ac.nz Thu Mar 27 00:19:34 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 27 Mar 2008 11:19:34 +1200 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47E9F3D2.6090400@gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> <47E9D9E9.8030905@canterbury.ac.nz> <47E9F3D2.6090400@gmail.com> Message-ID: <47EADA06.9090806@canterbury.ac.nz> Nick Coghlan wrote: > Greg Ewing wrote: > >> I thought Decimal was going to be replaced by a C >> implementation soon anyway. > > I believe that was found to be more trouble than it was worth. That's very disappointing. Was there any discussion of the problems that killed it? I don't remember seeing any showstoppers being mentioned. -- Greg From greg.ewing at canterbury.ac.nz Thu Mar 27 00:22:08 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 27 Mar 2008 11:22:08 +1200 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47E9F5FA.3030809@gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> <47E9E92E.4090401@v.loewis.de> <47E9F5FA.3030809@gmail.com> Message-ID: <47EADAA0.6060705@canterbury.ac.nz> Nick Coghlan wrote: > Yeah, this thread has convinced me that it would be better to start > rejecting bytes in int() and float() as well rather than implicitly > assuming an ASCII encoding. I had another thought -- would it be feasible to have some kind of wrapper object that would make a byte array containing ascii chars look like a string? Then cases like this could be handled without having to copy the data. -- Greg From ncoghlan at gmail.com Thu Mar 27 03:06:04 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 27 Mar 2008 12:06:04 +1000 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47EADA06.9090806@canterbury.ac.nz> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> <47E9D9E9.8030905@canterbury.ac.nz> <47E9F3D2.6090400@gmail.com> <47EADA06.9090806@canterbury.ac.nz> Message-ID: <47EB010C.9040608@gmail.com> Greg Ewing wrote: > Nick Coghlan wrote: >> Greg Ewing wrote: >> >>> I thought Decimal was going to be replaced by a C >>> implementation soon anyway. >> I believe that was found to be more trouble than it was worth. > > That's very disappointing. Was there any discussion of > the problems that killed it? I don't remember seeing > any showstoppers being mentioned. I believe the list of incompatibilities and kludges and the subsequent comments in the following file give the gist of the problems: http://svn.python.org/projects/sandbox/trunk/decimal-c/_decimal.c Basically, while it makes a lot of sense to move the *arithmetic* to C (as Mark mentioned in his other post), there's a lot of ancillary stuff related to flags and exceptions and context handling that is much easier to handle in Python. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From chrism at plope.com Thu Mar 27 03:34:14 2008 From: chrism at plope.com (Chris McDonough) Date: Wed, 26 Mar 2008 22:34:14 -0400 Subject: [Python-Dev] how to easily consume just the parts of eggs that are good for you In-Reply-To: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com> References: <913B68B1-C70C-4B56-8223-152954B86EBE@zooko.com> Message-ID: <47EB07A6.6040800@plope.com> zooko wrote: > Folks: > > Here is a simple proposal: make the standard Python "import" > mechanism notice eggs on the PYTHONPATH and insert them (into the > *same* location) on the sys.path. > > This eliminates the #1 problem with eggs -- that they don't easily > work when installing them into places other than your site-packages > and that if you allow any of them to be installed on your system then > they take precedence over your non-egg packages even you explicitly > put those other packages earlier in your PYTHONPATH. (That latter > behavior is very disagreeable to more than a few prorgammers.) Sorry if I'm out of the loop and there's some subtlety here that I'm disregarding, but it doesn't appear that either of the issues you mention is a actually problem with eggs. These are instead problems with how eggs get installed by easy_install (which uses a .pth file to extend sys.path). It's reasonable to put eggs on the PYTHONPATH manually (e.g. sys.path.append('/path/to/some.egg')) instead of using easy_install to install them. I don't think there would be any benefit to changing Python's import machinery to deal with them; they are essentially just directories (or zipfiles) that contain packages. - C From nnorwitz at gmail.com Thu Mar 27 07:56:31 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 26 Mar 2008 23:56:31 -0700 Subject: [Python-Dev] fixing broken build Message-ID: Christian, Please fix the build on the various buildbots that are failing or revert your changes for unicode literals. The build failures started to occur at r61953. There were several more (~5) follow up checkins. You can find all the failures here: http://www.python.org/dev/buildbot/all/ There seem to be at least two variations for how setup.py is failing. See below. n -- Traceback (most recent call last): File "./setup.py", line 6, in import sys, os, imp, re, optparse File "/opt/users/buildbot/slave/trunk.loewis-sun/build/Lib/optparse.py", line 71, in import textwrap File "/opt/users/buildbot/slave/trunk.loewis-sun/build/Lib/textwrap.py", line 32, in class TextWrapper: File "/opt/users/buildbot/slave/trunk.loewis-sun/build/Lib/textwrap.py", line 84, in TextWrapper r'(\s+|' # any whitespace File "/opt/users/buildbot/slave/trunk.loewis-sun/build/Lib/re.py", line 188, in compile return _compile(pattern, flags) File "/opt/users/buildbot/slave/trunk.loewis-sun/build/Lib/re.py", line 239, in _compile raise TypeError, "first argument must be string or compiled pattern" TypeError: first argument must be string or compiled pattern Traceback (most recent call last): File "./setup.py", line 13, in from distutils.core import Extension, setup File "/home/buildbot/slave/py-build/trunk.norwitz-amd64/build/Lib/distutils/core.py", line 21, in from distutils.dist import Distribution File "/home/buildbot/slave/py-build/trunk.norwitz-amd64/build/Lib/distutils/dist.py", line 21, in from distutils.fancy_getopt import FancyGetopt, translate_longopt File "/home/buildbot/slave/py-build/trunk.norwitz-amd64/build/Lib/distutils/fancy_getopt.py", line 32, in longopt_xlate = string.maketrans('-', '_') File "/home/buildbot/slave/py-build/trunk.norwitz-amd64/build/Lib/string.py", line 76, in maketrans return ''.join(L) UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 0: ordinal not in range(128) From greg.ewing at canterbury.ac.nz Thu Mar 27 08:04:05 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 27 Mar 2008 19:04:05 +1200 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47EB010C.9040608@gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> <47E9D9E9.8030905@canterbury.ac.nz> <47E9F3D2.6090400@gmail.com> <47EADA06.9090806@canterbury.ac.nz> <47EB010C.9040608@gmail.com> Message-ID: <47EB46E5.4060203@canterbury.ac.nz> Nick Coghlan wrote: > I believe the list of incompatibilities and kludges and the subsequent > comments in the following file give the gist of the problems: > http://svn.python.org/projects/sandbox/trunk/decimal-c/_decimal.c It sounds like some aspects of the API weren't thought through very well when the Python version was designed. The question now is whether to fix the API design, or leave it to become entrenched and lose all hope of ever having a fully C-coded implementation. -- Greg From lists at cheimes.de Thu Mar 27 09:20:07 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 27 Mar 2008 09:20:07 +0100 Subject: [Python-Dev] fixing broken build In-Reply-To: References: Message-ID: <47EB58B7.8060402@cheimes.de> Neal Norwitz schrieb: > Christian, > > Please fix the build on the various buildbots that are failing or > revert your changes for unicode literals. The build failures started > to occur at r61953. There were several more (~5) follow up checkins. > > You can find all the failures here: http://www.python.org/dev/buildbot/all/ > > There seem to be at least two variations for how setup.py is failing. > See below. I've already fixed the problem in r61956. I didn't noticed the issue with a non initialized var until I compiled Python without pydebug. In order to fix the problem on the build bots one has to remove all pyc and pyo files. Christian From g.brandl at gmx.net Thu Mar 27 09:46:28 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 27 Mar 2008 09:46:28 +0100 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47EB46E5.4060203@canterbury.ac.nz> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> <47E9D9E9.8030905@canterbury.ac.nz> <47E9F3D2.6090400@gmail.com> <47EADA06.9090806@canterbury.ac.nz> <47EB010C.9040608@gmail.com> <47EB46E5.4060203@canterbury.ac.nz> Message-ID: Greg Ewing schrieb: > Nick Coghlan wrote: >> I believe the list of incompatibilities and kludges and the subsequent >> comments in the following file give the gist of the problems: >> http://svn.python.org/projects/sandbox/trunk/decimal-c/_decimal.c > > It sounds like some aspects of the API weren't thought > through very well when the Python version was designed. > > The question now is whether to fix the API design, or > leave it to become entrenched and lose all hope of > ever having a fully C-coded implementation. As Nick said, a drop-in replacement in C isn't feasible But probably users of decimal won't really care if they have to slightly adapt their code if they get the speed increase instead. We had a SOC student working on decimal-c in the past, so it shouldn't be totally dead. What about this year's SOC? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From mal at egenix.com Thu Mar 27 10:19:51 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 27 Mar 2008 10:19:51 +0100 Subject: [Python-Dev] fixing broken build In-Reply-To: <47EB58B7.8060402@cheimes.de> References: <47EB58B7.8060402@cheimes.de> Message-ID: <47EB66B7.6070704@egenix.com> On 2008-03-27 09:20, Christian Heimes wrote: > Neal Norwitz schrieb: >> Christian, >> >> Please fix the build on the various buildbots that are failing or >> revert your changes for unicode literals. The build failures started >> to occur at r61953. There were several more (~5) follow up checkins. >> >> You can find all the failures here: http://www.python.org/dev/buildbot/all/ >> >> There seem to be at least two variations for how setup.py is failing. >> See below. > > I've already fixed the problem in r61956. I didn't noticed the issue > with a non initialized var until I compiled Python without pydebug. In > order to fix the problem on the build bots one has to remove all pyc and > pyo files. I'm not sure why that's necessary, but whenever you change something in the compiler, please remember to update the PYC magic. I'd also suggest that you run a non-debug build of Python to test any checkins before committing them. The debug builds change various ways the code is built. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 27 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From lists at cheimes.de Thu Mar 27 13:13:14 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 27 Mar 2008 13:13:14 +0100 Subject: [Python-Dev] fixing broken build In-Reply-To: <47EB66B7.6070704@egenix.com> References: <47EB58B7.8060402@cheimes.de> <47EB66B7.6070704@egenix.com> Message-ID: <47EB8F5A.20304@cheimes.de> M.-A. Lemburg schrieb: > I'm not sure why that's necessary, but whenever you change something > in the compiler, please remember to update the PYC magic. > > I'd also suggest that you run a non-debug build of Python to test > any checkins before committing them. The debug builds change various > ways the code is built. Changing the pyc magic isn't required here. Some files were compiled to bytecode with the wrong flags because I didn't initialize the flags correctly. I (ab)used a temporary change of the magic to force a recompilation of bytecode. All build bots are fine again. I usually don't test the new code with a non-debug build unless a release is immanent. A full test run already takes a considerable amount of time (>10 minutes) and debug builds usually catch more errors than plain builds. I've to draw the line somewhere ... Christian From christian at cheimes.de Thu Mar 27 13:14:26 2008 From: christian at cheimes.de (Christian Heimes) Date: Thu, 27 Mar 2008 13:14:26 +0100 Subject: [Python-Dev] UCS-4 build on build bots Message-ID: <47EB8FA2.6070608@cheimes.de> Can we have at least one build bot that compiles Python with UCS-4 unicode instead of UCS-2? Christian From asmodai at in-nomine.org Thu Mar 27 13:10:51 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Thu, 27 Mar 2008 13:10:51 +0100 Subject: [Python-Dev] UCS-4 build on build bots In-Reply-To: <47EB8FA2.6070608@cheimes.de> References: <47EB8FA2.6070608@cheimes.de> Message-ID: <20080327121051.GM80617@nexus.in-nomine.org> -On [20080327 13:09], Christian Heimes (christian at cheimes.de) wrote: >Can we have at least one build bot that compiles Python with UCS-4 >unicode instead of UCS-2? Feel free to add another configuration for my machine. Only caveat is the curses test, which I am still tracking. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ To fight and conquer in one hundred battles is not the highest skill. To subdue the enemy with no fight at all, that's the highest skill... From facundobatista at gmail.com Thu Mar 27 13:54:29 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 27 Mar 2008 09:54:29 -0300 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47EB010C.9040608@gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> <47E9D9E9.8030905@canterbury.ac.nz> <47E9F3D2.6090400@gmail.com> <47EADA06.9090806@canterbury.ac.nz> <47EB010C.9040608@gmail.com> Message-ID: 2008/3/26, Nick Coghlan : > Basically, while it makes a lot of sense to move the *arithmetic* to C > (as Mark mentioned in his other post), there's a lot of ancillary stuff > related to flags and exceptions and context handling that is much easier > to handle in Python. That's why we think that the most probably future move here is: 1. Code a small core in C. 2. Let the rest of Decimal in Py. "small core" and "rest of Decimal" are moving targets.... we (as python-dev) could decide that __add__ is really worthy to do it in C, but not quantize(), for example. Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From skip at pobox.com Thu Mar 27 01:52:34 2008 From: skip at pobox.com (skip at pobox.com) Date: Wed, 26 Mar 2008 19:52:34 -0500 Subject: [Python-Dev] [Python-3000] the release gods are angry at python In-Reply-To: <1afaf6160803261410n4e0067a3waa779d62a4fcc777@mail.gmail.com> References: <1afaf6160803261410n4e0067a3waa779d62a4fcc777@mail.gmail.com> Message-ID: <18410.61394.801284.968850@montanaro-dyndns-org.local> >> The next releases of 2.6/3.0 are planned for April 2, just over a >> week from now. There is much work that needs to be done. The >> buildbots are in a pretty sad state and the gods are seeing too much >> red. >> >> http://www.python.org/dev/buildbot/stable/ >> http://www.python.org/dev/buildbot/all/ Is there some reason the "g4 osx.4 trunk" buildbot isn't running? When I checked it this morning it was idle with two pending. I forced a run. I checked later today and it showed five pending. Now it shows 10 pending. It looks like its last run was about 18 hours ago. You would think by now it would have been able to make another run. It pings fine. Skip From dickinsm at gmail.com Thu Mar 27 14:47:38 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Thu, 27 Mar 2008 09:47:38 -0400 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: References: <20080325134617.GB2851@phd.pp.ru> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> <47E9D9E9.8030905@canterbury.ac.nz> <47E9F3D2.6090400@gmail.com> <47EADA06.9090806@canterbury.ac.nz> <47EB010C.9040608@gmail.com> <47EB46E5.4060203@canterbury.ac.nz> Message-ID: <5c6f2a5d0803270647n625f73d1we85e63e6a242888e@mail.gmail.com> On Thu, Mar 27, 2008 at 4:46 AM, Georg Brandl wrote: > As Nick said, a drop-in replacement in C isn't feasible > > But probably users of decimal won't really care if they have to slightly > adapt their code if they get the speed increase instead. > > We had a SOC student working on decimal-c in the past, so it shouldn't be > totally dead. What about this year's SOC? > I worry that rewriting Decimal in C in its entirety would make it significantly harder to maintain. The IBM Decimal Specification hasn't stabilised yet: there's another update to it expected some time after IEEE 754r is finally approved, so there are probably still significant changes to be made to Decimal in the future. I know that I would have contributed a lot less to Decimal had it been written in C, simply because it would have taken me much more time to understand and modify the code. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080327/f1ce3379/attachment.htm From janssen at parc.com Thu Mar 27 19:31:56 2008 From: janssen at parc.com (Bill Janssen) Date: Thu, 27 Mar 2008 11:31:56 PDT Subject: [Python-Dev] [Python-3000] the release gods are angry at python In-Reply-To: References: Message-ID: <08Mar27.113204pdt."58696"@synergy1.parc.xerox.com> > There > have been other tests that have also been flaky like test_asynchat, > test_smtplib, test_ssl, test_urllib2net, test_urllibnet, > test_xmlrpc_net and some of the tests that use networking. Some of the *other* tests that use networking, I think you mean. Sounds like networking tests in general are flaky. > - http://www.python.org/dev/buildbot/all/S-390%20Debian%20trunk/builds/255/step-test/0 Unfortunately, this log has no information about why the test is failing, and I do not have a Debian-unstable machine to try it on (much less a six-processor IBM S/390 mainframe running Debian-unstable -- cool!). I'm unsure about how to make progress here. It's interesting to see that most of the stable buildbots running Debian are doing so on big-endian (PPC, Sparc) hardware. I also can't tell which branch of Python is being tested, from this logfile. That would be nice to add. Is this 2.6 (known problems with SSL) or 3.0 (no known problems with SSL)? Is this information in the logfile somewhere and I'm not seeing it? Bill From fuzzyman at voidspace.org.uk Thu Mar 27 19:40:46 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 27 Mar 2008 18:40:46 +0000 Subject: [Python-Dev] [Python-3000] the release gods are angry at python In-Reply-To: <08Mar27.113204pdt."58696"@synergy1.parc.xerox.com> References: <08Mar27.113204pdt."58696"@synergy1.parc.xerox.com> Message-ID: <47EBEA2E.6020109@voidspace.org.uk> Bill Janssen wrote: >> There >> have been other tests that have also been flaky like test_asynchat, >> test_smtplib, test_ssl, test_urllib2net, test_urllibnet, >> test_xmlrpc_net and some of the tests that use networking. >> > > Some of the *other* tests that use networking, I think you mean. > Sounds like networking tests in general are flaky. > This patch was put together at PyCon (but has not been applied): http://bugs.python.org/issue2429 It moves some of the urllib2net tests to urllib2localnet - they use a local server instead of going out to external servers and should be more reliable. Michael > >> - http://www.python.org/dev/buildbot/all/S-390%20Debian%20trunk/builds/255/step-test/0 >> > > Unfortunately, this log has no information about why the test is > failing, and I do not have a Debian-unstable machine to try it on > (much less a six-processor IBM S/390 mainframe running Debian-unstable > -- cool!). I'm unsure about how to make progress here. It's > interesting to see that most of the stable buildbots running Debian > are doing so on big-endian (PPC, Sparc) hardware. > > I also can't tell which branch of Python is being tested, from this > logfile. That would be nice to add. Is this 2.6 (known problems with > SSL) or 3.0 (no known problems with SSL)? Is this information in the > logfile somewhere and I'm not seeing it? > > Bill > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > From nnorwitz at gmail.com Thu Mar 27 19:43:25 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 27 Mar 2008 11:43:25 -0700 Subject: [Python-Dev] [Python-3000] the release gods are angry at python In-Reply-To: <18410.61394.801284.968850@montanaro-dyndns-org.local> References: <1afaf6160803261410n4e0067a3waa779d62a4fcc777@mail.gmail.com> <18410.61394.801284.968850@montanaro-dyndns-org.local> Message-ID: On Wed, Mar 26, 2008 at 5:52 PM, wrote: > >> The next releases of 2.6/3.0 are planned for April 2, just over a > >> week from now. There is much work that needs to be done. The > >> buildbots are in a pretty sad state and the gods are seeing too much > >> red. > >> > >> http://www.python.org/dev/buildbot/stable/ > >> http://www.python.org/dev/buildbot/all/ > > Is there some reason the "g4 osx.4 trunk" buildbot isn't running? When I > checked it this morning it was idle with two pending. I forced a run. I > checked later today and it showed five pending. Now it shows 10 pending. > It looks like its last run was about 18 hours ago. You would think by now > it would have been able to make another run. It pings fine. I logged in to the machine an the buildbot is running. I think what happens is that sometimes the master loses track of the slaves. The fix requires restarting the master, but I didn't want to do that last night while there were still builds going on. I'll try to find a quiet time tonight and restart the master. I expect that will fix the problem. n From exarkun at divmod.com Thu Mar 27 20:00:37 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Thu, 27 Mar 2008 14:00:37 -0500 Subject: [Python-Dev] [Python-3000] the release gods are angry at python In-Reply-To: Message-ID: <20080327190037.6859.1618375469.divmod.quotient.23604@ohm> On Thu, 27 Mar 2008 11:43:25 -0700, Neal Norwitz wrote: >On Wed, Mar 26, 2008 at 5:52 PM, wrote: >> >> The next releases of 2.6/3.0 are planned for April 2, just over a >> >> week from now. There is much work that needs to be done. The >> >> buildbots are in a pretty sad state and the gods are seeing too much >> >> red. >> >> >> >> http://www.python.org/dev/buildbot/stable/ >> >> http://www.python.org/dev/buildbot/all/ >> >> Is there some reason the "g4 osx.4 trunk" buildbot isn't running? When I >> checked it this morning it was idle with two pending. I forced a run. I >> checked later today and it showed five pending. Now it shows 10 pending. >> It looks like its last run was about 18 hours ago. You would think by now >> it would have been able to make another run. It pings fine. > >I logged in to the machine an the buildbot is running. I think what >happens is that sometimes the master loses track of the slaves. The >fix requires restarting the master, but I didn't want to do that last >night while there were still builds going on. I'll try to find a >quiet time tonight and restart the master. I expect that will fix the >problem. > There's a bug in some configurations (I have never managed to track down the details) where the ping action actually prevents any further builds from happening on that slave until the master is restarted. Not sure if this is related to the problem you're seeing here or not. Jean-Paul From db3l.net at gmail.com Thu Mar 27 20:51:04 2008 From: db3l.net at gmail.com (David Bolen) Date: Thu, 27 Mar 2008 15:51:04 -0400 Subject: [Python-Dev] [Python-3000] the release gods are angry at python References: <20080327190037.6859.1618375469.divmod.quotient.23604@ohm> Message-ID: Jean-Paul Calderone writes: > There's a bug in some configurations (I have never managed to track down > the details) where the ping action actually prevents any further builds > from happening on that slave until the master is restarted. Not sure if > this is related to the problem you're seeing here or not. Over time, I've seen this occasionally in the status for my buildbots (idle, but builds shown as pending, and yes often the last action was a ping). At least in the cases I've observed into so far, restarting the buildbot has typically cleaned it up and let the builds proceed. -- David From musiccomposition at gmail.com Thu Mar 27 21:33:56 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 27 Mar 2008 15:33:56 -0500 Subject: [Python-Dev] Backport of bytearray type and io module In-Reply-To: References: <47E6C946.5020302@cheimes.de> <47E99335.5040306@cheimes.de> <47EA4365.7050908@cheimes.de> <47EA48D0.9060002@cheimes.de> Message-ID: <1afaf6160803271333o1b804498l1de3a06adacb1872@mail.gmail.com> On Wed, Mar 26, 2008 at 11:58 AM, Guido van Rossum wrote: > Yay indeed! Of course the new I/O module will undergo changes (Ping is > working on it still I believe). Try to keep an eye on it so the > improvements can be backported. So improvements to backported features will be merged into 2.6? > > > On Wed, Mar 26, 2008 at 6:00 AM, Facundo Batista > wrote: > > 2008/3/26, Christian Heimes : > > > > > > > Correct! > > > > > > The bytearray type and the new IO system are now backported to > Python 2.6. > > > > Thank you very much for this effort! > > > > Regards, > > > > > > > > -- > > . Facundo > > > > Blog: http://www.taniquetil.com.ar/plog/ > > PyAr: http://www.python.org/ar/ > > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/ > ) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -- Cheers, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080327/f93b8064/attachment.htm From schmir at gmail.com Thu Mar 27 22:20:01 2008 From: schmir at gmail.com (Ralf Schmitt) Date: Thu, 27 Mar 2008 22:20:01 +0100 Subject: [Python-Dev] [Python-3000] the release gods are angry at python In-Reply-To: References: Message-ID: <932f8baf0803271420y6288e25cu65e0980822cf9b50@mail.gmail.com> On Wed, Mar 26, 2008 at 7:21 AM, Neal Norwitz wrote: > * test_xmlrpc transient socket errors > - > http://www.python.org/dev/buildbot/stable/g4%20osx.4%20trunk/builds/3101/step-test/0 > > These are caused by the accept call returning a nonblocking socket, when the listening socket itself is nonblocking (which it is). see http://bugs.python.org/issue1503 for more details. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080327/b54e49b0/attachment.htm From musiccomposition at gmail.com Thu Mar 27 22:25:21 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 27 Mar 2008 16:25:21 -0500 Subject: [Python-Dev] Commit access request In-Reply-To: <1afaf6160803251327l26a67491yea3bf852ba5b9d2d@mail.gmail.com> References: <1afaf6160803240456u7ce0aa53t19514771e3434b30@mail.gmail.com> <1afaf6160803251327l26a67491yea3bf852ba5b9d2d@mail.gmail.com> Message-ID: <1afaf6160803271425t180be227t865b04b667c3ab27@mail.gmail.com> On Tue, Mar 25, 2008 at 3:27 PM, Benjamin Peterson < musiccomposition at gmail.com> wrote: > > > On Tue, Mar 25, 2008 at 12:40 PM, Georg Brandl wrote: > > > Georg Brandl schrieb: > > > Benjamin Peterson schrieb: > > >> Hi Python devs, > > >> I have been contributing to since December. (See me first issue on > > the > > >> tracker, #1828; it was a major learning experience.) :P In that time, > > I > > >> have contributed many patches and actively participated on this list. > > >> This will enable me to help triage bugs on the tracker, something > > I've > > >> been wanting to help with. > > > > > > I'm +1 to Benjamin getting commit privileges, letting commits sign off > > > by a senior developer. > > > > Since there are no objections, I've now added Benjamin's key, under that > > condition. > > Thanks! > > > > > > > Welcome on board, Benjamin! :) > > One more question: Do I get a @python.org email alias? > Glad to help! > > > > > > > Georg > > > > > > -- > > Thus spake the Lord: Thou shalt indent with four spaces. No more, no > > less. > > Four shall be the number of spaces thou shalt indent, and the number of > > thy > > indenting shall be four. Eight shalt thou not indent, nor either indent > > thou > > two, excepting that thou then proceed to four. Tabs are right out. > > > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > > > > > > -- > Cheers, > Benjamin Peterson -- Cheers, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080327/13b672e4/attachment.htm From brett at python.org Thu Mar 27 22:31:34 2008 From: brett at python.org (Brett Cannon) Date: Thu, 27 Mar 2008 14:31:34 -0700 Subject: [Python-Dev] Commit access request In-Reply-To: <1afaf6160803271425t180be227t865b04b667c3ab27@mail.gmail.com> References: <1afaf6160803240456u7ce0aa53t19514771e3434b30@mail.gmail.com> <1afaf6160803251327l26a67491yea3bf852ba5b9d2d@mail.gmail.com> <1afaf6160803271425t180be227t865b04b667c3ab27@mail.gmail.com> Message-ID: On Thu, Mar 27, 2008 at 2:25 PM, Benjamin Peterson wrote: > > > > On Tue, Mar 25, 2008 at 3:27 PM, Benjamin Peterson > wrote: > > > > > > > > > > On Tue, Mar 25, 2008 at 12:40 PM, Georg Brandl wrote: > > > > > Georg Brandl schrieb: > > > > > > > Benjamin Peterson schrieb: > > > >> Hi Python devs, > > > >> I have been contributing to since December. (See me first issue on > the > > > >> tracker, #1828; it was a major learning experience.) :P In that time, > I > > > >> have contributed many patches and actively participated on this list. > > > >> This will enable me to help triage bugs on the tracker, something > I've > > > >> been wanting to help with. > > > > > > > > I'm +1 to Benjamin getting commit privileges, letting commits sign off > > > > by a senior developer. > > > > > > Since there are no objections, I've now added Benjamin's key, under that > > > condition. > > > > Thanks! > > > > > > > > > > > Welcome on board, Benjamin! :) > > > One more question: Do I get a @python.org email alias? Not by default. That has typically been given to people who have contributed to the web site. -Brett From greg.ewing at canterbury.ac.nz Fri Mar 28 00:12:32 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 28 Mar 2008 11:12:32 +1200 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> <47E9D9E9.8030905@canterbury.ac.nz> <47E9F3D2.6090400@gmail.com> <47EADA06.9090806@canterbury.ac.nz> <47EB010C.9040608@gmail.com> <47EB46E5.4060203@canterbury.ac.nz> Message-ID: <47EC29E0.5020209@canterbury.ac.nz> Georg Brandl wrote: > As Nick said, a drop-in replacement in C isn't feasible Yes, but that appears to be so only because of some unfortunate features of the Python version's API. Seems to me it would be better to undergo a little pain now and get a well-designed C-friendly API. -- Greg From dickinsm at gmail.com Fri Mar 28 02:07:40 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Thu, 27 Mar 2008 21:07:40 -0400 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: References: <20080325134617.GB2851@phd.pp.ru> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> <47E9D9E9.8030905@canterbury.ac.nz> <47E9F3D2.6090400@gmail.com> <47EADA06.9090806@canterbury.ac.nz> <47EB010C.9040608@gmail.com> <47EB46E5.4060203@canterbury.ac.nz> Message-ID: <5c6f2a5d0803271807r608f0328m14af4c78c8e9d519@mail.gmail.com> On Thu, Mar 27, 2008 at 4:46 AM, Georg Brandl wrote: > > As Nick said, a drop-in replacement in C isn't feasible > > But probably users of decimal won't really care if they have to slightly > adapt their code if they get the speed increase instead. > Could you give me an example of the sort of adaptations that might be necessary, or the API changes that would be necessary to make a drop-in replacement possible? Are you just talking about things like removing the context argument from __add__ and friends, or is it more serious than this? Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080327/d04b6395/attachment-0001.htm From nnorwitz at gmail.com Fri Mar 28 06:41:26 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 27 Mar 2008 22:41:26 -0700 Subject: [Python-Dev] [Python-3000] the release gods are angry at python In-Reply-To: <8526353492909807871@unknownmsgid> References: <8526353492909807871@unknownmsgid> Message-ID: On Thu, Mar 27, 2008 at 11:31 AM, Bill Janssen wrote: > > There > > have been other tests that have also been flaky like test_asynchat, > > test_smtplib, test_ssl, test_urllib2net, test_urllibnet, > > test_xmlrpc_net and some of the tests that use networking. > > Some of the *other* tests that use networking, I think you mean. Yes, primarily networking tests. Also things like signals and threading. > Sounds like networking tests in general are flaky. Anything that connects to a remote host is definitely flaky. Most of the tests that connect to localhost are flaky for one reason or another. One reason used to be port allocation. That is mostly fixed now. The big reason that most tests fail now is EAGAIN or some other error, typically on non-blocking sockets. Switching to blocking sockets isn't really a great option since the tests are much more likely to hang. > > - http://www.python.org/dev/buildbot/all/S-390%20Debian%20trunk/builds/255/step-test/0 > > Unfortunately, this log has no information about why the test is > failing, and I do not have a Debian-unstable machine to try it on > (much less a six-processor IBM S/390 mainframe running Debian-unstable > -- cool!). I'm unsure about how to make progress here. It's > interesting to see that most of the stable buildbots running Debian > are doing so on big-endian (PPC, Sparc) hardware. > > I also can't tell which branch of Python is being tested, from this > logfile. That would be nice to add. Is this 2.6 (known problems with > SSL) or 3.0 (no known problems with SSL)? Is this information in the > logfile somewhere and I'm not seeing it? The information happens to be encoded in the URL. The URL above is for trunk. You can see more info here: http://www.python.org/dev/buildbot/all/ That is the landing page for all the info you could want. Well, mostly. The logs often don't really have enough info to debug the problem. In that case, it would be better to add more debugging to the tests. n From ncoghlan at gmail.com Fri Mar 28 08:46:47 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 28 Mar 2008 17:46:47 +1000 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47EC29E0.5020209@canterbury.ac.nz> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> <47E9D9E9.8030905@canterbury.ac.nz> <47E9F3D2.6090400@gmail.com> <47EADA06.9090806@canterbury.ac.nz> <47EB010C.9040608@gmail.com> <47EB46E5.4060203@canterbury.ac.nz> <47EC29E0.5020209@canterbury.ac.nz> Message-ID: <47ECA267.6010702@gmail.com> Greg Ewing wrote: > Georg Brandl wrote: > >> As Nick said, a drop-in replacement in C isn't feasible > > Yes, but that appears to be so only because of some > unfortunate features of the Python version's API. > > Seems to me it would be better to undergo a little > pain now and get a well-designed C-friendly API. What features do you find particularly unfortunate? Just because something isn't particularly amenable to implementation in C, doesn't make it a bad API for a Python library (e.g. the dicts to enable/signal the different error traps are a natural interface for Python code, even though C code would be inclined to use a bit field for the same thing). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From g.brandl at gmx.net Fri Mar 28 10:24:51 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 28 Mar 2008 10:24:51 +0100 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <5c6f2a5d0803271807r608f0328m14af4c78c8e9d519@mail.gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> <47E9D9E9.8030905@canterbury.ac.nz> <47E9F3D2.6090400@gmail.com> <47EADA06.9090806@canterbury.ac.nz> <47EB010C.9040608@gmail.com> <47EB46E5.4060203@canterbury.ac.nz> <5c6f2a5d0803271807r608f0328m14af4c78c8e9d519@mail.gmail.com> Message-ID: Mark Dickinson schrieb: > On Thu, Mar 27, 2008 at 4:46 AM, Georg Brandl > wrote: > > > As Nick said, a drop-in replacement in C isn't feasible > > But probably users of decimal won't really care if they have to slightly > adapt their code if they get the speed increase instead. > > > Could you give me an example of the sort of adaptations that might be > necessary, or the API changes that would be necessary to make a > drop-in replacement possible? > > Are you just talking about things like removing the context argument > from __add__ and friends, or is it more serious than this? One thing Nick already said is the handling of signal traps via a dict with class keys -- with that, it's necessary to check the dict all the time. Having e.g. a method to set traps would enable the implementation to work with a bit field. There were some other things I can't remember right now, but this was the most problematic one. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From lists at cheimes.de Fri Mar 28 10:48:28 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 28 Mar 2008 10:48:28 +0100 Subject: [Python-Dev] Windows build bot failure Message-ID: Some Windows build bots can't compile openssl. The MS assembler fails to assemble a file. I *think* the issue can be fixed by using nasm32 instead of masm. Christian ml /Cp /coff /c /Cx /Zi /Focrypto\sha\asm\s1_win32.obj .\crypto\sha\asm\s1_win32.asm Microsoft (R) Macro Assembler Version 7.10.6030 Copyright (C) Microsoft Corporation. All rights reserved. Assembling: .\crypto\sha\asm\s1_win32.asm ml /Cp /coff /c /Cx /Zi /Focrypto\sha\asm\sha512-sse2.obj .\crypto\sha\asm\sha512-sse2.asm Microsoft (R) Macro Assembler Version 7.10.6030 Copyright (C) Microsoft Corporation. All rights reserved. Assembling: .\crypto\sha\asm\sha512-sse2.asm .\crypto\sha\asm\sha512-sse2.asm(29) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(30) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(31) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(32) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(35) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(36) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(37) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(38) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(125) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(138) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(151) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(154) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(155) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(156) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(278) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(279) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(280) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(281) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(282) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(283) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(284) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(285) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(286) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(287) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(288) : error A2006: undefined symbol : XMMWORD .\crypto\sha\asm\sha512-sse2.asm(289) : error A2006: undefined symbol : XMMWORD NMAKE : fatal error U1077: 'ml' : return code '0x1' Stop. Executing ms\d32.mak failed 2 Build log was saved at "file://s:\buildbots\python\2.5.nelson-windows\build\PCbuild\x86-temp-debug\_ssl\BuildLog.htm" _ssl - 27 error(s), 0 warning(s) From asmodai at in-nomine.org Fri Mar 28 10:52:58 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Fri, 28 Mar 2008 10:52:58 +0100 Subject: [Python-Dev] FreeBSD test suite failure -> curses In-Reply-To: <47D46BB2.1080202@v.loewis.de> References: <20080308092434.GZ60713@nexus.in-nomine.org> <47D273B0.4020809@v.loewis.de> <20080309162900.GA60713@nexus.in-nomine.org> <47D43926.4030902@v.loewis.de> <20080309210210.GB60713@nexus.in-nomine.org> <47D46BB2.1080202@v.loewis.de> Message-ID: <20080328095258.GS80617@nexus.in-nomine.org> -On [20080309 23:59], "Martin v. L?wis" (martin at v.loewis.de) wrote: >So this now *is* a FreeBSD/ncurses expert's question. Actually, given my recent results and discussion with Thomas Dickey I am not so sure. Some basic debugging I did with Georg Brandl led to this: comment the requires('curses') line in test_curses.py. Then start python in the build root with ./pyton; from the interactive prompt I do import curses and import test.test_curses and python subsequently coredumps after some screen manipulation. Setting an awatch with gdb on newscr shows that at one point it gets set to 0. When I put import curses and import test.test_curses in a file and subsequently execute this with ./python the test passes without any problems. Thomas mentioned after seeing an ltrace: Perhaps it's failing on that: curses.setupterm(fd=sys.__stdout__.fileno()) That would have newscr null. The failure might be from closing stdout, e.g., if it was redirected. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ We've laid together in the sun before the mind-rape has begun... From asmodai at in-nomine.org Fri Mar 28 10:56:27 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Fri, 28 Mar 2008 10:56:27 +0100 Subject: [Python-Dev] Windows build bot failure In-Reply-To: References: Message-ID: <20080328095627.GT80617@nexus.in-nomine.org> -On [20080328 10:44], Christian Heimes (lists at cheimes.de) wrote: >Some Windows build bots can't compile openssl. The MS assembler fails to >assemble a file. I *think* the issue can be fixed by using nasm32 >instead of masm. A posting to the OpenSSL list on 2007-10-22: It's a problem with older versions of MASM. The following patch works around this issue: http://cvs.openssl.org/chngview?cn=16708 However MASM is being phased out in OpenSSL (it wont be supported at all in 0.9.9) so you are advised to switch to the free NASM instead which doesn't have such problems. (http://www.mail-archive.com/openssl-users at openssl.org/msg50725.html) -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ Atone me to my throes curtail... From g.brandl at gmx.net Fri Mar 28 11:50:13 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 28 Mar 2008 11:50:13 +0100 Subject: [Python-Dev] nested classes leaking in compiler Message-ID: While preparing the Python-AST compilation patch, I noticed that each class nested in a class leaks one reference (2.5 and trunk). It wasn't found by regrtest -R because it only happens on compiling, and it seems that all snippets compiled during the tests as opposed to on import didn't contain such a construct. The AST generation stage is fine, the leak happens somehwere after that (I suspect the symtable code). It would be nice if someone who understands more about that code than I do could fix this. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From skip at pobox.com Fri Mar 28 11:31:44 2008 From: skip at pobox.com (skip at pobox.com) Date: Fri, 28 Mar 2008 05:31:44 -0500 Subject: [Python-Dev] [Python-3000] the release gods are angry at python In-Reply-To: References: <8526353492909807871@unknownmsgid> Message-ID: <18412.51472.322043.137606@montanaro-dyndns-org.local> Neal> Anything that connects to a remote host is definitely flaky. Would it maybe help to set up a dedicated host (or virtual host) to serve as the sole target of all network tests? Skip From status at bugs.python.org Fri Mar 28 18:06:22 2008 From: status at bugs.python.org (Tracker) Date: Fri, 28 Mar 2008 18:06:22 +0100 (CET) Subject: [Python-Dev] Summary of Tracker Issues Message-ID: <20080328170622.889D6782F1@psf.upfronthosting.co.za> ACTIVITY SUMMARY (03/21/08 - 03/28/08) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1794 open (+40) / 12535 closed (+18) / 14329 total (+58) Open issues with patches: 528 Average duration of open issues: 715 days. Median duration of open issues: 1243 days. Open Issues Breakdown open 1770 (+40) pending 24 ( +0) Issues Created Or Reopened (58) _______________________________ 2to3 translates "import foobar" to "import .foobar" rather than 03/21/08 http://bugs.python.org/issue2446 created dangyogi Python 2.6 refleak test issues 03/21/08 http://bugs.python.org/issue2447 created tiran namedtuple breaks refleak tests 03/21/08 http://bugs.python.org/issue2448 created tiran Improved serialization error logging in xmlrpclib 03/21/08 http://bugs.python.org/issue2449 created blunck2 urllib2.Request constructor has no timeout parameter 03/21/08 CLOSED http://bugs.python.org/issue2450 created jjlee No way to disable socket timeouts in httplib, etc. 03/27/08 http://bugs.python.org/issue2451 reopened facundobatista inaccuracy in httplib timeout documentation 03/21/08 http://bugs.python.org/issue2452 created jjlee fix_except needs to allow for empty excepts 03/22/08 CLOSED http://bugs.python.org/issue2453 created loewis sha and md5 fixer 03/22/08 http://bugs.python.org/issue2454 created loewis stat.ST_CTIME and stat.ST_ATIME problem 03/22/08 http://bugs.python.org/issue2455 created sassas Make sysmodule.c compatible with Bazaar 03/22/08 http://bugs.python.org/issue2456 created barry add --help and -h options to pdb 03/22/08 CLOSED http://bugs.python.org/issue2457 created benjamin.peterson patch Allow Python code to change Py3k warning flag 03/22/08 http://bugs.python.org/issue2458 created benjamin.peterson patch speedup loops with better bytecode 03/22/08 http://bugs.python.org/issue2459 created pitrou patch Ellipsis not copyable 03/22/08 CLOSED http://bugs.python.org/issue2460 created aronacher patch test_util.py for distutils 03/23/08 http://bugs.python.org/issue2461 created tarek patch python.exe slowing my system 03/23/08 CLOSED http://bugs.python.org/issue2462 created FireSnake python.exe slowing my system 03/23/08 CLOSED http://bugs.python.org/issue2463 created FireSnake urllib2 can't handle http://www.wikispaces.com 03/23/08 http://bugs.python.org/issue2464 created weijie90 sphinx-quickstart.py still creates makefile even if user tells i 03/23/08 CLOSED http://bugs.python.org/issue2465 created varmaa patch os.path.ismount doesn't work for NTFS mounts 03/23/08 http://bugs.python.org/issue2466 created rossburton gc.DEBUG_STATS reports invalid "elapsed" times 03/23/08 http://bugs.python.org/issue2467 created exarkun patch izip fixer generates incorrect import statement 03/23/08 CLOSED http://bugs.python.org/issue2468 created loewis Fix build error in unicodeobject.c UCS4 03/23/08 CLOSED http://bugs.python.org/issue2469 created benjamin.peterson patch Need fixer for dl (removed) -> ctypes module 03/24/08 http://bugs.python.org/issue2470 created nnorwitz imp.get_magic() should return bytes, not bytearray 03/24/08 http://bugs.python.org/issue2471 created brett.cannon patch Fixed block ordering code in compiler.pyassem 03/24/08 http://bugs.python.org/issue2472 created pitrou patch logging.LogRecord should know how to handle dict-like args 03/24/08 http://bugs.python.org/issue2473 created wcmaier fset not working 03/24/08 CLOSED http://bugs.python.org/issue2474 created ryansturmer Popen.poll always returns None 03/24/08 http://bugs.python.org/issue2475 created jjcogliati optparse docs should mention %default being new in 2.4 03/24/08 CLOSED http://bugs.python.org/issue2476 created brodierao parser support for future import of unicode_strings 03/25/08 http://bugs.python.org/issue2477 created nnorwitz patch decimal.Decimal( 0 ).sqrt() fails 03/25/08 CLOSED http://bugs.python.org/issue2478 created glathoud Bytearray and io backport 03/25/08 http://bugs.python.org/issue2479 created tiran 26backport pickling of large recursive structures fails 03/25/08 http://bugs.python.org/issue2480 created cyhawk locale.strxfrm does not work with Unicode strings 03/25/08 http://bugs.python.org/issue2481 created cito Decimal(unicode) 03/25/08 CLOSED http://bugs.python.org/issue2482 created phd int and float accept bytes, complex does not 03/25/08 http://bugs.python.org/issue2483 created marketdickinson Cosmetic patch for warning "unused variable" 03/25/08 CLOSED http://bugs.python.org/issue2484 created ocean-city patch, patch Traceback changed in 2.6 for unhashable objects 03/25/08 http://bugs.python.org/issue2485 created mg Consider using bytes type instead of str to store Decimal coeffi 03/25/08 http://bugs.python.org/issue2486 created marketdickinson patch ldexp(x,n) misbehaves when abs(n) is large 03/25/08 http://bugs.python.org/issue2487 created fredrikj Backport sys.maxsize to Python 2.6 03/25/08 http://bugs.python.org/issue2488 created tiran patch, easy, 26backport Patch for bugs in pty.py 03/26/08 http://bugs.python.org/issue2489 created fergushenderson patch Assertion failure in datetime.strftime() 03/26/08 CLOSED http://bugs.python.org/issue2490 created genepi io.open() handles errors differently on different platforms 03/26/08 http://bugs.python.org/issue2491 created nnorwitz Check implementation of new buffer interface for PyString in 2.6 03/26/08 http://bugs.python.org/issue2492 created tiran 26backport Remove unused constants from optimized code objects 03/26/08 http://bugs.python.org/issue2493 created belopolsky patch Can't round-trip datetimes<->timestamps prior to 1970 on Windows 03/27/08 http://bugs.python.org/issue2494 created mark tokenize doesn't handle __future__.unicode_literals correctly 03/27/08 CLOSED http://bugs.python.org/issue2495 created tiran test_no_refcycle_through_target sometimes fails in test_threadin 03/27/08 CLOSED http://bugs.python.org/issue2496 created pitrou patch stdbool support 03/27/08 http://bugs.python.org/issue2497 created rolland patch bdb modernized 03/27/08 http://bugs.python.org/issue2498 created benjamin.peterson patch Fold unary + and not on constants 03/28/08 http://bugs.python.org/issue2499 created belopolsky patch Sync _sqlite module code 03/28/08 http://bugs.python.org/issue2500 created tiran xml.sax.parser() doesn't terminate when given a filename 03/28/08 http://bugs.python.org/issue2501 created mark Add enum() example for named tuples 03/28/08 CLOSED http://bugs.python.org/issue2502 created calvin patch Replace "== None/True/False" with "is" 03/28/08 http://bugs.python.org/issue2503 created calvin patch Issues Now Closed (63) ______________________ zlib.crc32() and adler32() return value 181 days http://bugs.python.org/issue1202 gregory.p.smith 64bit, easy Force BOM option in UTF output. 149 days http://bugs.python.org/issue1328 doerwalter UnicodeDecodeError that cannot be caught in narrow unicode build 125 days http://bugs.python.org/issue1477 amaury.forgeotdarc patch shutil.move() does not use os.rename() if dst is a directory 103 days http://bugs.python.org/issue1577 draghuram patch filehandle.write() does not return None (Python 3.0a2) 72 days http://bugs.python.org/issue1775 georg.brandl AST compile() patch 77 days http://bugs.python.org/issue1810 georg.brandl patch Add key argument to heapq functions 62 days http://bugs.python.org/issue1904 rhettinger patch weak references are removed before __del__ is called. 59 days http://bugs.python.org/issue1918 georg.brandl test_largefile.py converted to unittest 48 days http://bugs.python.org/issue2026 benjamin.peterson patch urllib2 basic auth handler doesn't handle realm names in single- 33 days http://bugs.python.org/issue2136 georg.brandl patch Document PyImport_GetImporter 29 days http://bugs.python.org/issue2160 georg.brandl patch unittest.findTestCases undocumented 28 days http://bugs.python.org/issue2162 georg.brandl Add doc-string to UserDict and DictMixin 30 days http://bugs.python.org/issue2172 belopolsky patch setitimer, getitimer wrapper 19 days http://bugs.python.org/issue2240 loewis patch quit() method of SMTP instance (of smtplib) doesn't return it's 20 days http://bugs.python.org/issue2248 georg.brandl patch test_decimal: possible thread lockup in test case 10 days http://bugs.python.org/issue2273 facundobatista patch Patch for "string without null bytes" check in getargs.c 6 days http://bugs.python.org/issue2298 georg.brandl patch make html fails 5 days http://bugs.python.org/issue2300 georg.brandl patch Py3K warn against using __members__ 4 days http://bugs.python.org/issue2346 georg.brandl patch, 26backport Py3K warn for using __methods__ 4 days http://bugs.python.org/issue2347 georg.brandl 26backport Py3K warn using file.softspace 4 days http://bugs.python.org/issue2348 georg.brandl patch, 26backport cmp argument to list.sort()/sorted() should raise a Py3K warning 4 days http://bugs.python.org/issue2354 rhettinger patch, 26backport Using buffer() should raise a Py3K warning 8 days http://bugs.python.org/issue2355 georg.brandl patch, 26backport Using sys.exc_clear should raise a Py3K warning 4 days http://bugs.python.org/issue2358 georg.brandl patch, 26backport A Py3K warning for array.array.{read,write} is needed 8 days http://bugs.python.org/issue2359 georg.brandl patch, 26backport Merge 2.6 ACKS with 3.0 ACKS 7 days http://bugs.python.org/issue2390 benjamin.peterson easy Improvement suggestions for the gzip module documentation 9 days http://bugs.python.org/issue2406 georg.brandl patch pysqlite provides no interface for database SQL dump 9 days http://bugs.python.org/issue2426 gregory.p.smith patch DictReader does not suport line_num 1 days http://bugs.python.org/issue2432 georg.brandl Undocumented features added to 2.6 1 days http://bugs.python.org/issue2442 georg.brandl easy urllib2.Request constructor has no timeout parameter 0 days http://bugs.python.org/issue2450 jjlee fix_except needs to allow for empty excepts 6 days http://bugs.python.org/issue2453 collinwinter add --help and -h options to pdb 4 days http://bugs.python.org/issue2457 benjamin.peterson patch Ellipsis not copyable 1 days http://bugs.python.org/issue2460 rhettinger patch python.exe slowing my system 1 days http://bugs.python.org/issue2462 FireSnake python.exe slowing my system 0 days http://bugs.python.org/issue2463 tiran sphinx-quickstart.py still creates makefile even if user tells i 0 days http://bugs.python.org/issue2465 georg.brandl patch izip fixer generates incorrect import statement 0 days http://bugs.python.org/issue2468 David Wolever Fix build error in unicodeobject.c UCS4 1 days http://bugs.python.org/issue2469 amaury.forgeotdarc patch fset not working 0 days http://bugs.python.org/issue2474 georg.brandl optparse docs should mention %default being new in 2.4 0 days http://bugs.python.org/issue2476 georg.brandl decimal.Decimal( 0 ).sqrt() fails 0 days http://bugs.python.org/issue2478 facundobatista Decimal(unicode) 0 days http://bugs.python.org/issue2482 marketdickinson Cosmetic patch for warning "unused variable" 1 days http://bugs.python.org/issue2484 georg.brandl patch, patch Assertion failure in datetime.strftime() 1 days http://bugs.python.org/issue2490 georg.brandl tokenize doesn't handle __future__.unicode_literals correctly 0 days http://bugs.python.org/issue2495 amaury.forgeotdarc test_no_refcycle_through_target sometimes fails in test_threadin 1 days http://bugs.python.org/issue2496 jyasskin patch Add enum() example for named tuples 0 days http://bugs.python.org/issue2502 amaury.forgeotdarc patch provide a documented serialization func 2362 days http://bugs.python.org/issue467384 gvanrossum default index for __getslice__ is not sys.maxint 1481 days http://bugs.python.org/issue908441 georg.brandl Heap class for heapq module 1107 days http://bugs.python.org/issue1162363 rhettinger patch compiler package: "global a; a=5" 964 days http://bugs.python.org/issue1251748 pitrou patch import dynamic library bug? 954 days http://bugs.python.org/issue1261390 georg.brandl bsddb: segfault on db.associate call with Txn and large data 793 days http://bugs.python.org/issue1413192 jcea bisect should support a custom comparison function 739 days http://bugs.python.org/issue1451588 rhettinger custom comparison function for bisect module 724 days http://bugs.python.org/issue1462228 rhettinger bisect on presorted list 460 days http://bugs.python.org/issue1619060 rhettinger Add triangular distribution to random 374 days http://bugs.python.org/issue1681432 rhettinger patch Duplicate "preferences" menu item/Tk Aqua 8.4.14 359 days http://bugs.python.org/issue1691411 georg.brandl patch audioop module - request to add a note 344 days http://bugs.python.org/issue1700821 georg.brandl slice type is unhashable 291 days http://bugs.python.org/issue1733184 belopolsky re.findall hangs python completely 281 days http://bugs.python.org/issue1737127 georg.brandl struni: test_urllib2, test_cookielib 238 days http://bugs.python.org/issue1762940 loewis patch Top Issues Most Discussed (10) ______________________________ 15 speedup loops with better bytecode 6 days open http://bugs.python.org/issue2459 10 Merge 2.6 ACKS with 3.0 ACKS 7 days closed http://bugs.python.org/issue2390 8 compiler package: "global a; a=5" 964 days closed http://bugs.python.org/issue1251748 7 test_no_refcycle_through_target sometimes fails in test_threadi 1 days closed http://bugs.python.org/issue2496 7 parser support for future import of unicode_strings 3 days open http://bugs.python.org/issue2477 7 Adding __iter__ to class Values of module optparse 7 days open http://bugs.python.org/issue2444 7 Semi autogenerated _types module 107 days open http://bugs.python.org/issue1605 7 test_xmlrpc is still flakey 123 days open http://bugs.python.org/issue1503 6 No way to disable socket timeouts in httplib, etc. 1 days open http://bugs.python.org/issue2451 6 uninitialized access to va_list 7 days open http://bugs.python.org/issue2443 From amauryfa at gmail.com Fri Mar 28 18:46:29 2008 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Fri, 28 Mar 2008 18:46:29 +0100 Subject: [Python-Dev] nested classes leaking in compiler In-Reply-To: References: Message-ID: On Fri, Mar 28, 2008 at 11:50 AM, Georg Brandl wrote: > While preparing the Python-AST compilation patch, I noticed that each > class nested in a class leaks one reference (2.5 and trunk). > > It wasn't found by regrtest -R because it only happens on compiling, > and it seems that all snippets compiled during the tests as opposed to > on import didn't contain such a construct. > > The AST generation stage is fine, the leak happens somehwere > after that (I suspect the symtable code). It would be nice if someone > who understands more about that code than I do could fix this. The problem is actually in compile.c, in compiler_class(): Each compilation scope holds the name of the enclosing class (for __name mangling). In the case of nested classes, the inner scope is first initialized with the outer class name, then replaced with the inner class name. But a Py_XDECREF(c->u->u_private) is missing here... I will submit the correction in a few hours. -- Amaury Forgeot d'Arc From nnorwitz at gmail.com Fri Mar 28 18:48:54 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Fri, 28 Mar 2008 10:48:54 -0700 Subject: [Python-Dev] [Python-3000] the release gods are angry at python In-Reply-To: <18412.51472.322043.137606@montanaro-dyndns-org.local> References: <8526353492909807871@unknownmsgid> <18412.51472.322043.137606@montanaro-dyndns-org.local> Message-ID: On Fri, Mar 28, 2008 at 3:31 AM, wrote: > > Neal> Anything that connects to a remote host is definitely flaky. > > Would it maybe help to set up a dedicated host (or virtual host) to serve as > the sole target of all network tests? It would help, but not fix the problem. There will still be transient network issues that come up. n From collinw at gmail.com Fri Mar 28 18:51:15 2008 From: collinw at gmail.com (Collin Winter) Date: Fri, 28 Mar 2008 10:51:15 -0700 Subject: [Python-Dev] Removing callable() from the stdlib Message-ID: <43aa6ff70803281051ufa612day856b9cf3c84e2fb7@mail.gmail.com> I've been running 2to3's fix_callable over 2.6's stdlib to remove -3 warnings due to callable() usage; 2to3 replaces callable(x) with has_attr(x, "__call__"). Unfortunately, this breaks code that wants to run callable() on old-style classes (like test_builtin), because old-style classes don't have a __call__ attribute (new-style classes do, however). How should this be handled? I'm tempted to just add __call__ to old-style classes for 2.6, but Neal Norwitz thought that might break some user code. Any other thoughts? Collin Winter From janssen at parc.com Fri Mar 28 19:15:52 2008 From: janssen at parc.com (Bill Janssen) Date: Fri, 28 Mar 2008 11:15:52 PDT Subject: [Python-Dev] [Python-3000] the release gods are angry at python In-Reply-To: References: <8526353492909807871@unknownmsgid> <18412.51472.322043.137606@montanaro-dyndns-org.local> Message-ID: <08Mar28.111601pdt."58696"@synergy1.parc.xerox.com> > On Fri, Mar 28, 2008 at 3:31 AM, wrote: > > > > Neal> Anything that connects to a remote host is definitely flaky. > > > > Would it maybe help to set up a dedicated host (or virtual host) to serve as > > the sole target of all network tests? > > It would help, but not fix the problem. There will still be transient > network issues that come up. We really need to be able to wrap a timeout around any individual test, perhaps throwing a TimeoutException if the timeout is reached, and have the ability to regard a TimeoutException as not being a "failure". Bill From g.brandl at gmx.net Fri Mar 28 19:01:46 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 28 Mar 2008 19:01:46 +0100 Subject: [Python-Dev] nested classes leaking in compiler In-Reply-To: References: Message-ID: Amaury Forgeot d'Arc schrieb: > On Fri, Mar 28, 2008 at 11:50 AM, Georg Brandl wrote: >> While preparing the Python-AST compilation patch, I noticed that each >> class nested in a class leaks one reference (2.5 and trunk). >> >> It wasn't found by regrtest -R because it only happens on compiling, >> and it seems that all snippets compiled during the tests as opposed to >> on import didn't contain such a construct. >> >> The AST generation stage is fine, the leak happens somehwere >> after that (I suspect the symtable code). It would be nice if someone >> who understands more about that code than I do could fix this. > > The problem is actually in compile.c, in compiler_class(): > Each compilation scope holds the name of the enclosing class (for > __name mangling). > In the case of nested classes, the inner scope is first initialized > with the outer class name, then replaced with the inner class name. > But a Py_XDECREF(c->u->u_private) is missing here... > > I will submit the correction in a few hours. Somehow I knew you'd be the one to find the problem :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From guido at python.org Fri Mar 28 20:58:03 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 28 Mar 2008 12:58:03 -0700 Subject: [Python-Dev] Removing callable() from the stdlib In-Reply-To: <43aa6ff70803281051ufa612day856b9cf3c84e2fb7@mail.gmail.com> References: <43aa6ff70803281051ufa612day856b9cf3c84e2fb7@mail.gmail.com> Message-ID: On Fri, Mar 28, 2008 at 10:51 AM, Collin Winter wrote: > I've been running 2to3's fix_callable over 2.6's stdlib to remove -3 > warnings due to callable() usage; 2to3 replaces callable(x) with > has_attr(x, "__call__"). Unfortunately, this breaks code that wants to > run callable() on old-style classes (like test_builtin), because > old-style classes don't have a __call__ attribute (new-style classes > do, however). > > How should this be handled? I'm tempted to just add __call__ to > old-style classes for 2.6, but Neal Norwitz thought that might break > some user code. Any other thoughts? I don't see how this would break user code, as long as the __call__ shows up for the *class* but not for the *instance*, and as long as it doesn't break users overriding __call__ in a classic class to make the instances callable. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From collinw at gmail.com Fri Mar 28 21:28:41 2008 From: collinw at gmail.com (Collin Winter) Date: Fri, 28 Mar 2008 13:28:41 -0700 Subject: [Python-Dev] Removing callable() from the stdlib In-Reply-To: References: <43aa6ff70803281051ufa612day856b9cf3c84e2fb7@mail.gmail.com> Message-ID: <43aa6ff70803281328p77ab1bb7q5c960f8525511d4f@mail.gmail.com> On Fri, Mar 28, 2008 at 12:58 PM, Guido van Rossum wrote: > > On Fri, Mar 28, 2008 at 10:51 AM, Collin Winter wrote: > > I've been running 2to3's fix_callable over 2.6's stdlib to remove -3 > > warnings due to callable() usage; 2to3 replaces callable(x) with > > has_attr(x, "__call__"). Unfortunately, this breaks code that wants to > > run callable() on old-style classes (like test_builtin), because > > old-style classes don't have a __call__ attribute (new-style classes > > do, however). > > > > How should this be handled? I'm tempted to just add __call__ to > > old-style classes for 2.6, but Neal Norwitz thought that might break > > some user code. Any other thoughts? > > I don't see how this would break user code, as long as the __call__ > shows up for the *class* but not for the *instance*, and as long as it > doesn't break users overriding __call__ in a classic class to make the > instances callable. Yep, I was only going to make it appear on the class. I'll go ahead and whip up a patch. Collin From greg.ewing at canterbury.ac.nz Fri Mar 28 22:57:06 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 29 Mar 2008 09:57:06 +1200 Subject: [Python-Dev] Decimal(unicode) In-Reply-To: <47ECA267.6010702@gmail.com> References: <20080325134617.GB2851@phd.pp.ru> <5c6f2a5d0803250747v165c79fchb59f9b87194a927a@mail.gmail.com> <47E91A6F.7090601@gmail.com> <5c6f2a5d0803250853g4814eac6n25b07326f2e7805d@mail.gmail.com> <47E99ACB.8010802@canterbury.ac.nz> <47E9BCB2.6090008@gmail.com> <47E9D9E9.8030905@canterbury.ac.nz> <47E9F3D2.6090400@gmail.com> <47EADA06.9090806@canterbury.ac.nz> <47EB010C.9040608@gmail.com> <47EB46E5.4060203@canterbury.ac.nz> <47EC29E0.5020209@canterbury.ac.nz> <47ECA267.6010702@gmail.com> Message-ID: <47ED69B2.4060408@canterbury.ac.nz> Nick Coghlan wrote: > What features do you find particularly unfortunate? Whichever ones are making people think that implementing it in C is infeasible. > Just because > something isn't particularly amenable to implementation in C, doesn't > make it a bad API for a Python library No, but for something like a number type, which benefits greatly from speed, making it actively C-hostile doesn't seem like a good idea. > (e.g. the dicts to enable/signal > the different error traps are a natural interface for Python code I don't see why there can't be an object with a mapping interface for this, that stores them internally as a bit field or whatever is convenient for the C code. -- Greg From jyasskin at gmail.com Sat Mar 29 04:31:13 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Fri, 28 Mar 2008 20:31:13 -0700 Subject: [Python-Dev] [Python-3000] the release gods are angry at python In-Reply-To: <1450237217404852410@unknownmsgid> References: <8526353492909807871@unknownmsgid> <18412.51472.322043.137606@montanaro-dyndns-org.local> <1450237217404852410@unknownmsgid> Message-ID: <5d44f72f0803282031i5552b67bj5d61efccf62c4a@mail.gmail.com> On Fri, Mar 28, 2008 at 11:15 AM, Bill Janssen wrote: > > On Fri, Mar 28, 2008 at 3:31 AM, wrote: > > > > > > Neal> Anything that connects to a remote host is definitely flaky. > > > > > > Would it maybe help to set up a dedicated host (or virtual host) to serve as > > > the sole target of all network tests? > > > > It would help, but not fix the problem. There will still be transient > > network issues that come up. > > We really need to be able to wrap a timeout around any individual > test, perhaps throwing a TimeoutException if the timeout is reached, > and have the ability to regard a TimeoutException as not being a > "failure". Something like: @contextmanager def timeout(max_time): def RaiseTimeout(signal, frame): raise TimeoutException # Why don't we have a context manager for the next line? old_alarm = signal.signal(signal.SIGALRM, RaiseTimeout) try: signal.alarm(max_time) try: yield finally: signal.alarm(0) finally: signal.signal(signal.SIGALRM, old_alarm) with appropriate fallbacks for Windows. I may not be poking at test_support for a bit, so anyone else is welcome to check that in, or a fixed version of it, whenever they're fixing a timing-out test. Someone else will have to implement the support in regrtest, although I wouldn't mind having it treat a timeout as a failure, as long as it's easier to diagnose than the SIGKILL it currently uses for timeouts. -- Namast?, Jeffrey Yasskin From musiccomposition at gmail.com Sat Mar 29 17:13:04 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sat, 29 Mar 2008 11:13:04 -0500 Subject: [Python-Dev] editing the docs Message-ID: <1afaf6160803290913x4353d4d3u493faddb1ad85328@mail.gmail.com> Hi, Now that I'm starting to examine and do some edits on the docs, I'd like to ask some guidance. What editor(s) do you guys use? I'm not one to cling to an editor, so all suggestions are fair game. -- Thanks, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080329/ad7baaf1/attachment.htm From g.brandl at gmx.net Sat Mar 29 17:36:52 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 29 Mar 2008 17:36:52 +0100 Subject: [Python-Dev] editing the docs In-Reply-To: <1afaf6160803290913x4353d4d3u493faddb1ad85328@mail.gmail.com> References: <1afaf6160803290913x4353d4d3u493faddb1ad85328@mail.gmail.com> Message-ID: Benjamin Peterson schrieb: > Hi, > Now that I'm starting to examine and do some edits on the docs, I'd like > to ask some guidance. What editor(s) do you guys use? I'm not one to > cling to an editor, so all suggestions are fair game. I use Emacs, for which the docutils bring an excellent rst mode. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From martin at v.loewis.de Sat Mar 29 17:51:27 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 29 Mar 2008 17:51:27 +0100 Subject: [Python-Dev] editing the docs In-Reply-To: <1afaf6160803290913x4353d4d3u493faddb1ad85328@mail.gmail.com> References: <1afaf6160803290913x4353d4d3u493faddb1ad85328@mail.gmail.com> Message-ID: <47EE738F.3060604@v.loewis.de> > Now that I'm starting to examine and do some edits on the docs, I'd like > to ask some guidance. What editor(s) do you guys use? I'm not one to > cling to an editor, so all suggestions are fair game. For the docs, I typically use Emacs as well. Regards, Martin From asmodai at in-nomine.org Sat Mar 29 18:25:02 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Sat, 29 Mar 2008 18:25:02 +0100 Subject: [Python-Dev] editing the docs In-Reply-To: <1afaf6160803290913x4353d4d3u493faddb1ad85328@mail.gmail.com> References: <1afaf6160803290913x4353d4d3u493faddb1ad85328@mail.gmail.com> Message-ID: <20080329172502.GX80617@nexus.in-nomine.org> -On [20080329 17:13], Benjamin Peterson (musiccomposition at gmail.com) wrote: >Now that I'm starting to examine and do some edits on the docs, I'd like to ask >some guidance. What editor(s) do you guys use? I'm not one to cling to an >editor, so all suggestions are fair game. I personally use vim. But this question is really dependent on what you think works best. Some prefer command line tools, others graphical tools. Just try a few and see what you personally find to work nicely enough. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ To love someone deeply gives you strength. Being loved by someone deeply gives you courage... From martin at v.loewis.de Sat Mar 29 18:37:08 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 29 Mar 2008 18:37:08 +0100 Subject: [Python-Dev] FreeBSD test suite failure -> curses In-Reply-To: <20080328095258.GS80617@nexus.in-nomine.org> References: <20080308092434.GZ60713@nexus.in-nomine.org> <47D273B0.4020809@v.loewis.de> <20080309162900.GA60713@nexus.in-nomine.org> <47D43926.4030902@v.loewis.de> <20080309210210.GB60713@nexus.in-nomine.org> <47D46BB2.1080202@v.loewis.de> <20080328095258.GS80617@nexus.in-nomine.org> Message-ID: <47EE7E44.1090008@v.loewis.de> > Actually, given my recent results and discussion with Thomas Dickey I am not > so sure. Even if there is a bug in the test or in Python that causes newscr to become NULL - why does that not show up in other releases, or on other systems? > Thomas mentioned after seeing an ltrace: > > Perhaps it's failing on that: > > curses.setupterm(fd=sys.__stdout__.fileno()) > > That would have newscr null. The failure might be from closing stdout, e.g., > if it was redirected. Can you debug this and confirm that this call indeed sets newscr to NULL? At the point the test is run, stdout is not closed; instead, it is a pipe going to the build slave. Usually, all curses output shows up in the buildbot log (which doesn't really automatically test whether the right thing is happening - but we don't want to test the system's curses library, anyway). Regards, Martin From musiccomposition at gmail.com Sat Mar 29 18:41:55 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sat, 29 Mar 2008 12:41:55 -0500 Subject: [Python-Dev] editing the docs In-Reply-To: <20080329172502.GX80617@nexus.in-nomine.org> References: <1afaf6160803290913x4353d4d3u493faddb1ad85328@mail.gmail.com> <20080329172502.GX80617@nexus.in-nomine.org> Message-ID: <1afaf6160803291041j79d78679r532ad1d5f7135a79@mail.gmail.com> On Sat, Mar 29, 2008 at 12:25 PM, Jeroen Ruigrok van der Werven < asmodai at in-nomine.org> wrote: > -On [20080329 17:13], Benjamin Peterson (musiccomposition at gmail.com) > wrote: > >Now that I'm starting to examine and do some edits on the docs, I'd like > to ask > >some guidance. What editor(s) do you guys use? I'm not one to cling to an > >editor, so all suggestions are fair game. > > I personally use vim. But this question is really dependent on what you > think works best. Some prefer command line tools, others graphical tools. > Just try a few and see what you personally find to work nicely enough. Well, I do use emacs for my coding, so I'm going to try that mode out now. > > > -- > Jeroen Ruigrok van der Werven / asmodai > ????? ?????? ??? ?? ?????? > http://www.in-nomine.org/ | http://www.rangaku.org/ > To love someone deeply gives you strength. Being loved by someone deeply > gives you courage... > -- Cheers, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080329/e45d6910/attachment.htm From asmodai at in-nomine.org Sat Mar 29 18:59:38 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Sat, 29 Mar 2008 18:59:38 +0100 Subject: [Python-Dev] FreeBSD test suite failure -> curses In-Reply-To: <47EE7E44.1090008@v.loewis.de> References: <20080308092434.GZ60713@nexus.in-nomine.org> <47D273B0.4020809@v.loewis.de> <20080309162900.GA60713@nexus.in-nomine.org> <47D43926.4030902@v.loewis.de> <20080309210210.GB60713@nexus.in-nomine.org> <47D46BB2.1080202@v.loewis.de> <20080328095258.GS80617@nexus.in-nomine.org> <47EE7E44.1090008@v.loewis.de> Message-ID: <20080329175938.GZ80617@nexus.in-nomine.org> -On [20080329 18:37], "Martin v. L?wis" (martin at v.loewis.de) wrote: >Even if there is a bug in the test or in Python that causes newscr to >become NULL - why does that not show up in other releases, or on other >systems? I have been breaking my head about that as well. But then again it is just as strange that from a script run it does not coredump whereas running from the interactive prompt or from the testsuite it does. That should, in principle, not be much different either. >Can you debug this and confirm that this call indeed sets newscr to NULL? Breakpoint 1, 0x283119f8 in setupterm () from /lib/libncursesw.so.6 (gdb) print newscr $1 = 0 (gdb) cont Continuing. Breakpoint 1, 0x283119f8 in setupterm () from /lib/libncursesw.so.6 (gdb) print newscr $2 = 0 (gdb) cont Continuing. Hardware access (read/write) watchpoint 2: {} 674382472 Value = 0 0x2836e102 in PyCurses_getsyx (self=0x0) at /usr/home/asmodai/projects/python/Modules/_cursesmodule.c:1770 1770 getsyx(y, x); (gdb) print newscr $3 = (WINDOW *) 0x0 If you have any specific debug guidance, please feel free to give so. :) -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ The superior person uses his mind like a mirror: it accepts all, it reflects all. It receives, but it does not keep... From lists at cheimes.de Sat Mar 29 19:09:49 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 29 Mar 2008 19:09:49 +0100 Subject: [Python-Dev] nested classes leaking in compiler In-Reply-To: References: Message-ID: <47EE85ED.7090503@cheimes.de> Georg Brandl schrieb: > Somehow I knew you'd be the one to find the problem :) Agreed! :) Amaury is really good in finding loose ends in the parser and AST code. I still can't wrap my brain around the AST code but it can see the light at the end of the code. From lists at cheimes.de Sat Mar 29 19:09:49 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 29 Mar 2008 19:09:49 +0100 Subject: [Python-Dev] nested classes leaking in compiler In-Reply-To: References: Message-ID: <47EE85ED.7090503@cheimes.de> Georg Brandl schrieb: > Somehow I knew you'd be the one to find the problem :) Agreed! :) Amaury is really good in finding loose ends in the parser and AST code. I still can't wrap my brain around the AST code but it can see the light at the end of the code. From amauryfa at gmail.com Sat Mar 29 21:27:03 2008 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Sat, 29 Mar 2008 21:27:03 +0100 Subject: [Python-Dev] nested classes leaking in compiler In-Reply-To: <47EE85ED.7090503@cheimes.de> References: <47EE85ED.7090503@cheimes.de> Message-ID: Christian Heimes wrote: > Georg Brandl schrieb: > > > Somehow I knew you'd be the one to find the problem :) > > Agreed! :) > > Amaury is really good in finding loose ends in the parser and AST code. > I still can't wrap my brain around the AST code but it can see the light > at the end of the code. Actually the ast compiler is the part I know least in CPython code. The approach I used is a bit brute-force, but it may work for other reference leaks: - First write a big leaking script: for x in xrange(100000): leaking_function() - Since memory usage does not increase dramatically, it's only a refcount problem - no new object every time. - Modify the Py_INCREF macro so that it generates a breakpoint in the debugger when the refcount is large: On Windows, it is something like: if (ob->refcount > 10000) { __asm int 3; } (with gdb I think you may signal SIGUSR1) - carefully inspect the code each time the debugger stops. Fortunately, many functions can be skipped: PyDict_SetItem and other bullet-proof parts of the code. The 20th break was the good one: a lack of symmetry between Py_INCREF and Py_DECREF. -- Amaury Forgeot d'Arc From musiccomposition at gmail.com Sat Mar 29 22:11:10 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sat, 29 Mar 2008 16:11:10 -0500 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> Message-ID: <1afaf6160803291411h39251770xd32846e7342ec8b6@mail.gmail.com> On Thu, Mar 20, 2008 at 2:42 PM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I'm happy to announce that we now have available for public > consumption, the Python source code for 2.5, 2.6 and 3.0 available > under the Bazaar distributed version control system. > > The current Subversion repository is still the master copy of the > source code. We have not made a decision to move to Bazaar > officially, nor have we made a decision to even move off of > Subversion. We're making these branches available exactly so that > you, the Python developer community, can kick the tires and see if it > makes sense to move to a different vcs. Nothing will happen until > after the Python 2.6/3.0 releases anyway. > > All the gory details are documented here: > > http://www.python.org/dev/bazaar > > These branches are available both for core Python developers with > commit privileges, and the wider world of developers without commit > privileges. It's this latter group that I think will find the most > compelling immediate benefit from using Bazaar, because they will no > longer need to maintain their own changes using a mass of patch files. > > For more information on Bazaar in general, see: > > http://bazaar-vcs.org > > You will probably be most interested in the Bazaar mirrors of the > Subversion master repository. We have a cron job that updates Bazaar > from Subversion every 15 minutes. It is also possible to push changes > made in your Bazaar branches into the Subversion master, so you can > keep reasonably up-to-date and interact with the Python source code > solely via Bazaar. Once you've pushed the branches, is there a way to remove them? > > > Please let me know if you have any questions or anything in the docs > referenced above aren't clear. I know I need to document the Bazaar- > >Subversion workflow in more detail. > > Huge thanks go out especially to Thomas Wouters who sprinted with me > yesterday on getting the whole infrastructure up and running. Thanks > also to Martin v. Loewis, Sean Reifschneider, and the folks here at > Pycon from the Bazaar project, Ian, Andrew, John, and Edwin. > > Enjoy, > - -Barry > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.8 (Darwin) > > iQCVAwUBR+K+H3EjvBPtnXfVAQL6bwP/d0XKx0mRPgzdOD6zCPwwRl8y2kxWV9Vl > zVwN07cK8IlwMa9M470MsERuXuD8hmeXnPgnrUcrX9HciwldmY8C33t0f8GaOk1J > iD4Od1midlIaJJiMd+JcFk9NbX3Rwc4HMj3s8jKjjdlGrFe77ei9DCZ/HMl/iG5K > fyyjXo4HLEE= > =Gcjq > -----END PGP SIGNATURE----- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -- Cheers, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080329/bdf93056/attachment.htm From barry at python.org Sat Mar 29 22:59:04 2008 From: barry at python.org (Barry Warsaw) Date: Sat, 29 Mar 2008 17:59:04 -0400 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: <1afaf6160803291411h39251770xd32846e7342ec8b6@mail.gmail.com> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <1afaf6160803291411h39251770xd32846e7342ec8b6@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 29, 2008, at 5:11 PM, Benjamin Peterson wrote: > Once you've pushed the branches, is there a way to remove them? Do you mean the local branches? If yes, then 'rm -rf mymergedbranch' does exactly what you want. :) - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR+67qXEjvBPtnXfVAQJ94AP8CA6OmnIlOluIfUuW1TolCpZQJTZS3TbL 4ZZS32CYvgF4gB366wAuaFGuCsaQlAV9punP5l0T9Ei6i4cHD1nIKheNY49JfLxX 83I5fXV9n+Mwe0uJEjgfP5GKTykawwTypG6zfy2lkvJm4wwhfHUH26ctafOlVXo1 z4bLDFZsMMs= =KQay -----END PGP SIGNATURE----- From musiccomposition at gmail.com Sat Mar 29 23:00:47 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sat, 29 Mar 2008 17:00:47 -0500 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <1afaf6160803291411h39251770xd32846e7342ec8b6@mail.gmail.com> Message-ID: <1afaf6160803291500j3099e1cfl654dcd637c3cc189@mail.gmail.com> On Sat, Mar 29, 2008 at 4:59 PM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Mar 29, 2008, at 5:11 PM, Benjamin Peterson wrote: > > > Once you've pushed the branches, is there a way to remove them? > > Do you mean the local branches? If yes, then 'rm -rf mymergedbranch' > does exactly what you want. :) No, I mean the pushed version on code.python.org. > > > - -Barry > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.8 (Darwin) > > iQCVAwUBR+67qXEjvBPtnXfVAQJ94AP8CA6OmnIlOluIfUuW1TolCpZQJTZS3TbL > 4ZZS32CYvgF4gB366wAuaFGuCsaQlAV9punP5l0T9Ei6i4cHD1nIKheNY49JfLxX > 83I5fXV9n+Mwe0uJEjgfP5GKTykawwTypG6zfy2lkvJm4wwhfHUH26ctafOlVXo1 > z4bLDFZsMMs= > =KQay > -----END PGP SIGNATURE----- > -- Cheers, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080329/81a2a601/attachment.htm From barry at python.org Sat Mar 29 23:10:46 2008 From: barry at python.org (Barry Warsaw) Date: Sat, 29 Mar 2008 18:10:46 -0400 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: <1afaf6160803291500j3099e1cfl654dcd637c3cc189@mail.gmail.com> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <1afaf6160803291411h39251770xd32846e7342ec8b6@mail.gmail.com> <1afaf6160803291500j3099e1cfl654dcd637c3cc189@mail.gmail.com> Message-ID: <05CF1392-B11E-4FA1-AB0C-DAD88E3980CE@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 29, 2008, at 6:00 PM, Benjamin Peterson wrote: > > No, I mean the pushed version on code.python.org. Not unless you have shell or sftp access, which you probably don't. It's not a big deal though except for a mild feeling of uncleanliness. If you ever want to push a completely unrelated branch overtop that one, just use --overwrite. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR+6+ZnEjvBPtnXfVAQIbdgP+KRv3Uhq9OQrhc6iRI1eEVR+1bbYbFSQs bwvwSf9SXNaf/YfO5Bm61YlJkHHkJd0237cf0Dn/MU8IFacRawJijhVbBYec2QmY 4CWeMiiPop+LL1z2MlXkErfbeVt9AZeiNQ/oDLYda9Bg7Tw20XKE3VYqJGAVGt0b XrucWkxI964= =ahtL -----END PGP SIGNATURE----- From skip at pobox.com Sun Mar 30 00:15:18 2008 From: skip at pobox.com (skip at pobox.com) Date: Sat, 29 Mar 2008 18:15:18 -0500 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: <1afaf6160803291411h39251770xd32846e7342ec8b6@mail.gmail.com> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <1afaf6160803291411h39251770xd32846e7342ec8b6@mail.gmail.com> Message-ID: <18414.52614.618794.737997@montanaro-dyndns-org.local> Benjamin> Once you've pushed the branches, is there a way to remove them? Related question: is there a way to view the various branches in a non-local repository? Skip From martin at v.loewis.de Sun Mar 30 09:35:42 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 30 Mar 2008 09:35:42 +0200 Subject: [Python-Dev] Python source code on Bazaar vcs In-Reply-To: <18414.52614.618794.737997@montanaro-dyndns-org.local> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <1afaf6160803291411h39251770xd32846e7342ec8b6@mail.gmail.com> <18414.52614.618794.737997@montanaro-dyndns-org.local> Message-ID: <47EF42CE.1040101@v.loewis.de> skip at pobox.com wrote: > Benjamin> Once you've pushed the branches, is there a way to remove them? > > Related question: is there a way to view the various branches in a non-local > repository? IIUC, conceptually, no. A branch is not *in* a repository; a branch *is* a repository (*). So your question is almost equivalent to "is there a way to find out all clones of a repository that have ever been made?", to which the answer is "no". Now, if you have a convention of where you put the branches, you should be able to find them later. E.g. if they are all in a directory tree that is exposed through http, you can use a web browser to see them, e.g. by going to http://code.python.org/python/users/skip/ Likewise, if you are accessing the repository over bzr+ssh, in general, you can just ssh to the machine, and do a regular ls. In the specific setup, regular ssh is restricted to running "bzr serve", which (apparently) has no support for directory browsing. Regards, Martin From martin at v.loewis.de Sun Mar 30 09:54:49 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 30 Mar 2008 09:54:49 +0200 Subject: [Python-Dev] [Python-3000] Python source code on Bazaar vcs In-Reply-To: References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <1afaf6160803291411h39251770xd32846e7342ec8b6@mail.gmail.com> Message-ID: <47EF4749.1060302@v.loewis.de> >> Once you've pushed the branches, is there a way to remove them? > > Do you mean the local branches? If yes, then 'rm -rf mymergedbranch' > does exactly what you want. :) Notice that, unlike subversion, this will cause the entire branch to go away, irrevocably (unless you had pushed it elsewhere before, or was branched from). IIUC, if this is a "shared repository", the actual revisions will still be stored, but unaccessible, as the branch files store the information what revision was the head revision. Regards, Martin From lists at cheimes.de Sun Mar 30 16:29:42 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 30 Mar 2008 16:29:42 +0200 Subject: [Python-Dev] nested classes leaking in compiler In-Reply-To: References: <47EE85ED.7090503@cheimes.de> Message-ID: <47EFA3D6.9050700@cheimes.de> Amaury Forgeot d'Arc schrieb: > The approach I used is a bit brute-force, but it may work for other > reference leaks: > - First write a big leaking script: > for x in xrange(100000): leaking_function() > - Since memory usage does not increase dramatically, it's only a > refcount problem - no new object every time. > - Modify the Py_INCREF macro so that it generates a breakpoint in the > debugger when the refcount is large: > On Windows, it is something like: > if (ob->refcount > 10000) { __asm int 3; } > (with gdb I think you may signal SIGUSR1) > - carefully inspect the code each time the debugger stops. > Fortunately, many functions can be skipped: PyDict_SetItem and other > bullet-proof parts of the code. > The 20th break was the good one: a lack of symmetry between > Py_INCREF and Py_DECREF. Nice! Can you write a short howto for the wiki? The idea is ingenious. Christian From lists at cheimes.de Sun Mar 30 19:46:37 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 30 Mar 2008 19:46:37 +0200 Subject: [Python-Dev] No time for svn merge Message-ID: For your information: I won't have time to do another svn merge before the next alphas of 2.6 and 3.0 are to be released. Somebody else has to do the merge. Christian From martin at v.loewis.de Sun Mar 30 22:51:48 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 30 Mar 2008 22:51:48 +0200 Subject: [Python-Dev] No time for svn merge In-Reply-To: References: Message-ID: <47EFFD64.8020006@v.loewis.de> > I won't have time to do another svn merge before the next alphas of 2.6 > and 3.0 are to be released. Somebody else has to do the merge. I merged a few revisions, but I'm done now (until Tuesday or so). Regards, Martin From musiccomposition at gmail.com Sun Mar 30 23:16:39 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sun, 30 Mar 2008 16:16:39 -0500 Subject: [Python-Dev] No time for svn merge In-Reply-To: <47EFFD64.8020006@v.loewis.de> References: <47EFFD64.8020006@v.loewis.de> Message-ID: <1afaf6160803301416n6e37df7cw5ff39a55ddb7341@mail.gmail.com> On Sun, Mar 30, 2008 at 3:51 PM, "Martin v. L?wis" wrote: > > I won't have time to do another svn merge before the next alphas of 2.6 > > and 3.0 are to be released. Somebody else has to do the merge. > > I merged a few revisions, but I'm done now (until Tuesday or so). If you'd like, I can merge the rest. > > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -- Cheers, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080330/e68edaab/attachment.htm From martin at v.loewis.de Sun Mar 30 23:54:59 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 30 Mar 2008 23:54:59 +0200 Subject: [Python-Dev] No time for svn merge In-Reply-To: <1afaf6160803301416n6e37df7cw5ff39a55ddb7341@mail.gmail.com> References: <47EFFD64.8020006@v.loewis.de> <1afaf6160803301416n6e37df7cw5ff39a55ddb7341@mail.gmail.com> Message-ID: <47F00C33.1050503@v.loewis.de> > If you'd like, I can merge the rest. If you have the time to figure it all out, sure. I found that quite a tedious task, and had to spent on some patches quite a long time to figure out what they do, and what the 3.x equivalent should be. Regards, Martin From dickinsm at gmail.com Mon Mar 31 00:07:33 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Sun, 30 Mar 2008 18:07:33 -0400 Subject: [Python-Dev] No time for svn merge In-Reply-To: <47F00C33.1050503@v.loewis.de> References: <47EFFD64.8020006@v.loewis.de> <1afaf6160803301416n6e37df7cw5ff39a55ddb7341@mail.gmail.com> <47F00C33.1050503@v.loewis.de> Message-ID: <5c6f2a5d0803301507m25455babm7254b9cb62905dfe@mail.gmail.com> On Sun, Mar 30, 2008 at 5:54 PM, "Martin v. L?wis" wrote: > If you have the time to figure it all out, sure. > I found that quite a tedious task, and had to spent > on some patches quite a long time to figure out what > they do, and what the 3.x equivalent should be. > Is there any easy way that the burden of trunk -> py3k merging could be moved to the original committers of the trunk patches? Presumably the original committer of a patch is well-placed to understand what the patch does and how best to translate it to 3.0. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080330/fe0f6db6/attachment.htm From martin at v.loewis.de Mon Mar 31 00:16:29 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 31 Mar 2008 00:16:29 +0200 Subject: [Python-Dev] No time for svn merge In-Reply-To: <5c6f2a5d0803301507m25455babm7254b9cb62905dfe@mail.gmail.com> References: <47EFFD64.8020006@v.loewis.de> <1afaf6160803301416n6e37df7cw5ff39a55ddb7341@mail.gmail.com> <47F00C33.1050503@v.loewis.de> <5c6f2a5d0803301507m25455babm7254b9cb62905dfe@mail.gmail.com> Message-ID: <47F0113D.8020408@v.loewis.de> > Is there any easy way that the burden of trunk -> py3k > merging could be moved to the original committers of > the trunk patches? I'm not sure I understand the question. If the committer of the original patch would do the merge himself, then certainly the burden would be on him, and that's an easy way. If you meant to say "an easy way to enforce...", then I cannot see how that could work, other than establishing that as a policy, and starting to revoke commit privileges to people who don't follow the policy. Rather than actually merging changes, one could start sending out messages automatically to committers who don't either merge or block their changes within 24 hours (or send a summary message every day to python-dev). Regards, Martin From musiccomposition at gmail.com Mon Mar 31 00:33:23 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sun, 30 Mar 2008 17:33:23 -0500 Subject: [Python-Dev] No time for svn merge In-Reply-To: <47F0113D.8020408@v.loewis.de> References: <47EFFD64.8020006@v.loewis.de> <1afaf6160803301416n6e37df7cw5ff39a55ddb7341@mail.gmail.com> <47F00C33.1050503@v.loewis.de> <5c6f2a5d0803301507m25455babm7254b9cb62905dfe@mail.gmail.com> <47F0113D.8020408@v.loewis.de> Message-ID: <1afaf6160803301533j32a97970tbee1bc422c4ec997@mail.gmail.com> On Sun, Mar 30, 2008 at 5:16 PM, "Martin v. L?wis" wrote: > > Is there any easy way that the burden of trunk -> py3k > > merging could be moved to the original committers of > > the trunk patches? > > I'm not sure I understand the question. If the committer > of the original patch would do the merge himself, then > certainly the burden would be on him, and that's an easy > way. > > If you meant to say "an easy way to enforce...", then I > cannot see how that could work, other than establishing > that as a policy, and starting to revoke commit privileges > to people who don't follow the policy. I think we could just ask politely and try it for a while. If things aren't working, we can reevaluate. > > > Rather than actually merging changes, one could start > sending out messages automatically to committers who > don't either merge or block their changes within 24 hours > (or send a summary message every day to python-dev). Like above, let's try a little before we start setting up new infrastructure left and right. > > > Regards, > Martin > > > -- Cheers, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080330/d492aba7/attachment.htm From dickinsm at gmail.com Mon Mar 31 00:33:52 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Sun, 30 Mar 2008 18:33:52 -0400 Subject: [Python-Dev] No time for svn merge In-Reply-To: <47F0113D.8020408@v.loewis.de> References: <47EFFD64.8020006@v.loewis.de> <1afaf6160803301416n6e37df7cw5ff39a55ddb7341@mail.gmail.com> <47F00C33.1050503@v.loewis.de> <5c6f2a5d0803301507m25455babm7254b9cb62905dfe@mail.gmail.com> <47F0113D.8020408@v.loewis.de> Message-ID: <5c6f2a5d0803301533l1d79be9bh6033bb3dfafb485a@mail.gmail.com> On Sun, Mar 30, 2008 at 6:16 PM, "Martin v. L?wis" wrote: > > Is there any easy way that the burden of trunk -> py3k > > merging could be moved to the original committers of > > the trunk patches? > > I'm not sure I understand the question. If the committer > of the original patch would do the merge himself, then > certainly the burden would be on him, and that's an easy > way. > Yes, that's all I meant: make it the committer's job to merge or block as appropriate. I just wasn't sure if there was some reason that this would be difficult or undesirable. I'm not suggesting that this be enforced (mechanically or otherwise); just that it might be worth considering instituting a policy to encourage individual committers to take responsibility for forward porting their commits. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080330/a418ecb4/attachment-0001.htm From musiccomposition at gmail.com Mon Mar 31 04:41:27 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sun, 30 Mar 2008 21:41:27 -0500 Subject: [Python-Dev] No time for svn merge In-Reply-To: <47F00C33.1050503@v.loewis.de> References: <47EFFD64.8020006@v.loewis.de> <1afaf6160803301416n6e37df7cw5ff39a55ddb7341@mail.gmail.com> <47F00C33.1050503@v.loewis.de> Message-ID: <1afaf6160803301941i605dfdd4nd19eb6336b5c361d@mail.gmail.com> On Sun, Mar 30, 2008 at 4:54 PM, "Martin v. L?wis" wrote: > > If you'd like, I can merge the rest. > > If you have the time to figure it all out, sure. > I found that quite a tedious task, and had to spent > on some patches quite a long time to figure out what > they do, and what the 3.x equivalent should be. Ok. I merged some more of the low hanging fruit. Somebody who understands AST better than me should probably do the merges with that. > > > Regards, > Martin > -- Cheers, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080330/2b438529/attachment.htm From nnorwitz at gmail.com Mon Mar 31 04:43:47 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 30 Mar 2008 19:43:47 -0700 Subject: [Python-Dev] No time for svn merge In-Reply-To: <1afaf6160803301941i605dfdd4nd19eb6336b5c361d@mail.gmail.com> References: <47EFFD64.8020006@v.loewis.de> <1afaf6160803301416n6e37df7cw5ff39a55ddb7341@mail.gmail.com> <47F00C33.1050503@v.loewis.de> <1afaf6160803301941i605dfdd4nd19eb6336b5c361d@mail.gmail.com> Message-ID: On Sun, Mar 30, 2008 at 7:41 PM, Benjamin Peterson wrote: > > > > On Sun, Mar 30, 2008 at 4:54 PM, "Martin v. L?wis" > wrote: > > > > > If you'd like, I can merge the rest. > > > > If you have the time to figure it all out, sure. > > I found that quite a tedious task, and had to spent > > on some patches quite a long time to figure out what > > they do, and what the 3.x equivalent should be. > Ok. I merged some more of the low hanging fruit. Somebody who understands > AST better than me should probably do the merges with that. Are you done for today/tonight? If so, I can merge the rest. The last checkin to regrtest I saw looked like it doesn't work. I thought it had print foo without parens. Did I miss something? n From josiah.carlson at gmail.com Mon Mar 31 04:44:03 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Sun, 30 Mar 2008 19:44:03 -0700 Subject: [Python-Dev] Py3k and asyncore/asynchat In-Reply-To: References: <95d2b6e8-2524-4464-b5a9-c4d56e5f4318@t16g2000hsc.googlegroups.com> Message-ID: (sorry for top posting) I haven't really had time to update the tests/documentation, but again, I wasn't experiencing any strange test failures with asyncore/asynchat, nor have I been able to find the buildbot failures that you are referring to. Could someone please link the failures that are not related to being unable to discover a port number? According to the release schedule, we should have at least a couple more months for documentation and tests to be updated (I can get patches ready for alpha 3). - Josiah On Wed, Mar 26, 2008 at 12:21 AM, Josiah Carlson wrote: > On Tue, Mar 25, 2008 at 11:26 PM, Neal Norwitz wrote: > > Any reason this was sent just to me and not the list? > > Because gmail only replies to the sender by default. I need to > remember to cc python-dev when I reply (I used the same email client > for 8 1/2 years, remembering the quirks of gmail may take some time). > > > > > On Tue, Mar 25, 2008 at 10:34 PM, Josiah Carlson > > wrote: > > > > > > On Tue, Mar 25, 2008 at 9:00 PM, Neal Norwitz wrote: > > > > On Thu, Feb 14, 2008 at 10:09 AM, Giampaolo Rodola' wrote: > > > > > On 14 Feb, 16:36, "Giampaolo Rodola'" wrote: > > > > > > Ok, I'll try to take a look at all asyncore/chat reports and try to > > > > > > summarize them by splitting patches which solve bugs and patches which > > > > > > add enhancements or functionalities. > > > > > > > > > > > > > > > > > > === Patches for existing issues === > > > > > > > > > > - 1736190 which includes fixes for the following issues among other > > > > > improvements: > > > > > - 1063924 (asyncore should handle ECONNRESET in send()) > > > > > - 1736101 (asyncore should handle ECONNABORTED in recv()) > > > > > - 760475 (handle_error() should call handle_close() instead of > > > > > close()) > > > > > - 1740572 (refill_buffer() should call handle_close() rather than > > > > > close()) > > > > > - 777588 (wrong "connection refused" behavior on Windows) > > > > > - 889153 (wrong connect() behavior) > > > > > - 953599 (asyncore misses socket closes when poll is used) > > > > > - 1025525 (asyncore.file_dispatcher should not take fd as argument) > > > > > > > > > > - 1519 (async_chat.__init__() and asyncore.dispatcher.__init__ > > > > > parameters inconsistency) > > > > > - 1541 (Bad OOB data management when using asyncore with > > > > > select.poll()) > > > > > - 2073 (asynchat push always sends 512 bytes (ignoring > > > > > ac_out_buffer_size)) > > > > > > > > > > > > > > > === Open issues with no patches (need review) === > > > > > > > > > > - 658749 (asyncore connect() and winsock errors) > > > > > - 1161031 (neverending warnings from asyncore) > > > > > > > > > > > > > > > === Enhancements & new features === > > > > > > > > > > - 1641 (add delayed calls feature) > > > > > - 1563 (conversion to py3k and some other changes) > > > > > > > > That's a lot of patches. My immediate concern is that test_asynchat > > > > is very flaky and fails often. Sometimes it causes other tests to > > > > fail. Is there a patch that addresses this? If you need examples, > > > > just look through the buildbot mails that mention test_asynchat in: > > > > http://mail.python.org/pipermail/python-checkins/ > > > > > > No, it's one patch. All of the fixes were performed mostly by myself > > > last spring, tested and verified in personal servers, then re-used by > > > Giampaolo in his async ftp server (which pointed out a few small bugs, > > > which have been fixed). > > > > > > > > > > Some platforms have more problems than others, but almost all > > > > platforms have failed test_asynchat at one point or another. > > > > > > Certainly that is the case. And according to my reading of a few > > > buildbot failures, aside from someone breaking asyncore itself, the > > > failures seem to stem from the test being unable to create a port to > > > listen on in order to test the server/client functionality. This is a > > > buildbot configuration issue (per host), not an asyncore issue. > > > > That was the case a long time (~3? months) ago, but hasn't been the > > case recently. See my recent message about the release. > > I'll look for it tomorrow. For reference, searches of > 'site:mail.python.org test_asynchat failure buildbot' only seem to > produce the socket listen error. If there is a better incantation to > get google to produce the proper errors (and/or a link), I would > appreciate the help. > > > > > > > Please break up the patches into 2 sets and prioritize the patches > > > > with the set. > > > > > > > > Set #1: Patches that have a test and doc updates if required > > > > Set #2: Patches that don't have a test or doc updates as required > > > > > > > > For the patches that fall into Set #1, list them in priority order. > > > > Top priority would be a problem that fixes failures seen in the > > > > buildbots. Next priority would go to the patches that solve more > > > > serious problems. Post the results here. I will try to look at them. > > > > > > > > For every patch you list, make sure that it conforms to the proper > > > > style (e.g, PEP 8) and is essentially perfect and ready for inclusion. > > > > This means that there is a single file to download that contains all > > > > the modifications. The changes are appropriately commented, lines are > > > > less than 80 characters, etc. One of the modifications should be an > > > > entry in Misc/NEWS. > > > > > > I lied earlier; really there are two patches. The first is a patch to > > > asyncore.py and asynchat.py . It addresses those bugs that Giampaolo > > > has listed, it is tested, and works. The second patch is to update > > > the documentation to mention the sample methods in asynchat for use as > > > examples, as well as any other changes to the language used in the > > > documentation that I had made last spring, but which are out of date > > > from my posting of the original patch. I can update the documentation > > > in the next week. > > > > Can you provide a link to the patches? Do the patches include changes > > to test_asyncore and test_asynchat? The next release is April 2. I > > would like to commit any patches before Monday to ensure they are > > stable. Can you get me the patches by this Saturday? > > See http://bugs.python.org/issue1736190 for an updated patch for the > modules. The current test cases pass without issue, though we may > want to add tests, which I need to look at the original patch and the > original file from which it was created against, then compare it with > the most recent changes to the tests from Facundo last June or July. > > I should have the time to get patches for tests and documentation by Monday. > > - Josiah > From musiccomposition at gmail.com Mon Mar 31 04:46:13 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sun, 30 Mar 2008 21:46:13 -0500 Subject: [Python-Dev] No time for svn merge In-Reply-To: References: <47EFFD64.8020006@v.loewis.de> <1afaf6160803301416n6e37df7cw5ff39a55ddb7341@mail.gmail.com> <47F00C33.1050503@v.loewis.de> <1afaf6160803301941i605dfdd4nd19eb6336b5c361d@mail.gmail.com> Message-ID: <1afaf6160803301946m170d93c2sab65bc346f889ef0@mail.gmail.com> On Sun, Mar 30, 2008 at 9:43 PM, Neal Norwitz wrote: > On Sun, Mar 30, 2008 at 7:41 PM, Benjamin Peterson > wrote: > > > > > > > > On Sun, Mar 30, 2008 at 4:54 PM, "Martin v. L?wis" > > wrote: > > > > > > > If you'd like, I can merge the rest. > > > > > > If you have the time to figure it all out, sure. > > > I found that quite a tedious task, and had to spent > > > on some patches quite a long time to figure out what > > > they do, and what the 3.x equivalent should be. > > Ok. I merged some more of the low hanging fruit. Somebody who > understands > > AST better than me should probably do the merges with that. > > Are you done for today/tonight? If so, I can merge the rest. Be my guest! I'm going to bed. > > > The last checkin to regrtest I saw looked like it doesn't work. I > thought it had print foo without parens. Did I miss something? I just merged it incorrectly, so I reverted it. > > > n > -- Cheers, Benjamin Peterson -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20080330/aee939b0/attachment.htm From martin at v.loewis.de Mon Mar 31 07:02:32 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 31 Mar 2008 07:02:32 +0200 Subject: [Python-Dev] No time for svn merge In-Reply-To: <5c6f2a5d0803301533l1d79be9bh6033bb3dfafb485a@mail.gmail.com> References: <47EFFD64.8020006@v.loewis.de> <1afaf6160803301416n6e37df7cw5ff39a55ddb7341@mail.gmail.com> <47F00C33.1050503@v.loewis.de> <5c6f2a5d0803301507m25455babm7254b9cb62905dfe@mail.gmail.com> <47F0113D.8020408@v.loewis.de> <5c6f2a5d0803301533l1d79be9bh6033bb3dfafb485a@mail.gmail.com> Message-ID: <47F07068.3060503@v.loewis.de> > Yes, that's all I meant: make it the committer's job > to merge or block as appropriate. I just wasn't sure if > there was some reason that this would be difficult or > undesirable. Ah, yes. It is indeed difficult or undesirable, or was so in the past: Some committers don't care (didn't care) at all about 3k. They would have to setup sandboxes, learn what the nature of changes is, and invest some regular time into forward-porting. Regards, Martin From nnorwitz at gmail.com Mon Mar 31 09:11:49 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 31 Mar 2008 00:11:49 -0700 Subject: [Python-Dev] Py3k and asyncore/asynchat In-Reply-To: References: <95d2b6e8-2524-4464-b5a9-c4d56e5f4318@t16g2000hsc.googlegroups.com> Message-ID: On Sun, Mar 30, 2008 at 7:44 PM, Josiah Carlson wrote: > > I haven't really had time to update the tests/documentation, but > again, I wasn't experiencing any strange test failures with > asyncore/asynchat, nor have I been able to find the buildbot failures > that you are referring to. Could someone please link the failures > that are not related to being unable to discover a port number? 3 different platforms, 3 different problems: http://www.python.org/dev/buildbot/all/alpha%20Tru64%205.1%20trunk/builds/2790/step-test/0 http://www.python.org/dev/buildbot/all/x86%20FreeBSD%203%203.0/builds/0/step-test/0 http://www.python.org/dev/buildbot/all/x86%20XP-4%203.0/builds/643 > According to the release schedule, we should have at least a couple > more months for documentation and tests to be updated (I can get > patches ready for alpha 3). When you get the patches with tests and doc, I'll be happy to check in. n From mhammond at skippinet.com.au Mon Mar 31 09:20:07 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Mon, 31 Mar 2008 18:20:07 +1100 Subject: [Python-Dev] FW: [issue2513] 64bit cross compilation on windows Message-ID: <002401c892ff$a8254630$f86fd290$@com.au> FYI, I've uploaded a patch that provides for cross-compilation on Windows between 32 and 64 bit platforms - all comments invited! Thanks, Mark -----Original Message----- From: Mark Hammond [mailto:report at bugs.python.org] Sent: Sunday, 30 March 2008 6:01 PM To: mhammond at users.sourceforge.net Subject: [issue2513] 64bit cross compilation on windows New submission from Mark Hammond : I've taken the liberty of adding Trent, Christian and Martin to the nosy list as I know they are actively, if reluctantly interested in this. This patch allows the distutils to cross-compile on Windows. It has been tested on x86 and amd64 platforms, with both platforms successfully able to natively and cross-compile extension modules and create binary distributions. To cross-compile, specify '--plat-name' to the build command (valid values are 'win32', 'win-amd64' and 'win-ia64'). This option name was chosen to be consistent with the bdist_dumb command. I've included the docs I added below (which are also in the patch), but note that as with native compilation using distutils, it's not necessary to set any environment variables or do anything else special with your environment to make this work. The patch also adds a x64 target for the 'bdist_wininst' target, which it creates as distutils/command/wininst-9.0-amd64.exe. This executable is necessary even for bdist_wininst to work natively on x64, but is still included here for simplicity. To assist with testing, I've also added a distutils setup.py script to the PC/example_nt directory. This is capable of creating bdist_wininst executables for both native and cross platforms; 'setup.py build --platname=win-amd64 bdist_wininst' will create an amd64 installer on an x86 machine. The patch has not been tested with a Visual Studio environment without cross-compile tools installed - it will obviously fail, but its not clear how ugly this failure will be. Below is the text I added to docs/distutils/builtdist.rst: Cross-compiling on Windows ===================== Starting with Python 2.6, distutils is capable of cross-compiling between Windows platforms. In practice, this means that with the correct tools installed, you can use a 32bit version of Windows to create 64bit extensions and vice-versa. To build for an alternate platform, specify the :option:`--plat-name` option to the build command. Valid values are currently 'win32', 'win-amd64' and 'win-ia64'. For example, on a 32bit version of Windows, you could execute:: python setup.py build --plat-name=win-amd64 to build a 64bit version of your extension. The Windows Installers also support this option, so the command:: python setup.py build --plat-name=win-amd64 bdist_wininst would create a 64bit installation executable on your 32bit version of Windows. Note that by default, Visual Studio 2008 does not install 64bit compilers or tools. You may need to reexecute the Visual Studio setup process and select these tools. ---------- assignee: mhammond components: Distutils files: windows-cross-compile.patch keywords: 64bit, patch messages: 64743 nosy: Trent.Nelson, ctheune, loewis, mhammond severity: normal status: open title: 64bit cross compilation on windows type: feature request versions: Python 2.6 Added file: http://bugs.python.org/file9900/windows-cross-compile.patch __________________________________ Tracker __________________________________ From facundobatista at gmail.com Mon Mar 31 14:09:14 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Mon, 31 Mar 2008 09:09:14 -0300 Subject: [Python-Dev] No time for svn merge In-Reply-To: <47F00C33.1050503@v.loewis.de> References: <47EFFD64.8020006@v.loewis.de> <1afaf6160803301416n6e37df7cw5ff39a55ddb7341@mail.gmail.com> <47F00C33.1050503@v.loewis.de> Message-ID: 2008/3/30, "Martin v. L?wis" : > > If you'd like, I can merge the rest. > > If you have the time to figure it all out, sure. > I found that quite a tedious task, and had to spent > on some patches quite a long time to figure out what > they do, and what the 3.x equivalent should be. Me for myself, I thought that the trunk -> 3k merge was easier! Sometimes I commited changes to the trunk, don't worrying about 3k at all, thinking it was a mostly automatic process. Now that I know this, I will start patching both trunk & 3k simultaneously... Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From lists at cheimes.de Mon Mar 31 14:25:01 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 31 Mar 2008 14:25:01 +0200 Subject: [Python-Dev] No time for svn merge In-Reply-To: References: <47EFFD64.8020006@v.loewis.de> <1afaf6160803301416n6e37df7cw5ff39a55ddb7341@mail.gmail.com> <47F00C33.1050503@v.loewis.de> Message-ID: <47F0D81D.6010903@cheimes.de> Facundo Batista schrieb: > Me for myself, I thought that the trunk -> 3k merge was easier! > > Sometimes I commited changes to the trunk, don't worrying about 3k at > all, thinking it was a mostly automatic process. > > Now that I know this, I will start patching both trunk & 3k simultaneously... In most cases it's easy. Usually it takes me less than 20 minutes per day to merge the chances from trunk -> py3k. In this particular case several obstacles come together. The changes in the AST and parser code aren't trivial, I'm not familiar with the internals of the AST and parser code and my free time is very limited. Please don't patch trunk and py3k simultaneously. It may confuse svn and may make future merges harder. You should apply the patch to the trunk and either merge or block the revision with svnmerge. Example: # Commit a patch to the trunk ~$ cd python/trunk trunk$ svn ci -m "Message" ... Committed revision 1234. # Merge the revision to the py3k branch trunk $ cd ../py3k py3k $ svnmerge.py merge -r 1234 ... # resolve conflicts ... # commit the merge py3k $ svn ci -F svnmerge-commit-message.txt Christian From barry at python.org Mon Mar 31 14:32:20 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 31 Mar 2008 08:32:20 -0400 Subject: [Python-Dev] [Python-3000] Python source code on Bazaar vcs In-Reply-To: <47EF42CE.1040101@v.loewis.de> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <1afaf6160803291411h39251770xd32846e7342ec8b6@mail.gmail.com> <18414.52614.618794.737997@montanaro-dyndns-org.local> <47EF42CE.1040101@v.loewis.de> Message-ID: <1F046B4C-05C5-42C7-B0A9-731B9A4A5AB4@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 30, 2008, at 3:35 AM, Martin v. L?wis wrote: > Likewise, if you are accessing the repository over bzr+ssh, in > general, > you can just ssh to the machine, and do a regular ls. In the specific > setup, regular ssh is restricted to running "bzr serve", which > (apparently) has no support for directory browsing. I suppose we could enable sftp access to user branches on code.python.org. You wouldn't want to use sftp:// protocol to do bzr, even though you could, because bzr+ssh is more efficient. - - -Barry - -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR+/04HEjvBPtnXfVAQI6gAQApMnG/h4qCoSx5rxd6US03DN01hh2ef2N l7tNnQCJMAd2+By5z/cn/BY2vehZmVZEFhgNsGzhAOHZNbsbFG2kBdJkT5PLCOQN ZGqdYCgbMbHhDLsPG0XfWorOgpeER/7WlesF/VXGbbGiZqbWYFZVbprM6M7n4O/y V5RzsxpHO/s= =Po3X - -----END PGP SIGNATURE----- -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR/DZ1XEjvBPtnXfVAQJm6gQAjnNOy8HjE90WjbzPgJtSS+S6W9Z2lBGm IFxSFElDv6D59G67qikdHHlJfn/eCSrl1QB158AucGMCIWDRFM/d5UOTfv6a7vcQ pLpJvWD1DqvvbvKZi7CkRIpcTn80YATAhJ4mOTaJ7zlC6m9OHJwkLZnxDRJdmrkJ xv5hZzqVTS0= =fcSk -----END PGP SIGNATURE----- From facundobatista at gmail.com Mon Mar 31 14:38:27 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Mon, 31 Mar 2008 09:38:27 -0300 Subject: [Python-Dev] No time for svn merge In-Reply-To: <47F0D81D.6010903@cheimes.de> References: <47EFFD64.8020006@v.loewis.de> <1afaf6160803301416n6e37df7cw5ff39a55ddb7341@mail.gmail.com> <47F00C33.1050503@v.loewis.de> <47F0D81D.6010903@cheimes.de> Message-ID: 2008/3/31, Christian Heimes : > In most cases it's easy. Usually it takes me less than 20 minutes per > day to merge the chances from trunk -> py3k. In this particular case > several obstacles come together. The changes in the AST and parser code > aren't trivial, I'm not familiar with the internals of the AST and > parser code and my free time is very limited. > > Please don't patch trunk and py3k simultaneously. It may confuse svn and > may make future merges harder. You should apply the patch to the trunk > and either merge or block the revision with svnmerge. Mmmm... so, I'll do this. I'll just commit on the trunk, but know people that I'm here if you need to ask anything about commited stuff or whatever ("here" means through IM, and normally hanging around in #python-dev or #pyar in freenode). Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From barry at python.org Mon Mar 31 15:06:24 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 31 Mar 2008 09:06:24 -0400 Subject: [Python-Dev] Next alphas, this Wednesday Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Just another heartbeat reminder that I intend to release the next alphas for Python 2.6 and 3.0 this Wednesday April 2nd, at approximately 6pm Eastern time (UTC 2200). Current status: the buildbots for both the trunk and 3.0 look relatively good, though a few are purple or red: http://www.python.org/dev/buildbot/stable/ There is currently one release blocker bug for 3.0: http://bugs.python.org/issue2507 I'll be looking at these in more detail later today. If you have time, feel free to comment on the bug or send a follow up about the stable buildbots. Please try to be very conservative in your commits to the trunk and 3.0 over the next few days. Concentrate on fixing existing code rather than committing new features. Your release manager thanks you for your diligence! :) Cheers, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR/Dh0HEjvBPtnXfVAQL3HgQAnk/eoRyfrF0RQcCKCfhNyfpfc7KGbPMW Y46xZy/yzySPbUDLP4YhTs3hhjt4hfZEHYagWV50Yy0Wtak0vDMj1Fr+8jxIptR0 Qh0zQhop2OQPZVT5S5TOgOVlXAj9nYITDpfnFfogoNo4PaEG8wNvlBZKcxN+8cfv KkSrM6Sm4Rw= =rOZw -----END PGP SIGNATURE----- From martin at v.loewis.de Mon Mar 31 19:08:53 2008 From: martin at v.loewis.de (=?iso-8859-1?Q?=22Martin_v._L=F6wis=22?=) Date: Mon, 31 Mar 2008 10:08:53 -0700 Subject: [Python-Dev] [Python-3000] Python source code on Bazaar vcs In-Reply-To: <18414.52614.618794.737997@montanaro-dyndns-org.local> References: <20C0AC37-D748-450E-B690-FBCA2ACFFC4E@python.org> <1afaf6160803291411h39251770xd32846e7342ec8b6@mail.gmail.com><18414.52614.618794.737997@montanaro-dyndns-org.local> Message-ID: skip at pobox.com wrote: > Benjamin> Once you've pushed the branches, is there a way to remove them? > > Related question: is there a way to view the various branches in a non-local > repository? IIUC, conceptually, no. A branch is not *in* a repository; a branch *is* a repository (*). So your question is almost equivalent to "is there a way to find out all clones of a repository that have ever been made?", to which the answer is "no". Now, if you have a convention of where you put the branches, you should be able to find them later. E.g. if they are all in a directory tree that is exposed through http, you can use a web browser to see them, e.g. by going to http://code.python.org/python/users/skip/ Likewise, if you are accessing the repository over bzr+ssh, in general, you can just ssh to the machine, and do a regular ls. In the specific setup, regular ssh is restricted to running "bzr serve", which (apparently) has no support for directory browsing. Regards, Martin _______________________________________________ Python-3000 mailing list Python-3000 at python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/fumanchu%40aminus.org From solipsis at pitrou.net Mon Mar 31 22:09:29 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 31 Mar 2008 20:09:29 +0000 (UTC) Subject: [Python-Dev] Thread-safe file objects, the return Message-ID: Hello, It seems this subject has had quite a bit of history. Tim Peters demonstrated the problem in 2003 in this message: http://mail.python.org/pipermail/python-dev/2003-June/036537.html In short, Python file objects release the GIL before calling any C stdlib function on their embedded FILE pointer. Unfortunately, if another thread calls fclose on the FILE pointer concurrently, the contents pointed to can become garbage and the interpreter process crashes. Just by using the same file object in two threads running pure Python code, you can crash the interpreter. (another, easier-to-solve problem is that the FILE pointer stored in the file object could become NULL at the point it is used by another thread. If that was the only problem you could just store the FILE pointer in a local variable before releasing the GIL et voil?) There was some discussion at the time about the possible resolution. I've tried to fix the problem, and I've come to what I think is a satisfying solution, which I can sum up as the following bullet points: * Each file object gets a dedicated counter, which is incremented before the bject releases the GIL and decremented after the GIL is taken again; thus this counter keeps track of how many running "unlocked" sections of code are using that particular file object. (please note the counter doesn't need its own lock, since it is only modified in GIL-protected sections) * In the close() method, if the aforementioned counter is greater than 0, we refuse to call fclose and instead raise an IOError. This may seem like a worrying semantic change, but I don't think it is, for the following reasons: 1) if we closed the FILE pointer anyway, the interpreter would likely crash because another thread would be using garbage data (that's what we are trying to fix after all!) 2) if close() raises an IOError, it can be called again later, or at worse fclose will be called when the file object is garbage collected 3) close() can already raise an IOError if fclose fails for whatever reason (although for sure it's probably very rare) 4) it doesn't seem wrong to notify the programmer that his code is very unsafe The patch is attached at http://bugs.python.org/issue815646 . It addresses (or at least I hope it does) all potential problems with pure Python code, threads, and the file object. It doesn't try to fix C extensions using the PyFile_AsFile API and doing their own dirty things with the FILE pointer. It could be a second step if the approach is accepted, but as noted in the 2003 discussions it would probably involve a new API. Whether we want to introduce such an API in Python 2.x while Python 3.0 has a different IO model anyway is left open to discussion :) Regards Antoine. From interstar at gmail.com Fri Mar 21 03:25:05 2008 From: interstar at gmail.com (phil jones) Date: Thu, 20 Mar 2008 23:25:05 -0300 Subject: [Python-Dev] [Distutils] Wow, I think I actually *get* it now! In-Reply-To: <79990c6b0803201612h5a6d57bt82faa4e71c451f07@mail.gmail.com> References: <20080319195438.A28DE3A40A5@sparrow.telecommunity.com> <79990c6b0803200233x5f9bd4b8tcac35b1a837a22fa@mail.gmail.com> <20080320103823.21558.491968946.divmod.xquotient.8438@joule.divmod.com> <20080320150640.379643A40A5@sparrow.telecommunity.com> <20080320220844.21558.1047009855.divmod.xquotient.8501@joule.divmod.com> <20080320224227.601543A4074@sparrow.telecommunity.com> <79990c6b0803201612h5a6d57bt82faa4e71c451f07@mail.gmail.com> Message-ID: Just a thought. What I'd really like to see is a quick 5 / 10 minute screen-cast of using setup tools. I've been looking for one for a couple of weeks but haven't found, Something like this buildout example, perhaps : http://video.google.com/videoplay?docid=3428163188647461098&total=7&start=0&num=10&so=0&type=search&plindex=0 cheers phil jones On Thu, Mar 20, 2008 at 8:12 PM, Paul Moore wrote: > On 20/03/2008, Phillip J. Eby wrote: > > Actually, this information is VERY helpful. It makes it blindingly > > obvious to me now that the difference between loving and hating > > setuptools is whether you're *intentionally* using it, or whether it > > shows up in your ecosystem uninvited. > > Exactly. I never thought to express that, but it precisely explains my > situation as well. > > > > Hm. So it seems to me that maybe one thing that would help is a > > "Setuptools Haters' Guide To Setuptools" -- that is, *short* > > documentation specifically written for people who don't want to use > > setuptools and want to minimize its impact on their systems. I could > > probably write something like that fairly easily, now that I have > > some idea of what to go in it, more than, "the existing documentation > > sucks". :) > > > > Can I count on some non-assimilated persons' help in critiquing such > > a document and suggesting any topics I miss? > > I would do so. (From a Windows user's perspective). > > Paul. > > > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG at python.org > http://mail.python.org/mailman/listinfo/distutils-sig > From waterbug at pangalactic.us Fri Mar 21 16:30:53 2008 From: waterbug at pangalactic.us (Stephen Waterbury) Date: Fri, 21 Mar 2008 11:30:53 -0400 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> Message-ID: <47E3D4AD.8000607@pangalactic.us> Phillip J. Eby wrote: > ... if tools exist and are distributed for such a [PEP 262] "database", > and *everybody* agrees to use it as an officially-blessed standard, > then it should be possible for setuptools to co-exist with that > framework, and we're all happy campers. I like this idea and the 3 items proposed to accomplish it. > 2. Update or replace the implementation as appropriate ... After some googling and digging around, I found: Is that what you meant by "the implementation"? > Questions, comments... volunteers? :) I'll try to help, if this is agreed to and if I'm able. Steve From status at bugs.python.org Fri Mar 21 18:06:17 2008 From: status at bugs.python.org (Tracker) Date: Fri, 21 Mar 2008 18:06:17 +0100 (CET) Subject: [Python-Dev] Summary of Tracker Issues Message-ID: <20080321170617.882F8780F7@psf.upfronthosting.co.za> ACTIVITY SUMMARY (03/14/08 - 03/21/08) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1795 open (+106) / 12476 closed (+50) / 14271 total (+156) Open issues with patches: 529 Average duration of open issues: 714 days. Median duration of open issues: 1217 days. Open Issues Breakdown open 1773 (+106) pending 22 ( +0) Issues Created Or Reopened (157) ________________________________ Add a factorial function 03/18/08 http://bugs.python.org/issue2138 reopened rhettinger os.path.normpath over-normalizes 03/14/08 CLOSED http://bugs.python.org/issue2289 created eljay451 [PATCH] Update Lib/distutils/sysconfig.py to handle x64 Windows 03/14/08 CLOSED http://bugs.python.org/issue2290 created Trent.Nelson patch Raise a Py3K warning for catching non-BaseException exceptions 03/14/08 CLOSED http://bugs.python.org/issue2291 created zotbar1234 Missing *-unpacking generalizations 03/15/08 http://bugs.python.org/issue2292 created twouters patch, patch Missing case / switch / evaluate 03/15/08 CLOSED http://bugs.python.org/issue2293 created dbodin Bug in Pickle protocol involving __setstate__ 03/15/08 CLOSED http://bugs.python.org/issue2294 created gpk cPickle corner case - docs or bug? 03/15/08 http://bugs.python.org/issue2295 created gpk [PATCH] Tcl/Tk 8.4.16 patches needed to get an x64 Windows build 03/15/08 http://bugs.python.org/issue2296 created Trent.Nelson patch, 64bit Patch for fatal stack overflow in Windows caused by -v 03/16/08 CLOSED http://bugs.python.org/issue2297 created dgreiman patch Patch for "string without null bytes" check in getargs.c 03/16/08 http://bugs.python.org/issue2298 created dgreiman patch Minor typos in newtypes.rst 03/16/08 CLOSED http://bugs.python.org/issue2299 created dgreiman patch make html fails 03/16/08 http://bugs.python.org/issue2300 created tiran [Py3k] No text shown when SyntaxError (when not UTF8) 03/16/08 CLOSED http://bugs.python.org/issue2301 created ocean-city Uses of SocketServer.BaseServer.shutdown have a race 03/16/08 http://bugs.python.org/issue2302 created jyasskin patch, patch isinstance is 4x as slow as in 2.5 because the common case raise 03/16/08 http://bugs.python.org/issue2303 created jyasskin patch subprocess under windows fails to quote properly when shell=True 03/16/08 http://bugs.python.org/issue2304 created tim.golden patch Update What's new in 2.6 03/16/08 http://bugs.python.org/issue2305 created gvanrossum Update What's new in 3.0 03/16/08 http://bugs.python.org/issue2306 created gvanrossum Decide what to do with bytes/str when transferring pickles betwe 03/16/08 CLOSED http://bugs.python.org/issue2307 created gvanrossum Make structseq more like collections.namedtuple 03/16/08 http://bugs.python.org/issue2308 created gvanrossum Add xturtle to the standard library? 03/16/08 CLOSED http://bugs.python.org/issue2309 created gvanrossum Reorganize the 3.0 Misc/NEWS file 03/16/08 http://bugs.python.org/issue2310 created gvanrossum Update the ACKS file 03/16/08 http://bugs.python.org/issue2311 created gvanrossum Update PEP 3135 (super()) 03/16/08 http://bugs.python.org/issue2312 created gvanrossum correct int / long object type casts 03/16/08 CLOSED http://bugs.python.org/issue2313 created JosephArmbruster patch Test issue 03/16/08 CLOSED http://bugs.python.org/issue2314 created loewis TimedRotatingFileHandler does not account for daylight savings t 03/17/08 http://bugs.python.org/issue2315 created ceder TimedRotatingFileHandler names files incorrectly if nothing is l 03/17/08 http://bugs.python.org/issue2316 created ceder TimedRotatingFileHandler logic for removing files wrong 03/17/08 http://bugs.python.org/issue2317 created ceder TimedRotatingFileHandler: rotate every month, or every year 03/17/08 http://bugs.python.org/issue2318 created ceder abc.py:ABCMeta.__instancecheck__ broken for old style classes 03/17/08 CLOSED http://bugs.python.org/issue2319 created schmir Race condition in subprocess using stdin 03/17/08 http://bugs.python.org/issue2320 created Pankrat return more memory from unicode objects to system 03/17/08 CLOSED http://bugs.python.org/issue2321 created nnorwitz patch, patch Clean up getargs.c and its formatting possibilities 03/17/08 http://bugs.python.org/issue2322 created brett.cannon Make structseq's API look more like a nametuple. 03/17/08 CLOSED http://bugs.python.org/issue2323 created brett.cannon Document that 2.6 pickles of strings turn into pickles of unicod 03/17/08 CLOSED http://bugs.python.org/issue2324 created brett.cannon isinstance(anything, MetaclassThatDefinesInstancecheck) raises i 03/17/08 http://bugs.python.org/issue2325 created jyasskin Doc isnumeric and isdecimal for the unicode object 03/17/08 CLOSED http://bugs.python.org/issue2326 created brett.cannon patch Backport keyword-only arguments to 2.6 03/17/08 http://bugs.python.org/issue2327 created brett.cannon 26backport Class **kwds broken (PEP 3115) 03/17/08 CLOSED http://bugs.python.org/issue2328 created jackdied ImportError: No module named _bsddb 03/17/08 CLOSED http://bugs.python.org/issue2329 created ddk Update PEP 3000 with new release schedule 03/17/08 http://bugs.python.org/issue2330 created gvanrossum Backport parameter annotations 03/17/08 http://bugs.python.org/issue2331 created brett.cannon 26backport Renaming of attributes on functions need to be backported. 03/17/08 CLOSED http://bugs.python.org/issue2332 created brett.cannon 26backport Backport dict comprehensions 03/17/08 http://bugs.python.org/issue2333 created brett.cannon 26backport Backport set comprehensions 03/17/08 http://bugs.python.org/issue2334 created brett.cannon 26backport Backport set literals 03/17/08 http://bugs.python.org/issue2335 created brett.cannon 26backport Backport PEP 3114 (__next__) 03/17/08 http://bugs.python.org/issue2336 created brett.cannon 26backport Backport oct() and hex() to use __index__ 03/17/08 http://bugs.python.org/issue2337 created brett.cannon 26backport Backport reload() moving to imp.reload() 03/17/08 http://bugs.python.org/issue2338 created brett.cannon 26backport Backport intern() -> sys.intern() 03/17/08 CLOSED http://bugs.python.org/issue2339 created brett.cannon patch, 26backport Backport PEP 3132 (extended iterable unpacking) 03/17/08 http://bugs.python.org/issue2340 created brett.cannon 26backport Raise a Py3K warning when raise non-BaseException exceptions 03/17/08 CLOSED http://bugs.python.org/issue2341 created brett.cannon patch, 26backport Comparing between disparate types should raise a Py3K warning 03/17/08 CLOSED http://bugs.python.org/issue2342 created brett.cannon patch, 26backport Raise a Py3K warning when using a float where an int is expected 03/17/08 http://bugs.python.org/issue2343 created brett.cannon 26backport Using an iteration variable outside a list comprehension needs a 03/17/08 http://bugs.python.org/issue2344 created brett.cannon 26backport Using an exception variable outside an 'except' clause should ra 03/17/08 http://bugs.python.org/issue2345 created brett.cannon 26backport Py3K warn against using __members__ 03/17/08 http://bugs.python.org/issue2346 created brett.cannon patch, 26backport Py3K warn for using __methods__ 03/17/08 http://bugs.python.org/issue2347 created brett.cannon 26backport Py3K warn using file.softspace 03/17/08 http://bugs.python.org/issue2348 created brett.cannon patch, 26backport Py3K warn against assigning to True/False 03/17/08 http://bugs.python.org/issue2349 created brett.cannon patch, 26backport Warn against importing 'exceptions' 03/17/08 http://bugs.python.org/issue2350 created brett.cannon 26backport Using __(get|set|del)slice__ needs a Py3K warning 03/17/08 http://bugs.python.org/issue2351 created brett.cannon 26backport Use of __oct__/__hex__ should raise a Py3K warning 03/17/08 http://bugs.python.org/issue2352 created brett.cannon 26backport Use of file.xreadlines() should raise a Py3K warning 03/17/08 http://bugs.python.org/issue2353 created brett.cannon patch, 26backport cmp argument to list.sort()/sorted() should raise a Py3K warning 03/19/08 http://bugs.python.org/issue2354 reopened rhettinger patch, 26backport Using buffer() should raise a Py3K warning 03/17/08 http://bugs.python.org/issue2355 created brett.cannon patch, 26backport sys.exitfunc should raise a Py3K warning 03/17/08 http://bugs.python.org/issue2356 created brett.cannon 26backport sys.exc_{type,values,traceback} should raise a Py3K warning 03/17/08 http://bugs.python.org/issue2357 created brett.cannon patch, 26backport Using sys.exc_clear should raise a Py3K warning 03/17/08 http://bugs.python.org/issue2358 created brett.cannon patch, 26backport A Py3K warning for array.array.{read,write} is needed 03/17/08 http://bugs.python.org/issue2359 created brett.cannon patch, 26backport Fixer for itertools.imap() -> map() 03/17/08 CLOSED http://bugs.python.org/issue2360 created brett.cannon 26backport Fixer for itertools.ifilter() -> filter() 03/17/08 CLOSED http://bugs.python.org/issue2361 created brett.cannon 26backport Fixer for itertools.izip() -> zip() 03/17/08 CLOSED http://bugs.python.org/issue2362 created brett.cannon 26backport Fixer for itertools.ifilterfalse() -> itertools.filterfalse() 03/17/08 CLOSED http://bugs.python.org/issue2363 created brett.cannon 26backport Patch to make 2to3 testing easier 03/17/08 CLOSED http://bugs.python.org/issue2364 created David Wolever patch Fixer for filter(None, ...) -> filter(bool, ...) 03/17/08 CLOSED http://bugs.python.org/issue2365 created brett.cannon 26backport Fixer for new metaclass syntax is needed 03/17/08 http://bugs.python.org/issue2366 created brett.cannon patch, 26backport Fixer to handle new places where parentheses are needed 03/17/08 http://bugs.python.org/issue2367 created brett.cannon patch, 26backport Fixer needed to change __builtin__ -> builtins 03/17/08 http://bugs.python.org/issue2368 created brett.cannon 26backport Fixer for new integer literals are needed 03/17/08 http://bugs.python.org/issue2369 created brett.cannon 26backport operator.{isCallable,sequenceIncludes} needs a fixer 03/17/08 http://bugs.python.org/issue2370 created brett.cannon patch, 26backport Patch for catching exceptions that do not inherit from BaseExcep 03/17/08 CLOSED http://bugs.python.org/issue2371 created taicki patch Pubkey 03/17/08 CLOSED http://bugs.python.org/issue2372 created David Wolever Raise Py3K warnings for comparisons that changed 03/17/08 http://bugs.python.org/issue2373 created bethard patch, 26backport Use of builtin file should give Py3k warning 03/18/08 CLOSED http://bugs.python.org/issue2374 created benjamin.peterson patch PYTHON3PATH environment variable to supersede PYTHONPATH for mul 03/18/08 http://bugs.python.org/issue2375 created glyph Set up "supported"-only buildbot waterfall view 03/18/08 http://bugs.python.org/issue2376 created exarkun Replace import.c with a pure Python implementation 03/18/08 http://bugs.python.org/issue2377 created brett.cannon UnboundLocalError when trying to raise exceptions inside execfil 03/18/08 http://bugs.python.org/issue2378 created jseutter Raise a Py3K warning for __getitem__ or __getslice__ on exceptio 03/18/08 CLOSED http://bugs.python.org/issue2379 created belopolsky patch Raise a Py3K warning for catching nested tuples with non-BaseExc 03/18/08 http://bugs.python.org/issue2380 created belopolsky patch, easy, 26backport test_subprocess fails if your sys.executable is on a path with a 03/18/08 http://bugs.python.org/issue2381 created lanny patch, easy [Py3k] SyntaxError cursor shifted if multibyte character is in l 03/18/08 http://bugs.python.org/issue2382 created ocean-city patch Remove old XXX comment in stat.py 03/18/08 CLOSED http://bugs.python.org/issue2383 created jseutter patch [Py3k] line number is wrong after encoding declaration 03/18/08 http://bugs.python.org/issue2384 created ocean-city run_setup can fail if the setup script uses __file__ 03/18/08 http://bugs.python.org/issue2385 created tarek patch os.strerror missing/HAVE_STRERROR not defined 03/18/08 CLOSED http://bugs.python.org/issue2386 created schmir patch cStringIO and unicode 03/18/08 CLOSED http://bugs.python.org/issue2387 created vdupras patch Compiler warnings when using UCS4 03/18/08 http://bugs.python.org/issue2388 created benjamin.peterson Array pickling exposes internal memory representation of element 03/18/08 http://bugs.python.org/issue2389 created hniksic Merge 2.6 ACKS with 3.0 ACKS 03/18/08 http://bugs.python.org/issue2390 created gvanrossum easy Sean is testing tracker bug. 03/18/08 CLOSED http://bugs.python.org/issue2392 created jafo Backport buffer interface in Python 3.0 to Python 2.6 03/18/08 http://bugs.python.org/issue2393 created teoliphant [Py3k] Finish the memoryview object implementation 03/18/08 http://bugs.python.org/issue2394 created teoliphant [Py3k] struct module changes of PEP 3118 03/18/08 http://bugs.python.org/issue2395 created teoliphant Backport memoryview object to Python 2.6 03/18/08 http://bugs.python.org/issue2396 created teoliphant Backport 3.0 struct module changes to 2.6 03/18/08 http://bugs.python.org/issue2397 created teoliphant test_errno fails with unexpected error value EREMOTEIO 03/18/08 CLOSED http://bugs.python.org/issue2398 created andybalaam patch Patches for Tools/msi 03/18/08 http://bugs.python.org/issue2399 created teoliphant patch from .foo import * should work 03/18/08 CLOSED http://bugs.python.org/issue2400 created nnorwitz Solaris: ctypes tests being skipped despite following #1516 03/18/08 http://bugs.python.org/issue2401 created jafo get rid of warnings in regrtest with -3 03/18/08 http://bugs.python.org/issue2402 created bethard Add figleaf coverage metrics 03/18/08 http://bugs.python.org/issue2403 created jseutter patch, patch Backport ctypes support for buffer protocol to Python 2.6 (ref i 03/18/08 http://bugs.python.org/issue2404 created teoliphant Drop w9xpopen and all dependencies 03/18/08 http://bugs.python.org/issue2405 created Trent.Nelson Improvement suggestions for the gzip module documentation 03/18/08 http://bugs.python.org/issue2406 created madarche warnings.filterwarnings() not isolated between tests 03/18/08 CLOSED http://bugs.python.org/issue2407 created jseutter patch, patch types module can be implemented only in Python 03/18/08 http://bugs.python.org/issue2408 created benjamin.peterson patch regrtest should not just skip imports that fail 03/18/08 http://bugs.python.org/issue2409 created nnorwitz patch absolute import doesn't work for standard python modules 03/18/08 http://bugs.python.org/issue2410 created dangyogi test_nis.py fails if NIS is not configured or used 03/18/08 CLOSED http://bugs.python.org/issue2411 created MichaelBishop patch Check 2to3 for support of print function. 03/18/08 http://bugs.python.org/issue2412 created eric.smith os.strerror does not check for out of range argument 03/19/08 http://bugs.python.org/issue2413 created belopolsky patch Fix implicit relative imports 03/19/08 CLOSED http://bugs.python.org/issue2414 created loewis bytes() should respect __bytes__ 03/19/08 http://bugs.python.org/issue2415 created barry % string formatting does not support %b 03/19/08 http://bugs.python.org/issue2416 created eric.smith [py3k] Integer floor division (//): small int check omitted 03/19/08 http://bugs.python.org/issue2417 created tjreedy patch Incorrect LaTeX generated (Python 2.6a1) 03/19/08 CLOSED http://bugs.python.org/issue2418 created vmanis1 Remove all IRIX dependant modules from aifc module 03/19/08 http://bugs.python.org/issue2419 created MichaelBishop Faq 4.28 -- Trailing comas 03/19/08 http://bugs.python.org/issue2420 created dubiel doc\make.bat fails for htmlhelp because of hardcoded filename 03/19/08 http://bugs.python.org/issue2421 created tim.golden patch Automatically disable pymalloc when running under valgrind 03/19/08 http://bugs.python.org/issue2422 created jamesh patch test_smtplib.py no longer butt slow 03/19/08 http://bugs.python.org/issue2423 created jseutter patch, patch Logging module hides user code errors (bare except) 03/19/08 http://bugs.python.org/issue2424 created bradallen test_py3kwarn doesn't use sys.py3kwarning 03/19/08 CLOSED http://bugs.python.org/issue2425 created jeff.balogh patch pysqlite provides no interface for database SQL dump 03/19/08 http://bugs.python.org/issue2426 created kippesp patch 2to3 should translate itertools.imap into generator expression, 03/19/08 CLOSED http://bugs.python.org/issue2427 created dangyogi 2to3 deletes # comments before "from __future__ import with_stat 03/19/08 CLOSED http://bugs.python.org/issue2428 created dangyogi Patch for test_urllib2_net moving tests to use a local server 03/19/08 http://bugs.python.org/issue2429 created fuzzyman patch Stop test_format.py from executing as side effect of import 03/20/08 http://bugs.python.org/issue2430 created jseutter patch, patch 2to3 is rather slow 03/20/08 http://bugs.python.org/issue2431 created David Wolever patch DictReader does not suport line_num 03/20/08 http://bugs.python.org/issue2432 created ivanoe Merge audio modules 03/20/08 http://bugs.python.org/issue2433 created MichaelBishop Improve platform.win32_ver() support for Python installations wi 03/20/08 CLOSED http://bugs.python.org/issue2434 created lemburg pybench does not run anymore 03/20/08 CLOSED http://bugs.python.org/issue2435 created pitrou Should __future__ print_function be valid in 3.0? 03/20/08 CLOSED http://bugs.python.org/issue2436 created eric.smith Distutils runtime_library_dirs broken on Windows 03/20/08 http://bugs.python.org/issue2437 created janssen subprocess.Popen with wildcard arguments 03/20/08 CLOSED http://bugs.python.org/issue2438 created pbrandt Patch to add a get_data function to pkgutil 03/20/08 http://bugs.python.org/issue2439 created pmoore patch Issues with getargs_n(), PyNumber_Index and PyLong_AsSize_t. 03/20/08 http://bugs.python.org/issue2440 created Trent.Nelson patch, patch Mac build_install.py script fetches unavailable SQLite version 03/20/08 http://bugs.python.org/issue2441 created carlosedp patch Undocumented features added to 2.6 03/21/08 http://bugs.python.org/issue2442 created akuchling uninitialized access to va_list 03/21/08 http://bugs.python.org/issue2443 created rolland Adding __iter__ to class Values of module optparse 03/21/08 http://bugs.python.org/issue2444 created gpolo patch Use The CygwinCCompiler Under Cygwin 03/21/08 http://bugs.python.org/issue2445 created dstanek patch Issues Now Closed (98) ______________________ zlib.crc32() and adler32() return value 174 days http://bugs.python.org/issue1202 gregory.p.smith 64bit, easy doctest EXCEPTION_RE can't handle preceding output 147 days http://bugs.python.org/issue1312 jafo patch XML codec 132 days http://bugs.python.org/issue1399 jafo patch func alloca inside ctypes lib needs #include on solar 112 days http://bugs.python.org/issue1506 theller patch os.system() fails for commands with multiple quoted file names 112 days http://bugs.python.org/issue1524 jafo IDLE installation problems and no message errors 106 days http://bugs.python.org/issue1544 jafo Backport PEP 3141 to 2.6 85 days http://bugs.python.org/issue1689 jyasskin VC6 build patch for release-maint25 78 days http://bugs.python.org/issue1720 ocean-city patch .pypirc not found on windows 74 days http://bugs.python.org/issue1741 jafo easy ZIP files with archive comments longer than 4k not recognized as 71 days http://bugs.python.org/issue1746 jafo OptionMenu class is defined both in Tkinter and Tix 38 days http://bugs.python.org/issue2059 jafo pprint._safe_repr() unsafe on ordering differently types objects 36 days http://bugs.python.org/issue2074 percivall patch __import__ with fromlist=[''] causes double initialization of mo 36 days http://bugs.python.org/issue2090 hauser Blocking sockets take entirely too long to timeout 31 days http://bugs.python.org/issue2132 jafo Pydoc interactive browser misses some docs 32 days http://bugs.python.org/issue2141 ping smtplib.SSLFakeFile hangs forever if "\n" is not encountered 29 days http://bugs.python.org/issue2143 jafo patch pydistutils.cfg won't be found on Windows 25 days http://bugs.python.org/issue2166 jafo Add map, filter, zip to future_builtins 24 days http://bugs.python.org/issue2171 David Wolever code objects should conserve memory 20 days http://bugs.python.org/issue2185 nnorwitz patch, patch [patch] urllib2 hint - disabled ProxyHandler() 24 days http://bugs.python.org/issue2188 jafo urllib.quote() throws KeyError when passed an iterator 24 days http://bugs.python.org/issue2189 jafo Further simplify dict literal bytecode 22 days http://bugs.python.org/issue2197 jafo patch code_hash() can be the same for different code objects 19 days http://bugs.python.org/issue2198 jyasskin ssl module getpeercert returns empty dict when cert_reqs=ssl.CER 20 days http://bugs.python.org/issue2203 jafo Problems using logging module with idle 14 days http://bugs.python.org/issue2216 vsajip patch Py30a3: Possibly confusing message when module detection fails 18 days http://bugs.python.org/issue2219 jafo Search in 2.6 docs does not work 17 days http://bugs.python.org/issue2229 jafo TypeError instead of SyntaxError for syntactically invalid gen e 16 days http://bugs.python.org/issue2238 jafo empty specifier for float.__format__ does not always print at le 7 days http://bugs.python.org/issue2264 eric.smith [Py30a3] xml.parsers.expat recognizes encoding="utf-8" but not e 5 days http://bugs.python.org/issue2278 georg.brandl TextIOWrapper.seekable() always returns False 5 days http://bugs.python.org/issue2282 ping patch Stack overflow exception caused by test_marshal on Windows x64 4 days http://bugs.python.org/issue2286 Trent.Nelson 64bit Problems using logging module with logging.basicConfig(level=log 2 days http://bugs.python.org/issue2287 vsajip Confusing documentation for urllib.urlopen 0 days http://bugs.python.org/issue2288 georg.brandl os.path.normpath over-normalizes 2 days http://bugs.python.org/issue2289 georg.brandl [PATCH] Update Lib/distutils/sysconfig.py to handle x64 Windows 4 days http://bugs.python.org/issue2290 Trent.Nelson patch Raise a Py3K warning for catching non-BaseException exceptions 3 days http://bugs.python.org/issue2291 gvanrossum Missing case / switch / evaluate 0 days http://bugs.python.org/issue2293 tiran Bug in Pickle protocol involving __setstate__ 0 days http://bugs.python.org/issue2294 georg.brandl Patch for fatal stack overflow in Windows caused by -v 3 days http://bugs.python.org/issue2297 Trent.Nelson patch Minor typos in newtypes.rst 0 days http://bugs.python.org/issue2299 georg.brandl patch [Py3k] No text shown when SyntaxError (when not UTF8) 1 days http://bugs.python.org/issue2301 loewis Decide what to do with bytes/str when transferring pickles betwe 1 days http://bugs.python.org/issue2307 gvanrossum Add xturtle to the standard library? 1 days http://bugs.python.org/issue2309 georg.brandl correct int / long object type casts 1 days http://bugs.python.org/issue2313 jyasskin patch Test issue 1 days http://bugs.python.org/issue2314 loewis abc.py:ABCMeta.__instancecheck__ broken for old style classes 0 days http://bugs.python.org/issue2319 jyasskin return more memory from unicode objects to system 1 days http://bugs.python.org/issue2321 nnorwitz patch, patch Make structseq's API look more like a nametuple. 0 days http://bugs.python.org/issue2323 rhettinger Document that 2.6 pickles of strings turn into pickles of unicod 0 days http://bugs.python.org/issue2324 gvanrossum Doc isnumeric and isdecimal for the unicode object 0 days http://bugs.python.org/issue2326 bethard patch Class **kwds broken (PEP 3115) 0 days http://bugs.python.org/issue2328 jackdied ImportError: No module named _bsddb 3 days http://bugs.python.org/issue2329 gregory.p.smith Renaming of attributes on functions need to be backported. 0 days http://bugs.python.org/issue2332 nnorwitz 26backport Backport intern() -> sys.intern() 0 days http://bugs.python.org/issue2339 rhettinger patch, 26backport Raise a Py3K warning when raise non-BaseException exceptions 0 days http://bugs.python.org/issue2341 gvanrossum patch, 26backport Comparing between disparate types should raise a Py3K warning 1 days http://bugs.python.org/issue2342 bethard patch, 26backport Fixer for itertools.imap() -> map() 0 days http://bugs.python.org/issue2360 David Wolever 26backport Fixer for itertools.ifilter() -> filter() 0 days http://bugs.python.org/issue2361 David Wolever 26backport Fixer for itertools.izip() -> zip() 0 days http://bugs.python.org/issue2362 David Wolever 26backport Fixer for itertools.ifilterfalse() -> itertools.filterfalse() 0 days http://bugs.python.org/issue2363 David Wolever 26backport Patch to make 2to3 testing easier 0 days http://bugs.python.org/issue2364 loewis patch Fixer for filter(None, ...) -> filter(bool, ...) 0 days http://bugs.python.org/issue2365 rhettinger 26backport Patch for catching exceptions that do not inherit from BaseExcep 0 days http://bugs.python.org/issue2371 belopolsky patch Pubkey 0 days http://bugs.python.org/issue2372 loewis Use of builtin file should give Py3k warning 0 days http://bugs.python.org/issue2374 benjamin.peterson patch Raise a Py3K warning for __getitem__ or __getslice__ on exceptio 0 days http://bugs.python.org/issue2379 gvanrossum patch Remove old XXX comment in stat.py 2 days http://bugs.python.org/issue2383 georg.brandl patch os.strerror missing/HAVE_STRERROR not defined 0 days http://bugs.python.org/issue2386 belopolsky patch cStringIO and unicode 0 days http://bugs.python.org/issue2387 georg.brandl patch Sean is testing tracker bug. 0 days http://bugs.python.org/issue2392 loewis test_errno fails with unexpected error value EREMOTEIO 0 days http://bugs.python.org/issue2398 andybalaam patch from .foo import * should work 0 days http://bugs.python.org/issue2400 loewis warnings.filterwarnings() not isolated between tests 1 days http://bugs.python.org/issue2407 brett.cannon patch, patch test_nis.py fails if NIS is not configured or used 1 days http://bugs.python.org/issue2411 brett.cannon patch Fix implicit relative imports 1 days http://bugs.python.org/issue2414 David Wolever Incorrect LaTeX generated (Python 2.6a1) 0 days http://bugs.python.org/issue2418 vmanis1 test_py3kwarn doesn't use sys.py3kwarning 0 days http://bugs.python.org/issue2425 brett.cannon patch 2to3 should translate itertools.imap into generator expression, 0 days http://bugs.python.org/issue2427 rhettinger 2to3 deletes # comments before "from __future__ import with_stat 0 days http://bugs.python.org/issue2428 David Wolever Improve platform.win32_ver() support for Python installations wi 0 days http://bugs.python.org/issue2434 lemburg pybench does not run anymore 0 days http://bugs.python.org/issue2435 amaury.forgeotdarc Should __future__ print_function be valid in 3.0? 0 days http://bugs.python.org/issue2436 eric.smith subprocess.Popen with wildcard arguments 0 days http://bugs.python.org/issue2438 skip.montanaro pydoc needs readline completion 2414 days http://bugs.python.org/issue448736 georg.brandl long double causes compiler warnings 2205 days http://bugs.python.org/issue525481 jyasskin Deprecate PyNumber_Check 1973 days http://bugs.python.org/issue628842 nascheme Mach-O gcc optimisation flag can boost performance up to 10% 1774 days http://bugs.python.org/issue735110 jvr test_coercion failing on Panther 1700 days http://bugs.python.org/issue775892 jyasskin sys.exec_prefix does not work 1529 days http://bugs.python.org/issue871747 georg.brandl TclError not a subclass of StandardError 1218 days http://bugs.python.org/issue1068881 nnorwitz add server.shutdown() method to SocketServer 1050 days http://bugs.python.org/issue1193577 jyasskin patch Elemental Security contribution - pgen2 package 874 days http://bugs.python.org/issue1337696 gvanrossum patch Python build fails for gcc 4.x from Gnu 730 days http://bugs.python.org/issue1450807 jyasskin from __future__ import print_function 432 days http://bugs.python.org/issue1633807 eric.smith patch VC6 build patch for trunk 341 days http://bugs.python.org/issue1700463 ocean-city patch chown broken on 64bit 258 days http://bugs.python.org/issue1747858 gregory.p.smith 64bit Make python build with gcc-4.2 on OS X 10.4.9 208 days http://bugs.python.org/issue1779871 jyasskin patch Top Issues Most Discussed (10) ______________________________ 16 Py3K warn against assigning to True/False 4 days open http://bugs.python.org/issue2349 15 tokenize module w/ coding cookie 1806 days open http://bugs.python.org/issue719888 14 Raise a Py3K warning for catching non-BaseException exceptions 3 days closed http://bugs.python.org/issue2291 14 improved allocation of PyUnicode objects 55 days open http://bugs.python.org/issue1943 12 Patch to add a get_data function to pkgutil 1 days open http://bugs.python.org/issue2439 9 [py3k] Integer floor division (//): small int check omitted 3 days open http://bugs.python.org/issue2417 8 Raise Py3K warnings for comparisons that changed 4 days open http://bugs.python.org/issue2373 8 [Py3k] No text shown when SyntaxError (when not UTF8) 1 days closed http://bugs.python.org/issue2301 8 Missing *-unpacking generalizations 6 days open http://bugs.python.org/issue2292 8 Add a factorial function 4 days open http://bugs.python.org/issue2138 From jinty at web.de Sun Mar 23 15:57:15 2008 From: jinty at web.de (Brian Sutherland) Date: Sun, 23 Mar 2008 15:57:15 +0100 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E46176.4090205@v.loewis.de> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <20080321224110.657C63A40A5@sparrow.telecommunity.com> <47E46176.4090205@v.loewis.de> Message-ID: <20080323145715.GD900@mobilista.local> On Sat, Mar 22, 2008 at 02:31:34AM +0100, "Martin v. L?wis" wrote: > > Tools which will need this data, in order to do their work. Hence, > > the reason for standardizing the data, instead of the tool(s). > > If there was a chance that the infrastructure being developed > actually helps these tools, *that* would be a reasonable goal, > IMO. > > However, I'm extremely skeptical that this can ever succeed > to the degree that whoever provides RPMs, .debs, or MSI > files will actually use such data, as they will find that > the data are incomplete, and they have to redo all of it, > anyway. I've found it extremely useful to have access to dependency information when making Debian packages automagically out of setuptools tarballs. It's not easy or robust to access/use it, but for my simple pure python packages, it's been working wonderfully. -- Brian Sutherland From jeff at drinktomi.com Fri Mar 28 09:12:52 2008 From: jeff at drinktomi.com (Jeff Younker) Date: Fri, 28 Mar 2008 01:12:52 -0700 Subject: [Python-Dev] [Distutils] How we can get rid of eggs for 2.6 and beyond In-Reply-To: <47E82AA2.1020502@palladion.com> References: <20080321134749.1C8993A40B0@sparrow.telecommunity.com> <47E40726.1090209@egenix.com> <20080321212127.791B63A4074@sparrow.telecommunity.com> <47E4330C.5070601@egenix.com> <47E4DE8E.6010809@holdenweb.com> <79990c6b0803220646r50e9b455y5bd9b955814f334a@mail.gmail.com> <47E53435.4090104@v.loewis.de> <47E82AA2.1020502@palladion.com> Message-ID: On Mar 24, 2008, at 3:26 PM, Tres Seaver wrote: > Sharing the system python is hugely problematic on a unix box which > actually *uses* python for its own tools: the application is not > "safe" > from additions / updates / removeals of the packages in > /usr/lib/python2.x/site-packages done to support those system tools. > The problem gets worse as multiple non-system applications install > files > there: e.g., the 'twisted' package on Debian boxes depends on an > ancient version of 'zope.interface', which can't be used with any > currently supported version of Zope. This is why versioning would be an useful solution. Each package would use the dependent packages that it requires. Foo 1.0 uses Bar 2.3 and Baz 3.2 uses Foo 1.4. An application can use both Foo 1.0 and Baz 3.2 without having to mediate between their requirements. While nobody is really requesting versioning, it seems to be the solution to many problems that plague us. -jeff