From steve at pearwood.info Sun Nov 1 12:21:15 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 1 Nov 2009 22:21:15 +1100 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it In-Reply-To: <874opgi8rd.fsf@uwakimon.sk.tsukuba.ac.jp> References: <200910231132.45504.w.richert@gmx.net> <200910310211.33590.steve@pearwood.info> <874opgi8rd.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <200911012221.16289.steve@pearwood.info> It seems that even those originally asking for set retrieval have gone silent, so I guess this isn't going anywhere. However, for the benefit of future discussions (because I'm sure this question will be raised again), I'd like to answer a couple of points raised by Stephen. On Sat, 31 Oct 2009 01:42:46 pm Stephen J. Turnbull wrote: > ?> If folks prefer a different name, by all means suggest it. I like > ?> the name suggested by Wikipedia, "pick". > > "Pick" has a connotation of removal in many contexts: "Pick a card, > any card". ? And in just as many, there is no connotation of removal. "Pick a colour". For what it's worth, Forth and similar stack-based languages usually have a function "pick" equivalent to pop-without-removal. pick seems to be a standard term for this behaviour. > Like get, to me it has a flavor of > "according to some criterion": "a band of picked men". ?I would > expect a pick or get operation to take an optional predicate. I think you are underestimating the power of context here. In practice, method names are interpreted in the context of the data being operated on. We don't get confused that ConfigParser.get() has a different signature to dict.get(), which is different again from Queue.get(). list.pop() takes an index; dict.pop() takes a key and an optional second argument; set.pop() takes no argument at all. Is anyone actually confused by this? If not, why would set.get() be more confusing? I think far too many people say "this is confusing" when what they really mean is "I don't like this". > The use case I have so far > observed people to have in mind is a cast from an equivalence class > to a representative member. The difficulty is that we actually have two related, but different, behaviours, and sadly we've conflated the two of them by using the same name "get" for both. One behaviour is what Wikipedia calls pick: set.pick() -> return an arbitrary element from set without removing it This is not useful for the cast you describe. For that, you need a method that takes an argument. I'm going to call it "retrieve", to avoid the baggage of "get": set.retrieve(x) -> return the element from set equal to x In the first case, it seems self-evident to me that having "arbitrary" mean "the same element each time it is called" really would condemn pick() to be unused, but I don't care enough to argue. In the second case, the retrieval isn't arbitrary, so this is not a problem that needs solving. > ?> No. Since sets are unordered, there's no way to seek to a specific > ?> element. > > I think people will realize that in fact *these* sets are ordered and > they will demand a seek function for various practical purposes. We can iterate over dicts, and the order is arbitrary but consistent. Where are the people clamouring for dict.seek()? -- Steven D'Aprano From solipsis at pitrou.net Sun Nov 1 12:33:19 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 1 Nov 2009 11:33:19 +0000 (UTC) Subject: [Python-Dev] Reworking the GIL References: <1256502140.5621.163.camel@localhost> Message-ID: Hello again, Brett Cannon python.org> writes: > > I think it's worth it. Removal of the GIL is a totally open-ended problem > with no solution in sight. This, on the other hand, is a performance benefit > now. I say move forward with this. If it happens to be short-lived because > some actually figures out how to remove the GIL then great, but is that > really going to happen between now and Python 3.2? I doubt it. Based on this whole discussion, I think I am going to merge the new GIL work into the py3k branch, with priority requests disabled. If you think this is premature or uncalled for, or if you just want to review the changes before making a judgement, please voice up :) Regards Antoine. From w.richert at gmx.net Sun Nov 1 14:55:30 2009 From: w.richert at gmx.net (Willi Richert) Date: Sun, 1 Nov 2009 14:55:30 +0100 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it In-Reply-To: <200911012221.16289.steve@pearwood.info> References: <200910231132.45504.w.richert@gmx.net> <874opgi8rd.fsf@uwakimon.sk.tsukuba.ac.jp> <200911012221.16289.steve@pearwood.info> Message-ID: <200911011455.30615.w.richert@gmx.net> Am Sonntag, 1. November 2009 12:21:15 schrieben Sie: > It seems that even those originally asking for set retrieval have gone > silent Nope. Stilll following and waiting for the verdict of the community after having filed the corpus delicti [1] wr [1]: http://bugs.python.org/issue7212 From solipsis at pitrou.net Sun Nov 1 16:23:19 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 1 Nov 2009 15:23:19 +0000 (UTC) Subject: [Python-Dev] Bizarre mtime behaviour Message-ID: Hello, I wondered if someone had a clue about the following behaviour. While debugging an erratic test_mailbox failure on RDM's buildbot (and other machines), it turned out that the system sometimes set the wrong mtime on a directory: $ date && python -c 'import os; os.link("setup.py", "t/c")' && stat t && date Sun Nov 1 09:49:04 EST 2009 File: `t' Size: 144 Blocks: 0 IO Block: 4096 directory Device: 811h/2065d Inode: 223152 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 1001/ pitrou) Gid: ( 1005/ pitrou) Access: 2009-11-01 09:10:11.000000000 -0500 Modify: 2009-11-01 09:49:03.000000000 -0500 Change: 2009-11-01 09:49:03.000000000 -0500 Sun Nov 1 09:49:04 EST 2009 As you see above, the mtime for directory 't' is set to a full second before the actual modification has happened. Sprinkling traces of time.time() and os.path.getmtime() on Lib/mailbox.py shows this is exactly what trips up test_mailbox. I've got posted a patch to fix it (see issue #6896), but I would like to know if such OS behaviour is normal. Regards Antoine. From rhamph at gmail.com Sun Nov 1 17:25:39 2009 From: rhamph at gmail.com (Adam Olsen) Date: Sun, 1 Nov 2009 09:25:39 -0700 Subject: [Python-Dev] Bizarre mtime behaviour In-Reply-To: References: Message-ID: On Sun, Nov 1, 2009 at 08:23, Antoine Pitrou wrote: > Hello, > > I wondered if someone had a clue about the following behaviour. > While debugging an erratic test_mailbox failure on RDM's buildbot (and other > machines), it turned out that the system sometimes set the wrong mtime on a > directory: > > $ date && python -c 'import os; os.link("setup.py", "t/c")' && stat t && date > > Sun Nov ?1 09:49:04 EST 2009 > ?File: `t' > ?Size: 144 ? ? ? ? ? ? Blocks: 0 ? ? ? ? ?IO Block: 4096 ? directory > Device: 811h/2065d ? ? ?Inode: 223152 ? ? ?Links: 2 > Access: (0755/drwxr-xr-x) ?Uid: ( 1001/ ?pitrou) ? Gid: ( 1005/ ?pitrou) > Access: 2009-11-01 09:10:11.000000000 -0500 > Modify: 2009-11-01 09:49:03.000000000 -0500 > Change: 2009-11-01 09:49:03.000000000 -0500 > Sun Nov ?1 09:49:04 EST 2009 > > As you see above, the mtime for directory 't' is set to a full second before the > actual modification has happened. > > Sprinkling traces of time.time() and os.path.getmtime() on Lib/mailbox.py shows > this is exactly what trips up test_mailbox. I've got posted a patch to fix it > (see issue #6896), but I would like to know if such OS behaviour is normal. Looks like an OS bug to me. Linux I'm guessing? -- Adam Olsen, aka Rhamphoryncus From solipsis at pitrou.net Sun Nov 1 17:29:11 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 1 Nov 2009 16:29:11 +0000 (UTC) Subject: [Python-Dev] Bizarre mtime behaviour References: Message-ID: Adam Olsen gmail.com> writes: > > Looks like an OS bug to me. Linux I'm guessing? Yes, but only on certain boxes. I could never reproduce on my home box. RDM (David)'s buildbot is a Gentoo vserver with a reiserfs filesystem. From eric at trueblade.com Sun Nov 1 18:03:36 2009 From: eric at trueblade.com (Eric Smith) Date: Sun, 01 Nov 2009 12:03:36 -0500 Subject: [Python-Dev] Bizarre mtime behaviour In-Reply-To: References: Message-ID: <4AEDBF68.2060902@trueblade.com> Antoine Pitrou wrote: > Adam Olsen gmail.com> writes: >> Looks like an OS bug to me. Linux I'm guessing? > > Yes, but only on certain boxes. I could never reproduce on my home box. > RDM (David)'s buildbot is a Gentoo vserver with a reiserfs filesystem. You'll occasionally see something similar on Windows boxes running FAT, because the timestamps stored in the filesystem have a 2 second granularity there. Not sure if this might be similar, or just a reiserfs issue. Eric. From exarkun at twistedmatrix.com Sun Nov 1 18:54:32 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Sun, 01 Nov 2009 17:54:32 -0000 Subject: [Python-Dev] EC2 buildslaves In-Reply-To: References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com> <4AE41E9E.4050706@v.loewis.de> <20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain> <4AEB6C06.3070907@v.loewis.de> Message-ID: <20091101175432.12668.1205553571.divmod.xquotient.130@localhost.localdomain> On 31 Oct, 08:13 pm, solipsis at pitrou.net wrote: >Martin v. L?wis v.loewis.de> writes: >> >>Not sure whether it's still relevant after the offers of individually >>donated hardware. > >We'll see, indeed. >>However, if you want to look into this, feel free to >>set up EC2 slaves. > >I only know to setup mainstream Linux distros though (Debian- or >Redhat-lookalikes :-)). I've just played a bit and, after the hassle of >juggling >with a bunch of different keys and credentials, setting up an instance >and >saving an image for future use is quite easy. Starting with a mainstream distro doesn't seem like a bad idea. For example, there isn't currently a 32bit Ubuntu (any version) slave. That would be a nice gap to fill in, right? Jean-Paul From robertc at robertcollins.net Sun Nov 1 22:02:32 2009 From: robertc at robertcollins.net (Robert Collins) Date: Mon, 02 Nov 2009 08:02:32 +1100 Subject: [Python-Dev] Bizarre mtime behaviour In-Reply-To: <4AEDBF68.2060902@trueblade.com> References: <4AEDBF68.2060902@trueblade.com> Message-ID: <1257109353.1235.23.camel@lifeless-64> On Sun, 2009-11-01 at 12:03 -0500, Eric Smith wrote: > Antoine Pitrou wrote: > > Adam Olsen gmail.com> writes: > >> Looks like an OS bug to me. Linux I'm guessing? > > > > Yes, but only on certain boxes. I could never reproduce on my home box. > > RDM (David)'s buildbot is a Gentoo vserver with a reiserfs filesystem. > > You'll occasionally see something similar on Windows boxes running FAT, > because the timestamps stored in the filesystem have a 2 second > granularity there. Not sure if this might be similar, or just a reiserfs > issue. Some years ago there was a bug where the dentry cache held timestamps more granular than those that the filesystem could represent. We saw this cause build failures which were troubling to reproduce :) I haven't seen Linux arbitrarily invent time travel so far:). The FAT rounding issue is a possibility, but I didn't think reiserfs was short that much precision. I'd check that the work area you had really was reiser, not a mounted AT partition, and if its not look up the ReiserFS precision for mtime. -Rob -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: From solipsis at pitrou.net Sun Nov 1 22:08:14 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 01 Nov 2009 22:08:14 +0100 Subject: [Python-Dev] Bizarre mtime behaviour In-Reply-To: <1257109353.1235.23.camel@lifeless-64> References: <4AEDBF68.2060902@trueblade.com> <1257109353.1235.23.camel@lifeless-64> Message-ID: <1257109694.30946.7.camel@localhost> Le lundi 02 novembre 2009 ? 08:02 +1100, Robert Collins a ?crit : > > The FAT rounding issue is a possibility, but I didn't think reiserfs was > short that much precision. > > I'd check that the work area you had really was reiser, not a mounted AT > partition, and if its not look up the ReiserFS precision for mtime. Yes it was reiserfs. As for the mtime granularity, since some displayed mtimes were even and some were odd, I assume it's one second or less. Another possibility would be that the filesystem would use a slightly different clock than time.time(), but why? From brett at python.org Sun Nov 1 22:17:24 2009 From: brett at python.org (Brett Cannon) Date: Sun, 1 Nov 2009 13:17:24 -0800 Subject: [Python-Dev] Reworking the GIL In-Reply-To: References: <1256502140.5621.163.camel@localhost> Message-ID: On Sun, Nov 1, 2009 at 03:33, Antoine Pitrou wrote: > > Hello again, > > Brett Cannon python.org> writes: >> >> I think it's worth it. Removal of the GIL is a totally open-ended problem >> with no solution in sight. This, on the other hand, is a performance benefit >> now. I say move forward with this. If it happens to be short-lived because >> some actually figures out how to remove the GIL then great, but is that >> really going to happen between now and Python 3.2? I doubt it. > > Based on this whole discussion, I think I am going to merge the new GIL work > into the py3k branch, with priority requests disabled. This will be a nice Py3K carrot! > > If you think this is premature or uncalled for, or if you just want to review > the changes before making a judgement, please voice up :) I know I personally trust you to not mess it up, Antoine, but that might also come from mental exhaustion and laziness. =) -Brett From regebro at gmail.com Sun Nov 1 22:26:58 2009 From: regebro at gmail.com (Lennart Regebro) Date: Sun, 1 Nov 2009 22:26:58 +0100 Subject: [Python-Dev] nonlocal keyword in 2.x? In-Reply-To: <4AE95F18.4040807@gmail.com> References: <4AE0BF83.4080903@v.loewis.de> <20091022204621.11571.1616253771.divmod.xquotient.1454@localhost.localdomain> <4AE0C73C.9070805@v.loewis.de> <319e029f0910272330me6847d4i5f9a4edb8f9359fc@mail.gmail.com> <19176.18618.662587.613771@montanaro.dyndns.org> <319e029f0910290115y49be844co78da6636452445f@mail.gmail.com> <4AE95F18.4040807@gmail.com> Message-ID: <319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com> 2009/10/29 Nick Coghlan : > Lennart Regebro wrote: >> 2009/10/28 Antoine Pitrou : >>> pobox.com> writes: >>>> ? ? >> So 2.7 support will for the most part be a case not of supporting >>>> ? ? >> Python versions, but Python *users*. >>>> >>>> ? ? Antoine> That's still not a good reason to backport nonlocal. The same >>>> ? ? Antoine> reasoning could be used to backport new features to the 2.6 >>>> ? ? Antoine> branch after all. >>>> >>>> No, because 2.6 is in feature freeze (bug fixes only). ?2.7 is the current >>>> version of 2.x where new features are allowed to be added. >>> That was precisely my point. >> >> Then I don't understand what you are saying. Obviously we shouldn't >> backport to the 2.6 branch, it's in bugfix mode. This is about 2.7. I >> don't see what 2.6 has to do with it. >> >>> There are development practices which mitigate the >>> idea that backporting is always helpful to the user. >> >> And those are? > > You said it above yourself: "bugfix mode" Again: Then I don't understand what you are saying. Obviously we shouldn't backport to the 2.6 branch, it's in bugfix mode. This is about 2.7. I don't see what 2.6 has to do with it. > That's all Antoine's point was - backporting of new features to previous > branches is not automatically a good idea. Obviously we shouldn't backport to the 2.6 branch, it's in bugfix mode. This is about 2.7. I don't see what 2.6 has to do with it. > In the case of 3.2 -> 2.7 backports, there are issues with the initial > development time investment to do the backport, future double-keying of > additional maintenance issues, consideration of possible poor > interaction with legacy features in the 2.x series. It's a bunch of > additional work that isn't going to happen without someone volunteering > to do it. Yes? I feel that you are saying things that are obvious, and that you then expect me to draw a conclusion from this, that you probably find obvious too, but I don't. So I still don't understand what you are saying, or how this in any way contradicts what I said, or clarified or expanded on the matter. As I said: Python 2 support is not only about supporting old versions of Python, but also supporting users of Python2-only modules. So 2.7 support will for the most part be a case not of supporting Python versions, but Python *users*. And contrary to what Antoine said, that *is* a good reason to backport it. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From python at rcn.com Sun Nov 1 22:39:36 2009 From: python at rcn.com (Raymond Hettinger) Date: Sun, 1 Nov 2009 13:39:36 -0800 Subject: [Python-Dev] nonlocal keyword in 2.x? References: <4AE0BF83.4080903@v.loewis.de><20091022204621.11571.1616253771.divmod.xquotient.1454@localhost.localdomain><4AE0C73C.9070805@v.loewis.de><319e029f0910272330me6847d4i5f9a4edb8f9359fc@mail.gmail.com><19176.18618.662587.613771@montanaro.dyndns.org><319e029f0910290115y49be844co78da6636452445f@mail.gmail.com><4AE95F18.4040807@gmail.com> <319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com> Message-ID: <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1> > As I said: Python 2 support is not only about supporting old versions of Python, > but also supporting users of Python2-only modules. So 2.7 support will > for the most part be a case not of supporting Python versions, but > Python *users*. And contrary to what Antoine said, that *is* a good > reason to backport it. FWIW, I support backporting the nonlocal-keyword in 2.7. All of the reasons for introducting nonlocal to 3.x also apply to 2.x. Using the nonlocal keyword in clear and explicit, especially when compared to the existing workarounds which are not pretty. Raymond From lists at cheimes.de Sun Nov 1 23:12:16 2009 From: lists at cheimes.de (Christian Heimes) Date: Sun, 01 Nov 2009 23:12:16 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: References: <1256502140.5621.163.camel@localhost> Message-ID: <4AEE07C0.5090006@cheimes.de> Antoine Pitrou wrote: > Based on this whole discussion, I think I am going to merge the new GIL work > into the py3k branch, with priority requests disabled. > > If you think this is premature or uncalled for, or if you just want to review > the changes before making a judgement, please voice up :) +1 from me. I trust you like Brett does. How much work would it cost to make your patch optional at compile time? For what it's worth we could compare your work on different machines and on different platforms before it gets enabled by default. Can you imagine scenarios where your implementation might be slower than the current GIL implementation? From solipsis at pitrou.net Sun Nov 1 23:27:35 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 1 Nov 2009 22:27:35 +0000 (UTC) Subject: [Python-Dev] Reworking the GIL References: <1256502140.5621.163.camel@localhost> <4AEE07C0.5090006@cheimes.de> Message-ID: Christian Heimes cheimes.de> writes: > > +1 from me. I trust you like Brett does. > > How much work would it cost to make your patch optional at compile time? Quite a bit, because it changes the logic for processing asynchronous pending calls (signals) and asynchronous exceptions in the eval loop. The #defines would get quite convoluted, I think; I'd prefer not to do that. > For what it's worth we could compare your work on different machines and > on different platforms before it gets enabled by default. Can you > imagine scenarios where your implementation might be slower than the > current GIL implementation? I don't really think so. The GIL is taken and released much more predictably than it was before. The thing that might be worth checking is a workload with many threads (say 50 or 100). Does anyone have that? Regards Antoine. From lists at cheimes.de Sun Nov 1 23:43:50 2009 From: lists at cheimes.de (Christian Heimes) Date: Sun, 01 Nov 2009 23:43:50 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: References: <1256502140.5621.163.camel@localhost> <4AEE07C0.5090006@cheimes.de> Message-ID: <4AEE0F26.2000209@cheimes.de> Antoine Pitrou wrote: > Christian Heimes cheimes.de> writes: >> +1 from me. I trust you like Brett does. >> >> How much work would it cost to make your patch optional at compile time? > > Quite a bit, because it changes the logic for processing asynchronous pending > calls (signals) and asynchronous exceptions in the eval loop. The #defines would > get quite convoluted, I think; I'd prefer not to do that. Based on the new piece of information I totally agree. > I don't really think so. The GIL is taken and released much more predictably > than it was before. The thing that might be worth checking is a workload with > many threads (say 50 or 100). Does anyone have that? I don't have an application that works on Python 3 and uses that many threads, sorry. Christian From brett at python.org Mon Nov 2 00:57:21 2009 From: brett at python.org (Brett Cannon) Date: Sun, 1 Nov 2009 15:57:21 -0800 Subject: [Python-Dev] nonlocal keyword in 2.x? In-Reply-To: <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1> References: <4AE0C73C.9070805@v.loewis.de> <319e029f0910272330me6847d4i5f9a4edb8f9359fc@mail.gmail.com> <19176.18618.662587.613771@montanaro.dyndns.org> <319e029f0910290115y49be844co78da6636452445f@mail.gmail.com> <4AE95F18.4040807@gmail.com> <319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com> <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1> Message-ID: On Sun, Nov 1, 2009 at 13:39, Raymond Hettinger wrote: >> As I said: Python 2 support is not only about supporting old versions of >> Python, >> but also supporting users of Python2-only modules. So 2.7 support will >> for the most part be a case not of supporting Python versions, but >> Python *users*. And contrary to what Antoine said, that *is* a good >> reason to backport it. > > FWIW, I support backporting the nonlocal-keyword in 2.7. > All of the reasons for introducting nonlocal to 3.x also apply to 2.x. > Using the nonlocal keyword in clear and explicit, especially > when compared to the existing workarounds which are not pretty. We are voting, I'm -0. -Brett From sturla at molden.no Mon Nov 2 04:22:52 2009 From: sturla at molden.no (Sturla Molden) Date: Mon, 02 Nov 2009 04:22:52 +0100 Subject: [Python-Dev] Integer behaviour in Python 2.6.4 Message-ID: <4AEE508C.1070807@molden.no> Why does this happen? >>> type(2**31-1) It seems to have broken NumPy's RNG on Win32. From curt at hagenlocher.org Mon Nov 2 04:49:18 2009 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Sun, 1 Nov 2009 20:49:18 -0700 Subject: [Python-Dev] Integer behaviour in Python 2.6.4 In-Reply-To: <4AEE508C.1070807@molden.no> References: <4AEE508C.1070807@molden.no> Message-ID: On Sun, Nov 1, 2009 at 8:22 PM, Sturla Molden wrote: > > Why does this happen? > > >>> type(2**31-1) > Does that not happen on non-Windows platforms? 2**31 can't be represented as a 32-bit signed integer, so it's automatically promoted to a long. -- Curt Hagenlocher curt at hagenlocher.org From sturla at molden.no Mon Nov 2 05:21:06 2009 From: sturla at molden.no (Sturla Molden) Date: Mon, 02 Nov 2009 05:21:06 +0100 Subject: [Python-Dev] Integer behaviour in Python 2.6.4 In-Reply-To: References: <4AEE508C.1070807@molden.no> Message-ID: <4AEE5E32.1090506@molden.no> Curt Hagenlocher skrev: > Does that not happen on non-Windows platforms? 2**31 can't be > represented as a 32-bit signed integer, so it's automatically promoted > to a long. > > Yes you are right. I've now traced down the problem to an integer overflow in NumPy. It seems to have this Pyrex code: cdef long lo, hi, diff [...] diff = hi - lo - 1 :-D Sturla From cs at zip.com.au Mon Nov 2 06:29:00 2009 From: cs at zip.com.au (Cameron Simpson) Date: Mon, 2 Nov 2009 16:29:00 +1100 Subject: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it In-Reply-To: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> Message-ID: <20091102052900.GA25160@cskk.homeip.net> On 30Oct2009 20:43, Chris Bergstresser wrote: | On Fri, Oct 30, 2009 at 8:29 PM, Steven D'Aprano wrote: | >> > Iterating over an iterable is | >> > what iterators are for. | > | > set.get(), or set.pick() as Wikipedia calls it, isn't for iterating over | > sets. It is for getting an arbitrary element from the set. [...] | > The purpose is to | > return an arbitrary item each time it is called. If two threads get the | > same element, that's not a problem; if one thread misses an element | > because another thread grabbed it first, that's not a problem either. | > If people prefer a random element instead, I have no problem with | > that -- personally I think that's overkill, but maybe that's just me. | | I also think returning a random one is overkill, in the base set. | And I'd have the base implementation do the simplest thing possible: | return a fixed element (either the first returned when iterated over, | or the last stored, or whatever's easiest based on the internals). | For me, leaving the choice of *which* element to return on each call | is a feature. It allows subclasses to change the behavior to support | other use cases, like a random or round-robin behavior. Personally, I'm for the iteration spec in a lot of ways. Firstly, a .get()/.pick() that always returns the same element feels horrible. Is there anyone here who _likes_ it? Secondly, I think the thread-unsafe arguments are weak. Question: is the existing set iteration scheme thread safe? i.e. does it return a fresh iterator that a thread can happily use concurrently while another thread uses its own iterator? (Absent set modifications). If the answer is yes, then I don't buy the thread-unsafe complaints - there are already plenty of thread unsafe things much as lots of iterators will break in the face of changes to the data structures over which they're iterating. If iter(set) gets you a safe iterator (absent set modification and multiple users of that iterator) then thread safe methods exist and are easy to use. No presently thread-safe program is going to be broken by adding an iterating .get() method. Thirdly, an iteration-like .get() is dead easy to provide: you just keep a _single_, cycling, internal iterator made on demand the same way __iter__ already works. So why not do just do it? There's no need to construct it before anyone calls .get() the first time. Somewhat like: def get(self): for x in self: return x raise something here but not making a new iterator for every caller. Indeed, making a new iterater per caller, in addition to being expensive, might easily return the same element to every caller. Do anyone feel an iteration capable .get() unduely burdens subclasses that want to impement different .get()s? Both the suggested potential subclass choices (round robin and random) suggest iteration capable .get()s (presuming "random" means "shuffle order" rather than potentially returning the same element twice). Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Why doesn't DOS ever say EXCELLENT command or filename? From greg at krypto.org Mon Nov 2 07:39:21 2009 From: greg at krypto.org (Gregory P. Smith) Date: Sun, 1 Nov 2009 22:39:21 -0800 Subject: [Python-Dev] Reworking the GIL In-Reply-To: References: <1256502140.5621.163.camel@localhost> Message-ID: <52dc1c820911012239m627036ccx2b5eca83c0b4b94f@mail.gmail.com> On Sun, Nov 1, 2009 at 3:33 AM, Antoine Pitrou wrote: > > Hello again, > > Brett Cannon python.org> writes: >> >> I think it's worth it. Removal of the GIL is a totally open-ended problem >> with no solution in sight. This, on the other hand, is a performance benefit >> now. I say move forward with this. If it happens to be short-lived because >> some actually figures out how to remove the GIL then great, but is that >> really going to happen between now and Python 3.2? I doubt it. > > Based on this whole discussion, I think I am going to merge the new GIL work > into the py3k branch, with priority requests disabled. > > If you think this is premature or uncalled for, or if you just want to review > the changes before making a judgement, please voice up :) +1 Good idea. Thats the best way to make sure this work gets anywhere. It can be iterated on from there if anyone has objections. From fuzzyman at voidspace.org.uk Mon Nov 2 09:29:22 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 02 Nov 2009 09:29:22 +0100 Subject: [Python-Dev] nonlocal keyword in 2.x? In-Reply-To: References: <4AE0C73C.9070805@v.loewis.de> <319e029f0910272330me6847d4i5f9a4edb8f9359fc@mail.gmail.com> <19176.18618.662587.613771@montanaro.dyndns.org> <319e029f0910290115y49be844co78da6636452445f@mail.gmail.com> <4AE95F18.4040807@gmail.com> <319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com> <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1> Message-ID: <4AEE9862.8010201@voidspace.org.uk> Brett Cannon wrote: > On Sun, Nov 1, 2009 at 13:39, Raymond Hettinger wrote: > >>> As I said: Python 2 support is not only about supporting old versions of >>> Python, >>> but also supporting users of Python2-only modules. So 2.7 support will >>> for the most part be a case not of supporting Python versions, but >>> Python *users*. And contrary to what Antoine said, that *is* a good >>> reason to backport it. >>> >> FWIW, I support backporting the nonlocal-keyword in 2.7. >> All of the reasons for introducting nonlocal to 3.x also apply to 2.x. >> Using the nonlocal keyword in clear and explicit, especially >> when compared to the existing workarounds which are not pretty. >> > > We are voting, I'm -0. > nonlocal is really useful for tests so I'd love to have this in 2.7, but obviously someone has to do the work so I'm only +0. Michael > -Brett > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ From martin at v.loewis.de Mon Nov 2 10:08:48 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 02 Nov 2009 10:08:48 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: <1256502140.5621.163.camel@localhost> References: <1256502140.5621.163.camel@localhost> Message-ID: <4AEEA1A0.8050908@v.loewis.de> > The new GIL does away with this by ditching _Py_Ticker entirely and > instead using a fixed interval (by default 5 milliseconds, but settable) > after which we ask the main thread to release the GIL and let another > thread be scheduled. I've looked at this part of the implementation, and have a few comments. a) why is gil_interval a double? Make it an int, counting in microseconds. b) notice that, on Windows, minimum wait resolution may be as large as 15ms (e.g. on XP, depending on the hardware). Not sure what this means for WaitForMultipleObjects; most likely, if you ask for a 5ms wait, it waits until the next clock tick. It would be bad if, on some systems, a wait of 5ms would mean that it immediately returns. c) is the gil_drop_request handling thread-safe? If your answer is "yes", you should add a comment as to what the assumptions are of this code (ISTM that multiple threads may simultaneously attempt to set the drop request, overlapping with the holding thread actually dropping the GIL). There is also the question whether it is thread-safe to write into a "volatile int"; I keep forgetting the answer. Regards, Martin From dickinsm at gmail.com Mon Nov 2 10:18:49 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 2 Nov 2009 09:18:49 +0000 Subject: [Python-Dev] thanks to everyone cleaning up the tests In-Reply-To: References: Message-ID: <5c6f2a5d0911020118h4cfee72cl430ef45a1cbf9bc7@mail.gmail.com> On Sat, Oct 31, 2009 at 9:47 PM, Brett Cannon wrote: > Just wanted to publicly thank everyone who has been causing all the > checkins to fix and stabilize the test suite (I think it's mostly > Antoine and Mark, but I could be missing somebody; I'm under a > deadline so only have marginal higher brain functionality). Not guilty, your honour. Blame Antoine and David. :) David's new buildslave seems to be making a difference, too. Mark From solipsis at pitrou.net Mon Nov 2 11:21:05 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 2 Nov 2009 10:21:05 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Retrieve_an_arbitrary_element_from_a_set?= =?utf-8?q?=09withoutremoving_it?= References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <20091102052900.GA25160@cskk.homeip.net> Message-ID: Cameron Simpson zip.com.au> writes: > > Personally, I'm for the iteration spec in a lot of ways. > > Firstly, a .get()/.pick() that always returns the same element feels > horrible. Is there anyone here who _likes_ it? I do. Since the caller is asking for an arbitrary element, returning the same element is legitimate. It's funny how people seem to have a problem with the word "arbitrary" :) And I'm -1 on any implicit iteration attaching some state to the object. If you want to iterate, there's already an obvious way to it. Regards Antoine. From sturla at molden.no Mon Nov 2 11:28:39 2009 From: sturla at molden.no (Sturla Molden) Date: Mon, 02 Nov 2009 11:28:39 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: <4AEEA1A0.8050908@v.loewis.de> References: <1256502140.5621.163.camel@localhost> <4AEEA1A0.8050908@v.loewis.de> Message-ID: <4AEEB457.6010808@molden.no> Martin v. L?wis skrev: > b) notice that, on Windows, minimum wait resolution may be as large as > 15ms (e.g. on XP, depending on the hardware). Not sure what this > means for WaitForMultipleObjects; most likely, if you ask for a 5ms > wait, it waits until the next clock tick. It would be bad if, on > some systems, a wait of 5ms would mean that it immediately returns. > Which is why one should use multimedia timers with QPC on Windows. To get a wait function with much better resolution than Windows' default, do this: 1. Set a high resolution with timeBeginPeriod. 2. Loop using a time-out of 0 for WaitForMultipleObjects and put a Sleep(0) in the loop not to burn the CPU. Call QPF to get a precise timing, and break the loop when the requested time-out has been reached. 3. When you are done, call timeBeginPeriod to turn the multimedia timer off. This is how you create usleep() in Windows as well: Just loop on QPF and Sleep(0) after setting timeBeginPeriod(1). From martin at v.loewis.de Mon Nov 2 11:42:47 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 02 Nov 2009 11:42:47 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: <4AEEB457.6010808@molden.no> References: <1256502140.5621.163.camel@localhost> <4AEEA1A0.8050908@v.loewis.de> <4AEEB457.6010808@molden.no> Message-ID: <4AEEB7A7.3020102@v.loewis.de> Sturla Molden wrote: > Martin v. L?wis skrev: >> b) notice that, on Windows, minimum wait resolution may be as large as >> 15ms (e.g. on XP, depending on the hardware). Not sure what this >> means for WaitForMultipleObjects; most likely, if you ask for a 5ms >> wait, it waits until the next clock tick. It would be bad if, on >> some systems, a wait of 5ms would mean that it immediately returns. >> > Which is why one should use multimedia timers with QPC on Windows. Maybe you should study the code under discussion before making such a proposal. Regards, Martin From solipsis at pitrou.net Mon Nov 2 11:53:37 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 2 Nov 2009 10:53:37 +0000 (UTC) Subject: [Python-Dev] Reworking the GIL References: <1256502140.5621.163.camel@localhost> <4AEEA1A0.8050908@v.loewis.de> Message-ID: Martin v. L?wis v.loewis.de> writes: > > I've looked at this part of the implementation, and have a few comments. > a) why is gil_interval a double? Make it an int, counting in > microseconds. Ok, I'll do that. > b) notice that, on Windows, minimum wait resolution may be as large as > 15ms (e.g. on XP, depending on the hardware). Not sure what this > means for WaitForMultipleObjects; most likely, if you ask for a 5ms > wait, it waits until the next clock tick. It would be bad if, on > some systems, a wait of 5ms would mean that it immediately returns. I'll let someone else give an authoritative answer. I did the Windows version mainly by reading online MSDN docs, and testing it worked fine in an XP VM. Anyway, here is what the online docs have to say : ? The system clock "ticks" at a constant rate. If the time-out interval is less than the resolution of the system clock, the wait may time out in less than the specified length of time. If the time-out interval is greater than one tick but less than two, the wait can be anywhere between one and two ticks, and so on. ? So it seems that the timeout always happens on a Windows tick boundary, which means that it can immediately return, but only very rarely so and never more than once in a row... > c) is the gil_drop_request handling thread-safe? If your answer is > "yes", you should add a comment as to what the assumptions are of > this code (ISTM that multiple threads may simultaneously attempt > to set the drop request, overlapping with the holding thread actually > dropping the GIL). The gil_drop_request is volatile mainly out of precaution, but it's probably not necessary. It is only written (set/cleared) when holding the gil_mutex; however, it can be read while not holding that mutex. Exceptionally reading the "wrong" value would not have any adverse consequences, it would just decrease the switching quality at the said instant. Regards Antoine. From w.richert at gmx.net Mon Nov 2 12:00:24 2009 From: w.richert at gmx.net (Willi Richert) Date: Mon, 2 Nov 2009 12:00:24 +0100 Subject: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it In-Reply-To: <20091102052900.GA25160@cskk.homeip.net> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <20091102052900.GA25160@cskk.homeip.net> Message-ID: <200911021200.24249.w.richert@gmx.net> Hi, all your points are valid -- for the experienced Python programmer who has stumbled across this issue already and solved it in one of several ways. All your points, however, do not support the "one obvious way to do it" philosophy of Python. It's all about making Python even more clean and beautiful. wr Am Montag, 2. November 2009 06:29:00 schrieb Cameron Simpson: > On 30Oct2009 20:43, Chris Bergstresser wrote: > | On Fri, Oct 30, 2009 at 8:29 PM, Steven D'Aprano wrote: > | >> > Iterating over an iterable is > | >> > what iterators are for. > | > > | > set.get(), or set.pick() as Wikipedia calls it, isn't for iterating > | > over sets. It is for getting an arbitrary element from the set. > > [...] > > | > The purpose is to > | > return an arbitrary item each time it is called. If two threads get the > | > same element, that's not a problem; if one thread misses an element > | > because another thread grabbed it first, that's not a problem either. > | > If people prefer a random element instead, I have no problem with > | > that -- personally I think that's overkill, but maybe that's just me. > | > | I also think returning a random one is overkill, in the base set. > | And I'd have the base implementation do the simplest thing possible: > | return a fixed element (either the first returned when iterated over, > | or the last stored, or whatever's easiest based on the internals). > | For me, leaving the choice of *which* element to return on each call > | is a feature. It allows subclasses to change the behavior to support > | other use cases, like a random or round-robin behavior. > > Personally, I'm for the iteration spec in a lot of ways. > > Firstly, a .get()/.pick() that always returns the same element feels > horrible. Is there anyone here who _likes_ it? > > Secondly, I think the thread-unsafe arguments are weak. Question: is the > existing set iteration scheme thread safe? i.e. does it return a fresh > iterator that a thread can happily use concurrently while another thread > uses its own iterator? (Absent set modifications). If the answer is yes, > then I don't buy the thread-unsafe complaints - there are already plenty > of thread unsafe things much as lots of iterators will break in the face > of changes to the data structures over which they're iterating. If > iter(set) gets you a safe iterator (absent set modification and multiple > users of that iterator) then thread safe methods exist and are easy to > use. No presently thread-safe program is going to be broken by adding an > iterating .get() method. > > Thirdly, an iteration-like .get() is dead easy to provide: you just keep a > _single_, cycling, internal iterator made on demand the same way __iter__ > already works. So why not do just do it? There's no need to construct it > before anyone calls .get() the first time. Somewhat like: > > def get(self): > for x in self: > return x > raise something here > > but not making a new iterator for every caller. Indeed, making a new > iterater per caller, in addition to being expensive, might easily return > the same element to every caller. > > Do anyone feel an iteration capable .get() unduely burdens subclasses > that want to impement different .get()s? Both the suggested potential > subclass choices (round robin and random) suggest iteration capable > .get()s (presuming "random" means "shuffle order" rather than potentially > returning the same element twice). > > Cheers, > From solipsis at pitrou.net Mon Nov 2 12:30:56 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 2 Nov 2009 11:30:56 +0000 (UTC) Subject: [Python-Dev] EC2 buildslaves References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com> <4AE41E9E.4050706@v.loewis.de> <20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain> <4AEB6C06.3070907@v.loewis.de> <20091101175432.12668.1205553571.divmod.xquotient.130@localhost.localdomain> Message-ID: twistedmatrix.com> writes: > > Starting with a mainstream distro doesn't seem like a bad idea. For > example, there isn't currently a 32bit Ubuntu (any version) slave. That > would be a nice gap to fill in, right? I've setup a buildslave on an EC2 Ubuntu Karmic instance here: http://www.python.org/dev/buildbot/all/buildslaves/pitrou-ubuntu However, since it's right now billed on my credit card, I'll shut it down in a couple of days. I wonder how we can get the PSF to be billed instead of me, if the PSF accepts to fund such an instance (which, given EC2 prices, is perhaps not the best use of money?). Regards Antoine. From martin at v.loewis.de Mon Nov 2 12:42:19 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 02 Nov 2009 12:42:19 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: References: <1256502140.5621.163.camel@localhost> <4AEEA1A0.8050908@v.loewis.de> Message-ID: <4AEEC59B.1040902@v.loewis.de> >> c) is the gil_drop_request handling thread-safe? If your answer is >> "yes", you should add a comment as to what the assumptions are of >> this code (ISTM that multiple threads may simultaneously attempt >> to set the drop request, overlapping with the holding thread actually >> dropping the GIL). > > The gil_drop_request is volatile mainly out of precaution, but it's probably not > necessary. It is only written (set/cleared) when holding the gil_mutex; however, > it can be read while not holding that mutex. Exceptionally reading the "wrong" > value would not have any adverse consequences, it would just decrease the > switching quality at the said instant. I think it then needs definitely to be volatile - otherwise, the compiler may chose to cache it in a register (assuming enough registers are available), instead of re-reading it from memory each time (which is exactly what volatile then forces it to). Even if it is read from memory, I still wonder what might happen on systems that require explicit memory barriers to synchronize across CPUs. What if CPU 1 keeps reading a 0 value out of its cache, even though CPU 1 has written an 1 value a long time ago? IIUC, any (most?) pthread calls would cause synchronization in that case, which is why applications that also use locks for reading: http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap04.html#tag_04_10 Of course, on x86, you won't see any issues, because it's cache-coherent anyway. Regards, Martin From solipsis at pitrou.net Mon Nov 2 13:15:28 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 2 Nov 2009 12:15:28 +0000 (UTC) Subject: [Python-Dev] Reworking the GIL References: <1256502140.5621.163.camel@localhost> <4AEEA1A0.8050908@v.loewis.de> <4AEEC59B.1040902@v.loewis.de> Message-ID: Martin v. L?wis v.loewis.de> writes: > [gil_drop_request] > Even if it is read from memory, I still wonder what might happen on > systems that require explicit memory barriers to synchronize across > CPUs. What if CPU 1 keeps reading a 0 value out of its cache, even > though CPU 1 has written an 1 value a long time ago? > > IIUC, any (most?) pthread calls would cause synchronization in that > case, which is why applications that also use locks for reading: > > http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap04.html#tag_04_10 > > Of course, on x86, you won't see any issues, because it's cache-coherent > anyway. I think there are two things here: - all machines Python runs on should AFAIK be cache-coherent: CPUs synchronize their views of memory in a rather timely fashion. - memory ordering: writes made by a CPU can be seen in a different order by another CPU (i.e. CPU 1 writes A before B, but CPU 2 sees B written before A). I don't see how this can apply to gil_drop_request, since it's a single variable (and, moreover, only a single bit of it is significant). (there's an explanation of memory ordering issues here: http://www.linuxjournal.com/article/8211) As a side note, I remember Jeffrey Yasskin trying to specify an ordering model for Python code (see http://code.google.com/p/unladen-swallow/wiki/MemoryModel). Regards Antoine. From ncoghlan at gmail.com Mon Nov 2 13:27:48 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 02 Nov 2009 22:27:48 +1000 Subject: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it In-Reply-To: References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <20091102052900.GA25160@cskk.homeip.net> Message-ID: <4AEED044.4020405@gmail.com> Antoine Pitrou wrote: > Cameron Simpson zip.com.au> writes: >> Personally, I'm for the iteration spec in a lot of ways. >> >> Firstly, a .get()/.pick() that always returns the same element feels >> horrible. Is there anyone here who _likes_ it? > > I do. Since the caller is asking for an arbitrary element, returning the same > element is legitimate. It's funny how people seem to have a problem with the > word "arbitrary" :) Agreed. Arbitrary is arbitrary - for a stateless method that returns an arbitrary result, return the same value every time is fine. > And I'm -1 on any implicit iteration attaching some state to the object. If you > want to iterate, there's already an obvious way to it. Also agreed. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From martin at v.loewis.de Mon Nov 2 13:31:25 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 02 Nov 2009 13:31:25 +0100 Subject: [Python-Dev] EC2 buildslaves In-Reply-To: References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com> <4AE41E9E.4050706@v.loewis.de> <20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain> <4AEB6C06.3070907@v.loewis.de> <20091101175432.12668.1205553571.divmod.xquotient.130@localhost.localdomain> Message-ID: <4AEED11D.3000602@v.loewis.de> > I've setup a buildslave on an EC2 Ubuntu Karmic instance here: > http://www.python.org/dev/buildbot/all/buildslaves/pitrou-ubuntu > > However, since it's right now billed on my credit card, I'll shut it down in a > couple of days. I wonder how we can get the PSF to be billed instead of me, if > the PSF accepts to fund such an instance (which, given EC2 prices, is perhaps > not the best use of money?). Send a request to psf-board at python.org. Such request should include: - contact information (who'll be running this project) - project purpose/description - estimated expenses (in case of doubt, round up rather than rounding down) - a proposal of how payment should proceed. I'm not quite sure whether it could be billed on the PSF credit card (you may ask Kurt Kaiser in advance); traditionally, it worked best when we received invoices. There will be a board meeting next Monday, so it might be useful to have a proposal by then. As for whether that's good use of the money, I'm skeptical as well. I don't actually know what EC2 prices *are*, or what the current pricing for a root server is (plus there had been various offers from people donating hardware - from people who would be unable to also donate time). There was discussion that an EC2 instance can be turned on only when needed, so we could try to set up something like that (the build master could then trigger activation of the machine, IIUC). However, it might be that the machine would have to be up most of the day, as there would be one build going on always, anyway. Regards, Martin From solipsis at pitrou.net Mon Nov 2 13:41:20 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 02 Nov 2009 13:41:20 +0100 Subject: [Python-Dev] EC2 buildslaves In-Reply-To: <4AEED11D.3000602@v.loewis.de> References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com> <4AE41E9E.4050706@v.loewis.de> <20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain> <4AEB6C06.3070907@v.loewis.de> <20091101175432.12668.1205553571.divmod.xquotient.130@localhost.localdomain> <4AEED11D.3000602@v.loewis.de> Message-ID: <1257165680.5521.11.camel@localhost> Le lundi 02 novembre 2009 ? 13:31 +0100, "Martin v. L?wis" a ?crit : > > There was discussion that an EC2 instance can be turned on only when > needed, so we could try to set up something like that (the build master > could then trigger activation of the machine, IIUC). However, it might > be that the machine would have to be up most of the day, as there would > be one build going on always, anyway. Yes, I think that would be the case. We have frequent commits on each of the 4 branches under test, with a test suite that takes quite a bit of time to run in debug mode with -uall. Moreover, a standard ("small", which also means cheapest) EC2 instance apparently provides (based on a quick test) between 25% and 50% of the power of a full CPU core, which makes builds longer. I thought a full CPU core was provided, but it is not. An always-on "small" EC2 instance is at least 500$ a year, with a small storage cost added to that. Therefore, I think EC2 buildslaves would be interesting under the condition that they are donated rather than paid for. I don't know whether anyone has contacts at Amazon. (but then any donated piece of hardware would be good, not necessary an EC2 instance) I assume Jean-Paul made his original suggestion under the premise that the EC2 instances would only be run when necessary, which is probably very economical with Twisted's development model (few commits on trunk) but not with ours. Regards Antoine. From ssteinerx at gmail.com Mon Nov 2 13:42:35 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Mon, 2 Nov 2009 07:42:35 -0500 Subject: [Python-Dev] EC2 buildslaves In-Reply-To: References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com> <4AE41E9E.4050706@v.loewis.de> <20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain> <4AEB6C06.3070907@v.loewis.de> <20091101175432.12668.1205553571.divmod.xquotient.130@localhost.localdomain> Message-ID: <008C8A20-798A-45F0-81D4-C90785EEC8CA@gmail.com> On Nov 2, 2009, at 6:30 AM, Antoine Pitrou wrote: > twistedmatrix.com> writes: >> >> Starting with a mainstream distro doesn't seem like a bad idea. For >> example, there isn't currently a 32bit Ubuntu (any version) slave. >> That >> would be a nice gap to fill in, right? > > I've setup a buildslave on an EC2 Ubuntu Karmic instance here: > http://www.python.org/dev/buildbot/all/buildslaves/pitrou-ubuntu > If you could send me the script that you used to set it up, I could give it a shot on RackSpace where it's cheaper (and I have a temporary developer account). S From solipsis at pitrou.net Mon Nov 2 13:49:10 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 02 Nov 2009 13:49:10 +0100 Subject: [Python-Dev] EC2 buildslaves In-Reply-To: <008C8A20-798A-45F0-81D4-C90785EEC8CA@gmail.com> References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com> <4AE41E9E.4050706@v.loewis.de> <20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain> <4AEB6C06.3070907@v.loewis.de> <20091101175432.12668.1205553571.divmod.xquotient.130@localhost.localdomain> <008C8A20-798A-45F0-81D4-C90785EEC8CA@gmail.com> Message-ID: <1257166150.5521.19.camel@localhost> Le lundi 02 novembre 2009 ? 07:42 -0500, ssteinerX at gmail.com a ?crit : > > If you could send me the script that you used to set it up, I could > give it a shot on RackSpace where it's cheaper (and I have a temporary > developer account). There's no need for a special script, really. Install Python, buildbot, subversion (all packaged by your Linux provider), then a couple of development libraries so that enough extension modules get built: dev headers for zlib, bz2, openssl and sqlite3 would be enough IMO. When you have done that, go to http://wiki.python.org/moin/BuildBot and follow the instructions at the end. Skip "install buildbot from source" if you've installed it from your distro's package repo. Also, you may have to replace "buildbot slave" with "buildbot create-slave". From martin at v.loewis.de Mon Nov 2 13:51:43 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 02 Nov 2009 13:51:43 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: References: <1256502140.5621.163.camel@localhost> <4AEEA1A0.8050908@v.loewis.de> <4AEEC59B.1040902@v.loewis.de> Message-ID: <4AEED5DF.8040105@v.loewis.de> > - all machines Python runs on should AFAIK be cache-coherent: CPUs synchronize > their views of memory in a rather timely fashion. Ok. I thought that Itanium was an example where this assumption is actually violated (as many web pages claim such a restriction), however, it seems that on Itanium, caches are indeed synchronized using MESI. So claims wrt. lack of cache consistency on Itanium, and the need for barrier instruction, seem to be caused by the Itanium feature that allows the processor to fetch memory out-of-order, i.e. an earlier read may see a later memory state. This is apparently used to satisfy reads as soon as the cache line is read (so that the cache line can be discarded earlier). Wrt. to your requirement ("rather timely fashion", this still seems to be fine). Still, this all needs to be documented in the code :-) Regards, Martin From martin at v.loewis.de Mon Nov 2 13:59:37 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 02 Nov 2009 13:59:37 +0100 Subject: [Python-Dev] EC2 buildslaves In-Reply-To: <1257165680.5521.11.camel@localhost> References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com> <4AE41E9E.4050706@v.loewis.de> <20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain> <4AEB6C06.3070907@v.loewis.de> <20091101175432.12668.1205553571.divmod.xquotient.130@localhost.localdomain> <4AEED11D.3000602@v.loewis.de> <1257165680.5521.11.camel@localhost> Message-ID: <4AEED7B9.2000209@v.loewis.de> > An always-on "small" EC2 instance is at least 500$ a year, with a small > storage cost added to that. OTOH, that isn't that expensive (compared to the other PSF expenses), plus people keep donating money, so when we say what we use it for, there may be a larger return than just the test results. OTTH, the same could be achieved by buying a hosted server elsewhere. Regards, Martin P.S. Perhaps this *is* the time for Steve to ask for "always-on" machines. From solipsis at pitrou.net Mon Nov 2 14:06:14 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 02 Nov 2009 14:06:14 +0100 Subject: [Python-Dev] EC2 buildslaves In-Reply-To: <4AEED7B9.2000209@v.loewis.de> References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com> <4AE41E9E.4050706@v.loewis.de> <20091025140512.11571.1513702964.divmod.xquotient.1543@localhost.localdomain> <4AEB6C06.3070907@v.loewis.de> <20091101175432.12668.1205553571.divmod.xquotient.130@localhost.localdomain> <4AEED11D.3000602@v.loewis.de> <1257165680.5521.11.camel@localhost> <4AEED7B9.2000209@v.loewis.de> Message-ID: <1257167174.5521.23.camel@localhost> > OTOH, that isn't that expensive (compared to the other PSF expenses), > plus people keep donating money, so when we say what we use it for, > there may be a larger return than just the test results. > > OTTH, the same could be achieved by buying a hosted server elsewhere. One advantage of a real hosted server is that we could have a bunch of our own VMs on it, which is probably not possible (and perhaps not allowed) on an EC2 instance (not to mention it could really be slow). (I'm not volunteering to install and manage VMs, however; I don't think I'm competent to do that) From jnoller at gmail.com Mon Nov 2 15:04:03 2009 From: jnoller at gmail.com (Jesse Noller) Date: Mon, 2 Nov 2009 09:04:03 -0500 Subject: [Python-Dev] EC2 buildslaves In-Reply-To: <1257167174.5521.23.camel@localhost> References: <5c6f2a5d0910250154nfcdabafyc051389c956ef258@mail.gmail.com> <4AEB6C06.3070907@v.loewis.de> <20091101175432.12668.1205553571.divmod.xquotient.130@localhost.localdomain> <4AEED11D.3000602@v.loewis.de> <1257165680.5521.11.camel@localhost> <4AEED7B9.2000209@v.loewis.de> <1257167174.5521.23.camel@localhost> Message-ID: <4222a8490911020604s73de78bj44c197b43b30e38b@mail.gmail.com> On Mon, Nov 2, 2009 at 8:06 AM, Antoine Pitrou wrote: > >> OTOH, that isn't that expensive (compared to the other PSF expenses), >> plus people keep donating money, so when we say what we use it for, >> there may be a larger return than just the test results. >> >> OTTH, the same could be achieved by buying a hosted server elsewhere. > > One advantage of a real hosted server is that we could have a bunch of > our own VMs on it, which is probably not possible (and perhaps not > allowed) on an EC2 instance (not to mention it could really be slow). > > (I'm not volunteering to install and manage VMs, however; I don't think > I'm competent to do that) Now that the bulk of my pycon work is "completed" - I'll thread swap to my VM host proposal and the moratorium PEP. So don't worry about the VM host. jesse From sturla at molden.no Mon Nov 2 15:07:14 2009 From: sturla at molden.no (Sturla Molden) Date: Mon, 02 Nov 2009 15:07:14 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: <4AEEB7A7.3020102@v.loewis.de> References: <1256502140.5621.163.camel@localhost> <4AEEA1A0.8050908@v.loewis.de> <4AEEB457.6010808@molden.no> <4AEEB7A7.3020102@v.loewis.de> Message-ID: <4AEEE792.9080108@molden.no> Martin v. L?wis skrev: > Maybe you should study the code under discussion before making such > a proposal. > I did, and it does nothing of what I suggested. I am sure I can make the Windows GIL in ceval_gil.h and the mutex in thread_nt.h at lot more precise and efficient. This is the kind of code I was talking about, from ceval_gil.h: r = WaitForMultipleObjects(2, objects, TRUE, milliseconds); I would turn on multimedia timer (it is not on by default), and replace this call with a loop, approximately like this: for (;;) { r = WaitForMultipleObjects(2, objects, TRUE, 0); /* blah blah blah */ QueryPerformanceCounter(&cnt); if (cnt > timeout) break; Sleep(0); } And the timeout "milliseconds" would now be computed from querying the performance counter, instead of unreliably by the Windows NT kernel. Sturla From solipsis at pitrou.net Mon Nov 2 15:16:58 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 2 Nov 2009 14:16:58 +0000 (UTC) Subject: [Python-Dev] Reworking the GIL References: <1256502140.5621.163.camel@localhost> <4AEEA1A0.8050908@v.loewis.de> <4AEEB457.6010808@molden.no> <4AEEB7A7.3020102@v.loewis.de> <4AEEE792.9080108@molden.no> Message-ID: Sturla Molden molden.no> writes: > > And the timeout "milliseconds" would now be computed from querying the > performance > counter, instead of unreliably by the Windows NT kernel. Could you actually test your proposal under Windows and report what kind of concrete benefits it brings? Thank you Antoine. From sturla at molden.no Mon Nov 2 15:22:02 2009 From: sturla at molden.no (Sturla Molden) Date: Mon, 02 Nov 2009 15:22:02 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: <4AEEE792.9080108@molden.no> References: <1256502140.5621.163.camel@localhost> <4AEEA1A0.8050908@v.loewis.de> <4AEEB457.6010808@molden.no> <4AEEB7A7.3020102@v.loewis.de> <4AEEE792.9080108@molden.no> Message-ID: <4AEEEB0A.9070604@molden.no> Sturla Molden skrev: > > I would turn on multimedia timer (it is not on by default), and > replace this > call with a loop, approximately like this: > > for (;;) { > r = WaitForMultipleObjects(2, objects, TRUE, 0); > /* blah blah blah */ QueryPerformanceCounter(&cnt); if (cnt > > timeout) break; > Sleep(0); > } And just so you don't ask: There should not just be a Sleep(0) in the loop, but a sleep that gets shorter and shorter until a lower threshold is reached, where it skips to Sleep(0). That way we avoid hammering om WaitForMultipleObjects and QueryPerformanceCounter more than we need. And for all that to work better than just giving a timeout to WaitForMultipleObjects, we need the multimedia timer turned on. Sturla From sturla at molden.no Mon Nov 2 15:40:43 2009 From: sturla at molden.no (Sturla Molden) Date: Mon, 02 Nov 2009 15:40:43 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: <4AEEEB0A.9070604@molden.no> References: <1256502140.5621.163.camel@localhost> <4AEEA1A0.8050908@v.loewis.de> <4AEEB457.6010808@molden.no> <4AEEB7A7.3020102@v.loewis.de> <4AEEE792.9080108@molden.no> <4AEEEB0A.9070604@molden.no> Message-ID: <4AEEEF6B.80100@molden.no> Sturla Molden skrev: > > And just so you don't ask: There should not just be a Sleep(0) in the > loop, but a sleep that gets shorter and shorter until a lower > threshold is reached, where it skips to Sleep(0). That way we avoid > hammering om WaitForMultipleObjects and QueryPerformanceCounter more > than we need. And for all that to work better than just giving a > timeout to WaitForMultipleObjects, we need the multimedia timer turned > on. The important thing about multimedia timer is that the granularity of Sleep() and WaitForMultipleObjects() by default is "10 ms or at most 20 ms". But if we call timeBeginPeriod(1); the MM timer is on and granularity becomes 1 ms or at most 2 ms. But we can get even more precise than that by hammering on Sleep(0) for timeouts less than 2 ms. We can get typical granularity in the order of 10 ?s, with the occational 100 ?s now and then. I know this because I was using Windows 2000 to generate TTL signals on the LPT port some years ago, and watched them on the oscilloscope. ~ 15 ms granularity is Windows default. But that is brain dead. By the way Antoine, if you think granularity of 1-2 ms is sufficient, i.e. no need for ?s precision, then just calling timeBeginPeriod(1) will be sufficient. Sturla From solipsis at pitrou.net Mon Nov 2 15:53:30 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 2 Nov 2009 14:53:30 +0000 (UTC) Subject: [Python-Dev] Reworking the GIL References: <1256502140.5621.163.camel@localhost> <4AEEA1A0.8050908@v.loewis.de> <4AEEB457.6010808@molden.no> <4AEEB7A7.3020102@v.loewis.de> <4AEEE792.9080108@molden.no> <4AEEEB0A.9070604@molden.no> <4AEEEF6B.80100@molden.no> Message-ID: Sturla Molden molden.no> writes: > > By the way Antoine, if you think granularity of 1-2 ms is sufficient, It certainly is. But once again, I'm no Windows developer and I don't have a native Windost host to test on; therefore someone else (you?) has to try. Also, the MSDN doc (*) says timeBeginPeriod() can have a detrimental effect on system performance; I don't know how much of it is true. (*) http://msdn.microsoft.com/en-us/library/dd757624(VS.85).aspx From sturla at molden.no Mon Nov 2 16:07:06 2009 From: sturla at molden.no (Sturla Molden) Date: Mon, 02 Nov 2009 16:07:06 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: References: <1256502140.5621.163.camel@localhost> <4AEEA1A0.8050908@v.loewis.de> <4AEEB457.6010808@molden.no> <4AEEB7A7.3020102@v.loewis.de> <4AEEE792.9080108@molden.no> <4AEEEB0A.9070604@molden.no> <4AEEEF6B.80100@molden.no> Message-ID: <4AEEF59A.10504@molden.no> Antoine Pitrou skrev: > It certainly is. > But once again, I'm no Windows developer and I don't have a native Windost host > to test on; therefore someone else (you?) has to try. > I'd love to try, but I don't have VC++ to build Python, I use GCC on Windows. Anyway, the first thing to try then is to call timeBeginPeriod(1); once on startup, and leave the rest of the code as it is. If 2-4 ms is sufficient we can use timeBeginPeriod(2), etc. Microsoft is claiming Windows performs better with high granularity, which is why it is 10 ms by default. Sturla From john.arbash.meinel at gmail.com Mon Nov 2 16:22:52 2009 From: john.arbash.meinel at gmail.com (John Arbash Meinel) Date: Mon, 02 Nov 2009 09:22:52 -0600 Subject: [Python-Dev] Reworking the GIL In-Reply-To: <4AEEF59A.10504@molden.no> References: <1256502140.5621.163.camel@localhost> <4AEEA1A0.8050908@v.loewis.de> <4AEEB457.6010808@molden.no> <4AEEB7A7.3020102@v.loewis.de> <4AEEE792.9080108@molden.no> <4AEEEB0A.9070604@molden.no> <4AEEEF6B.80100@molden.no> <4AEEF59A.10504@molden.no> Message-ID: <4AEEF94C.4090701@gmail.com> Sturla Molden wrote: > Antoine Pitrou skrev: >> It certainly is. >> But once again, I'm no Windows developer and I don't have a native >> Windost host >> to test on; therefore someone else (you?) has to try. >> > I'd love to try, but I don't have VC++ to build Python, I use GCC on > Windows. > > Anyway, the first thing to try then is to call > > timeBeginPeriod(1); > > once on startup, and leave the rest of the code as it is. If 2-4 ms is > sufficient we can use timeBeginPeriod(2), etc. Microsoft is claiming > Windows performs better with high granularity, which is why it is 10 ms > by default. > > > Sturla That page claims: Windows uses the lowest value (that is, highest resolution) requested by any process. I would posit that the chance of having some random process on your machine request a high-speed timer is high enough that the overhead for Python doing the same is probably low. John =:-> From solipsis at pitrou.net Mon Nov 2 16:28:12 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 2 Nov 2009 15:28:12 +0000 (UTC) Subject: [Python-Dev] Reworking the GIL References: <1256502140.5621.163.camel@localhost> <4AEEA1A0.8050908@v.loewis.de> <4AEEB457.6010808@molden.no> <4AEEB7A7.3020102@v.loewis.de> <4AEEE792.9080108@molden.no> <4AEEEB0A.9070604@molden.no> <4AEEEF6B.80100@molden.no> <4AEEF59A.10504@molden.no> Message-ID: Sturla Molden molden.no> writes: > > I'd love to try, but I don't have VC++ to build Python, I use GCC on > Windows. You can use Visual Studio Express, which is free (gratis). From martin at v.loewis.de Mon Nov 2 18:27:40 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 02 Nov 2009 18:27:40 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: <4AEEE792.9080108@molden.no> References: <1256502140.5621.163.camel@localhost> <4AEEA1A0.8050908@v.loewis.de> <4AEEB457.6010808@molden.no> <4AEEB7A7.3020102@v.loewis.de> <4AEEE792.9080108@molden.no> Message-ID: <4AEF168C.1020102@v.loewis.de> > I did, and it does nothing of what I suggested. I am sure I can make the > Windows GIL in ceval_gil.h and the mutex in thread_nt.h at lot more precise and > efficient. Hmm. I'm skeptical that your code makes it more accurate, and I completely fail to see that it makes it more efficient (by what measurement of efficiency?) Also, why would making it more accurate make it better? IIUC, accuracy is completely irrelevant here, though efficiency (low overhead) does matter. > This is the kind of code I was talking about, from ceval_gil.h: > > r = WaitForMultipleObjects(2, objects, TRUE, milliseconds); > > I would turn on multimedia timer (it is not on by default), and replace > this > call with a loop, approximately like this: > > for (;;) { > r = WaitForMultipleObjects(2, objects, TRUE, 0); > /* blah blah blah */ QueryPerformanceCounter(&cnt); if (cnt > > timeout) break; > Sleep(0); > } > > And the timeout "milliseconds" would now be computed from querying the > performance counter, instead of unreliably by the Windows NT kernel. Hmm. This creates a busy wait loop; if you add larger sleep values, then it loses accuracy. Why not just call timeBeginPeriod, and then rely on the higher clock rate for WaitForMultipleObjects? Regards, Martin From daniel at stutzbachenterprises.com Mon Nov 2 18:40:29 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 2 Nov 2009 11:40:29 -0600 Subject: [Python-Dev] Reworking the GIL In-Reply-To: <4AEF168C.1020102@v.loewis.de> References: <1256502140.5621.163.camel@localhost> <4AEEA1A0.8050908@v.loewis.de> <4AEEB457.6010808@molden.no> <4AEEB7A7.3020102@v.loewis.de> <4AEEE792.9080108@molden.no> <4AEF168C.1020102@v.loewis.de> Message-ID: On Mon, Nov 2, 2009 at 11:27 AM, "Martin v. L?wis" wrote: > Hmm. This creates a busy wait loop; if you add larger sleep values, > then it loses accuracy. > I thought that at first, too, but then I checked the documentation for Sleep(0): "A value of zero causes the thread to relinquish the remainder of its time slice to any other thread of equal priority that is ready to run." (this is not to say that I think the solution with Sleep is worthwhile, though...) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From jyasskin at gmail.com Mon Nov 2 18:56:11 2009 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Mon, 2 Nov 2009 09:56:11 -0800 Subject: [Python-Dev] Reworking the GIL In-Reply-To: References: <1256502140.5621.163.camel@localhost> <4AEEA1A0.8050908@v.loewis.de> <4AEEC59B.1040902@v.loewis.de> Message-ID: <5d44f72f0911020956t4e21bf7bp8fa3f096cbb894fd@mail.gmail.com> On Mon, Nov 2, 2009 at 4:15 AM, Antoine Pitrou wrote: > Martin v. L?wis v.loewis.de> writes: >> > [gil_drop_request] >> Even if it is read from memory, I still wonder what might happen on >> systems that require explicit memory barriers to synchronize across >> CPUs. What if CPU 1 keeps reading a 0 value out of its cache, even >> though CPU 1 has written an 1 value a long time ago? >> >> IIUC, any (most?) pthread calls would cause synchronization in that >> case, which is why applications that also use locks for reading: >> >> http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap04.html#tag_04_10 >> >> Of course, on x86, you won't see any issues, because it's cache-coherent >> anyway. > > I think there are two things here: > - all machines Python runs on should AFAIK be cache-coherent: CPUs synchronize > their views of memory in a rather timely fashion. > - memory ordering: writes made by a CPU can be seen in a different order by > another CPU (i.e. CPU 1 writes A before B, but CPU 2 sees B written before A). I > don't see how this can apply to gil_drop_request, since it's a single variable > (and, moreover, only a single bit of it is significant). > > (there's an explanation of memory ordering issues here: > http://www.linuxjournal.com/article/8211) > > As a side note, I remember Jeffrey Yasskin trying to specify an ordering model > for Python code > (see http://code.google.com/p/unladen-swallow/wiki/MemoryModel). Note that that memory model was only for Python code; the C code implementing it is subject to the C memory model, which is weaker (and not even fully defined until the next C standard comes out). To be really safe, we ought to have a couple primitives implementing "atomic" rather than just "volatile" instructions, but until then a signal that's just saying "do something" rather than "here's some data you should look at" should be ok as a volatile int. I'd like to look at the patch in detail, but I can't guarantee that I'll get to it in a timely manner. I'd say check it in and let more threading experts look at it in the tree. We've got some time before a release for people to fix problems and make further improvements. +1 to Martin's request for detailed documentation though. :) From guido at python.org Mon Nov 2 19:01:54 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 2 Nov 2009 10:01:54 -0800 Subject: [Python-Dev] nonlocal keyword in 2.x? In-Reply-To: <4AEE9862.8010201@voidspace.org.uk> References: <19176.18618.662587.613771@montanaro.dyndns.org> <319e029f0910290115y49be844co78da6636452445f@mail.gmail.com> <4AE95F18.4040807@gmail.com> <319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com> <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1> <4AEE9862.8010201@voidspace.org.uk> Message-ID: I'm -0 on backporting nonlocal to 2.7. I could be +0 if we added "from __future__ import nonlocal_keyword" (or some such phrasing) to enable it. -- --Guido van Rossum (python.org/~guido) From python at rcn.com Mon Nov 2 19:06:27 2009 From: python at rcn.com (Raymond Hettinger) Date: Mon, 2 Nov 2009 10:06:27 -0800 Subject: [Python-Dev] nonlocal keyword in 2.x? References: <19176.18618.662587.613771@montanaro.dyndns.org> <319e029f0910290115y49be844co78da6636452445f@mail.gmail.com> <4AE95F18.4040807@gmail.com><319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com> <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1> <4AEE9862.8010201@voidspace.org.uk> Message-ID: <5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1> [Guido van Rossum] > I'm -0 on backporting nonlocal to 2.7. I could be +0 if we added "from > __future__ import nonlocal_keyword" (or some such phrasing) to enable > it. With the "from __future__" option, what keeps you from being a full +1 on nonlocal? Is there something that makes it a better solution for 3.x than 2.x? Just curious about the pros and cons from your point of view. Raymond From guido at python.org Mon Nov 2 19:09:40 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 2 Nov 2009 10:09:40 -0800 Subject: [Python-Dev] nonlocal keyword in 2.x? In-Reply-To: <5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1> References: <319e029f0910290115y49be844co78da6636452445f@mail.gmail.com> <4AE95F18.4040807@gmail.com> <319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com> <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1> <4AEE9862.8010201@voidspace.org.uk> <5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1> Message-ID: On Mon, Nov 2, 2009 at 10:06 AM, Raymond Hettinger wrote: > > [Guido van Rossum] >> >> I'm -0 on backporting nonlocal to 2.7. I could be +0 if we added "from >> __future__ import nonlocal_keyword" (or some such phrasing) to enable >> it. > > With the "from __future__" option, what keeps you from being > a full +1 on nonlocal? ? Is there something that makes it a better > solution for 3.x than 2.x? ?Just curious about the pros and cons > from your point of view. I think the number of projects that can afford to drop support for 2.6 is pretty limited, so I think the utility of the feature is thereby also limited. -- --Guido van Rossum (python.org/~guido) From sturla at molden.no Mon Nov 2 19:10:12 2009 From: sturla at molden.no (Sturla Molden) Date: Mon, 02 Nov 2009 19:10:12 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: <4AEF168C.1020102@v.loewis.de> References: <1256502140.5621.163.camel@localhost> <4AEEA1A0.8050908@v.loewis.de> <4AEEB457.6010808@molden.no> <4AEEB7A7.3020102@v.loewis.de> <4AEEE792.9080108@molden.no> <4AEF168C.1020102@v.loewis.de> Message-ID: <4AEF2084.2040905@molden.no> Martin v. L?wis skrev: >> I did, and it does nothing of what I suggested. I am sure I can make the >> Windows GIL in ceval_gil.h and the mutex in thread_nt.h at lot more precise and >> efficient. >> > > Hmm. I'm skeptical that your code makes it more accurate, and I > completely fail to see that it makes it more efficient (by what > measurement of efficiency?) > > Also, why would making it more accurate make it better? IIUC, > accuracy is completely irrelevant here, though efficiency > (low overhead) does matter. > > >> This is the kind of code I was talking about, from ceval_gil.h: >> >> r = WaitForMultipleObjects(2, objects, TRUE, milliseconds); >> >> I would turn on multimedia timer (it is not on by default), and replace >> this >> call with a loop, approximately like this: >> >> for (;;) { >> r = WaitForMultipleObjects(2, objects, TRUE, 0); >> /* blah blah blah */ QueryPerformanceCounter(&cnt); if (cnt > >> timeout) break; >> Sleep(0); >> } >> >> And the timeout "milliseconds" would now be computed from querying the >> performance counter, instead of unreliably by the Windows NT kernel. >> > > Hmm. This creates a busy wait loop; if you add larger sleep values, > then it loses accuracy. > > Actually an usleep lookes like this, and the call to the wait function must go into the for loop. But no, it's not a busy sleep. static int inited = 0; static __int64 hz; static double dhz; const double sleep_granularity = 2.0E10-3; void usleep( long us ) { __int64 cnt, end; double diff; if (!inited) { timeBeginPeriod(1); QueryPerformanceFrequency((LARGE_INTEGER*)&hz); dhz = (double)hz; inited = 1; } QueryPerformanceCounter((LARGE_INTEGER*)&cnt); end = cnt + (__int64)(1.0E10-6 * (double)(us) * dhz); for (;;) { QueryPerformanceCounter((LARGE_INTEGER*)&cnt); if (cnt >= end) break; diff = (double)(end - cnt)/dhz; if (diff > sleep_granularity) Sleep((DWORD)(diff - sleep_granularity)); else Sleep(0); } } > Why not just call timeBeginPeriod, and then rely on the higher clock > rate for WaitForMultipleObjects? > > That is what I suggested when Antoine said 1-2 ms was enough. Sturla From martin at v.loewis.de Mon Nov 2 19:39:53 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 02 Nov 2009 19:39:53 +0100 Subject: [Python-Dev] language summit topic: issue tracker In-Reply-To: <4AE17C12.1080007@gmail.com> References: <4AE17C12.1080007@gmail.com> Message-ID: <4AEF2779.5040402@v.loewis.de> > +lots on adding a module field (independent of automatically adding > maintainers to the nosy list, it would assist in "I just did a major > cleanup of module X, are there any old bugs I can kill off"). Link (1:1) or Multilink (1:n)? What is the impact on the Component field? Would you be willing to manage the field (in the sense of managing the set of values)? If so, please send me a list of values. Regards, Martin From martin at v.loewis.de Mon Nov 2 19:42:22 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 02 Nov 2009 19:42:22 +0100 Subject: [Python-Dev] language summit topic: issue tracker In-Reply-To: <19169.43241.70138.170239@montanaro.dyndns.org> References: <19169.43241.70138.170239@montanaro.dyndns.org> Message-ID: <4AEF280E.6040503@v.loewis.de> skip at pobox.com wrote: > Brett> Another summit, another potential time to see if people want to > Brett> change anything about the issue tracker. > > I have no idea how hard this would be to implement and won't be at the > language summit to formally present the idea, but it seems to me that some > integration between the issue tracker and Rietveld would be beneficial. See http://psf.upfronthosting.co.za/roundup/meta/issue247 Regards, Martin From rdmurray at bitdance.com Mon Nov 2 20:18:48 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 2 Nov 2009 14:18:48 -0500 (EST) Subject: [Python-Dev] nonlocal keyword in 2.x? In-Reply-To: References: <319e029f0910290115y49be844co78da6636452445f@mail.gmail.com> <4AE95F18.4040807@gmail.com> <319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com> <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1> <4AEE9862.8010201@voidspace.org.uk> <5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1> Message-ID: On Mon, 2 Nov 2009 at 10:09, Guido van Rossum wrote: > On Mon, Nov 2, 2009 at 10:06 AM, Raymond Hettinger wrote: >> >> [Guido van Rossum] >>> >>> I'm -0 on backporting nonlocal to 2.7. I could be +0 if we added "from >>> __future__ import nonlocal_keyword" (or some such phrasing) to enable >>> it. >> >> With the "from __future__" option, what keeps you from being >> a full +1 on nonlocal? ? Is there something that makes it a better >> solution for 3.x than 2.x? ?Just curious about the pros and cons >> from your point of view. > > I think the number of projects that can afford to drop support for 2.6 > is pretty limited, so I think the utility of the feature is thereby > also limited. I don't currently have an opinion on this backport proposal, but in regard to this argument: if we do not do any 2.x releases after 2.7, then over time the number of packages that can afford to drop 2.6 support will grow, yet many will need to retain 2.7 support for much longer. --David From benjamin at python.org Mon Nov 2 21:00:21 2009 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 2 Nov 2009 14:00:21 -0600 Subject: [Python-Dev] 2.7/3.2 release schedule Message-ID: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> I've updated PEP 373 with my proposed release schedule: - 2.7/3.2 alpha 1 2009-12-05 - 2.7/3.2 alpha 2 2010-01-09 - 2.7/3.2 alpha 3 2010-02-06 - 2.7/3.2 alpha 4 2010-03-06 - 2.7/3.2 beta 1 2010-04-03 - 2.7/3.2 beta 2 2010-05-01 - 2.7/3.2 rc1 2010-05-29 - 2.7/3.2 rc2 2010-06-12 - 2.7/3.2 final 2010-06-26 -- Regards, Benjamin From martin at v.loewis.de Mon Nov 2 22:17:07 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 02 Nov 2009 22:17:07 +0100 Subject: [Python-Dev] nonlocal keyword in 2.x? In-Reply-To: References: <319e029f0910290115y49be844co78da6636452445f@mail.gmail.com> <4AE95F18.4040807@gmail.com> <319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com> <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1> <4AEE9862.8010201@voidspace.org.uk> <5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1> Message-ID: <4AEF4C53.7090902@v.loewis.de> > I don't currently have an opinion on this backport proposal, but in > regard to this argument: if we do not do any 2.x releases after 2.7, > then over time the number of packages that can afford to drop 2.6 support > will grow, yet many will need to retain 2.7 support for much longer. I don't think the argument applies to 2.7 as much as it applied to earlier releases: 2.7 will have a life time of 18 months perhaps (I think we still need to decide formally against 2.8, and also decide when to make the last 2.7 bug fix release). There is some likelihood that by the time people can agree to drop 2.6 support, 2.7 will be out of bug fix maintenance already (*). I'm fairly skeptical that people will be interested in adding new code to specifically clean up their 2.x codebase. Regards, Martin (*) just as it happened with any other release: people are now dropping support for 2.3, when this got out of security fixes a year ago. By that measure, people will drop 2.6 support in 2015. I think there are *really* good chances that many packages will drop 2.6 support along with dropping 2.x altogether. From rdmurray at bitdance.com Mon Nov 2 22:27:10 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 2 Nov 2009 16:27:10 -0500 (EST) Subject: [Python-Dev] nonlocal keyword in 2.x? In-Reply-To: <4AEF4C53.7090902@v.loewis.de> References: <319e029f0910290115y49be844co78da6636452445f@mail.gmail.com> <4AE95F18.4040807@gmail.com> <319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com> <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1> <4AEE9862.8010201@voidspace.org.uk> <5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1> <4AEF4C53.7090902@v.loewis.de> Message-ID: On Mon, 2 Nov 2009 at 22:17, "Martin v. L?wis" wrote: >> I don't currently have an opinion on this backport proposal, but in >> regard to this argument: if we do not do any 2.x releases after 2.7, >> then over time the number of packages that can afford to drop 2.6 support >> will grow, yet many will need to retain 2.7 support for much longer. > > I don't think the argument applies to 2.7 as much as it applied to > earlier releases: 2.7 will have a life time of 18 months perhaps (I > think we still need to decide formally against 2.8, and also decide > when to make the last 2.7 bug fix release). There is some likelihood I was under the impression that if 2.7 was the last release that it would be maintained (ie: bugfixed) until we decided 3.x uptake was "sufficient", and that that might be considerably longer than 18 months. If that is not the case, then what you say is true. --David From guido at python.org Mon Nov 2 22:33:10 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 2 Nov 2009 13:33:10 -0800 Subject: [Python-Dev] nonlocal keyword in 2.x? In-Reply-To: References: <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1> <4AEE9862.8010201@voidspace.org.uk> <5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1> <4AEF4C53.7090902@v.loewis.de> Message-ID: On Mon, Nov 2, 2009 at 1:27 PM, R. David Murray wrote: > On Mon, 2 Nov 2009 at 22:17, "Martin v. L?wis" wrote: >>> >>> I don't currently have an opinion on this backport proposal, but in >>> regard to this argument: ?if we do not do any 2.x releases after 2.7, >>> then over time the number of packages that can afford to drop 2.6 support >>> will grow, yet many will need to retain 2.7 support for much longer. >> >> I don't think the argument applies to 2.7 as much as it applied to >> earlier releases: 2.7 will have a life time of 18 months perhaps (I >> think we still need to decide formally against 2.8, and also decide >> when to make the last 2.7 bug fix release). There is some likelihood > > I was under the impression that if 2.7 was the last release that it > would be maintained (ie: bugfixed) until we decided 3.x uptake was > "sufficient", and that that might be considerably longer than 18 months. > If that is not the case, then what you say is true. Is it even wort doing a 2.7 release? Isn't the effort better spent on 3.2 alone? (Note, these aren't rhetorical questions. It's well possible that there are good reasons for pushing along with 2.7. Maybe considering those reasons will also help answering questions about whether to backport things like nonlocal.) -- --Guido van Rossum (python.org/~guido) From martin at v.loewis.de Mon Nov 2 22:33:38 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 02 Nov 2009 22:33:38 +0100 Subject: [Python-Dev] nonlocal keyword in 2.x? In-Reply-To: References: <319e029f0910290115y49be844co78da6636452445f@mail.gmail.com> <4AE95F18.4040807@gmail.com> <319e029f0911011326i45bf52ew35b9d7e438df6570@mail.gmail.com> <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1> <4AEE9862.8010201@voidspace.org.uk> <5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1> <4AEF4C53.7090902@v.loewis.de> Message-ID: <4AEF5032.6020403@v.loewis.de> R. David Murray wrote: > On Mon, 2 Nov 2009 at 22:17, "Martin v. L?wis" wrote: >>> I don't currently have an opinion on this backport proposal, but in >>> regard to this argument: if we do not do any 2.x releases after 2.7, >>> then over time the number of packages that can afford to drop 2.6 >>> support >>> will grow, yet many will need to retain 2.7 support for much longer. >> >> I don't think the argument applies to 2.7 as much as it applied to >> earlier releases: 2.7 will have a life time of 18 months perhaps (I >> think we still need to decide formally against 2.8, and also decide >> when to make the last 2.7 bug fix release). There is some likelihood > > I was under the impression that if 2.7 was the last release that it > would be maintained (ie: bugfixed) until we decided 3.x uptake was > "sufficient", and that that might be considerably longer than 18 months. > If that is not the case, then what you say is true. I think that's as-yet undecided. My understanding was that we would decide, at some point, whether to create a 2.8 release or not, and if not, that 2.7 would be the final release. To me, this always implied that there wouldn't be any bug fix releases after 18 months, and no security releases after five years. If we would decide to continue doing 2.x releases, the we definitely wouldn't go beyond 2.9 (because there can't be a 2.10 release, numerically). At a rate of a release every 18 month, this would mean we would make bug fix releases for 2.x for no more than 5 years (from now). But I would really hope that we stop before 2.9. Regards, Martin From solipsis at pitrou.net Mon Nov 2 22:41:45 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 2 Nov 2009 21:41:45 +0000 (UTC) Subject: [Python-Dev] nonlocal keyword in 2.x? References: <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1> <4AEE9862.8010201@voidspace.org.uk> <5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1> <4AEF4C53.7090902@v.loewis.de> Message-ID: Guido van Rossum python.org> writes: > > Is it even wort doing a 2.7 release? Isn't the effort better spent on > 3.2 alone? (Note, these aren't rhetorical questions. It's well > possible that there are good reasons for pushing along with 2.7. Maybe > considering those reasons will also help answering questions about > whether to backport things like nonlocal.) 2.7 has an up-to-date backport of the C IO lib (as well as the memoryview object), which means it is better for people wanting to ease transition to 3.x. But of course, as Martin said, few people will want to support 2.7 only and not 2.6. From martin at v.loewis.de Mon Nov 2 22:42:40 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 02 Nov 2009 22:42:40 +0100 Subject: [Python-Dev] nonlocal keyword in 2.x? In-Reply-To: References: <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1> <4AEE9862.8010201@voidspace.org.uk> <5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1> <4AEF4C53.7090902@v.loewis.de> Message-ID: <4AEF5250.9090905@v.loewis.de> > Is it even wort doing a 2.7 release? Isn't the effort better spent on > 3.2 alone? (Note, these aren't rhetorical questions. It's well > possible that there are good reasons for pushing along with 2.7. Maybe > considering those reasons will also help answering questions about > whether to backport things like nonlocal.) If no 2.7 is produced, people will ask for the life time of 2.6. I think users will appreciate not being forced to 3.x as long as possible, which essentially means as long as we create 2.x releases. It may be that users could also "live" with 2.6 bug fix releases only - in this case, we would have to decide for how long these will have to be created. There is also the issue of committers who already added stuff to the trunk after the 2.6 release - some may be unhappy to learn that their stuff is not going to be released (at least not on that branch). In the end, the EOL for 2.x will need to be decided by BDFL pronouncement. Users would continue to ask for 2.x releases until 2020 and beyond if all they have to do is ask. Some committers would likely also want to continue creating 2.x releases for a few more years (not sure whether that's because they see themselves also as users, or because they sympathize with the users, or for other reasons). In any case, a decision not to release 2.7 should be made before Benjamin produces the first alpha release. Regards, Martin From olemis at gmail.com Mon Nov 2 22:43:07 2009 From: olemis at gmail.com (Olemis Lang) Date: Mon, 2 Nov 2009 16:43:07 -0500 Subject: [Python-Dev] A new way to configure logging In-Reply-To: <24ea26600910070926g6b742492xd6be4fa07a932a0c@mail.gmail.com> References: <24ea26600910070819r364f6715l9164a754bc20b8f4@mail.gmail.com> <24ea26600910070926g6b742492xd6be4fa07a932a0c@mail.gmail.com> Message-ID: <24ea26600911021343t4f26a319hbcc8093aeb016937@mail.gmail.com> On Wed, Oct 7, 2009 at 11:26 AM, Olemis Lang wrote: > On Wed, Oct 7, 2009 at 10:49 AM, Vinay Sajip wrote: >> Olemis Lang gmail.com> writes: >> >>> This kind of problems is similar to the one mentioned in another >>> thread about modifying config options after executing commands. In >>> that case I mentioned that the same dict-like interface also holds for >>> WinReg and so on ... >>> >>> So thinking big (yes ! I have a big head ! ) I think that the best >>> approach in this case is to build an adaptation (simple) layer on top >>> of ConfigParser, JSON, WinReg, PyCode, YAML, ... and build specific >>> extensions for these formats . Perhaps the proper interfaces are >>> already there (e.g. `dict`, `shelve` ) and I'm just blind and looking >>> somewhere else ;o) >> >> Sorry, you've lost me :-) >> > > Never mind . I was just trying to say that using `dict`, an adapter > could be implemented (if not already there ;o) for multiple formats > like the ones I mentioned above and the solution would cover many > config formats ... and also I was saying that I was not sure about > whether this is correct , I mean for *ANY* config formats, Intention = answer myself + sharing with you => for a better Pyland Maybe we could learn something from Augeas [1]_ and use their experience to include some of those features to `stdlib` . It seems to be an ambitious attempt to put all the pieces in config-land together; and AFAICS they handle (a lof of) config formats and have an API for that, and ... CMIIW anyway .. [1] Augeas (http://augeas.net/) -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: From greg.ewing at canterbury.ac.nz Mon Nov 2 23:19:38 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 03 Nov 2009 11:19:38 +1300 Subject: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it In-Reply-To: <20091102052900.GA25160@cskk.homeip.net> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <20091102052900.GA25160@cskk.homeip.net> Message-ID: <4AEF5AFA.5060806@canterbury.ac.nz> Cameron Simpson wrote: > Personally, I'm for the iteration spec in a lot of ways. > > Firstly, a .get()/.pick() that always returns the same element feels > horrible. Is there anyone here who _likes_ it? It doesn't sound very useful to me. However, an iterating version of it doesn't sound any more useful. What would it gain you? Why not just iterate over the set? Or make a copy and repeatedly pop() it? I completely fail to see a use case for this. -- Greg From ssteinerx at gmail.com Tue Nov 3 00:24:54 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Mon, 2 Nov 2009 18:24:54 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? Message-ID: Not that anyone has asked yet, but here's my opinion on two issues that have been raised on the python-dev mailing list lately: +1 on 2.7 release with as much 3.0 "easy-port goo" as is practicable without delaying the product beyond the tentative schedule. Sooner would, of course, be better but I'm sure PEP 373 was produced after due consideration of all pertinent factors. +1 on 2.7 being the last of the 2.x series. Enough already! I, personally, haven't even written my first line of 3.x code, nor have I had any good reason to. If I saw the actual end of the line at 2.7, I would actually start looking for 3.x versions of my favorite tools and would be much more inclined to help push them along ASAP. Right now, so much that I use on a daily basis doesn't even have a 3.x roadmap, much less any sort of working implementation, that I don't see switching to 3.x ever unless the 2.x line ends, and soon! Just my one vote. S From foom at fuhm.net Tue Nov 3 01:26:29 2009 From: foom at fuhm.net (James Y Knight) Date: Mon, 2 Nov 2009 19:26:29 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: Message-ID: On Nov 2, 2009, at 6:24 PM, ssteinerX at gmail.com wrote: > +1 on 2.7 being the last of the 2.x series. Enough already! -1. (not that it matters) > I, personally, haven't even written my first line of 3.x code, nor > have I had any good reason to. Me neither. > If I saw the actual end of the line at 2.7, I would actually start > looking for 3.x versions of my favorite tools and would be much more > inclined to help push them along ASAP. I'd probably keep using 2.7 to be able to keep using those tools, instead. > Right now, so much that I use on a daily basis doesn't even have a > 3.x roadmap, much less any sort of working implementation, that I > don't see switching to 3.x ever unless the 2.x line ends, and soon! I don't see switching to 3.x anytime soon either. But what's the rush? 2.x seems to be a fine edition of Python, why not let it keep going to 2.8 and beyond? Then you wouldn't have to switch to 3.x at all, and that'd save you a ton of work. (and save all the people you will have to convince to make a 3.x roadmap and do the port a ton of work too!) It really sounds like you're saying that switching to 3.x isn't worth the cost to you, but you want to force people (including yourself) to do so anyways, because ...? James From skip at pobox.com Tue Nov 3 02:56:18 2009 From: skip at pobox.com (skip at pobox.com) Date: Mon, 2 Nov 2009 19:56:18 -0600 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: Message-ID: <19183.36290.623351.810217@montanaro.dyndns.org> James> 2.x seems to be a fine edition of Python, why not let it keep James> going to 2.8 and beyond? Resources would be my guess. In much the same way that people move on to newer releases of Windows, Mac OSX or Linux leaving an ever-dwindling group available to support old versions, the same will be true of Python. Over time more and more of the core developers (and end users) will switch to 3.x leaving fewer and fewer people with the time or inclination to support 2.x. I think there are probably enough functional differences between 2.6 and 2.7 that releasing 2.7 makes sense. The discussion at this point should probably be what to do when 2.7 is out the door. It makes sense to me to merge the py3k branch to trunk coincident with the 2.7/3.2 releases and creation of the release27-maint and release32-maint branches. 3.3 and future versions would then be released from trunk and there would be no further 2.x releases. Skip From skip at pobox.com Tue Nov 3 03:26:02 2009 From: skip at pobox.com (skip at pobox.com) Date: Mon, 2 Nov 2009 20:26:02 -0600 Subject: [Python-Dev] language summit topic: issue tracker In-Reply-To: <4AEF280E.6040503@v.loewis.de> References: <19169.43241.70138.170239@montanaro.dyndns.org> <4AEF280E.6040503@v.loewis.de> Message-ID: <19183.38074.390745.534425@montanaro.dyndns.org> >> ... it seems to me that some integration between the issue tracker >> and Rietveld would be beneficial. Martin> See Martin> http://psf.upfronthosting.co.za/roundup/meta/issue247 Cool. I still haven't used Rietveld for anything, though I am getting comfortable with Review Board and like the tool support for code reviews. Skip From ssteinerx at gmail.com Tue Nov 3 04:48:46 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Mon, 2 Nov 2009 22:48:46 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: Message-ID: On Nov 2, 2009, at 7:26 PM, James Y Knight wrote: > > It really sounds like you're saying that switching to 3.x isn't > worth the cost to you, but you want to force people (including > yourself) to do so anyways, because ...? Because that's the future of Python, where the developers who make real base language improvements are focusing their attention, and because the language would advance faster without this split attention. A better language, i.e. Python 3.x, will become better faster without dragging the 2.x series out any longer. S From brett at python.org Tue Nov 3 05:22:56 2009 From: brett at python.org (Brett Cannon) Date: Mon, 2 Nov 2009 20:22:56 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: Message-ID: On Mon, Nov 2, 2009 at 16:26, James Y Knight wrote: > > On Nov 2, 2009, at 6:24 PM, ssteinerX at gmail.com wrote: > >> ? ? ? ?+1 on 2.7 being the last of the 2.x series. ?Enough already! > > -1. (not that it matters) > >> I, personally, haven't even written my first line of 3.x code, nor have I >> had any good reason to. > > Me neither. > >> If I saw the actual end of the line at 2.7, I would actually start looking >> for 3.x versions of my favorite tools and would be much more inclined to >> help push them along ASAP. > > I'd probably keep using 2.7 to be able to keep using those tools, instead. > >> Right now, so much that I use on a daily basis doesn't even have a 3.x >> roadmap, much less any sort of working implementation, that I don't see >> switching to 3.x ever unless the 2.x line ends, and soon! > > > I don't see switching to 3.x anytime soon either. But what's the rush? > > 2.x seems to be a fine edition of Python, why not let it keep going to 2.8 > and beyond? Then you wouldn't have to switch to 3.x at all, and that'd save > you a ton of work. (and save all the people you will have to convince to > make a 3.x roadmap and do the port a ton of work too!) > > It really sounds like you're saying that switching to 3.x isn't worth the > cost to you, but you want to force people (including yourself) to do so > anyways, because ...? ... I think a decent number of us no longer want to maintain the 2.x series. Honestly, if we go past 2.7 I am simply going to stop backporting features and bug fixes. It's just too much work keeping so many branches fixed. -Brett From barry at python.org Tue Nov 3 05:28:27 2009 From: barry at python.org (Barry Warsaw) Date: Mon, 2 Nov 2009 23:28:27 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: Message-ID: On Nov 2, 2009, at 10:48 PM, ssteinerX at gmail.com wrote: > A better language, i.e. Python 3.x, will become better faster > without dragging the 2.x series out any longer. If Python 2.7 becomes the last of the 2.x series, then I personally favor back porting as many features from Python 3 as possible. I still think doing so will help people migrate to Python 3 by getting their Python 2 code base as close to Python 3 as possible without biting the ultimate bullet. E.g. for me "from __future__ import absolute_import, unicode_literals" in Python 2.6 has helped quite a bit. I also think Guido's call for feature freeze makes a lot more sense when 2.7 is the EOL. Let's give people migrating to Python 3 a nice big stable target to hit. Improving the stdlib also gives people a big carrot to move. I think it's also necessary to give third party library and application authors as much help as possible to provide Python 3 compatible software. Putting together Python tools involves so many dependencies in a fairly deep stack that even one unconverted library can cause everything above it to stall on Python 2. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From ssteinerx at gmail.com Tue Nov 3 05:51:06 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Mon, 2 Nov 2009 23:51:06 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: Message-ID: On Nov 2, 2009, at 11:28 PM, Barry Warsaw wrote: > On Nov 2, 2009, at 10:48 PM, ssteinerX at gmail.com wrote: > >> A better language, i.e. Python 3.x, will become better faster >> without dragging the 2.x series out any longer. > > If Python 2.7 becomes the last of the 2.x series, then I personally > favor back porting as many features from Python 3 as possible. I > still think doing so will help people migrate to Python 3 by getting > their Python 2 code base as close to Python 3 as possible without > biting the ultimate bullet. E.g. for me "from __future__ import > absolute_import, unicode_literals" in Python 2.6 has helped quite a > bit. I agree as long as: A> 2.7 comes out as soon as possible, even if it's missing helpful porting features. B> 2.7 will get ONLY new features that make it easier to port to 3.x, not every feature added to 3.x or all you've done is make "Python 2.7, the Python 3 Version." and core developer time will continue to be wasted on Python 2.7 instead of moving forward. > I also think Guido's call for feature freeze makes a lot more sense > when 2.7 is the EOL. Let's give people migrating to Python 3 a nice > big stable target to hit. Improving the stdlib also gives people a > big carrot to move. Agreed. And specifically NOT porting every shiny new toy from Python 3 back to 2.7 makes sure the carrots are only in the 3.x series. > I think it's also necessary to give third party library and > application authors as much help as possible to provide Python 3 > compatible software. Putting together Python tools involves so many > dependencies in a fairly deep stack that even one unconverted > library can cause everything above it to stall on Python 2. And that's one of the reasons my explorations into Python 3 have been limited to pretty much nothing. I don't have time to do a bunch of work only to find out that the tool I absolutely have to have to finish a project doesn't have a Python 3 version or has been crippled to make a Python 3 version. BeautifulSoup, which I use every day, is one such product. Since the crappy old SMGL parser's gone, BeautifulSoup uses the one that's left in Python 3 and it makes BeautifulSoup completely useless for my daily work. That's not to say I can't fix that one particular project, but customers get cranky when their project is taking longer than expected and "Oh, I'm having to convert a lot of things to use Python 3" doesn't seem to improve their mood much. Thanks, S From guido at python.org Tue Nov 3 06:06:18 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 2 Nov 2009 22:06:18 -0700 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: Message-ID: On Mon, Nov 2, 2009 at 9:51 PM, ssteinerX at gmail.com wrote: > BeautifulSoup, which I use every day, is one such product. ?Since the crappy > old SMGL parser's gone, BeautifulSoup uses the one that's left in Python 3 > and it makes BeautifulSoup completely useless for my daily work. This sounds an area where some help might be useful. Perhaps the quickest solution would simply be to copy the old crappy "sgml" based html parser into a new version of BeautifulSoup. Though I imagine what it really needs is a "quirks mode" parser that is compatible with the HTML dialect accepted by, say, IE6. Maybe a summer of code project? -- --Guido van Rossum (python.org/~guido) From foom at fuhm.net Tue Nov 3 06:29:15 2009 From: foom at fuhm.net (James Y Knight) Date: Tue, 3 Nov 2009 00:29:15 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: Message-ID: <24CE11F0-415E-4D0B-B241-19E1763A695D@fuhm.net> On Nov 3, 2009, at 12:06 AM, Guido van Rossum wrote: > Though I imagine what > it really needs is a "quirks mode" parser that is compatible with the > HTML dialect accepted by, say, IE6. Maybe a summer of code project? Already exists: html5lib. http://code.google.com/p/html5lib/ Or if you want a faster (yet I think less exact) HTML parser, libxml2's HTML parser, via lxml: http://codespeak.net/lxml/parsing.html#parsing-html James From martin at v.loewis.de Tue Nov 3 07:07:03 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Nov 2009 07:07:03 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: Message-ID: <4AEFC887.6020801@v.loewis.de> Barry Warsaw wrote: > On Nov 2, 2009, at 10:48 PM, ssteinerX at gmail.com wrote: > >> A better language, i.e. Python 3.x, will become better faster without >> dragging the 2.x series out any longer. > > If Python 2.7 becomes the last of the 2.x series, then I personally > favor back porting as many features from Python 3 as possible. And if *2.6* becomes the last of the 2.x series? Regards, Martin From sturla at molden.no Tue Nov 3 08:20:14 2009 From: sturla at molden.no (Sturla Molden) Date: Tue, 03 Nov 2009 08:20:14 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: Message-ID: <4AEFD9AE.7090104@molden.no> I'd just like to mention that the scientific community is highly dependent on NumPy. As long as NumPy is not ported to Py3k, migration is out of the question. Porting NumPy is not a trivial issue. It might take a complete rewrite of the whole C base using Cython. NumPy's ABI is not even PEP 3118 compliant. Changing the ABI for Py3k might break extension code written for NumPy using C. And scientists tend to write CPU-bound routines in languages like C and Fortran, not Python, so that is a major issue as well. If we port NumPy to Py3k, everyone using NumPy will have to port their C code to the new ABI. There are lot of people stuck with Python 2.x for this reason. It does not just affect individual scientists, but also large projects like IBM and CERN's blue brain and NASA's space telecope. So please, do not cancel 2.x support before we have ported NumPy, Matplotlib and most of their dependant extensions to Py3k. The community of scientists and engineers using Python is growing, but shutting down 2.x support might bring an end to that. Sturla Molden From cs at zip.com.au Tue Nov 3 08:30:46 2009 From: cs at zip.com.au (Cameron Simpson) Date: Tue, 3 Nov 2009 18:30:46 +1100 Subject: [Python-Dev] Python-Dev Retrieve an arbitrary element from a set?withoutremoving it In-Reply-To: References: Message-ID: <20091103073046.GA31310@cskk.homeip.net> On 02Nov2009 10:21, Antoine Pitrou wrote: | Cameron Simpson zip.com.au> writes: | > | > Personally, I'm for the iteration spec in a lot of ways. | > | > Firstly, a .get()/.pick() that always returns the same element feels | > horrible. Is there anyone here who _likes_ it? | | I do. Since the caller is asking for an arbitrary element, returning the same | element is legitimate. It's funny how people seem to have a problem with the | word "arbitrary" :) | | And I'm -1 on any implicit iteration attaching some state to the object. If you | want to iterate, there's already an obvious way to it. Good point. Colour me convinced by this. I suggest the doco is really clear about the word arbitrary:-) -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ I must really get a thinner pencil. I can't manage this one a bit. It writes all manner of things I don't intend. - rheney at csugrad.cs.vt.edu (Bolo Mk XXXIX) From nnorwitz at gmail.com Tue Nov 3 08:41:09 2009 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 3 Nov 2009 00:41:09 -0700 Subject: [Python-Dev] thanks to everyone cleaning up the tests In-Reply-To: <5c6f2a5d0911020118h4cfee72cl430ef45a1cbf9bc7@mail.gmail.com> References: <5c6f2a5d0911020118h4cfee72cl430ef45a1cbf9bc7@mail.gmail.com> Message-ID: On Mon, Nov 2, 2009 at 2:18 AM, Mark Dickinson wrote: > On Sat, Oct 31, 2009 at 9:47 PM, Brett Cannon wrote: >> Just wanted to publicly thank everyone who has been causing all the >> checkins to fix and stabilize the test suite (I think it's mostly >> Antoine and Mark, but I could be missing somebody; I'm under a >> deadline so only have marginal higher brain functionality). > > Not guilty, your honour. ?Blame Antoine and David. ?:) > > David's new buildslave seems to be making a difference, too. It's really been awesome to see all the work improving the test suite. This will make it easier to implement change without fear of breaking everything. These changes will reap benefits for a long time to come. Kudos for picking up this work and to Brett for calling you out. :-) Thanks, n From ubershmekel at gmail.com Tue Nov 3 09:20:52 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Tue, 3 Nov 2009 10:20:52 +0200 Subject: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it In-Reply-To: <4AEF5AFA.5060806@canterbury.ac.nz> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <20091102052900.GA25160@cskk.homeip.net> <4AEF5AFA.5060806@canterbury.ac.nz> Message-ID: <9d153b7c0911030020n336fd15bobeea4ea713cd8b23@mail.gmail.com> On Tue, Nov 3, 2009 at 12:19 AM, Greg Ewing wrote: > Cameron Simpson wrote: > > Personally, I'm for the iteration spec in a lot of ways. >> >> Firstly, a .get()/.pick() that always returns the same element feels >> horrible. Is there anyone here who _likes_ it? >> > > State might cause people to use this to iterate which would be just plain wrong. The 2 things I have a bad feeling about is: 1. random.choice could be a pythonic alternative to what some have mentioned here but it doesn't work on sets. The following code raises TypeError: 'set' object does not support indexing: import random random.choice(set([1,2,3,4,5,6])) this is kinda ironic: http://en.wikipedia.org/wiki/Axiom_of_choice 2. If I store objects in a set and modify their attributes I have no O(1) way of getting the objects back if I stumble upon an equivalent object. In cases like that I'd have to either iterate over the set or use a dict with key == value. I have a feeling the "get" or "peek" api could cater to this need. A use case for this could be an implementation of a cookie jar with a set of cookies where equivalence is defined by the name and domain. --yuv -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg at krypto.org Tue Nov 3 09:29:15 2009 From: greg at krypto.org (Gregory P. Smith) Date: Tue, 3 Nov 2009 00:29:15 -0800 Subject: [Python-Dev] language summit topic: issue tracker In-Reply-To: <4AE17C12.1080007@gmail.com> References: <4AE17C12.1080007@gmail.com> Message-ID: <52dc1c820911030029m72b10895xae947b60068ad0c4@mail.gmail.com> On Fri, Oct 23, 2009 at 1:49 AM, Nick Coghlan wrote: > Brett Cannon wrote: >> Another summit, another potential time to see if people want to change >> anything about the issue tracker. I would bring up: >> >> - Dropping Stage in favor of some keywords (e.g. 'needs unit test', >> 'needs docs') >> - Adding a freestyle text box to delineate which, if any, stdlib module >> is the cause of a bug and tie that into Misc/maintainers.rst; would >> potentially scale back the Component box > > +lots on adding a module field (independent of automatically adding > maintainers to the nosy list, it would assist in "I just did a major > cleanup of module X, are there any old bugs I can kill off"). yet another feature request or two to be lost to a mailing list thread along those lines: Maintainer or not i'd like to be able to setup triggers so that i'm automatically cc'ed on any bugs matching a simple search query i specify. The email sent out to people cc'ed when a new bug is opened and unassigned should have a simple links in it when cc'ed to someone who can be assigned bugs: 'Assign to me' that if followed will assign that bug to them without requiring a login. > > Of course, it will take a while for the field to be filled in on > existing issues, but even having it be possible would be very nice. > > Cheers, > Nick. > > -- > Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia > --------------------------------------------------------------- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/greg%40krypto.org > From mal at egenix.com Tue Nov 3 10:10:59 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 03 Nov 2009 10:10:59 +0100 Subject: [Python-Dev] nonlocal keyword in 2.x? In-Reply-To: References: <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1> <4AEE9862.8010201@voidspace.org.uk> <5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1> <4AEF4C53.7090902@v.loewis.de> Message-ID: <4AEFF3A3.3020103@egenix.com> Antoine Pitrou wrote: > Guido van Rossum python.org> writes: >> >> Is it even wort doing a 2.7 release? Isn't the effort better spent on >> 3.2 alone? (Note, these aren't rhetorical questions. It's well >> possible that there are good reasons for pushing along with 2.7. Maybe >> considering those reasons will also help answering questions about >> whether to backport things like nonlocal.) > > 2.7 has an up-to-date backport of the C IO lib (as well as the memoryview > object), which means it is better for people wanting to ease transition to 3.x. > > But of course, as Martin said, few people will want to support 2.7 only and not > 2.6. Since 2.7 will be closer to 3.2 than 2.6, the result will more likely be: many people will want to support 2.7 and 3.x - which is really what we should be after. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 03 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From fuzzyman at voidspace.org.uk Tue Nov 3 10:13:52 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 03 Nov 2009 10:13:52 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AEFD9AE.7090104@molden.no> References: <4AEFD9AE.7090104@molden.no> Message-ID: <4AEFF450.9060007@voidspace.org.uk> Sturla Molden wrote: > I'd just like to mention that the scientific community is highly > dependent on NumPy. As long as NumPy is not ported to Py3k, migration > is out of the question. Porting NumPy is not a trivial issue. It might > take a complete rewrite of the whole C base using Cython. NumPy's ABI > is not even PEP 3118 compliant. Changing the ABI for Py3k might break > extension code written for NumPy using C. And scientists tend to write > CPU-bound routines in languages like C and Fortran, not Python, so > that is a major issue as well. If we port NumPy to Py3k, everyone > using NumPy will have to port their C code to the new ABI. There are > lot of people stuck with Python 2.x for this reason. It does not just > affect individual scientists, but also large projects like IBM and > CERN's blue brain and NASA's space telecope. So please, do not cancel > 2.x support before we have ported NumPy, Matplotlib and most of their > dependant extensions to Py3k. What will it take to *start* the port? (Or is it already underway?) For many projects I fear that it is only the impending obsolescence (real rather than theoretical) of Python 2 that will convince projects to port. Python 2.X is not about to 'stop working', but there will come a point where it will 'stop being worked on'. All the best, Michael > The community of scientists and engineers using Python is growing, but > shutting down 2.x support might bring an end to that. > > Sturla Molden > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ From ncoghlan at gmail.com Tue Nov 3 10:21:43 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 03 Nov 2009 19:21:43 +1000 Subject: [Python-Dev] language summit topic: issue tracker In-Reply-To: <4AEF2779.5040402@v.loewis.de> References: <4AE17C12.1080007@gmail.com> <4AEF2779.5040402@v.loewis.de> Message-ID: <4AEFF627.4060601@gmail.com> Martin v. L?wis wrote: >> +lots on adding a module field (independent of automatically adding >> maintainers to the nosy list, it would assist in "I just did a major >> cleanup of module X, are there any old bugs I can kill off"). > > Link (1:1) or Multilink (1:n)? What is the impact on the Component field? I was thinking multilink, and leaving component alone - the module field would largely come into play when the component was just the "Lib" catch-all. > Would you be willing to manage the field (in the sense of managing the > set of values)? If so, please send me a list of values. I would suggest just using the module index from the documentation to seed any such list of modules in the tracker: http://docs.python.org/modindex.html Packages could generally be left as a single entry in the list. The only exception I think is that there should be an "xml.etree" entry separate from the main "xml" entry, and perhaps a separate entry for "os.path". Deprecated modules could either be left out of the list, or else moved to appear at the end. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From cournape at gmail.com Tue Nov 3 10:55:39 2009 From: cournape at gmail.com (David Cournapeau) Date: Tue, 3 Nov 2009 18:55:39 +0900 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AEFF450.9060007@voidspace.org.uk> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> Message-ID: <5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com> On Tue, Nov 3, 2009 at 6:13 PM, Michael Foord wrote: > Sturla Molden wrote: >> >> I'd just like to mention that the scientific community is highly dependent >> on NumPy. As long as NumPy is not ported to Py3k, migration is out of the >> question. Porting NumPy is not a trivial issue. It might take a complete >> rewrite of the whole C base using Cython. NumPy's ABI is not even PEP 3118 >> compliant. Changing the ABI for Py3k might break extension code written for >> NumPy using C. And scientists tend to write CPU-bound routines in languages >> like C and Fortran, not Python, so that is a major issue as well. If we port >> NumPy to Py3k, everyone using NumPy will have to port their C code to the >> new ABI. There are lot of people stuck with Python 2.x for this reason. It >> does not just affect individual scientists, but also large projects like IBM >> and CERN's blue brain and NASA's space telecope. So please, do not cancel >> 2.x support before we have ported NumPy, Matplotlib and most of their >> dependant extensions to Py3k. > > What will it take to *start* the port? (Or is it already underway?) For many > projects I fear that it is only the impending obsolescence (real rather than > theoretical) of Python 2 that will convince projects to port. I feel the same way. Given how much resources it will take to port to py3k, I doubt the port will happen soon. I don't know what other numpy developers think, but I consider py3k to simply not worth the hassle - I know we will have to port eventually, though. To answer your question, the main issues are: - are two branches are necessary or not ? If two branches are necessary, I think we simply do not have the resources at the moment. - how to maintain a compatible C API across 2.x and 3.x - is it practically possible to support and maintain numpy from 2.4 to 3.x ? For example, I don't think the python 2.6 py3k warnings are very useful when you need to maintain compatibility with 2.4 and 2.5. There is also little documentation on how to port a significant C codebase to py3k. David From python at rcn.com Tue Nov 3 10:58:27 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 3 Nov 2009 01:58:27 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> Message-ID: <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> [Michael Foord] > What will it take to *start* the port? (Or is it already underway?) For many projects I fear that it is only the impending > obsolescence (real rather than theoretical) of Python 2 that will convince projects to port. FWIW, I do not buy into the several premises that have arisen in this thread: * For 3.x to succeed, something bad has to happen to 2.x. (which in my book translates to intentionally harming 2.x users, either through neglect or force, in order to bait them into switching to 3.x). * Core developers will are losing time supporting 2.x. (backports take some time but it is small in comparison to getting a patch to work in the first place -- if anyone can comment on this assertion, it is the people who have been doing it already (such as AP, MD, BP, GB, and myself)). * That 3.x has proven its readiness to supplant 2.x. (It hasn't been exercised that heavily and there are a lot of things that may or may not prove to be successful in the end -- bytes/text issues, tuple comparison challenges, new io, mapping views with set operations, etc). In all these matters, I think the users should get a vote. And that vote should be cast with their decision to stay with 2.x, or switch to 3.x, or try to support both. We should not muck with their rational decision making by putting "carrots" in one pile and abandoning the other. Raymond P.S. I found it curious that one of the strongest proponents of killing 2.x also mentioned that he has never written a line of 3.x code. Since this discussion is a matter of great consequence, I would hope that advocates will only take informed positions -- this isn't really time for shooting from the hip and killing 2.x. From p.f.moore at gmail.com Tue Nov 3 12:23:33 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 3 Nov 2009 11:23:33 +0000 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> Message-ID: <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> 2009/11/3 Raymond Hettinger : > In all these matters, I think the users should get a vote. ?And that vote > should be cast with their decision to stay with 2.x, or switch to 3.x, or > try to support both. ?We should not muck with their rational decision making > by putting "carrots" in one pile and abandoning the other. Agreed (up to a point). The biggest issue to my mind is that adoption by the ultimate end users is significantly hampered by the fact that big projects like Twisted, numpy and the like, have no current plans to move to Python 3. Even end users with a reasonable level of coding expertise don't have the time or resources to offer much in the way of help with a port, when the project as a whole isn't interested in starting the process. At the moment, it seems to me that this is the biggest blocker to Python 3 adoption. And it's a chicken and egg situation - I don't use Python 3, so I don't test the new features, so the projects I need see little take-up, so I can't use them in Python 3, so I don't use Python 3... And while I know I can run Python 2.x and Python 3.x side by side, at the end of the day, I want to just be able to type "python" to get my interpreter. I don't know how to solve this (assuming that "just wait" isn't going to do it). Maybe the core devs will have to offer resource to some of the key projects to get things moving (but as this is a volunteer effort, that isn't something that "just happens" either...) Paul. From solipsis at pitrou.net Tue Nov 3 12:40:38 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 3 Nov 2009 11:40:38 +0000 (UTC) Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? References: <4AEFD9AE.7090104@molden.no> Message-ID: Sturla Molden molden.no> writes: > > Porting NumPy is not a trivial issue. It might take > a complete rewrite of the whole C base using Cython. I don't see why they would need a rewrite. Little of the C API has changed between 2.x and 3.x. Cython itself is supposed to support both 2.x and 3.x, isn't it? > NumPy's ABI is not even PEP 3118 compliant. That's interesting, because PEP 3118 was pushed mainly by a prominent member of the NumPy community and some of its features are almost dedicated to NumPy. Regards Antoine. From solipsis at pitrou.net Tue Nov 3 12:43:30 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 3 Nov 2009 11:43:30 +0000 (UTC) Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com> Message-ID: David Cournapeau gmail.com> writes: > > To answer your question, the main issues are: > - are two branches are necessary or not ? If two branches are > necessary, I think we simply do not have the resources at the moment. > - how to maintain a compatible C API across 2.x and 3.x > - is it practically possible to support and maintain numpy from 2.4 > to 3.x ? For example, I don't think the python 2.6 py3k warnings are > very useful when you need to maintain compatibility with 2.4 and 2.5. You should ask all those questions on the dedicated mailing-list: http://mail.python.org/mailman/listinfo/python-porting Regards Antoine. From rdmurray at bitdance.com Tue Nov 3 13:41:04 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 3 Nov 2009 07:41:04 -0500 (EST) Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: Message-ID: On Mon, 2 Nov 2009 at 22:06, Guido van Rossum wrote: > On Mon, Nov 2, 2009 at 9:51 PM, ssteinerX at gmail.com wrote: >> BeautifulSoup, which I use every day, is one such product. ?Since the crappy >> old SMGL parser's gone, BeautifulSoup uses the one that's left in Python 3 >> and it makes BeautifulSoup completely useless for my daily work. > > This sounds an area where some help might be useful. Perhaps the > quickest solution would simply be to copy the old crappy "sgml" based > html parser into a new version of BeautifulSoup. Though I imagine what > it really needs is a "quirks mode" parser that is compatible with the > HTML dialect accepted by, say, IE6. Maybe a summer of code project? It's not a matter of quirks. It's a matter of being able to parse truly broken html/xml, which browsers unfortunately do too well for everyone else's sanity. So, call it a "sloppy mode" parser, and then yes, that would solve the problem. --David (RDM) From barry at python.org Tue Nov 3 13:55:34 2009 From: barry at python.org (Barry Warsaw) Date: Tue, 3 Nov 2009 07:55:34 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AEFC887.6020801@v.loewis.de> References: <4AEFC887.6020801@v.loewis.de> Message-ID: <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> On Nov 3, 2009, at 1:07 AM, Martin v. L?wis wrote: > Barry Warsaw wrote: >> On Nov 2, 2009, at 10:48 PM, ssteinerX at gmail.com wrote: >> >>> A better language, i.e. Python 3.x, will become better faster >>> without >>> dragging the 2.x series out any longer. >> >> If Python 2.7 becomes the last of the 2.x series, then I personally >> favor back porting as many features from Python 3 as possible. > > And if *2.6* becomes the last of the 2.x series? Then clearly we can't back port features. I'd like to read some case studies of people who have migrated applications from 2.6 to 3.0. Having just gone through a 2 week sprint to migrate Launchpad from 2.4 to 2.6, and only making it to 2.5, I can say that I was unpleasantly surprised at the amount of work it took. A lot of that was working out the dependency upgrades, with some amount of fixing our code (mostly tests) for things that have changed (e.g. exception print/str format). We didn't make it to Python 2.6 because dealing with deprecation warnings for sha, md5, and sets (a little in our code but tons in our dependencies) consumed most of our remaining time. Given another week or so I think we would have made it to Python 2.6, but I'm not at all confident that that would have been a good enough platform to attempt an upgrade to Python 3, even if all of our very numerous large dependencies were available for Python 3. Maybe it wouldn't be so bad, but my recent experience informs me that I'm probably being too optimistic rather than too pessimistic. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From barry at python.org Tue Nov 3 14:09:06 2009 From: barry at python.org (Barry Warsaw) Date: Tue, 3 Nov 2009 08:09:06 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: Message-ID: On Nov 2, 2009, at 11:51 PM, ssteinerX at gmail.com wrote: > I agree as long as: > A> 2.7 comes out as soon as possible, even if it's missing helpful > porting features. > B> 2.7 will get ONLY new features that make it easier to port to > 3.x, not every feature added to 3.x or all you've done is make > "Python 2.7, the Python 3 Version." and core developer time will > continue to be wasted on Python 2.7 instead of moving forward. I'm not sure I agree with that core developer time will be "wasted on Python 2.7 instead of moving forward". However, I completely understand core developers seeing Python 2.x as a dead end and not wanting to work on it. If that's a real issue, we should acknowledge that as a factor in the decision. This is a volunteer organization and if the majority of developers are sick and tired of Python 2, then it absolutely makes no sense to slog through a Python 2.7 release. I'd much rather take all the enthusiastic energy that decision will reclaim and focus it on, oh, Python 3's email package instead . >> I also think Guido's call for feature freeze makes a lot more sense >> when 2.7 is the EOL. Let's give people migrating to Python 3 a >> nice big stable target to hit. Improving the stdlib also gives >> people a big carrot to move. > > Agreed. And specifically NOT porting every shiny new toy from Python > 3 back to 2.7 makes sure the carrots are only in the 3.x series. Python 3's got enough carrots and sticks already. One huge carrot that will never make it back into Python 2 is bytes/strings of course. The huge stick is that Python 2 is end-of-lifed, if not now, eventually. It isn't going to get a reprieve. Everyone knows that everyone will have to get to Python 3. The question is, what can we as a community do to make that inevitability as easy to swallow as possible? >> I think it's also necessary to give third party library and >> application authors as much help as possible to provide Python 3 >> compatible software. Putting together Python tools involves so >> many dependencies in a fairly deep stack that even one unconverted >> library can cause everything above it to stall on Python 2. > > And that's one of the reasons my explorations into Python 3 have > been limited to pretty much nothing. > > I don't have time to do a bunch of work only to find out that the > tool I absolutely have to have to finish a project doesn't have a > Python 3 version or has been crippled to make a Python 3 version. Unfortunately, I think we have to do those explorations, fail, hit roadblocks, complain, etc. but most importantly identify the packages that need to be ported. Then work with those package authors to make the upgrades happen. And improve Python and Pythonic tools so that migrations can go smoothly. Speaking as a package author, I know how much work it is just to get a bug fix release out. The three lines of code fix means 50 lines of test writing, a half a day of documenting, packaging, uploading, and announcing. Porting even one of my packages to Python 3 is a significant undertaking which frankly I don't have the cycles for. Anything large and complicated is hopeless. Witness how long and difficult it's been just to get a standard library module updated (email) and you get a sense of how much work it will be to get an entire stack of code onto Python 3. > BeautifulSoup, which I use every day, is one such product. Since > the crappy old SMGL parser's gone, BeautifulSoup uses the one that's > left in Python 3 and it makes BeautifulSoup completely useless for > my daily work. > > That's not to say I can't fix that one particular project, but > customers get cranky when their project is taking longer than > expected and "Oh, I'm having to convert a lot of things to use > Python 3" doesn't seem to improve their mood much. I completely agree. What happens when your application depends on a half dozen Zope packages, Twisted, and 15 or 20 other established, mature packages? It's a daunting prospect. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From mal at egenix.com Tue Nov 3 14:21:34 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 03 Nov 2009 14:21:34 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> Message-ID: <4AF02E5E.4020501@egenix.com> Raymond Hettinger wrote: > In all these matters, I think the users should get a vote. And that > vote should be cast with their decision to stay with 2.x, or switch to > 3.x, or try to support both. We should not muck with their rational > decision making by putting "carrots" in one pile and abandoning the other. I don't think users will really go for carrots. Python 2.x is mature enough already and at least the users I know are really happy with it (including myself). IMHO, the main benefit of backporting features from 3.x to 2.x is to make the transition from 2.x to 3.x a gradual one based on evolution rather than revolution. The other aspect is maintenance. Users do care about bug fixes and security patches. They also care about fixes needed to make Python work on new platforms (such as Windows 7) - mainly to keep their existing code running on these new platforms. The question is: Who is going to continue working on such 2.x releases, review patches, etc. ? If there are no core developers willing to do this, it's likely not going to happen. OTOH, I'm sure that companies who have invested in Python 2.x applications will gladly pay a yearly fee to the PSF to have this done for them. It's simply a whole lot cheaper than to port a few 100K lines of Python/C code, not to mention having to wait for all the used 3rd party libs to get ported as well. The PSF could then pay a core developer to take care of the extra Python 2.x releases. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 03 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From cournape at gmail.com Tue Nov 3 14:26:00 2009 From: cournape at gmail.com (David Cournapeau) Date: Tue, 3 Nov 2009 22:26:00 +0900 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> Message-ID: <5b8d13220911030526y7dce8ce0gecb427ea009eaf1a@mail.gmail.com> On Tue, Nov 3, 2009 at 8:40 PM, Antoine Pitrou wrote: > Sturla Molden molden.no> writes: >> >> Porting NumPy is not a trivial issue. It might take >> a complete rewrite of the whole C base using Cython. > > I don't see why they would need a rewrite. (let me know if those numpy-specific discussions are considered OT0 There is certainly no need for a full rewrite, no. I am still unclear on the range of things to change for 3.x, but the C changes are not small, especially since numpy uses "dark" areas of python C extension. The long vs int, strings vs bytes will take some time. AFAIK, the only thing which has been attempted so far is porting our own distutils extension to python 3.x, but I have not integrated those changes yet. > between 2.x and 3.x. Cython itself is supposed to support both 2.x and 3.x, > isn't it? Yes - but no numpy code use cython ATM, except for the random generators, which would almost certainly be trivial to convert. The idea which has been discussed so far is that for *some* code which need significant changes or rewrite, using cython instead of C may be beneficial, as it would give the 3.x code "for free". Having more cython and less C could also bring more contributors - that would actually be the biggest incentive, as the number of people who know the core C code of numpy is too small. > That's interesting, because PEP 3118 was pushed mainly by a prominent member of > the NumPy community and some of its features are almost dedicated to NumPy. I have not been involved with PEP 3118 discussion, so cannot comment on the reason why it is not fully supported yet by numpy. But I think that's a different issue altogether - PEP 3118 goal is for interoperation with other packages. We can port to PEP 3118 without porting to 3.x, and we can port to 3.x without taking care of PEP 3118. David From cournape at gmail.com Tue Nov 3 14:40:04 2009 From: cournape at gmail.com (David Cournapeau) Date: Tue, 3 Nov 2009 22:40:04 +0900 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> References: <4AEFC887.6020801@v.loewis.de> <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> Message-ID: <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com> On Tue, Nov 3, 2009 at 9:55 PM, Barry Warsaw wrote: > > Then clearly we can't back port features. > > I'd like to read some case studies of people who have migrated applications > from 2.6 to 3.0. +1, especially for packages which have a lot of C code: the current documentation is sparse :) The only helpful reference I have found so far is an email by MvL concerning psycopg2 port. David From ssteinerx at gmail.com Tue Nov 3 14:40:48 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Tue, 3 Nov 2009 08:40:48 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: Message-ID: On Nov 3, 2009, at 12:06 AM, Guido van Rossum wrote: > On Mon, Nov 2, 2009 at 9:51 PM, ssteinerX at gmail.com > wrote: >> BeautifulSoup, which I use every day, is one such product. Since >> the crappy >> old SMGL parser's gone, BeautifulSoup uses the one that's left in >> Python 3 >> and it makes BeautifulSoup completely useless for my daily work. > > This sounds an area where some help might be useful. Perhaps the > quickest solution would simply be to copy the old crappy "sgml" based > html parser into a new version of BeautifulSoup. That is what we're discussing doing on the old-soup branch at http://github.com/adevore/old-beautiful-soup . I'm not exactly sure why the old SGML parser was dropped but it seems that porting it to Python 3 would be enough of an effort that it caused the Python library to drop it, and the current developer of the mainline of Beautiful Soup to decide to just use what was available in Python 3 natively. > Though I imagine what it really needs is a "quirks mode" parser that > is compatible with the > HTML dialect accepted by, say, IE6. Maybe a summer of code project? I think it just relies on the old SGML parser's not blowing up on completely bogus HTML (like most of the web) and does the best it can with the 'chunks' that come back; nothing to do with quirks mode per se. As for a Summer of Code project, I have no idea what would be involved. I know there are lots of users for Beautiful soup; as far as I know it is the best scraper of HTML code, valid or not, that's out there and it's been around a long time and I see it in projects in the "html scraping" realm all the time. At any rate, it's just one example of where the developer has taken the easy route out with a 3.0 port and has produced a product that's "Python 3" but, instead of getting better with Python's new features, has actually become useless for the majority of use-cases where formerly it shined. S From ssteinerx at gmail.com Tue Nov 3 14:43:06 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Tue, 3 Nov 2009 08:43:06 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AEFC887.6020801@v.loewis.de> References: <4AEFC887.6020801@v.loewis.de> Message-ID: <5B548FA6-995A-4519-B614-982794FA85DE@gmail.com> On Nov 3, 2009, at 1:07 AM, Martin v. L?wis wrote: > Barry Warsaw wrote: >> On Nov 2, 2009, at 10:48 PM, ssteinerX at gmail.com wrote: >> >>> A better language, i.e. Python 3.x, will become better faster >>> without >>> dragging the 2.x series out any longer. >> >> If Python 2.7 becomes the last of the 2.x series, then I personally >> favor back porting as many features from Python 3 as possible. > > And if *2.6* becomes the last of the 2.x series? Then I'd guess that that would annoy the crap out of everyone who's put so much effort into 2.7 and all of us who have been waiting for what I hope will finally be the "now ports way more easily to 3.0" version. S From ssteinerx at gmail.com Tue Nov 3 14:45:59 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Tue, 3 Nov 2009 08:45:59 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AEFD9AE.7090104@molden.no> References: <4AEFD9AE.7090104@molden.no> Message-ID: On Nov 3, 2009, at 2:20 AM, Sturla Molden wrote: > I'd just like to mention that the scientific community is highly > dependent on NumPy. As long as NumPy is not ported to Py3k, > migration is out of the question. Porting NumPy is not a trivial > issue. It might take a complete rewrite of the whole C base using > Cython. NumPy's ABI is not even PEP 3118 compliant. Changing the ABI > for Py3k might break extension code written for NumPy using C. And > scientists tend to write CPU-bound routines in languages like C and > Fortran, not Python, so that is a major issue as well. If we port > NumPy to Py3k, everyone using NumPy will have to port their C code > to the new ABI. There are lot of people stuck with Python 2.x for > this reason. > It does not just affect individual scientists, but also large > projects like IBM and CERN's blue brain and NASA's space telecope. Then, perhaps, if 2.7 is known to be the last release of the 2.x line, some of those deep pockets can cough up some $$$ or developers to actually _do_ the port. A Python 3 version of NumPy might be enough of an improvement to bring *more* scientists and engineers onboard if the Python 3.x version shows what great productivity gains are to be had with Python 3.x over 2.x. S From ssteinerx at gmail.com Tue Nov 3 14:47:58 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Tue, 3 Nov 2009 08:47:58 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com> Message-ID: <30C6A234-DC87-45CD-A82D-62733B2BD9E3@gmail.com> On Nov 3, 2009, at 4:55 AM, David Cournapeau wrote: > On Tue, Nov 3, 2009 at 6:13 PM, Michael Foord > wrote: > > There is also little documentation on how to port a significant C > codebase to py3k. Now there's a good Summer of Code project: to produce a pre-processor that will flag all C constructs that need to be modified in some way, with direct pointers to the relevant documentation, and "code suggestions" wherever practicable. S From ssteinerx at gmail.com Tue Nov 3 14:55:13 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Tue, 3 Nov 2009 08:55:13 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> Message-ID: <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> On Nov 3, 2009, at 4:58 AM, Raymond Hettinger wrote: > > P.S. I found it curious that one of the strongest proponents of > killing 2.x also mentioned that he has never written a line of 3.x > code. Since this discussion is a matter of great consequence, I > would hope that advocates will only take informed positions -- this > isn't really time for shooting from the hip and killing 2.x. Uh, that would be me. I'm only a proponent of making a decision. I *want* to have a better development language, library, and add-on tools. If 3.x is where future core development time is going to be focused, then I have faith that they will be able to make it the compelling path that it will become with Guido having put as much effort into it as he has. Or, maybe he's completely lost his mind as sometimes happens with "dictators for life" benevolent or other-wise. ;-) In any case, splitting time between 2.x and 3.x, with limited developer resources is going to lead to slower progress on both fronts. And, as you point out, if 3.x doesn't start getting the crap beat out of it in the real world sooner rather than later, we may find ourselves, collectively with a stale 2.x, an under battle-tested 3.x, and nowhere to go. S From solipsis at pitrou.net Tue Nov 3 15:04:06 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 3 Nov 2009 14:04:06 +0000 (UTC) Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? References: <4AEFD9AE.7090104@molden.no> <5b8d13220911030526y7dce8ce0gecb427ea009eaf1a@mail.gmail.com> Message-ID: David Cournapeau gmail.com> writes: > > We can port to PEP 3118 without > porting to 3.x, and we can port to 3.x without taking care of PEP > 3118. I'm not sure you can do the latter. The old buffer API (the one PEP 3118 replaces) doesn't exist in py3k. Antoine. From daniel at stutzbachenterprises.com Tue Nov 3 15:14:46 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 3 Nov 2009 08:14:46 -0600 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com> Message-ID: On Tue, Nov 3, 2009 at 3:55 AM, David Cournapeau wrote: > - are two branches are necessary or not ? If two branches are > necessary, I think we simply do not have the resources at the moment. > - how to maintain a compatible C API across 2.x and 3.x > - is it practically possible to support and maintain numpy from 2.4 > to 3.x ? For example, I don't think the python 2.6 py3k warnings are > very useful when you need to maintain compatibility with 2.4 and 2.5. > I've already ported some of my Python extension modules to Python 3. Here's how I would answer your questions based on my experience. Writing C code that compiles with Python 2.4 through 3.1 is pretty easy. Python's C API hasn't changed much and you can use #ifdef and #define for any bits that must be version-specific. It's pretty easy to make Python source that works under 2.6 and 3.x. It's basically impossible to make Python source that works under 2.4/2.5 and 3.x. You may be able to write code that works under 2.4/2.5 and works cleanly with 2to3 to produce 3.x code. I haven't tried that route, though in theory it should work. All you really need is syntax compatibility. For the rest, you can check sys.version_info. In a nutshell, I don't think you need two branches to support an extension module on Python 2 and Python 3. YMMV. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From skip at pobox.com Tue Nov 3 16:23:39 2009 From: skip at pobox.com (skip at pobox.com) Date: Tue, 3 Nov 2009 09:23:39 -0600 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AEFC887.6020801@v.loewis.de> References: <4AEFC887.6020801@v.loewis.de> Message-ID: <19184.19195.484852.432070@montanaro.dyndns.org> Martin> And if *2.6* becomes the last of the 2.x series? With all due respect, I don't think you can make that decision now. The time to have stated 2.6 would be the end of the 2.x line was when 2.6 was released. At that point instead of opening up the trunk for changes you would have closed it off or merged the py3k branch to trunk. 2.6.0 was released over a year ago and there has been no effort to suppress bug fix or feature additions to trunk since then. If you call 2.6 "the end of 2.x" you'll have wasted a year of work on 2.7 with about a month to go before the first 2.7 alpha release. If you want to accelerate release of 2.7 (fewer alphas, compressed schedule, etc) that's fine, but I don't think you can turn back the clock at this point and decree that 2.7 is dead. Skip From skip at pobox.com Tue Nov 3 16:32:04 2009 From: skip at pobox.com (skip at pobox.com) Date: Tue, 3 Nov 2009 09:32:04 -0600 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF02E5E.4020501@egenix.com> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <4AF02E5E.4020501@egenix.com> Message-ID: <19184.19700.781398.906305@montanaro.dyndns.org> mal> I don't think users will really go for carrots. Python 2.x is mal> mature enough already and at least the users I know are really mal> happy with it (including myself). Taking that thought further back one step (or three), from http://effbot.org/tkinterbook/listbox.htm I pull this quote: In versions before Python 1.5, use string.atoi instead of int. :-) Skip From guido at python.org Tue Nov 3 16:54:32 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Nov 2009 07:54:32 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> References: <4AEFC887.6020801@v.loewis.de> <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> Message-ID: On Tue, Nov 3, 2009 at 4:55 AM, Barry Warsaw wrote: > I'd like to read some case studies of people who have migrated applications > from 2.6 to 3.0. ?Having just gone through a 2 week sprint to migrate > Launchpad from 2.4 to 2.6, and only making it to 2.5, I can say that I was > unpleasantly surprised at the amount of work it took. ?A lot of that was > working out the dependency upgrades, with some amount of fixing our code > (mostly tests) for things that have changed (e.g. exception print/str > format). ?We didn't make it to Python 2.6 because dealing with deprecation > warnings for sha, md5, and sets (a little in our code but tons in our > dependencies) consumed most of our remaining time. Ouch. sets. I'm guessing you are referring to code that was still using the pre-2.4 sets.py module? Yes, that must have been painful. > Given another week or so I think we would have made it to Python 2.6, but > I'm not at all confident that that would have been a good enough platform to > attempt an upgrade to Python 3, even if all of our very numerous large > dependencies were available for Python 3. ?Maybe it wouldn't be so bad, but > my recent experience informs me that I'm probably being too optimistic > rather than too pessimistic. There are two stages of porting to 2.6 you have to go through. The first one, which you would probably have reached in that week, is running on 2.6 period. You can then release your code for the benefit of others wanting to use it on 2.6. But the second stage, which can take much more time (depending on the state of your code base) is to run on 2.6 *free of warnings with the -3 flag on*. You pretty much have to consider this a separate port, and it is here where you do much of the prep work for 3.x (at least for Python code -- for C extensions it's not so clear). The good news is that you can claim 2.6 support before you've even started this stage; the other good news is that doing this right will really help you prepare for 3.x. And in most cases you can even (with some effort) maintain compatibility with 2.5 or even 2.4 -- though you may have to work around some things like the md5 and sha warnings. The bad news is that this stage may well take more time than porting from 2.4 to 2.6. -- --Guido van Rossum (python.org/~guido) From eric at trueblade.com Tue Nov 3 16:56:24 2009 From: eric at trueblade.com (Eric Smith) Date: Tue, 03 Nov 2009 10:56:24 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <19184.19700.781398.906305@montanaro.dyndns.org> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <4AF02E5E.4020501@egenix.com> <19184.19700.781398.906305@montanaro.dyndns.org> Message-ID: <4AF052A8.6060007@trueblade.com> skip at pobox.com wrote: > mal> I don't think users will really go for carrots. Python 2.x is > mal> mature enough already and at least the users I know are really > mal> happy with it (including myself). > > Taking that thought further back one step (or three), from > > http://effbot.org/tkinterbook/listbox.htm > > I pull this quote: > > In versions before Python 1.5, use string.atoi instead of int. Which reminds me: I've been meaning to add -3 warnings for these string module functions! Eric. From foom at fuhm.net Tue Nov 3 18:04:00 2009 From: foom at fuhm.net (James Y Knight) Date: Tue, 3 Nov 2009 12:04:00 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> Message-ID: <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> On Nov 3, 2009, at 8:55 AM, ssteinerX at gmail.com wrote: > And, as you point out, if 3.x doesn't start getting the crap beat > out of it in the real world sooner rather than later, we may find > ourselves, collectively with a stale 2.x, an under battle-tested > 3.x, and nowhere to go. If that happens, it's not true that there's *nowhere* to go. A solution would be to discard 3.x as a failed experiment, take everything that is useful from it and port it to 2.x, and simply continue development from the last 2.x release. And from there, features can be deprecated and then removed a few releases later, as is the usual policy. Been there, done that, on a couple other projects. It's unfortunate when you have to throw out work you've done because it failed to gain traction over the thing you tried to replace, but sometimes that's life. James From debatem1 at gmail.com Tue Nov 3 18:10:17 2009 From: debatem1 at gmail.com (geremy condra) Date: Tue, 3 Nov 2009 12:10:17 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <30C6A234-DC87-45CD-A82D-62733B2BD9E3@gmail.com> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com> <30C6A234-DC87-45CD-A82D-62733B2BD9E3@gmail.com> Message-ID: On Tue, Nov 3, 2009 at 8:47 AM, ssteinerX at gmail.com wrote: > > On Nov 3, 2009, at 4:55 AM, David Cournapeau wrote: > >> On Tue, Nov 3, 2009 at 6:13 PM, Michael Foord >> wrote: >> >> There is also little documentation on how to port a significant C >> codebase to py3k. > > Now there's a good Summer of Code project: ?to produce a pre-processor that > will flag all C constructs that need to be modified in some way, with direct > pointers to the relevant documentation, and "code suggestions" wherever > practicable. > > S How much interest is there in this? Geremy Condra From guido at python.org Tue Nov 3 18:23:16 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Nov 2009 09:23:16 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> Message-ID: On Tue, Nov 3, 2009 at 9:04 AM, James Y Knight wrote: > If that happens, it's not true that there's *nowhere* to go. A solution > would be to discard 3.x as a failed experiment, take everything that is > useful from it and port it to 2.x, and simply continue development from the > last 2.x release. And from there, features can be deprecated and then > removed a few releases later, as is the usual policy. > > Been there, done that, on a couple other projects. It's unfortunate when you > have to throw out work you've done because it failed to gain traction over > the thing you tried to replace, but sometimes that's life. I'm not ready for that yet. I think there's plenty of time before we have to agree to such a bleak view. In the mean time let's do something practical like help NumPy port to Py3k. -- --Guido van Rossum (python.org/~guido) From martin at v.loewis.de Tue Nov 3 18:27:37 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Nov 2009 18:27:37 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com> Message-ID: <4AF06809.3040009@v.loewis.de> > To answer your question, the main issues are: > - are two branches are necessary or not ? If two branches are > necessary, I think we simply do not have the resources at the moment. No, it should be well possible to have the same source being used in both 2.x and 3.x. I've ported ZODB to Python 3 (which includes both C and Python code), and it works quite well. > - how to maintain a compatible C API across 2.x and 3.x Not sure what the "C API" is: if it is the actual implementation of the C modules, then I recommend to use preprocessor macros a lot. If you need specific solutions, you'll have to ask specific questions. > - is it practically possible to support and maintain numpy from 2.4 > to 3.x ? Absolutely, yes. > For example, I don't think the python 2.6 py3k warnings are > very useful when you need to maintain compatibility with 2.4 and 2.5. These I don't know. I found that I had little use for the 3k warnings in porting to 3k; I usually rely on tests to find out whether the code still works correctly. > There is also little documentation on how to port a significant C > codebase to py3k. Please do ask specific questions. Regards, Martin From arcriley at gmail.com Tue Nov 3 18:28:47 2009 From: arcriley at gmail.com (Arc Riley) Date: Tue, 3 Nov 2009 12:28:47 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> Message-ID: +1 on ending with 2.6. I'm the maintainer of 3rd party Python 3-only packages and have ported a few modules that we needed with some help from the 2to3 tool. It's really not a big deal - and Py3 really is a massive improvement. The main thing holding back the community are lazy and/or obstinate package maintainers. If they spent half the time they've put into complaining about Py3 into actually working to upgrade their code they'd be done now. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.bethard at gmail.com Tue Nov 3 18:33:20 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 3 Nov 2009 09:33:20 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> References: <4AEFC887.6020801@v.loewis.de> <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> Message-ID: On Tue, Nov 3, 2009 at 4:55 AM, Barry Warsaw wrote: > I'd like to read some case studies of people who have migrated applications > from 2.6 to 3.0. For what it's worth, it was pretty easy to migrate argparse: http://code.google.com/p/argparse/source/detail?r=12 It was mostly just adding a few aliases, and doing a little sys.exc_info() dance in a couple places because argparse supports Python 2.3 - 3.1 all in the same code base. Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From martin at v.loewis.de Tue Nov 3 18:34:04 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Nov 2009 18:34:04 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com> References: <4AEFC887.6020801@v.loewis.de> <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com> Message-ID: <4AF0698C.2030706@v.loewis.de> >> I'd like to read some case studies of people who have migrated applications >> from 2.6 to 3.0. > > +1, especially for packages which have a lot of C code: the current > documentation is sparse :) The only helpful reference I have found so > far is an email by MvL concerning psycopg2 port. You may also want to take a look at my ZODB port: https://www.dcl.hpi.uni-potsdam.de/home/loewis/zodb/ Regards, Martin From ssteinerx at gmail.com Tue Nov 3 18:35:16 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Tue, 3 Nov 2009 12:35:16 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> Message-ID: <3B364FE0-7616-41F8-82C7-528EA1282E40@gmail.com> On Nov 3, 2009, at 12:23 PM, Guido van Rossum wrote: > On Tue, Nov 3, 2009 at 9:04 AM, James Y Knight wrote: >> If that happens, it's not true that there's *nowhere* to go. A >> solution >> would be to discard 3.x as a failed experiment, take everything >> that is >> useful from it and port it to 2.x, and simply continue development >> from the >> last 2.x release. And from there, features can be deprecated and then >> removed a few releases later, as is the usual policy. >> >> Been there, done that, on a couple other projects. It's unfortunate >> when you >> have to throw out work you've done because it failed to gain >> traction over >> the thing you tried to replace, but sometimes that's life. > > I'm not ready for that yet. I think there's plenty of time before we > have to agree to such a bleak view. In the mean time let's do > something practical like help NumPy port to Py3k. Or, for example, Django... S From guido at python.org Tue Nov 3 18:35:10 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Nov 2009 09:35:10 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium Message-ID: I've checked draft (!) PEP 3003, "Python Language Moratorium", into SVN. As authors I've listed Jesse, Brett and myself. On python-ideas the moratorium idea got fairly positive responses (more positive than I'd expected, in fact) but I'm bracing myself for fierce discussion here on python-dev. It's important to me that if if this is accepted it is a "rough consensus" decision (working code we already have plenty of :-), not something enforced by a vocal minority or an influential individual such as myself. If there's too much opposition I'll withdraw the PEP so as not to waste everybody's time with a fruitless discussion. The PEP tries to spell out some gray areas but I'm sure there will be others; that's life. Do note that the PEP proposes to be *retroactive* back to the 3.1 release, i.e. the "frozen" version of the language is the state in which it was released as 3.1. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- PEP: 3003 Title: Python Language Moratorium Version: $Revision$ Last-Modified: $Date$ Author: Brett Cannon, Jesse Noller, Guido van Rossum Status: Draft Type: Process Content-Type: text/x-rst Created: 21-Oct-2009 Post-History: Abstract ======== This PEP proposes a temporary moratorium (suspension) of all changes to the Python language syntax, semantics, and built-ins for a period of at least two years from the release of Python 3.1. This suspension of features is designed to allow non-CPython implementations to "catch up" to the core implementation of the language, help ease adoption of Python 3.x, and provide a more stable base for the community. Rationale ========= This idea was proposed by Guido van Rossum on the python-ideas [1]_ mailing list. The premise of his email was to slow the alteration of the Python core syntax, builtins and semantics to allow non-CPython implementations to catch up to the current state of Python, both 2.x and 3.x. Python, as a language is more than the core implementation -- CPython -- with a rich, mature and vibrant community of implementations, such as Jython [2]_, IronPython [3]_ and PyPy [4]_ that are a benefit not only to the community, but to the language itself. Still others, such as Unladen Swallow [5]_ (a branch of CPython) seek not to create an alternative implementation, but rather they seek to enhance the performance and implementation of CPython itself. Python 3.x was a large part of the last several years of Python's development. Its release, as well as a bevy of changes to the language introduce by it and the previous 2.6.x releases, puts alternative implementations at a severe disadvantage in "keeping pace" with core python development. Additionally, many of the changes put into the recent releases of the language as implemented by CPython have not yet seen widespread usage amongst the general user population. For example, most users are beholden to the version of the interpreter (typically CPython) which comes pre-installed with their operating system. Most OS vendors are just barely beginning to ship Python 2.6 -- even fewer are shipping Python 3.x. As it is expected that Python 2.7 be the effective "end of life" of the Python 2.x code line, with Python 3.x being the future, it is in the best interest of Python core development to temporarily suspend the alteration of the language itself to allow all of these external entities to catch up and to assist in the adoption of, and migration to, Python 3.x Finally, the moratorium is intended to free up cycles within core development to focus on other issues, such as the CPython interpreter and improvements therein, the standard library, etc. This moratorium does not allow for exceptions -- once accepted, any pending changes to the syntax or semantics of the language will be postponed until the moratorium is lifted. This moratorium does not attempt to apply to any other Python implementation meaning that if desired other implementations may add features which deviate from the standard implementation. Details ======= Cannot Change ------------- * New built-ins * Language syntax The grammar file essentially becomes immutable apart from ambiguity fixes. * General language semantics The language operates as-is with only specific exemptions (see below). Case-by-Case Exemptions ----------------------- * New methods on built-ins The case for adding a method to a built-in object can be made. * Incorrect language semantics If the language semantics turn out to be ambiguous or improperly implemented based on the intention of the original design then the semantics may change. * Language semantics that are difficult to implement Because other VMs have not begun implementing Python 3.x semantics there is a possibility that certain semantics are too difficult to replicate. In those cases they can be changed to ease adoption of Python 3.x by the other VMs. Allowed to Change ----------------- * C API It is entirely acceptable to change the underlying C code of CPython as long as other restrictions of this moratorium are not broken. E.g. removing the GIL would be fine assuming certain operations that are currently atomic remain atomic. * The standard library As the standard library is not directly tied to the language definition it is not covered by this moratorium. * Backports of 3.x features to to 2.x The moratorium only affects features that would be new in 3.x. Retroactive =========== It is important to note that the moratorium covers all changes since the release of Python 3.1. This rule is intended to avoid features being rushed or smuggled into the CPython source tree while the moratorium is being discussed. Copyright ========= This document has been placed in the public domain. References ========== .. [1] http://mail.python.org/pipermail/python-ideas/2009-October/006305.html .. [2] http://www.jython.org/ .. [3] http://www.codeplex.com/IronPython .. [4] http://codespeak.net/pypy/ .. [5] http://code.google.com/p/unladen-swallow/ .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 80 coding: utf-8 End: From solipsis at pitrou.net Tue Nov 3 18:35:57 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 3 Nov 2009 17:35:57 +0000 (UTC) Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> Message-ID: Arc Riley gmail.com> writes: > > +1 on ending with 2.6.I'm the maintainer of 3rd party Python 3-only packages > and have ported a few modules that we needed with some help from the 2to3 > tool.? It's really not a big deal - and Py3 really is a massive improvement. > The main thing holding back the community are lazy and/or obstinate package > maintainers.? If they spent half the time they've put into complaining about > Py3 into actually working to upgrade their code they'd be done now. One thing you could do is explain (do you have a blog?) how Py3 is a massive improvement for you as a developer and package maintainer. We core developers obviously agree that py3k is better than 2.x, but the same opinion coming from a third-party developer would carry a different weight. Regards Antoine. From martin at v.loewis.de Tue Nov 3 18:37:35 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Nov 2009 18:37:35 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> Message-ID: <4AF06A5F.6090501@v.loewis.de> > A Python 3 version of NumPy might be enough of an improvement to bring > *more* scientists and engineers onboard if the Python 3.x version shows > what great productivity gains are to be had with Python 3.x over 2.x. I would be really surprised if 2.7 would simplify porting to 3.x. How could that possibly work? (and no, adding things like nonlocal to 2.7 doesn't making porting of a real application or library any easier, since the existing application or library simply doesn't use that keyword. In fact, no change to 2.x can reasonably simplify porting - only changes to 3.x might - except for changes to 2to3, which can simplify porting a lot. But 2to3 should be run under 3.x, IMO.) Regards, Martin From guido at python.org Tue Nov 3 18:38:09 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Nov 2009 09:38:09 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <3B364FE0-7616-41F8-82C7-528EA1282E40@gmail.com> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> <3B364FE0-7616-41F8-82C7-528EA1282E40@gmail.com> Message-ID: On Tue, Nov 3, 2009 at 9:35 AM, ssteinerX at gmail.com wrote: > > On Nov 3, 2009, at 12:23 PM, Guido van Rossum wrote: > >> On Tue, Nov 3, 2009 at 9:04 AM, James Y Knight wrote: >>> >>> If that happens, it's not true that there's *nowhere* to go. A solution >>> would be to discard 3.x as a failed experiment, take everything that is >>> useful from it and port it to 2.x, and simply continue development from >>> the >>> last 2.x release. And from there, features can be deprecated and then >>> removed a few releases later, as is the usual policy. >>> >>> Been there, done that, on a couple other projects. It's unfortunate when >>> you >>> have to throw out work you've done because it failed to gain traction >>> over >>> the thing you tried to replace, but sometimes that's life. >> >> I'm not ready for that yet. I think there's plenty of time before we >> have to agree to such a bleak view. In the mean time let's do >> something practical like help NumPy port to Py3k. > > Or, for example, Django... Wasn't Django ported to Py3k by MvL as a demo? The problem seems to be more to port the Django *community* to Py3k... -- --Guido van Rossum (python.org/~guido) From g.brandl at gmx.net Tue Nov 3 18:39:21 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 03 Nov 2009 18:39:21 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <19184.19195.484852.432070@montanaro.dyndns.org> References: <4AEFC887.6020801@v.loewis.de> <19184.19195.484852.432070@montanaro.dyndns.org> Message-ID: skip at pobox.com schrieb: > Martin> And if *2.6* becomes the last of the 2.x series? > > With all due respect, I don't think you can make that decision now. The > time to have stated 2.6 would be the end of the 2.x line was when 2.6 was > released. At that point instead of opening up the trunk for changes you > would have closed it off or merged the py3k branch to trunk. 2.6.0 was > released over a year ago and there has been no effort to suppress bug fix or > feature additions to trunk since then. If you call 2.6 "the end of 2.x" > you'll have wasted a year of work on 2.7 with about a month to go before the > first 2.7 alpha release. +1. Georg From martin at v.loewis.de Tue Nov 3 18:39:53 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Tue, 03 Nov 2009 18:39:53 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com> Message-ID: <4AF06AE9.2010706@v.loewis.de> > It's pretty easy to make Python source that works under 2.6 and 3.x. > It's basically impossible to make Python source that works under 2.4/2.5 > and 3.x. You may be able to write code that works under 2.4/2.5 and > works cleanly with 2to3 to produce 3.x code. I haven't tried that > route, though in theory it should work. All you really need is syntax > compatibility. I have tried that route for a number of projects, and I think it works really well. It is also supported by distribute. > In a nutshell, I don't think you need two branches to support an > extension module on Python 2 and Python 3. > > YMMV. Exactly my experience as well. Regards, Martin From martin at v.loewis.de Tue Nov 3 18:43:29 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Nov 2009 18:43:29 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <3B364FE0-7616-41F8-82C7-528EA1282E40@gmail.com> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> <3B364FE0-7616-41F8-82C7-528EA1282E40@gmail.com> Message-ID: <4AF06BC1.6060004@v.loewis.de> >> I'm not ready for that yet. I think there's plenty of time before we >> have to agree to such a bleak view. In the mean time let's do >> something practical like help NumPy port to Py3k. > > Or, for example, Django... See http://wiki.python.org/moin/PortingDjangoTo3k Regards, Martin From ssteinerx at gmail.com Tue Nov 3 18:44:08 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Tue, 3 Nov 2009 12:44:08 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> <3B364FE0-7616-41F8-82C7-528EA1282E40@gmail.com> Message-ID: <35DEB5D3-41C5-482A-8D84-577606EE2C70@gmail.com> On Nov 3, 2009, at 12:38 PM, Guido van Rossum wrote: > On Tue, Nov 3, 2009 at 9:35 AM, ssteinerX at gmail.com > wrote: >> >> On Nov 3, 2009, at 12:23 PM, Guido van Rossum wrote: >> >>> On Tue, Nov 3, 2009 at 9:04 AM, James Y Knight >>> wrote: >>>> >>>> If that happens, it's not true that there's *nowhere* to go. A >>>> solution >>>> would be to discard 3.x as a failed experiment, take everything >>>> that is >>>> useful from it and port it to 2.x, and simply continue >>>> development from >>>> the >>>> last 2.x release. And from there, features can be deprecated and >>>> then >>>> removed a few releases later, as is the usual policy. >>>> >>>> Been there, done that, on a couple other projects. It's >>>> unfortunate when >>>> you >>>> have to throw out work you've done because it failed to gain >>>> traction >>>> over >>>> the thing you tried to replace, but sometimes that's life. >>> >>> I'm not ready for that yet. I think there's plenty of time before we >>> have to agree to such a bleak view. In the mean time let's do >>> something practical like help NumPy port to Py3k. >> >> Or, for example, Django... > > Wasn't Django ported to Py3k by MvL as a demo? The problem seems to be > more to port the Django *community* to Py3k... I do remember seeing something about that somewhere but it sure isn't jumping into my workflow at the moment. If I can get a Python 3 version of Django, that's keeping up with trunk, I hereby declare that I will start using it on my current project as soon as the client takes the blowtorch off my toes for the current deliverable. And...I'll help keep it up to date with trunk as best I can and also help pull along all the modules I need (incidentally, including Beautiful Soup). I really _want_ Python 3 to be better, I hope actual use convinces me that it is... There, now I'm committed (or, maybe I should _be_ committed). Thanks, S From p.f.moore at gmail.com Tue Nov 3 18:44:24 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 3 Nov 2009 17:44:24 +0000 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF0698C.2030706@v.loewis.de> References: <4AEFC887.6020801@v.loewis.de> <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com> <4AF0698C.2030706@v.loewis.de> Message-ID: <79990c6b0911030944r26fd668ev37e56a4a7d79559b@mail.gmail.com> 2009/11/3 "Martin v. L?wis" : >>> I'd like to read some case studies of people who have migrated applications >>> from 2.6 to 3.0. >> >> +1, especially for packages which have a lot of C code: the current >> documentation is sparse :) The only helpful reference I have found so >> far is an email by MvL concerning psycopg2 port. > > You may also want to take a look at my ZODB port: > > https://www.dcl.hpi.uni-potsdam.de/home/loewis/zodb/ Has that port been integrated back into the zodb project? If not, it would be interesting to know the reasons (zodb project maintainers not interested, barriers to contribution are too high, port is not 100% complete and no-one willing to take it over and complete it...?) Paul. From g.brandl at gmx.net Tue Nov 3 18:44:58 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 03 Nov 2009 18:44:58 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> Message-ID: Raymond Hettinger schrieb: > [Michael Foord] >> What will it take to *start* the port? (Or is it already underway?) For >> many projects I fear that it is only the impending obsolescence (real >> rather than theoretical) of Python 2 that will convince projects to port. > > FWIW, I do not buy into the several premises that have arisen in this thread: > > > * For 3.x to succeed, something bad has to happen to 2.x. (which in my book > translates to intentionally harming 2.x users, either through neglect or > force, in order to bait them into switching to 3.x). > > * Core developers will are losing time supporting 2.x. (backports take some > time but it is small in comparison to getting a patch to work in the first > place -- if anyone can comment on this assertion, it is the people who have > been doing it already (such as AP, MD, BP, GB, and myself)). I agree. However I wouldn't want to lose the amount of work I've put into 2.7. While reviewing the 2.6 "svnmerge avail" output, I also got the impression that a *significant* number of fixes were not backported to 2.6. I don't have the time to go through a 300+ kb change log and find out what to backport, just based on commit messages that are not always clear on whether a fix or feature was added. So if we kill 2.7, we at least need to make sure no real improvements that should have been in 2.x are lost. Georg From solipsis at pitrou.net Tue Nov 3 18:47:01 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 3 Nov 2009 17:47:01 +0000 (UTC) Subject: [Python-Dev] PEP 3003 - Python Language Moratorium References: Message-ID: Guido van Rossum python.org> writes: > > The PEP tries to spell out some gray areas but I'm sure there will be > others; that's life. Do note that the PEP proposes to be *retroactive* > back to the 3.1 release, i.e. the "frozen" version of the language is > the state in which it was released as 3.1. I've done a quick check through the NEWS file and it seems no post-3.1 change would be affected by this PEP. I'm for the PEP myself. Antoine. From charlesc-lists-python-dev at pyropus.ca Tue Nov 3 18:41:32 2009 From: charlesc-lists-python-dev at pyropus.ca (Charles Cazabon) Date: Tue, 3 Nov 2009 11:41:32 -0600 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> Message-ID: <20091103174131.GA10645@pyropus.ca> Arc Riley wrote: > +1 on ending with 2.6. That seems precipitous. > I'm the maintainer of 3rd party Python 3-only packages and have ported a few > modules that we needed with some help from the 2to3 tool. It's really not a > big deal - and Py3 really is a massive improvement. > > The main thing holding back the community are lazy and/or obstinate package > maintainers. I wouldn't say that. For instance, I'm just starting a refactoring that will result in getmail v.5, but I need to target Python 2.5 and up, so there's essentially no way the code will run in Python 3.x (as another list member posted). Why do I need to target Python 2.5? Because that's the most current default version of Python shipped in Debian stable and various other distributions that don't stay on the bleeding edge. getmail v.4 targeted Python 2.3 and up, getmail v.3 targeted Python 1.5.2 and up. I may be able to target Python 2.6 in a year or two, at which point Python 3 compatibility becomes a reasonable goal. Saying "2.6 is the last Python 2.x" seems to me to be a death sentence for Python 3. People will stay with 2.x much longer than you seem to want them to, and making it harder for them to upgrade will only hurt Python 3. Charles -- ----------------------------------------------------------------------- Charles Cazabon GPL'ed software available at: http://pyropus.ca/software/ ----------------------------------------------------------------------- From martin at v.loewis.de Tue Nov 3 18:51:33 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Nov 2009 18:51:33 +0100 Subject: [Python-Dev] language summit topic: issue tracker In-Reply-To: <52dc1c820911030029m72b10895xae947b60068ad0c4@mail.gmail.com> References: <4AE17C12.1080007@gmail.com> <52dc1c820911030029m72b10895xae947b60068ad0c4@mail.gmail.com> Message-ID: <4AF06DA5.6080508@v.loewis.de> > yet another feature request or two to be lost to a mailing list thread > along those lines: > > Maintainer or not i'd like to be able to setup triggers so that i'm > automatically cc'ed on any bugs matching a simple search query i > specify. Please add that to the meta tracker (if you really want it, rather than just losing it on the mailing list :-) Implementing it would be quite involved, I think. It should probably go into the saved query feature. Evaluating them on every change would might be expensive, so doing so only regularly (e.g. hourly) would be ok? > The email sent out to people cc'ed when a new bug is opened and > unassigned should have a simple links in it when cc'ed to someone who > can be assigned bugs: 'Assign to me' that if followed will assign > that bug to them without requiring a login. Unfortunately, this is now defeated by the fear of XSS attacks. If such a link was possible, it would be also possible to embed it into a XSS attack, right? It would be possible to protect the link with a token, but then, efficient token validation might be tricky. So this would also need to go into the meta tracker. If you are really interested in these, it would be best to add them as feature requests to roundup itself (since that really is the place where they should be provided): http://issues.roundup-tracker.org/ [but then, roundup could also use more contributors] Regards, Martin From martin at v.loewis.de Tue Nov 3 18:53:36 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Nov 2009 18:53:36 +0100 Subject: [Python-Dev] nonlocal keyword in 2.x? In-Reply-To: <4AEFF3A3.3020103@egenix.com> References: <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1> <4AEE9862.8010201@voidspace.org.uk> <5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1> <4AEF4C53.7090902@v.loewis.de> <4AEFF3A3.3020103@egenix.com> Message-ID: <4AF06E20.7050307@v.loewis.de> >> 2.7 has an up-to-date backport of the C IO lib (as well as the memoryview >> object), which means it is better for people wanting to ease transition to 3.x. >> >> But of course, as Martin said, few people will want to support 2.7 only and not >> 2.6. > > Since 2.7 will be closer to 3.2 than 2.6, the result will more likely > be: many people will want to support 2.7 and 3.x - which is really > what we should be after. Supporting 2.7 doesn't mean that you have to use the nonlocal keyword (nor does support for 3.2 require that keyword). So I'm not sure how the backport of a feature to 2.x could encourage people to forward-port to 3.x. Regards, Martin From martin at v.loewis.de Tue Nov 3 18:56:11 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Nov 2009 18:56:11 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> <3B364FE0-7616-41F8-82C7-528EA1282E40@gmail.com> Message-ID: <4AF06EBB.8060100@v.loewis.de> > Wasn't Django ported to Py3k by MvL as a demo? The problem seems to be > more to port the Django *community* to Py3k... Exactly so. At the last Pycon, we then agreed that I would get a branch in the Django code repository, but then progress stalled due to inactivity on boths sides. Regards, Martin From zookog at gmail.com Tue Nov 3 18:59:42 2009 From: zookog at gmail.com (Zooko O'Whielacronx) Date: Tue, 3 Nov 2009 10:59:42 -0700 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> Message-ID: Folks: I really don't want to make anyone feel bad or to criticize, but I should mention that I have no plans to use Python 3 or to support Python 3. My best guess at this time is that the current projects that I'm involved in will still require Python 2 for the forseeable future (let's say 5 years. I can see 5 years into the future.), and that as I start new projects I will probably try out interesting alternative programming languages like Haskell, Newspeak [1], Jacaranda [2], and other new things that appear in the coming years. Of course, I reserve the right to change my mind and start using and supporting Python 3. That might happen if there is some combination of: 1. my users start asking for it (no-one has yet), 2. my dependencies start providing it (I use Python because it has Twisted. Twisted requires Python 2.), 3. it becomes more possible for me to write code which is still Python-2-compatible and also is more and more close to being Python-3-compatible. By the way, one significant detail which makes Python 3 less interesting to me is [3]. Those two languages that I mentioned -- Newspeak and Jacaranda -- both have object-capability nature. If that issue in [3] were fixed then Python 3 would join Python 2 as a language that can (with the CapPython extension) have object-capability nature. Regards, Zooko [1] http://bracha.org/Site/Newspeak.html [2] http://jacaranda.org [3] http://lackingrhoticity.blogspot.com/2008/09/cappython-unbound-methods-and-python-30.html --- Your cloud storage provider does not need access to your data. Tahoe-LAFS -- http://allmydata.org From flashk at gmail.com Tue Nov 3 19:01:22 2009 From: flashk at gmail.com (Farshid Lashkari) Date: Tue, 3 Nov 2009 10:01:22 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF06A5F.6090501@v.loewis.de> References: <4AEFD9AE.7090104@molden.no> <4AF06A5F.6090501@v.loewis.de> Message-ID: <978d1eac0911031001o40d69d9evfae42e7921fbf00a@mail.gmail.com> On Tue, Nov 3, 2009 at 9:37 AM, "Martin v. L?wis" wrote: > > (and no, adding things like nonlocal to 2.7 doesn't making porting of > a real application or library any easier, since the existing application > or library simply doesn't use that keyword. In fact, no change to 2.x > can reasonably simplify porting - only changes to 3.x might - except > for changes to 2to3, which can simplify porting a lot. But 2to3 should > be run under 3.x, IMO.) > I agree that adding these features doesn't really help much for people that are porting from 2.x to 3.x. However, I can see it being useful for package developers who want to support both 3.x and 2.x. The more these two versions have in common the easier it will be to support them both. Obviously, this means the package might not work with 2.6 and earlier. But some users out there might be in some situation where they cannot upgrade to 3.x but can jump from 2.4/5/6 to 2.7. -Farshid -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Tue Nov 3 19:03:25 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 3 Nov 2009 18:03:25 +0000 (UTC) Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? References: <4AEFC887.6020801@v.loewis.de> <19184.19195.484852.432070@montanaro.dyndns.org> Message-ID: Georg Brandl gmx.net> writes: > > skip pobox.com schrieb: > > Martin> And if *2.6* becomes the last of the 2.x series? > > > > With all due respect, I don't think you can make that decision now. The > > time to have stated 2.6 would be the end of the 2.x line was when 2.6 was > > released. At that point instead of opening up the trunk for changes you > > would have closed it off or merged the py3k branch to trunk. 2.6.0 was > > released over a year ago and there has been no effort to suppress bug fix or > > feature additions to trunk since then. If you call 2.6 "the end of 2.x" > > you'll have wasted a year of work on 2.7 with about a month to go before the > > first 2.7 alpha release. > > +1. +1 as well. Besides, it is much better communication to release 2.7 and say up front that it will be the last major release in the 2.x line. Announcing that there won't be a 2.7, however, would give the impression of poor planning. Regards Antoine. From martin at v.loewis.de Tue Nov 3 19:09:29 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Nov 2009 19:09:29 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <79990c6b0911030944r26fd668ev37e56a4a7d79559b@mail.gmail.com> References: <4AEFC887.6020801@v.loewis.de> <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com> <4AF0698C.2030706@v.loewis.de> <79990c6b0911030944r26fd668ev37e56a4a7d79559b@mail.gmail.com> Message-ID: <4AF071D9.5000900@v.loewis.de> >> You may also want to take a look at my ZODB port: >> >> https://www.dcl.hpi.uni-potsdam.de/home/loewis/zodb/ > > Has that port been integrated back into the zodb project? Only partially so. > If not, it > would be interesting to know the reasons (zodb project maintainers not > interested, barriers to contribution are too high, port is not 100% > complete and no-one willing to take it over and complete it...?) ZODB is a lot of layers, really (zope.interfaces, zope.lockfile, zope.proxy, ZConfig, etc..., finally ZODB). I failed to port buildout and zope.testing. With zope.testing not ported, it is not easy to actually run the test suites for all of these; this is where the integration stalled. In any case, I only did the port in September; at the DZUG sprint, people from gocept then started looking into integrating it. We didn't get through at the sprint, and I think there has been little progress since. One specific issue was about specifying the root of the zope.interfaces hierarchy. In current zope.interfaces, None could be used (implicitly) instead of Interface, to indicate the root interface type. In 3.x, this would break tests, because you can't sort a list of interfaces anymore if that list also contains None. So people decided that zope.interfaces needs to be changed to disallow None, and that was a significant change that has not yet been fully understood. Regards, Martin From eric at trueblade.com Tue Nov 3 19:13:18 2009 From: eric at trueblade.com (Eric Smith) Date: Tue, 03 Nov 2009 13:13:18 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF06A5F.6090501@v.loewis.de> References: <4AEFD9AE.7090104@molden.no> <4AF06A5F.6090501@v.loewis.de> Message-ID: <4AF072BE.2010603@trueblade.com> Martin v. L?wis wrote: >> A Python 3 version of NumPy might be enough of an improvement to bring >> *more* scientists and engineers onboard if the Python 3.x version shows >> what great productivity gains are to be had with Python 3.x over 2.x. > > I would be really surprised if 2.7 would simplify porting to 3.x. How > could that possibly work? The only things I can think of that would go into this category are features like: - PEP 3118, revised buffer protocol. If the buffer API that numpy uses is not present in py3k (I'm no expert on the subject, but it seems this way from a recent thread on python-dev), then if they could move to PEP 3118 in 2.7 their migration to 3.x would be easier - short float repr. This would remove a class of hard-to-find problems from a migration from 2.7 to 3.x. - Maybe io, but I don't know enough about it to say. But I definitely agree that backporting language features new to 3.x don't make it easier. Examples are nonlocal and required keyword args. Eric. From ssteinerx at gmail.com Tue Nov 3 19:15:18 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Tue, 3 Nov 2009 13:15:18 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> Message-ID: <3FE75D51-199C-4221-9B38-94FD2FE9BC4E@gmail.com> On Nov 3, 2009, at 12:28 PM, Arc Riley wrote: > > The main thing holding back the community are lazy and/or obstinate > package maintainers. If they spent half the time they've put into > complaining about Py3 into actually working to upgrade their code > they'd be done now. That's an inflammatory, defamatory, unsubstantiated, hyperbolic, sweeping overgeneralization. S -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Tue Nov 3 19:15:38 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Nov 2009 19:15:38 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <20091103174131.GA10645@pyropus.ca> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> <20091103174131.GA10645@pyropus.ca> Message-ID: <4AF0734A.3080705@v.loewis.de> > I wouldn't say that. For instance, I'm just starting a refactoring that will > result in getmail v.5, but I need to target Python 2.5 and up, so there's > essentially no way the code will run in Python 3.x (as another list member > posted). That's a common myth. It is very well possible, using 2to3. You don't have to wait until you can drop 2.5 to start supporting 3.x, out of a single code base. > Why do I need to target Python 2.5? Because that's the most current default > version of Python shipped in Debian stable and various other distributions > that don't stay on the bleeding edge. getmail v.4 targeted Python 2.3 and up, > getmail v.3 targeted Python 1.5.2 and up. I may be able to target Python 2.6 > in a year or two, at which point Python 3 compatibility becomes a reasonable > goal. Are you saying that it doesn't *run* on 2.6? Why? (not sure what you mean by "targetting") Regards, Martin From g.brandl at gmx.net Tue Nov 3 19:16:14 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 03 Nov 2009 18:16:14 +0000 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF06AE9.2010706@v.loewis.de> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com> <4AF06AE9.2010706@v.loewis.de> Message-ID: Martin v. L?wis schrieb: >> It's pretty easy to make Python source that works under 2.6 and 3.x. >> It's basically impossible to make Python source that works under 2.4/2.5 >> and 3.x. You may be able to write code that works under 2.4/2.5 and >> works cleanly with 2to3 to produce 3.x code. I haven't tried that >> route, though in theory it should work. All you really need is syntax >> compatibility. > > I have tried that route for a number of projects, and I think it works > really well. It is also supported by distribute. I've ported both Docutils and Pygments using that strategy, and I'll gladly agree to that. Georg From ssteinerx at gmail.com Tue Nov 3 19:18:46 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Tue, 3 Nov 2009 13:18:46 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF06BC1.6060004@v.loewis.de> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> <3B364FE0-7616-41F8-82C7-528EA1282E40@gmail.com> <4AF06BC1.6060004@v.loewis.de> Message-ID: <4A0DF493-B46D-436A-AEF3-BF1EC8DFE0A8@gmail.com> On Nov 3, 2009, at 12:43 PM, Martin v. L?wis wrote: >>> I'm not ready for that yet. I think there's plenty of time before we >>> have to agree to such a bleak view. In the mean time let's do >>> something practical like help NumPy port to Py3k. >> >> Or, for example, Django... > > See > > http://wiki.python.org/moin/PortingDjangoTo3k Well, that's certainly a start. I guess the logical question is: Now what? S From ssteinerx at gmail.com Tue Nov 3 19:19:58 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Tue, 3 Nov 2009 13:19:58 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> Message-ID: <5877F924-3E2B-4B18-BE55-4543C08BA149@gmail.com> On Nov 3, 2009, at 12:35 PM, Antoine Pitrou wrote: > Arc Riley gmail.com> writes: >> >> +1 on ending with 2.6.I'm the maintainer of 3rd party Python 3-only >> packages >> and have ported a few modules that we needed with some help from >> the 2to3 >> tool. It's really not a big deal - and Py3 really is a massive >> improvement. >> The main thing holding back the community are lazy and/or obstinate >> package >> maintainers. If they spent half the time they've put into >> complaining about >> Py3 into actually working to upgrade their code they'd be done now. > > One thing you could do is explain (do you have a blog?) how Py3 is a > massive > improvement for you as a developer and package maintainer. > We core developers obviously agree that py3k is better than 2.x, but > the same > opinion coming from a third-party developer would carry a different > weight. Maybe I haven't been looking, but has anyone collected the "Here's why 3.x is better and here's how it saved my bacon on project XYZ" stories? S From barry at python.org Tue Nov 3 19:20:18 2009 From: barry at python.org (Barry Warsaw) Date: Tue, 3 Nov 2009 13:20:18 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFC887.6020801@v.loewis.de> <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> Message-ID: <22B9056E-E74A-40C2-B7B6-BC75FD6D8F4E@python.org> On Nov 3, 2009, at 10:54 AM, Guido van Rossum wrote: > Ouch. sets. I'm guessing you are referring to code that was still > using the pre-2.4 sets.py module? Yes, that must have been painful. Indeed. :) What's nice though is that these changes could be made for the Python 2.5 branch and didn't have to wait until 2.6 (e.g. sha/md5 - > hashlib, sets -> builtin-set). What was actually most painful about this were all the tests that were checking stdout/stderr for clean subprocess runs. The DeprecationWarnings that were produced killed us for a long while because so many of those tests failed. It's actually quite difficult to suppress DeprecationWarnings across the board, especially when you have lots of subprocess that call third party code (where most of the deprecated code lived in, and which we couldn't change). We added filterwarnings where we could, and called subprocesses with -W every place we thought we needed to but we never did kill them all off. I think a $PYTHONWARNING environment variable might have helped here. >> Given another week or so I think we would have made it to Python >> 2.6, but >> I'm not at all confident that that would have been a good enough >> platform to >> attempt an upgrade to Python 3, even if all of our very numerous >> large >> dependencies were available for Python 3. Maybe it wouldn't be so >> bad, but >> my recent experience informs me that I'm probably being too >> optimistic >> rather than too pessimistic. > > There are two stages of porting to 2.6 you have to go through. The > first one, which you would probably have reached in that week, is > running on 2.6 period. You can then release your code for the benefit > of others wanting to use it on 2.6. Yep. > But the second stage, which can take much more time (depending on the > state of your code base) is to run on 2.6 *free of warnings with the > -3 flag on*. You pretty much have to consider this a separate port, > and it is here where you do much of the prep work for 3.x (at least > for Python code -- for C extensions it's not so clear). The good news > is that you can claim 2.6 support before you've even started this > stage; the other good news is that doing this right will really help > you prepare for 3.x. And in most cases you can even (with some effort) > maintain compatibility with 2.5 or even 2.4 -- though you may have to > work around some things like the md5 and sha warnings. > > The bad news is that this stage may well take more time than porting > from 2.4 to 2.6. We have another window for this (since it's all open source I don't mind talking about it) when the next version of Ubuntu comes out. Running with the -3 is a great idea, and something I will definitely try after finishing the straight 2.6 port. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From martin at v.loewis.de Tue Nov 3 19:20:49 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Nov 2009 19:20:49 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: <4AF07481.3080106@v.loewis.de> > * General language semantics > The language operates as-is with only specific exemptions (see > below). Would PEP 382 (namespace packages) fall under the moratorium? Regards, Martin From martin at v.loewis.de Tue Nov 3 19:25:14 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Nov 2009 19:25:14 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF072BE.2010603@trueblade.com> References: <4AEFD9AE.7090104@molden.no> <4AF06A5F.6090501@v.loewis.de> <4AF072BE.2010603@trueblade.com> Message-ID: <4AF0758A.6070108@v.loewis.de> >> I would be really surprised if 2.7 would simplify porting to 3.x. How >> could that possibly work? > > The only things I can think of that would go into this category are > features like: > - PEP 3118, revised buffer protocol. If the buffer API that numpy > uses is not present in py3k (I'm no expert on the subject, but > it seems this way from a recent thread on python-dev), then if > they could move to PEP 3118 in 2.7 their migration to 3.x would > be easier But only if NumPy would drop support for 2.x, for x < 7, right? That would probably be many years in the future. Regards, Martin From martin at v.loewis.de Tue Nov 3 19:25:55 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Nov 2009 19:25:55 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4A0DF493-B46D-436A-AEF3-BF1EC8DFE0A8@gmail.com> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> <3B364FE0-7616-41F8-82C7-528EA1282E40@gmail.com> <4AF06BC1.6060004@v.loewis.de> <4A0DF493-B46D-436A-AEF3-BF1EC8DFE0A8@gmail.com> Message-ID: <4AF075B3.2060301@v.loewis.de> >>>> I'm not ready for that yet. I think there's plenty of time before we >>>> have to agree to such a bleak view. In the mean time let's do >>>> something practical like help NumPy port to Py3k. >>> >>> Or, for example, Django... >> >> See >> >> http://wiki.python.org/moin/PortingDjangoTo3k > > Well, that's certainly a start. > > I guess the logical question is: Now what? Use it, and report bugs. Regards, Martin From eric at trueblade.com Tue Nov 3 19:26:34 2009 From: eric at trueblade.com (Eric Smith) Date: Tue, 03 Nov 2009 13:26:34 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF0758A.6070108@v.loewis.de> References: <4AEFD9AE.7090104@molden.no> <4AF06A5F.6090501@v.loewis.de> <4AF072BE.2010603@trueblade.com> <4AF0758A.6070108@v.loewis.de> Message-ID: <4AF075DA.5080607@trueblade.com> Martin v. L?wis wrote: >>> I would be really surprised if 2.7 would simplify porting to 3.x. How >>> could that possibly work? >> The only things I can think of that would go into this category are >> features like: >> - PEP 3118, revised buffer protocol. If the buffer API that numpy >> uses is not present in py3k (I'm no expert on the subject, but >> it seems this way from a recent thread on python-dev), then if >> they could move to PEP 3118 in 2.7 their migration to 3.x would >> be easier > > But only if NumPy would drop support for 2.x, for x < 7, right? > That would probably be many years in the future. Right. But that might be their best migration strategy: wait for 2.7 to be available everywhere, port to 2.7, then port to 3.4 (or whatever the current version of 3.x would be, then). From guido at python.org Tue Nov 3 19:39:56 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Nov 2009 10:39:56 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF06A5F.6090501@v.loewis.de> References: <4AEFD9AE.7090104@molden.no> <4AF06A5F.6090501@v.loewis.de> Message-ID: On Tue, Nov 3, 2009 at 9:37 AM, "Martin v. L?wis" wrote: > (and no, adding things like nonlocal to 2.7 doesn't making porting of > a real application or library any easier, since the existing application > or library simply doesn't use that keyword. Agreed. > In fact, no change to 2.x > can reasonably simplify porting - only changes to 3.x might - except > for changes to 2to3, which can simplify porting a lot. But 2to3 should > be run under 3.x, IMO.) Disagreed. Better "-3" warnings could make porting easier. (Not just more warnings -- "better" might mean fewer false positives for warnings already issued.) FWIW, it doesn't sound like killing 2.7 is a productive thing to do. However making 2.7 the end of the line (though with indefinite bugfix releases) might be. (Indefinite != infinite.) -- --Guido van Rossum (python.org/~guido) From g.brandl at gmx.net Tue Nov 3 19:40:56 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 03 Nov 2009 18:40:56 +0000 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFC887.6020801@v.loewis.de> <19184.19195.484852.432070@montanaro.dyndns.org> Message-ID: Antoine Pitrou schrieb: > Georg Brandl gmx.net> writes: >> >> skip pobox.com schrieb: >> > Martin> And if *2.6* becomes the last of the 2.x series? >> > >> > With all due respect, I don't think you can make that decision now. The >> > time to have stated 2.6 would be the end of the 2.x line was when 2.6 was >> > released. At that point instead of opening up the trunk for changes you >> > would have closed it off or merged the py3k branch to trunk. 2.6.0 was >> > released over a year ago and there has been no effort to suppress bug fix or >> > feature additions to trunk since then. If you call 2.6 "the end of 2.x" >> > you'll have wasted a year of work on 2.7 with about a month to go before the >> > first 2.7 alpha release. >> >> +1. > > +1 as well. Besides, it is much better communication to release 2.7 and say up > front that it will be the last major release in the 2.x line. Announcing that > there won't be a 2.7, however, would give the impression of poor planning. And of a rush to get rid of 2.x. Georg From g.brandl at gmx.net Tue Nov 3 19:42:12 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 03 Nov 2009 18:42:12 +0000 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <3FE75D51-199C-4221-9B38-94FD2FE9BC4E@gmail.com> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> <3FE75D51-199C-4221-9B38-94FD2FE9BC4E@gmail.com> Message-ID: ssteinerX at gmail.com schrieb: > > On Nov 3, 2009, at 12:28 PM, Arc Riley wrote: >> >> The main thing holding back the community are lazy and/or obstinate >> package maintainers. If they spent half the time they've put into >> complaining about Py3 into actually working to upgrade their code >> they'd be done now. > > That's an inflammatory, defamatory, unsubstantiated, hyperbolic, > sweeping overgeneralization. I know a few maintainers, and I have no problem seeing how Arc came to that conclusion. Georg From guido at python.org Tue Nov 3 19:44:58 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Nov 2009 10:44:58 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <4AF07481.3080106@v.loewis.de> References: <4AF07481.3080106@v.loewis.de> Message-ID: On Tue, Nov 3, 2009 at 10:20 AM, "Martin v. L?wis" wrote: >> * General language semantics >> ? ? The language operates as-is with only specific exemptions (see >> ? ? below). > > Would PEP 382 (namespace packages) fall under the moratorium? Import semantics are a bit of a gray area. However I think the moratorium shouldn't be interpreted to prohibit this. I think there was a question on python-ideas about import semantics, and my response was something like "import semantics vary between Python implementations anyway." I'll add this to the PEP. -- --Guido van Rossum (python.org/~guido) From charlesc-lists-python-dev at pyropus.ca Tue Nov 3 19:56:41 2009 From: charlesc-lists-python-dev at pyropus.ca (Charles Cazabon) Date: Tue, 3 Nov 2009 12:56:41 -0600 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF0734A.3080705@v.loewis.de> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> <20091103174131.GA10645@pyropus.ca> <4AF0734A.3080705@v.loewis.de> Message-ID: <20091103185641.GA13969@pyropus.ca> "Martin v. L?wis" wrote: > > I wouldn't say that. For instance, I'm just starting a refactoring that will > > result in getmail v.5, but I need to target Python 2.5 and up, so there's > > essentially no way the code will run in Python 3.x (as another list member > > posted). > > That's a common myth. It is very well possible, using 2to3. You don't > have to wait until you can drop 2.5 to start supporting 3.x, out of a > single code base. I haven't tried this, but I was relying on Daniel Stutzbach's opinion posted here earlier: http://mail.python.org/pipermail/python-dev/2009-November/093608.html It's pretty easy to make Python source that works under 2.6 and 3.x. It's basically impossible to make Python source that works under 2.4/2.5 and 3.x. > > Why do I need to target Python 2.5? Because that's the most current > > default version of Python shipped in Debian stable and various other > > distributions that don't stay on the bleeding edge. > > Are you saying that it doesn't *run* on 2.6? No. getmail v.4 runs fine on Python 2.3.4 through 2.6.x; getmail's code has always been pretty forward-compatible. > Why? (not sure what you mean by "targetting") By target, I mean backwards compatibility -- the minimum version of Python which is required to run getmail. getmail v.4 came out of beta about five years ago targetting Python 2.3 and higher, and 2.3 was too bleeding-edge for many users -- it wasn't shipped by many Linux distributions for a long time after getmail v.4 was released. Debian *still* shipps getmail v3 (which supports back to Python 1.5.2) today, although they also ship v4. getmail v.5 will be released in a month or three. And many of its users will still have Python 2.5, so that's what getmail has to run on. Perhaps Daniel's comment is incorrect (I have no evidence either way), but if it is true that having a single getmail codebase run on Python 2.5 and Python 3.x is "basically impossible", then I won't be too concerned about 3.x for a while yet. I've been an avid Python user and promoter since 1.2, but saying "drop Python 2.x and switch to 3 now" is simply not realistic in any of the environments in which I use Python daily. My $0.02. Charles -- ----------------------------------------------------------------------- Charles Cazabon GPL'ed software available at: http://pyropus.ca/software/ ----------------------------------------------------------------------- From guido at python.org Tue Nov 3 19:56:44 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Nov 2009 10:56:44 -0800 Subject: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it In-Reply-To: <9d153b7c0911030020n336fd15bobeea4ea713cd8b23@mail.gmail.com> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <20091102052900.GA25160@cskk.homeip.net> <4AEF5AFA.5060806@canterbury.ac.nz> <9d153b7c0911030020n336fd15bobeea4ea713cd8b23@mail.gmail.com> Message-ID: On Tue, Nov 3, 2009 at 12:20 AM, Yuvgoog Greenle wrote: > On Tue, Nov 3, 2009 at 12:19 AM, Greg Ewing > wrote: >> >> Cameron Simpson wrote: >> >>> Personally, I'm for the iteration spec in a lot of ways. >>> >>> Firstly, a .get()/.pick() that always returns the same element feels >>> horrible. Is there anyone here who _likes_ it? >> > > State might cause people to use this to iterate which would be just plain > wrong. The 2 things I have a bad feeling about is: > 1. random.choice could be a pythonic alternative to what some have mentioned > here but it doesn't work on sets. The following code raises TypeError: 'set' > object does not support indexing: > ?? ?import random > ?? ?random.choice(set([1,2,3,4,5,6])) There is a huge difference between picking a random element of a set and picking an arbitrary one. Algorithms that need randomness *must* use the random generator (and because the hash table implementation doesn't provide O(1) access they will have to use a list first). Algorithms that don't need randomness should not be forced to pay for the (considerable!) cost of calling the random number generator, and must accept that the element selected may be predictable. > this is kinda ironic:?http://en.wikipedia.org/wiki/Axiom_of_choice Also irrelevant. That Axiom is only interesting for infinite sets, which Python does not support (at least not using the set builtin -- you can of course write your own symbolic algebra package in Python that can be used to represent certain infinite sets, though you still won't be able to iterate over all of their elements :-). > 2. If I store objects in a set and modify their attributes I have no O(1) > way of getting the objects back if I stumble upon an equivalent object. In > cases like that I'd have to either iterate over the set or use a dict with > key == value. I have a feeling the "get" or "peek" api could cater to this > need. A use case for this could be an implementation of a cookie jar with a > set of cookies where equivalence is defined by the name and domain. Sets are what they are. If they don't fit your requirements, don't use them. Don't get fooled by the name -- a dict also isn't a very good data structure to implement an actual Chinese-English dictionary, for example. :-) -- --Guido van Rossum (python.org/~guido) From mal at egenix.com Tue Nov 3 20:15:54 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 03 Nov 2009 20:15:54 +0100 Subject: [Python-Dev] nonlocal keyword in 2.x? In-Reply-To: <4AF06E20.7050307@v.loewis.de> References: <02A812F3B69E46D180B45133D52D1F2B@RaymondLaptop1> <4AEE9862.8010201@voidspace.org.uk> <5C4388816CE0428FB27EA69CE77CFCC0@RaymondLaptop1> <4AEF4C53.7090902@v.loewis.de> <4AEFF3A3.3020103@egenix.com> <4AF06E20.7050307@v.loewis.de> Message-ID: <4AF0816A.706@egenix.com> "Martin v. L?wis" wrote: >>> 2.7 has an up-to-date backport of the C IO lib (as well as the memoryview >>> object), which means it is better for people wanting to ease transition to 3.x. >>> >>> But of course, as Martin said, few people will want to support 2.7 only and not >>> 2.6. >> >> Since 2.7 will be closer to 3.2 than 2.6, the result will more likely >> be: many people will want to support 2.7 and 3.x - which is really >> what we should be after. > > Supporting 2.7 doesn't mean that you have to use the nonlocal keyword > (nor does support for 3.2 require that keyword). So I'm not sure how > the backport of a feature to 2.x could encourage people to forward-port > to 3.x. True. This particular backport is not all that useful - it is well possible to write code that doesn't need the feature, both in 2.7 and 3.2. However, if a 3.2 feature is required for code to work out of the box without changes, then I'd regard such a feature as potential backport candidate. memoryviews are a good example. In 3.x the buffer object is gone and memoryviews are the new replacement for it. Code written to support both 2.7 and 3.2 would need to have access to these memoryviews in order to avoid having to write special support for buffer objects (for 2.7) and memoryviews (for 3.x). By backporting memoryviews to 2.7, users no longer have to write support code for buffer objects and that makes things easier for them. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 03 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From brett at python.org Tue Nov 3 20:21:06 2009 From: brett at python.org (Brett Cannon) Date: Tue, 3 Nov 2009 11:21:06 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: On Tue, Nov 3, 2009 at 09:35, Guido van Rossum wrote: > I've checked draft (!) PEP 3003, "Python Language Moratorium", into > SVN. As authors I've listed Jesse, Brett and myself. > > On python-ideas the moratorium idea got fairly positive responses > (more positive than I'd expected, in fact) but I'm bracing myself for > fierce discussion here on python-dev. It's important to me that if if > this is accepted it is a "rough consensus" decision (working code we > already have plenty of :-), not something enforced by a vocal minority > or an influential individual such as myself. If there's too much > opposition I'll withdraw the PEP so as not to waste everybody's time > with a fruitless discussion. > Are you going to gauge it roughly from python-dev feedback, or should we take a more formal vote on python-committers once the PEP has settled? > The PEP tries to spell out some gray areas but I'm sure there will be > others; that's life. Do note that the PEP proposes to be *retroactive* > back to the 3.1 release, i.e. the "frozen" version of the language is > the state in which it was released as 3.1. I'm obviously +1 for this. I think one thing to decide is how long the moratorium is in effect. As of right now the PEP's abstract says for "at least two years", which taking into account the release schedule Benjamin is proposing and assuming acceptance at the end of the year puts us at roughly the release of Python 3.3. Should we just say this will be in affect until development is open for Python 3.4? -Brett From martin at v.loewis.de Tue Nov 3 20:22:41 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Nov 2009 20:22:41 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <20091103185641.GA13969@pyropus.ca> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> <20091103174131.GA10645@pyropus.ca> <4AF0734A.3080705@v.loewis.de> <20091103185641.GA13969@pyropus.ca> Message-ID: <4AF08301.6070000@v.loewis.de> >>> I wouldn't say that. For instance, I'm just starting a refactoring that will >>> result in getmail v.5, but I need to target Python 2.5 and up, so there's >>> essentially no way the code will run in Python 3.x (as another list member >>> posted). >> That's a common myth. It is very well possible, using 2to3. You don't >> have to wait until you can drop 2.5 to start supporting 3.x, out of a >> single code base. > > I haven't tried this, but I was relying on Daniel Stutzbach's opinion posted > here earlier: > http://mail.python.org/pipermail/python-dev/2009-November/093608.html > > It's pretty easy to make Python source that works under 2.6 and 3.x. It's > basically impossible to make Python source that works under 2.4/2.5 and > 3.x. But that's a completely different statement. "make code that works on both 2.x and 3.x" is *entirely* different from "make code written for 2.x also run on 3.x". The latter is less constrained: to run it on 3.x, you can convert it to 3.x first, using 2to3. So it would not be the *same* code that runs, but you can have single source code for both 2.x and 3.x - only that what you run is not literally the source code. > getmail v.5 will be released in a month or three. And many of its users will > still have Python 2.5, so that's what getmail has to run on. Perhaps Daniel's > comment is incorrect (I have no evidence either way), but if it is true that > having a single getmail codebase run on Python 2.5 and Python 3.x is > "basically impossible", then I won't be too concerned about 3.x for a while > yet. No no no. Daniels comment is correct, in that you can't run the code *unmodified*. However, it is well possible to have a single code base for both 2.x and 3.x - you just don't *run* the code on 3.x (but convert it first, e.g. at installation time). > I've been an avid Python user and promoter since 1.2, but saying "drop Python > 2.x and switch to 3 now" is simply not realistic in any of the environments in > which I use Python daily. And I wouldn't say that. Instead, I say "support both 2.x and 3.x from a single code base". That approach can work for supporting everything from 2.3 to 3.2 (perhaps even from 2.0 to 3.x). Regards, Martin From mal at egenix.com Tue Nov 3 20:23:49 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 03 Nov 2009 20:23:49 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: <4AF08345.6040603@egenix.com> Guido van Rossum wrote: > I've checked draft (!) PEP 3003, "Python Language Moratorium", into > SVN. As authors I've listed Jesse, Brett and myself. > > On python-ideas the moratorium idea got fairly positive responses > (more positive than I'd expected, in fact) but I'm bracing myself for > fierce discussion here on python-dev. It's important to me that if if > this is accepted it is a "rough consensus" decision (working code we > already have plenty of :-), not something enforced by a vocal minority > or an influential individual such as myself. If there's too much > opposition I'll withdraw the PEP so as not to waste everybody's time > with a fruitless discussion. > > The PEP tries to spell out some gray areas but I'm sure there will be > others; that's life. Do note that the PEP proposes to be *retroactive* > back to the 3.1 release, i.e. the "frozen" version of the language is > the state in which it was released as 3.1. +1. > This suspension of features is designed to allow non-CPython implementations > to "catch up" to the core implementation of the language, help ease adoption > of Python 3.x, and provide a more stable base for the community. I'd also add "to allow users to catch up"... ;-) One question: There are currently number of patch waiting on the tracker for additional Unicode feature support and it's also likely that we'll want to upgrade to a more recent Unicode version within the next few years. How would such indirect changes be seen under the moratorium ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 03 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From skip at pobox.com Tue Nov 3 20:41:06 2009 From: skip at pobox.com (skip at pobox.com) Date: Tue, 3 Nov 2009 13:41:06 -0600 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: <19184.34642.648427.211546@montanaro.dyndns.org> Guido> I've checked draft (!) PEP 3003, "Python Language Moratorium", Guido> into SVN. LGTM. Skip From brett at python.org Tue Nov 3 20:53:02 2009 From: brett at python.org (Brett Cannon) Date: Tue, 3 Nov 2009 11:53:02 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com> References: <4AEFC887.6020801@v.loewis.de> <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com> Message-ID: On Tue, Nov 3, 2009 at 05:40, David Cournapeau wrote: > On Tue, Nov 3, 2009 at 9:55 PM, Barry Warsaw wrote: > >> >> Then clearly we can't back port features. >> >> I'd like to read some case studies of people who have migrated applications >> from 2.6 to 3.0. > > +1, especially for packages which have a lot of C code: the current > documentation is sparse :) The only helpful reference I have found so > far is an email by MvL concerning psycopg2 port. There used to be a page on the wiki of case studies, but I couldn't find it. But one C-specific case study is the port of Wing's extension modules to support from Python 2.0 to 3.0: http://pythonology.blogspot.com/2009/02/making-code-run-on-python-20-through-30.html . There is also a wiki pages explicitly about porting C code: http://wiki.python.org/moin/cporting and http://wiki.python.org/moin/Py3kExtensionModules . Those latter pages include links to the official porting docs: http://docs.python.org/3.1/howto/cporting.html . And as Georg pointed out there is always the python-porting mailing list: http://mail.python.org/mailman/listinfo/python-porting . I'm afraid there is some FUD going around here, which is understandable since no one wants to burn a ton of time on something that will be difficult or take a lot of time. But I have not heard anyone in this email thread (or anywhere for that matter) say that they tried a port in earnest and it turned out to be difficult. -Brett From barry at python.org Tue Nov 3 20:56:28 2009 From: barry at python.org (Barry Warsaw) Date: Tue, 3 Nov 2009 14:56:28 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: On Nov 3, 2009, at 12:35 PM, Guido van Rossum wrote: > I've checked draft (!) PEP 3003, "Python Language Moratorium", into > SVN. As authors I've listed Jesse, Brett and myself. > > On python-ideas the moratorium idea got fairly positive responses > (more positive than I'd expected, in fact) but I'm bracing myself for > fierce discussion here on python-dev. It's important to me that if if > this is accepted it is a "rough consensus" decision (working code we > already have plenty of :-), not something enforced by a vocal minority > or an influential individual such as myself. If there's too much > opposition I'll withdraw the PEP so as not to waste everybody's time > with a fruitless discussion. > > The PEP tries to spell out some gray areas but I'm sure there will be > others; that's life. Do note that the PEP proposes to be *retroactive* > back to the 3.1 release, i.e. the "frozen" version of the language is > the state in which it was released as 3.1. I think this is a great idea. I'd love to see the energy normally put into evolving the language into making the stdlib really kick ass. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From jnoller at gmail.com Tue Nov 3 21:01:22 2009 From: jnoller at gmail.com (Jesse Noller) Date: Tue, 3 Nov 2009 15:01:22 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: <4222a8490911031201y3a70087dv531255a73241dab3@mail.gmail.com> On Tue, Nov 3, 2009 at 12:35 PM, Guido van Rossum wrote: > I've checked draft (!) PEP 3003, "Python Language Moratorium", into > SVN. As authors I've listed Jesse, Brett and myself. > > On python-ideas the moratorium idea got fairly positive responses > (more positive than I'd expected, in fact) but I'm bracing myself for > fierce discussion here on python-dev. It's important to me that if if > this is accepted it is a "rough consensus" decision (working code we > already have plenty of :-), not something enforced by a vocal minority > or an influential individual such as myself. If there's too much > opposition I'll withdraw the PEP so as not to waste everybody's time > with a fruitless discussion. > > The PEP tries to spell out some gray areas but I'm sure there will be > others; that's life. Do note that the PEP proposes to be *retroactive* > back to the 3.1 release, i.e. the "frozen" version of the language is > the state in which it was released as 3.1. > > -- > --Guido van Rossum (python.org/~guido) > +1 - I'll work on some grammar kinks in my prose asap From guido at python.org Tue Nov 3 21:01:07 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Nov 2009 12:01:07 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: On Tue, Nov 3, 2009 at 11:21 AM, Brett Cannon wrote: > Are you going to gauge it roughly from python-dev feedback, or should > we take a more formal vote on python-committers once the PEP has > settled? I'll not take a formal vote unless the discussion suggests there's a lot of pushback. So far I've seen very little. >> The PEP tries to spell out some gray areas but I'm sure there will be >> others; that's life. Do note that the PEP proposes to be *retroactive* >> back to the 3.1 release, i.e. the "frozen" version of the language is >> the state in which it was released as 3.1. > > I'm obviously +1 for this. > > I think one thing to decide is how long the moratorium is in effect. > As of right now the PEP's abstract says for "at least two years", Yeah, I changed that from Jesse's and your "not to exceed two years". > which taking into account the release schedule Benjamin is proposing > and assuming acceptance at the end of the year puts us at roughly the > release of Python 3.3. Should we just say this will be in affect until > development is open for Python 3.4? I like the actual time limit, because it doesn't allow the backdoor of releasing 3.4 early. I'm fine with saying the moratorium effectively ends after 3.3 is released. -- --Guido van Rossum (python.org/~guido) From guido at python.org Tue Nov 3 21:03:13 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Nov 2009 12:03:13 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <4AF08345.6040603@egenix.com> References: <4AF08345.6040603@egenix.com> Message-ID: On Tue, Nov 3, 2009 at 11:23 AM, M.-A. Lemburg wrote: >> This suspension of features is designed to allow non-CPython implementations >> to "catch up" to the core implementation of the language, help ease adoption >> of Python 3.x, and provide a more stable base for the community. > > I'd also add "to allow users to catch up"... ;-) IMO that's implied by "help ease adoption". > One question: > > There are currently number of patch waiting on the tracker for > additional Unicode feature support and it's also likely that we'll > want to upgrade to a more recent Unicode version within the > next few years. > > How would such indirect changes be seen under the moratorium ? That would fall under the Case-by-Case Exemptions section. "Within the next few years" sounds like it might well wait until the moratorium is ended though. :-) -- --Guido van Rossum (python.org/~guido) From tjreedy at udel.edu Tue Nov 3 21:23:38 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 03 Nov 2009 15:23:38 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> Message-ID: James Y Knight wrote: > If that happens, it's not true that there's *nowhere* to go. A solution > would be to discard 3.x as a failed experiment, take everything that is > useful from it and port it to 2.x, and simply continue development from > the last 2.x release. And from there, features can be deprecated and > then removed a few releases later, as is the usual policy. The once 'usual policy' of removal was changed several years ago to 'defer removals until 3.0' because people wanted a more stable language and claimed that they would prefer to deal with several removals all at once. So old-style classes were kept around long past when they would have been removed under the old 'usual policy'. Ditto for old-style int / int and some others. Or one can simply recognize that 3.0 was the 'few releases later' release of that policy. The other big change was switching to unicode strings from ascii strings with optional unicode string add-on. That was/is/will-be a hassle regardless of when and what name, but necessary for Python to be a modern world language. tjr From facundobatista at gmail.com Tue Nov 3 21:27:24 2009 From: facundobatista at gmail.com (Facundo Batista) Date: Tue, 3 Nov 2009 17:27:24 -0300 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: On Tue, Nov 3, 2009 at 2:35 PM, Guido van Rossum wrote: > fierce discussion here on python-dev. It's important to me that if if > this is accepted it is a "rough consensus" decision (working code we +1 to the PEP. -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From fuzzyman at voidspace.org.uk Tue Nov 3 21:42:09 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 03 Nov 2009 21:42:09 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: <4AF095A1.6040406@voidspace.org.uk> Barry Warsaw wrote: > On Nov 3, 2009, at 12:35 PM, Guido van Rossum wrote: > >> I've checked draft (!) PEP 3003, "Python Language Moratorium", into >> SVN. As authors I've listed Jesse, Brett and myself. >> >> On python-ideas the moratorium idea got fairly positive responses >> (more positive than I'd expected, in fact) but I'm bracing myself for >> fierce discussion here on python-dev. It's important to me that if if >> this is accepted it is a "rough consensus" decision (working code we >> already have plenty of :-), not something enforced by a vocal minority >> or an influential individual such as myself. If there's too much >> opposition I'll withdraw the PEP so as not to waste everybody's time >> with a fruitless discussion. >> >> The PEP tries to spell out some gray areas but I'm sure there will be >> others; that's life. Do note that the PEP proposes to be *retroactive* >> back to the 3.1 release, i.e. the "frozen" version of the language is >> the state in which it was released as 3.1. > > I think this is a great idea. I'd love to see the energy normally put > into evolving the language into making the stdlib really kick ass. > +lots Michael > -Barry > > ------------------------------------------------------------------------ > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ From g.brandl at gmx.net Tue Nov 3 22:00:21 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 03 Nov 2009 21:00:21 +0000 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> Message-ID: Terry Reedy schrieb: > James Y Knight wrote: > >> If that happens, it's not true that there's *nowhere* to go. A solution >> would be to discard 3.x as a failed experiment, take everything that is >> useful from it and port it to 2.x, and simply continue development from >> the last 2.x release. And from there, features can be deprecated and >> then removed a few releases later, as is the usual policy. > > The once 'usual policy' of removal was changed several years ago to > 'defer removals until 3.0' because people wanted a more stable language > and claimed that they would prefer to deal with several removals all at > once. So old-style classes were kept around long past when they would > have been removed under the old 'usual policy'. Ditto for old-style int > / int and some others. Or one can simply recognize that 3.0 was the > 'few releases later' release of that policy. > > The other big change was switching to unicode strings from ascii strings > with optional unicode string add-on. That was/is/will-be a hassle > regardless of when and what name, but necessary for Python to be a > modern world language. >From my experience, string to unicode migration really is the biggest pain when porting anything that handles real-world data. An interesting experiment would have been to split the big changes in two parts, e.g., a 2.95 that only has the string to unicode changes, and a 3.0 that has all the rest. Of course, people would have complained about having to port twice :) Georg From mike.klaas at gmail.com Tue Nov 3 22:05:19 2009 From: mike.klaas at gmail.com (Mike Klaas) Date: Tue, 3 Nov 2009 13:05:19 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> <3FE75D51-199C-4221-9B38-94FD2FE9BC4E@gmail.com> Message-ID: <3d2ce8cb0911031305y330b832fo58e6907011e69efb@mail.gmail.com> On Tue, Nov 3, 2009 at 10:42 AM, Georg Brandl wrote: > ssteinerX at gmail.com schrieb: > > > > On Nov 3, 2009, at 12:28 PM, Arc Riley wrote: > >> > >> The main thing holding back the community are lazy and/or obstinate > >> package maintainers. If they spent half the time they've put into > >> complaining about Py3 into actually working to upgrade their code > >> they'd be done now. > > > > That's an inflammatory, defamatory, unsubstantiated, hyperbolic, > > sweeping overgeneralization. > > I know a few maintainers, and I have no problem seeing how Arc came > to that conclusion. Be that as it may, the only way python 3 will be widely adopted if people have motivation to (need to be compatible with other libs, pressure from users, their own interest in fostering python 3.0, etc.). Deriding them as "lazy" accomplishes nothing and obscures the fact that it is the python maintainers responsibility to bring about this motivation if they want python 3.0 to be adopted. No-one is going to convert to python 3.0 because you called them lazy. -Mike > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Tue Nov 3 22:10:14 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Nov 2009 13:10:14 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> Message-ID: On Tue, Nov 3, 2009 at 1:00 PM, Georg Brandl wrote: > From my experience, string to unicode migration really is the biggest > pain when porting anything that handles real-world data. Of course, handling Unicode right is also the biggest pain when writing code for 2.x in the first place -- writing greenfield code targeted at 3.x that does Unicode right is a lot easier. Alas, this does nothing for those folks who already have working Unicode handling code for 2.x... -- --Guido van Rossum (python.org/~guido) From solipsis at pitrou.net Tue Nov 3 22:56:19 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 3 Nov 2009 21:56:19 +0000 (UTC) Subject: [Python-Dev] Porting C extensions References: <4AEFC887.6020801@v.loewis.de> <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com> Message-ID: Brett Cannon python.org> writes: > > I'm afraid there is some FUD going around here, which is > understandable since no one wants to burn a ton of time on something > that will be difficult or take a lot of time. But I have not heard > anyone in this email thread (or anywhere for that matter) say that > they tried a port in earnest and it turned out to be difficult. One obvious example is the C extensions for the interpreter itself. The code for those C extensions is actually almost the same from 2.x to 3.x. The two main changes are the module initialization sequence (see PEP 3121) and the bytes->unicode change for special methods such as tp_repr. Regards Antoine. From p.f.moore at gmail.com Tue Nov 3 23:16:51 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 3 Nov 2009 22:16:51 +0000 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFC887.6020801@v.loewis.de> <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com> Message-ID: <79990c6b0911031416r36c6947ldd19984108d46f7e@mail.gmail.com> 2009/11/3 Brett Cannon : > I'm afraid there is some FUD going around here, which is > understandable since no one wants to burn a ton of time on something > that will be difficult or take a lot of time. But I have not heard > anyone in this email thread (or anywhere for that matter) say that > they tried a port in earnest and it turned out to be difficult. Sadly, what I've heard a lot of is people (projects) saying that they *won't* try a port (yet, for the forseeable future, take your pick) because they *expect* it to be difficult. The worst thing is that even Martin's (and probably others') efforts in actually producing ports, and demonstrating that it's not difficult, don't seem to be changing minds. FWIW, I did a quick survey of some packages (a sampling of packages I've used or considered using in the past): Twisted - no plans yet for Python 3 wxPython - no mention of Python 3 numpy - no plans yet for Python 3 pyQt - supports Python 3 cx_Oracle - supports Python 3 pywin32 - supports Python 3 pygame - python 3 support "mostly completed" Django - not yet, FAQ suggests it will be "a year or two" TurboGears - Python 3 "currently unsupported", no timescale given PIL - 1.1.7 (due very soon) supports Python 3 lxml - supports Python 3 pyCrypto - doesn't appear to support Python 3 yet gmpy - 1.10 beta supports Python 3 pyYaml - supports Python 3 mod_wsgi - 3.0 RC 5 supports Python 3 (but see below) Here, "supports Python 3" either means that explicit support is mentioned on the website, or Windows binaries exist. I guess there's also some pure python code that "might work", possibly only requiring testing to confirm this. That's a lot better picture than I expected. How come this message isn't getting across? I suspect the big answer is that there's no web framework (that I'm aware of) which works with Python 3 - mod_wsgi supports Python 3, but from what I've seen on web-sig, the WSGI picture for Python 3 is unclear, to say the least. It seems to me that Python 3 adoption is pretty healthy, at least in these terms. If a credible Python 3 web development framework appeared (ie, one of the "big ones" got a port done) things would start to be in pretty good shape. Paul. From solipsis at pitrou.net Tue Nov 3 23:23:55 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 3 Nov 2009 22:23:55 +0000 (UTC) Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? References: <4AEFC887.6020801@v.loewis.de> <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com> <79990c6b0911031416r36c6947ldd19984108d46f7e@mail.gmail.com> Message-ID: Paul Moore gmail.com> writes: > > FWIW, I did a quick survey of some packages (a sampling of packages > I've used or considered using in the past): > > Twisted - no plans yet for Python 3 Well Twisted depends on zope.interface which is not ported yet. Twisted apparently has plans to reduce or remove the warnings generated with the "-3" option, we'll see if the patches get committed: http://twistedmatrix.com/trac/ticket/2484 http://twistedmatrix.com/trac/ticket/4053 http://twistedmatrix.com/trac/ticket/4065 > TurboGears - Python 3 "currently unsupported", no timescale given TurboGears is Pylons-based, so I suppose the actual question is when Pylons gets ported. Regards Antoine. From skippy.hammond at gmail.com Tue Nov 3 23:39:38 2009 From: skippy.hammond at gmail.com (Mark Hammond) Date: Wed, 04 Nov 2009 09:39:38 +1100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: <4AF0B12A.9000406@gmail.com> On 4/11/2009 4:35 AM, Guido van Rossum wrote: > I've checked draft (!) PEP 3003, "Python Language Moratorium", into > SVN. As authors I've listed Jesse, Brett and myself. Good move, +1. Cheers, Mark From steve at pearwood.info Tue Nov 3 23:46:15 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 4 Nov 2009 09:46:15 +1100 Subject: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it In-Reply-To: <4AEF5AFA.5060806@canterbury.ac.nz> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <20091102052900.GA25160@cskk.homeip.net> <4AEF5AFA.5060806@canterbury.ac.nz> Message-ID: <200911040946.15805.steve@pearwood.info> On Tue, 3 Nov 2009 09:19:38 am Greg Ewing wrote: > Cameron Simpson wrote: > > Personally, I'm for the iteration spec in a lot of ways. > > > > Firstly, a .get()/.pick() that always returns the same element > > feels horrible. Is there anyone here who _likes_ it? > > It doesn't sound very useful to me. However, an iterating > version of it doesn't sound any more useful. What would it > gain you? Why not just iterate over the set? Or make a > copy and repeatedly pop() it? Since I was the person who decided that "arbitrary" meant "give a different result each time", I should answer that. My intention was not that people should use repeated calls to pick() for iteration. I described that as an abuse of the method. But I can imagine scenarios where the caller calls set.pick() sequentially: def pick_two_cards(hand): assert isinstance(hand, (set, frozenset)) assert len(hand) == 5 return (hand.pick(), hand.pick()) As I said in a reply to Raymond, optimization was not my primary concern, but as a fundamental set operation[1] pick() should be efficient. It may be called on a set with millions of items, not just small sets. Copying to another set, or to a list, will be O(N) and potentially slow -- at the very least, it is wasteful to copy millions of elements in order to access one. I don't know how expensive it is to create a set iterator, but (my reasoning went) surely we can do better? The set already has access to its own elements, it just needs to return one of them. pop() also knows how to retrieve an arbitrary element. pick() is just like pop(), except without removal. > I completely fail to see a use case for this. Nevertheless, people keep requesting it, so obviously they have a use for it. None of the standard solutions are obvious or easily discoverable, and in my experience the usual solution people come up with is to pop() an element, then add() it back in, but of course that's not just inelegant but it fails on frozensets. [1] Obviously there is disagreement on whether or not pick() is a fundamental operation or not. As Raymond points out, it is uncommon in other languages. But Wikipedia lists it as fundamental, and it is not just Python users who ask for it: http://stackoverflow.com/questions/124671/picking-a-random-element-from-a-set -- Steven D'Aprano From jackdied at gmail.com Wed Nov 4 00:07:39 2009 From: jackdied at gmail.com (Jack Diederich) Date: Tue, 3 Nov 2009 18:07:39 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: +1. There are no compelling language changes on the horizon ("yield from" is nice but not necessary). I see the main benefit of a moratorium as social rather than technical by encouraging people to work on the lib instead of the language. Plus, I'd gladly proxy my vote to any one of the three PEP authors so 3/3 is a no-brainer. -Jack From ssteinerx at gmail.com Wed Nov 4 00:10:00 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Tue, 3 Nov 2009 18:10:00 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> <3FE75D51-199C-4221-9B38-94FD2FE9BC4E@gmail.com> Message-ID: On Nov 3, 2009, at 1:42 PM, Georg Brandl wrote: > ssteinerX at gmail.com schrieb: >> >> On Nov 3, 2009, at 12:28 PM, Arc Riley wrote: >>> >>> The main thing holding back the community are lazy and/or obstinate >>> package maintainers. If they spent half the time they've put into >>> complaining about Py3 into actually working to upgrade their code >>> they'd be done now. >> >> That's an inflammatory, defamatory, unsubstantiated, hyperbolic, >> sweeping overgeneralization. > > I know a few maintainers, and I have no problem seeing how Arc came > to that conclusion. Yah, me neither, but the other two are still cool in my book ;-) S From guido at python.org Wed Nov 4 00:10:30 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Nov 2009 15:10:30 -0800 Subject: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it In-Reply-To: <200911040946.15805.steve@pearwood.info> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <20091102052900.GA25160@cskk.homeip.net> <4AEF5AFA.5060806@canterbury.ac.nz> <200911040946.15805.steve@pearwood.info> Message-ID: On Tue, Nov 3, 2009 at 2:46 PM, Steven D'Aprano wrote: > On Tue, 3 Nov 2009 09:19:38 am Greg Ewing wrote: >> Cameron Simpson wrote: >> > Personally, I'm for the iteration spec in a lot of ways. >> > >> > Firstly, a .get()/.pick() that always returns the same element >> > feels horrible. Is there anyone here who _likes_ it? >> >> It doesn't sound very useful to me. However, an iterating >> version of it doesn't sound any more useful. What would it >> gain you? Why not just iterate over the set? Or make a >> copy and repeatedly pop() it? > > Since I was the person who decided that "arbitrary" meant "give a > different result each time", I should answer that. > > My intention was not that people should use repeated calls to pick() for > iteration. I described that as an abuse of the method. But I can > imagine scenarios where the caller calls set.pick() sequentially: > > def pick_two_cards(hand): > ? ?assert isinstance(hand, (set, frozenset)) > ? ?assert len(hand) == 5 > ? ?return (hand.pick(), hand.pick()) > > > As I said in a reply to Raymond, optimization was not my primary > concern, but as a fundamental set operation[1] pick() should be > efficient. It may be called on a set with millions of items, not just > small sets. Copying to another set, or to a list, will be O(N) and > potentially slow -- at the very least, it is wasteful to copy millions > of elements in order to access one. > > I don't know how expensive it is to create a set iterator, but (my > reasoning went) surely we can do better? The set already has access to > its own elements, it just needs to return one of them. pop() also knows > how to retrieve an arbitrary element. pick() is just like pop(), except > without removal. > > >> I completely fail to see a use case for this. > > Nevertheless, people keep requesting it, so obviously they have a use > for it. None of the standard solutions are obvious or easily > discoverable, and in my experience the usual solution people come up > with is to pop() an element, then add() it back in, but of course > that's not just inelegant but it fails on frozensets. > > > > [1] Obviously there is disagreement on whether or not pick() is a > fundamental operation or not. As Raymond points out, it is uncommon in > other languages. But Wikipedia lists it as fundamental, and it is not > just Python users who ask for it: > > http://stackoverflow.com/questions/124671/picking-a-random-element-from-a-set You're obviously talking about a *random* element. This is a separate use case (though I agree many people don't know the difference). Picking a random element can be done in O(1) only if the data structure supports access by index, which Python's hash tables don't. -- --Guido van Rossum (python.org/~guido) From daniel at stutzbachenterprises.com Wed Nov 4 00:11:47 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 3 Nov 2009 17:11:47 -0600 Subject: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it In-Reply-To: <200911040946.15805.steve@pearwood.info> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <20091102052900.GA25160@cskk.homeip.net> <4AEF5AFA.5060806@canterbury.ac.nz> <200911040946.15805.steve@pearwood.info> Message-ID: On Tue, Nov 3, 2009 at 4:46 PM, Steven D'Aprano wrote: > def pick_two_cards(hand): > assert isinstance(hand, (set, frozenset)) > assert len(hand) == 5 > return (hand.pick(), hand.pick()) > Even if pick() chose random, you still might end up picking the same card twice. Is that really what you intended? FWIW, I've been working on an extension module that supplies a "sortedset" type [1]. In most ways, it's similar to a set except it's indexable like a list. The items are kept in sorted order, so index 0 is always the lowest item, index 1 is the next-to-lowest, etc. Because they're indexable, it's easy and efficient to retrieve random elements using the standard library's "random" module. With the sortedset type, that function would become: def pick_two_cards(hand): assert isinstance(hand, (set, frozenset)) assert len(hand) == 5 return random.choice(hand), random.choice(hand) or if you want to avoid duplicates: return random.sample(hand, 2) Would something like that fit your needs? [1] It's already implemented, along with sortedlist, weaksortedlist, and weaksortedset types. I'm just putting them through the paces in my own products before releasing them. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Wed Nov 4 00:21:56 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 3 Nov 2009 23:21:56 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Retrieve_an_arbitrary_element_from_a_set?= =?utf-8?q?=09withoutremoving_it?= References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <20091102052900.GA25160@cskk.homeip.net> <4AEF5AFA.5060806@canterbury.ac.nz> <200911040946.15805.steve@pearwood.info> Message-ID: Guido van Rossum python.org> writes: > > You're obviously talking about a *random* element. This is a separate > use case (though I agree many people don't know the difference). > > Picking a random element can be done in O(1) only if the data > structure supports access by index, which Python's hash tables don't. Well, at the implementation level, they can. You'd just have to pick a new random index until it points to a non-empty slot. Regards Antoine. From amauryfa at gmail.com Wed Nov 4 00:57:32 2009 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Wed, 4 Nov 2009 00:57:32 +0100 Subject: [Python-Dev] No buildbot to test wide unicode? Message-ID: Hello, It seems that there is no buildbot to test a wide unicode build. On http://www.python.org/dev/buildbot/3.x/ all outputs of the "configure" steps show this message:: checking what type to use for str... unsigned short which looks like a ucs2 build to me. Since wide unicode is the standard chosen by some Linux distributions, it would make sense to have at least one buildbot running with --with-wide-unicode (3.x) or --enable-unicode=ucs4 (2.x). -- Amaury Forgeot d'Arc From cournape at gmail.com Wed Nov 4 01:06:07 2009 From: cournape at gmail.com (David Cournapeau) Date: Wed, 4 Nov 2009 09:06:07 +0900 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF0758A.6070108@v.loewis.de> References: <4AEFD9AE.7090104@molden.no> <4AF06A5F.6090501@v.loewis.de> <4AF072BE.2010603@trueblade.com> <4AF0758A.6070108@v.loewis.de> Message-ID: <5b8d13220911031606g7a1c40d9jbfa51137661a6d4@mail.gmail.com> On Wed, Nov 4, 2009 at 3:25 AM, "Martin v. L?wis" wrote: > But only if NumPy would drop support for 2.x, for x < 7, right? > That would probably be many years in the future. Yes. Today, given the choice of supporting py 3.x and dropping python < 2.7 and continue support for 2.4, the latter is by far my preferred choice today (RHEL still require 2.4, for example). cheers, David From amcnabb at mcnabbs.org Tue Nov 3 22:28:43 2009 From: amcnabb at mcnabbs.org (Andrew McNabb) Date: Tue, 3 Nov 2009 14:28:43 -0700 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> Message-ID: <20091103212843.GA2444@mcnabbs.org> On Tue, Nov 03, 2009 at 12:28:47PM -0500, Arc Riley wrote: > > The main thing holding back the community are lazy and/or obstinate package > maintainers. If they spent half the time they've put into complaining about > Py3 into actually working to upgrade their code they'd be done now. The main reason the package maintainers are so "lazy and/or obstinate" is that Python 2 is much more available than Python 3. For example, work is being completed on a python3 RPM, but it will first appear in Fedora 13, not Fedora 12. RHEL is still using Python 2.4 and won't even get Python 2.6 until RHEL 6 comes out. I don't think it's worth worrying about packages being upgraded to Python 3 when many or most users still don't have access to Python 3. In my opinion, the best place for current efforts is on infrastructure: 1) Python 3 packages for all Linux distributions (I've recently been involved with bringing Python 3 to Fedora) 2) WSGI/mod_python, etc.: according to http://wsgi.org/wsgi/Amendments_1.0 and various blog posts, the WSGI standard isn't defined for Python 3 yet, and the modwsgi page at http://code.google.com/p/modwsgi/ does not yet list a Python 3 release. 3) Numpy/scipy: as others have mentioned, numpy is essential for scientific work, and apparently the project needs help to add support for Python 3 Before basic infrastructure is available for Python 3, it's absurd to expect package maintainers to flock to it. -- Andrew McNabb http://www.mcnabbs.org/andrew/ PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868 From bugbee at seanet.com Wed Nov 4 00:27:43 2009 From: bugbee at seanet.com (Larry Bugbee) Date: Tue, 3 Nov 2009 15:27:43 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com> References: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com> Message-ID: <93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com> This is my problem to solve... I work with a lot of [non-production] crypto where byte strings are a normal way of manipulating and storing keys and other artifacts. Under Python2 I have few/no Unicode issues. With Python3's default string type being Unicode, there are a lot of subtle ways where my codes go south. Like I say, it is my problem. I will convert to Py3, but not likely before summer. Besides 2to3, any other tools/suggestions would be welcome. (Perhaps there is a Master switch to disable Unicode? ;-) Changing the subject... I say -1 on backporting more 3.x features to 2.6/2.7. As previously mentioned, leave the carrot. And best intentions aside, backports can introduce problems of their own. From ben+python at benfinney.id.au Wed Nov 4 01:42:29 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 04 Nov 2009 11:42:29 +1100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? References: <4AEFC887.6020801@v.loewis.de> <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com> <79990c6b0911031416r36c6947ldd19984108d46f7e@mail.gmail.com> Message-ID: <87y6mnf7d6.fsf@benfinney.id.au> Antoine Pitrou writes: > Paul Moore gmail.com> writes: > > TurboGears - Python 3 "currently unsupported", no timescale given > > TurboGears is Pylons-based, so I suppose the actual question is when > Pylons gets ported. And there's the rub. I expect this general problem of ?project FOO depends on library BAR, so there's no point thinking about migration of FOO until BAR is ready for Python 3? will prove to be quite common in widespread projects. -- \ ?If you do not trust the source do not use this program.? | `\ ?Microsoft Vista security dialogue | _o__) | Ben Finney From ssteinerx at gmail.com Wed Nov 4 01:50:36 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Tue, 3 Nov 2009 19:50:36 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <87y6mnf7d6.fsf@benfinney.id.au> References: <4AEFC887.6020801@v.loewis.de> <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com> <79990c6b0911031416r36c6947ldd19984108d46f7e@mail.gmail.com> <87y6mnf7d6.fsf@benfinney.id.au> Message-ID: On Nov 3, 2009, at 7:42 PM, Ben Finney wrote: > Antoine Pitrou writes: > >> Paul Moore gmail.com> writes: >>> TurboGears - Python 3 "currently unsupported", no timescale given >> >> TurboGears is Pylons-based, so I suppose the actual question is when >> Pylons gets ported. > > And there's the rub. I expect this general problem of ?project FOO > depends on library BAR, so there's no point thinking about migration > of > FOO until BAR is ready for Python 3? will prove to be quite common in > widespread projects. There's no "will prove", it is. S From greg.ewing at canterbury.ac.nz Wed Nov 4 01:54:47 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 04 Nov 2009 13:54:47 +1300 Subject: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it In-Reply-To: <200911040946.15805.steve@pearwood.info> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <20091102052900.GA25160@cskk.homeip.net> <4AEF5AFA.5060806@canterbury.ac.nz> <200911040946.15805.steve@pearwood.info> Message-ID: <4AF0D0D7.3090403@canterbury.ac.nz> Steven D'Aprano wrote: > I don't know how expensive it is to create a set iterator, Not expensive enough to justify burdening the set type with extra functionality that will be extremely rarely used. -- Greg From greg.ewing at canterbury.ac.nz Wed Nov 4 02:00:12 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 04 Nov 2009 14:00:12 +1300 Subject: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it In-Reply-To: References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <20091102052900.GA25160@cskk.homeip.net> <4AEF5AFA.5060806@canterbury.ac.nz> <200911040946.15805.steve@pearwood.info> Message-ID: <4AF0D21C.8040909@canterbury.ac.nz> Antoine Pitrou wrote: > Guido van Rossum python.org> writes: > >>Picking a random element can be done in O(1) only if the data >>structure supports access by index, which Python's hash tables don't. > > Well, at the implementation level, they can. You'd just have to pick a new > random index until it points to a non-empty slot. But that's hardly O(1). -- Greg From solipsis at pitrou.net Wed Nov 4 02:10:58 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 4 Nov 2009 01:10:58 +0000 (UTC) Subject: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <20091102052900.GA25160@cskk.homeip.net> <4AEF5AFA.5060806@canterbury.ac.nz> <200911040946.15805.steve@pearwood.info> <4AF0D21C.8040909@canterbury.ac.nz> Message-ID: Greg Ewing canterbury.ac.nz> writes: > > > >>Picking a random element can be done in O(1) only if the data > >>structure supports access by index, which Python's hash tables don't. > > > > Well, at the implementation level, they can. You'd just have to pick a new > > random index until it points to a non-empty slot. > > But that's hardly O(1). It is, assuming that the set has a built-in minimum fill level (e.g. it always keeps at least x% of its entries filled), and the random generator is good. (of course, it is "statistically O(1)", like lookups in a hash table actually) Regards Antoine. From stephen at xemacs.org Wed Nov 4 02:39:23 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 04 Nov 2009 10:39:23 +0900 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4AF08345.6040603@egenix.com> Message-ID: <87tyxb131w.fsf@uwakimon.sk.tsukuba.ac.jp> Guido van Rossum writes: > On Tue, Nov 3, 2009 at 11:23 AM, M.-A. Lemburg wrote: > > One question: > > > > There are currently number of patch waiting on the tracker for > > additional Unicode feature support and it's also likely that we'll > > want to upgrade to a more recent Unicode version within the > > next few years. > > > > How would such indirect changes be seen under the moratorium ? > > That would fall under the Case-by-Case Exemptions section. "Within the > next few years" sounds like it might well wait until the moratorium is > ended though. :-) (1) Unicode "feature support" (eg, implementing access to various databases, implementation of new algorithms) strikes me as worth a hearing but very dubious under case-by-case (the case I have in mind is when necessary to support (2) or (3)). (2) Adjusting existing implementations of Unicode features (eg, universal newline, which currently does not conform to the relevant TR AFAIK, thus dubious IMO) to conform to (new versions of) Unicode seems worthy of consideration on a cost/benefit basis. I would expect that in most cases the costs that lead to a moratorium would outweigh prospective benefits, though. I don't know of an example where I'd support an exception to the moratorium for this, though. It's hard to imagine that things like the algorithm for UTF-16 are going to change, and support for things like bidi are currently outside of the scope of the moratorium as I understand it -- they'd be stdlib facilities. (3) Upgrading the character repertoire and property databases (eg, case and composition) seems like a shoo-in to me. That's why the databases are versioned. Of course there are costs to upgrading (backward compatibility in some corner cases, eg) that might outweigh the benefits at any given time, but I don't think this should fall under the moratorium if someone finds a benefit to upgrading the databases within the moratorium period. From guido at python.org Wed Nov 4 03:11:02 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Nov 2009 18:11:02 -0800 Subject: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it In-Reply-To: References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <20091102052900.GA25160@cskk.homeip.net> <4AEF5AFA.5060806@canterbury.ac.nz> <200911040946.15805.steve@pearwood.info> <4AF0D21C.8040909@canterbury.ac.nz> Message-ID: On Tue, Nov 3, 2009 at 5:10 PM, Antoine Pitrou wrote: > Greg Ewing canterbury.ac.nz> writes: >> > >> >>Picking a random element can be done in O(1) only if the data >> >>structure supports access by index, which Python's hash tables don't. >> > >> > Well, at the implementation level, they can. You'd just have to pick a new >> > random index until it points to a non-empty slot. >> >> But that's hardly O(1). > > It is, assuming that the set has a built-in minimum fill level (e.g. it always > keeps at least x% of its entries filled), and the random generator is good. > > (of course, it is "statistically O(1)", like lookups in a hash table actually) Clever. But I don't think the set implementation should have a dependency on random(), so it would have to expose an "indexing" operation, which smells like it would expose too much of the implementation for comfort. -- --Guido van Rossum (python.org/~guido) From tjreedy at udel.edu Wed Nov 4 03:26:25 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 03 Nov 2009 21:26:25 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> <4AF06A5F.6090501@v.loewis.de> Message-ID: Guido van Rossum wrote: > On Tue, Nov 3, 2009 at 9:37 AM, "Martin v. L?wis" wrote: >> (and no, adding things like nonlocal to 2.7 doesn't making porting of >> a real application or library any easier, since the existing application >> or library simply doesn't use that keyword. > > Agreed. > >> In fact, no change to 2.x >> can reasonably simplify porting - only changes to 3.x might - except >> for changes to 2to3, which can simplify porting a lot. But 2to3 should >> be run under 3.x, IMO.) > > Disagreed. Better "-3" warnings could make porting easier. (Not just > more warnings -- "better" might mean fewer false positives for > warnings already issued.) There is also Eric Smith's list to consider: PEP3118 new buffer protocol, short float repr, and maybe io. > FWIW, it doesn't sound like killing 2.7 is a productive thing to do. > However making 2.7 the end of the line (though with indefinite bugfix > releases) might be. (Indefinite != infinite.) I think you should decide and announce something like the following: ''' Python 2.7 will be the final, stable release in the Python 2 line. It will be released in mid-2010 with the first alpha scheduled for December 2009. It will not intentionally break valid older 2.x code; this means no removals. (Valid == not exploiting a bug.) Being the last of its line, there will be no deprecation warnings unless explicitly requested with the -3 flag to warn about incompatibilities with 3.x. There will be lots of bug fixes since 2.6. There will be only a few new features, with those aimed at easing eventual transition of libraries to 3.x. The period of 2.7.z bugfix releases should be longer than for previous x.y releases (as long as there are volunteers to write and review patches and prepare distributions). The developers urge people with 2.x code, especially library maintainers, to test it with preliminary alpha or beta releases so 2.7 can be as good as possible. The developers hope OS distributions can move to including 2.7 as soon as feasible, even if it means 'skipping' 2.6 as a default version. ''' [My intention with the last is to promote 2.7 as the definitive version of 2.x, though there might be better wording.] Terry Jan Reedy From guido at python.org Wed Nov 4 04:11:47 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Nov 2009 19:11:47 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com> References: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com> <93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com> Message-ID: On Tue, Nov 3, 2009 at 3:27 PM, Larry Bugbee wrote: > This is my problem to solve... ?I work with a lot of [non-production] crypto > where byte strings are a normal way of manipulating and storing keys and > other artifacts. ?Under Python2 I have few/no Unicode issues. ?With > Python3's default string type being Unicode, there are a lot of subtle ways > where my codes go south. ?Like I say, it is my problem. It may be interesting for others if you posted some examples of what you are used to doing in 2.x. Maybe somebody will suggest a better way of doing it in 3.x than you have thought of. :-) > I will convert to Py3, but not likely before summer. ?Besides 2to3, any > other tools/suggestions would be welcome. ?(Perhaps there is a Master switch > to disable Unicode? ;-) Yes, go back to 1.5.2. :-) -- --Guido van Rossum (python.org/~guido) From debatem1 at gmail.com Wed Nov 4 04:46:41 2009 From: debatem1 at gmail.com (geremy condra) Date: Tue, 3 Nov 2009 22:46:41 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com> References: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com> <93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com> Message-ID: On Tue, Nov 3, 2009 at 6:27 PM, Larry Bugbee wrote: > This is my problem to solve... ?I work with a lot of [non-production] crypto > where byte strings are a normal way of manipulating and storing keys and > other artifacts. ?Under Python2 I have few/no Unicode issues. ?With > Python3's default string type being Unicode, there are a lot of subtle ways > where my codes go south. ?Like I say, it is my problem. > Interesting- I've been doing crypto in python 3 for quite some time and haven't had any issues except with getting things to play nicely with C. Any specific examples? Geremy Condra From steven.bethard at gmail.com Wed Nov 4 05:57:27 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 3 Nov 2009 20:57:27 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <5b8d13220911030155x57615540x1f59e4b95b6c2f56@mail.gmail.com> Message-ID: On Tue, Nov 3, 2009 at 6:14 AM, Daniel Stutzbach wrote: > It's pretty easy to make Python source that works under 2.6 and 3.x.? It's > basically impossible to make Python source that works under 2.4/2.5 and > 3.x. This keeps getting quoted later in the thread so I just wanted to say again that this is not true. The argparse module supports Python 2.3-3.1 all from the same codebase, and it really wasn't all that hard: http://code.google.com/p/argparse/source/detail?r=12 Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From glyph at twistedmatrix.com Wed Nov 4 06:20:18 2009 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Wed, 4 Nov 2009 00:20:18 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <79990c6b0911031416r36c6947ldd19984108d46f7e@mail.gmail.com> References: <4AEFC887.6020801@v.loewis.de> <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com> <79990c6b0911031416r36c6947ldd19984108d46f7e@mail.gmail.com> Message-ID: On Nov 3, 2009, at 5:16 PM, Paul Moore wrote: > 2009/11/3 Brett Cannon : >> I'm afraid there is some FUD going around here, which is >> understandable since no one wants to burn a ton of time on something >> that will be difficult or take a lot of time. But I have not heard >> anyone in this email thread (or anywhere for that matter) say that >> they tried a port in earnest and it turned out to be difficult. > FWIW, I did a quick survey of some packages (a sampling of packages > I've used or considered using in the past): > > Twisted - no plans yet for Python 3 Speaking of FUD, we've had a plan for Python 3 support for some time: http://twistedmatrix.com/trac/ticket/2484 http://stackoverflow.com/questions/172306/how-are-you-planning-on-handling-the-migration-to-python-3/214601#214601 Not only that, but progress is actually being made on that plan, as it is being slowly executed by contributors from the community, a sampling of which you can see on these tickets, linked from the bottom of the "master plan" ticket I mentioned above (#2484): http://twistedmatrix.com/trac/ticket/4053 http://twistedmatrix.com/trac/ticket/4065 http://twistedmatrix.com/trac/ticket/4066 If you're interested in helping, our core team has all not had much time for Twisted lately and we need volunteers who are interested in doing code reviews and becoming a committer to help shepherd these tickets through the process. From cs at zip.com.au Wed Nov 4 06:26:40 2009 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 4 Nov 2009 16:26:40 +1100 Subject: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it In-Reply-To: <200911040946.15805.steve@pearwood.info> References: <200911040946.15805.steve@pearwood.info> Message-ID: <20091104052640.GA15378@cskk.homeip.net> On 04Nov2009 09:46, Steven D'Aprano wrote: | On Tue, 3 Nov 2009 09:19:38 am Greg Ewing wrote: | > Cameron Simpson wrote: | > > Personally, I'm for the iteration spec in a lot of ways. For the record, I've since, in mere hours or day, been convinced my preference was misguided. I _do_ still feel that a hypothetical .pick() (a name I think better than .get()) would be "nicer" if it didn't always return the same item. Not that I've imagined any use case not equally well served by the existing iterator facility :-( Greg Ewing: | > I completely fail to see a use case for this. Steven D'Aprano: | Nevertheless, people keep requesting it, so obviously they have a use | for it. None of the standard solutions are obvious or easily | discoverable, and in my experience the usual solution people come up | with is to pop() an element, then add() it back in, but of course | that's not just inelegant but it fails on frozensets. Question: has anyone asked for .pick()/.get() with a use case _not_ well served by the existing facilities? Personally, I have found it useful in doco I write to have a section on "Common Tasks", with recommended/suggested examples of how to do them and short rationale for the chosen method. It seems to me that if .pick() is frequently desired and "None of the standard solutions are obvious or easily discoverable" then they should be _made_ so with documentation. Comments? -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Politics, n. Strife of interests masquerading as a contest of principles. - Ambrose Bierce, _The_Devil's_Dictionary_ From glyph at twistedmatrix.com Wed Nov 4 06:51:09 2009 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Wed, 4 Nov 2009 00:51:09 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> Message-ID: <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> On Nov 3, 2009, at 6:23 AM, Paul Moore wrote: > 2009/11/3 Raymond Hettinger : >> In all these matters, I think the users should get a vote. And >> that vote >> should be cast with their decision to stay with 2.x, or switch to >> 3.x, or >> try to support both. We should not muck with their rational >> decision making >> by putting "carrots" in one pile and abandoning the other. > The biggest issue to my mind is that adoption by the ultimate end > users is significantly hampered by the fact that big projects like > Twisted, numpy and the like, have no current plans to move to Python > 3. Even end users with a reasonable level of coding expertise don't > have the time or resources to offer much in the way of help with a > port, when the project as a whole isn't interested in starting the > process. For what it's worth, the official position of the Twisted project is not that we have "no plan" to move to Python 3. It's that our plan is to do exactly as Raymond suggests, and give the users a vote - in this case, you vote with your patches :). We are actively and frequently encouraging our users to contribute patches that clean up warnings, which is the biggest impediment to a py3 port of Twisted. Some of you would probably expecting to whinge about how people never contribute anything, but actually, users *have* shown up and started doing this. Our biggest problem at the moment is that we don't have enough people doing code reviews so the contributions are starting to pile up. As I said in my other message, if someone would like to help, signing up to do code reviews would be a good way. Despite this progress, my hope is that there will be a robust 2.x series up through 2.9. For one thing, we have a very long row to hoe here. The migration to 3.0 is a long, tedious process with little tangible benefit. I hope that sometime in the next decade Twisted can accelerate the process of dropping old 2.x versions, but I seriously doubt we could do a feature- complete 3.1/2.6 version. I get the general impression that a 3.2/2.7 port would be more feasible; hopefully a 3.3/2.8 would be even moreso. Also, the benefits of migrating to python 3.x are still negligible, as far as I can tell. On the one hand, you've got a Python with no old- style classes and a clear unicode/bytes situation, and that's great. On the other hand, you've got NumPy, PyGTK, Unladen Swallow, PyPy, Jython, IronPython, and so on and so forth. Since I started using it, the strength of Python has been in its ecosystem, and the 3.x ecosystem is not yet viable. As long as we're tossing out modest proposals here, I still think that (as I believe James Knight already proposed) abandoning the current 3.x branch, backporting everything to 2.7, and continuing forward with a migration strategy that introduces individual deprecations every major version until 2.x == 3.x is the way to go. For example, 2.8 could emit a deprecation warning for every old-style class that was defined, 2.9 could emit a deprecation warning for every string constant declared without a 'b' or 'u' prefix unless the module in question were in "3.x mode" (i.e. no-prefix == 'u'). (I leave the determination of whether the parser should be in 3.x mode for a particular module as an exercise for the reader, but a 'from __future__' import would suffice.) I realize that there are other issues here, like the C ABI changes some NumPy folks have raised. Also, I'm not planning to actually do any *work* on this suggestion, so you can take it for what it is, which is to say, armchair quarterbacking. There have been some other comments in this thread indicating that this was not the case because some users indicated that they'd rather deal with lots of changes "all at once". My understanding is that it was done this way so that the *developers* of Python could make a clean break, and design and implement a new version of Python without being constrained by compatibility concerns. If you can show me an actual application or library developer in Python who wanted this one- big-jump migration, I will show you a crazy person. The main reason I want a long 2.x series is that I believe it would more easily allow us infrastructure folks to drop support for *older* versions. With this big 2.x->3.x chasm, I can't really see an end in sight for Twisted using Python 2.x as its _source_ language, translating with 2to3. Some projects which depend on Twisted and want new versions (and security fixes, etc) are going to want Python 2.x for a really long time. Maybe they're just really conservative, maybe they don't have a lot of maintenance energy, or maybe they have other dependencies which haven't got a port; it doesn't really matter, empirically speaking people want older versions of Python. Keep in mind also that the 2.x translation process is extremely slow and results in a clunky development process. There's no '2to3 -- interactive' commandline that lets me type python 2 at a >>> prompt and get python 3 results out so that I can try experiments on the 3.x interpreter; I have to actually put my experiments into my unit tests and wait 10 minutes to see if it works. It's like writing C++. With the 2.x series, users and operating systems seem to move on fairly rapidly, because dependencies generally continue to work if you upgrade just one version. This isn't quite as formal a requirement as I would like (warnings get generated, unit tests fail, things do break) but in practice, users can rely on it for most functionality. If 3.x could be broken into a series of transitions like that, where you can upgrade one version, fix some stuff, then upgrade another version, even if you couldn't actually support more than 2 versions at once, I think that we could pick up the migration pace to the point where we might actually be using 3.x syntax in a few years. Having a 2.x series which goes to 2.9 and then stops isn't *quite* the same thing as having one that moves over continuously to some 3.x version, but it does seem to me that by that point the chasm between versions will have narrowed to a crack, and the migration will be a little hop over it rather than the currently-required great flying leap. (To be fair, that leap is not the surface-to-orbit rocket-propelled jump that I originally expected it to be. Still, it's still not a small effort, especially if you're interfacing deeply with a lot of I/ O APIs.) From regebro at gmail.com Wed Nov 4 07:05:02 2009 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 4 Nov 2009 07:05:02 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: Message-ID: <319e029f0911032205y56f52984q433e8e8d4221a4c5@mail.gmail.com> 2009/11/3 ssteinerX at gmail.com : > Not that anyone has asked yet, but here's my opinion on two issues that have > been raised on the python-dev mailing list lately: > > ? ? ? ?+1 on 2.7 release with as much 3.0 "easy-port goo" as is practicable > without delaying the product beyond the tentative schedule. ?Sooner would, > of course, be better but I'm sure PEP 373 was produced after due > consideration of all pertinent factors. +1 > ? ? ? ?+1 on 2.7 being the last of the 2.x series. ?Enough already! -1 The last of the series should be the last one needed, and we don't know that until after it's been made. No official decision should be taken on this. > I, personally, haven't even written my first line of 3.x code, nor have I > had any good reason to. > > If I saw the actual end of the line at 2.7, I would actually start looking > for 3.x versions of my favorite tools and would be much more inclined to > help push them along ASAP. That's a terrible reason. It's a stick, and sticks suck. If Python 3 is not enough of a carrot, then people should not migrate at all. > Right now, so much that I use on a daily basis doesn't even have a 3.x > roadmap, much less any sort of working implementation, that I don't see > switching to 3.x ever unless the 2.x line ends, and soon! Wrong. Most things do not have a 3.x roadmap or a working implementation, because most things depend on another thing, that also doesn't have an implementation, and therefore can't be ported or planned. The major hangup for this has been setuptools. As you know, distribute came with a Python 3 release fairly recently, which means that most projects out there could not be ported or even planned (as setuptools had no Python 3 plans) until something like a month ago. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From regebro at gmail.com Wed Nov 4 07:06:09 2009 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 4 Nov 2009 07:06:09 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: Message-ID: <319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com> 2009/11/3 ssteinerX at gmail.com : > > On Nov 2, 2009, at 7:26 PM, James Y Knight wrote: >> >> It really sounds like you're saying that switching to 3.x isn't worth the >> cost to you, but you want to force people (including yourself) to do so >> anyways, because ...? > > Because that's the future of Python Or not. Maybe it's a dead branch of Python? -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From martin at v.loewis.de Wed Nov 4 07:07:24 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Wed, 04 Nov 2009 07:07:24 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <5b8d13220911031606g7a1c40d9jbfa51137661a6d4@mail.gmail.com> References: <4AEFD9AE.7090104@molden.no> <4AF06A5F.6090501@v.loewis.de> <4AF072BE.2010603@trueblade.com> <4AF0758A.6070108@v.loewis.de> <5b8d13220911031606g7a1c40d9jbfa51137661a6d4@mail.gmail.com> Message-ID: <4AF11A1C.9080107@v.loewis.de> >> But only if NumPy would drop support for 2.x, for x < 7, right? >> That would probably be many years in the future. > > Yes. Today, given the choice of supporting py 3.x and dropping python > < 2.7 and continue support for 2.4, the latter is by far my preferred > choice today (RHEL still require 2.4, for example). However, you don't have to make that choice. You can support 2.4 and 3.x simultaneously; IMO, any simplification resulting from possible backports is really minor compared to the rest of the work that you would have to do, anyway. For example, porting to the new buffer interface is may be difficult and/or a lot of work, and you'll have to do it, anyway. Making it then conditional (with preprocessor statements), and maintaining both APIs in parallel for some time, is really not that hard, IMO. Regards, Martin From regebro at gmail.com Wed Nov 4 07:31:27 2009 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 4 Nov 2009 07:31:27 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <79990c6b0911030944r26fd668ev37e56a4a7d79559b@mail.gmail.com> References: <4AEFC887.6020801@v.loewis.de> <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com> <4AF0698C.2030706@v.loewis.de> <79990c6b0911030944r26fd668ev37e56a4a7d79559b@mail.gmail.com> Message-ID: <319e029f0911032231l5fb294dbic4fe0ae996dd7c10@mail.gmail.com> 2009/11/3 Paul Moore : > Has that port been integrated back into the zodb project? That is on the way. Setuptools has been ported via the distribute project, I'm hoping to make a test release of the zope.interface branch (to test Python 2 compatibility mainly) soon, and then hopefully merge it, etc. It's slow, but moving. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From python at rcn.com Wed Nov 4 07:35:57 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 3 Nov 2009 22:35:57 -0800 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it References: <200911040946.15805.steve@pearwood.info> <20091104052640.GA15378@cskk.homeip.net> Message-ID: <31742B2077EC4E57836315709D494E34@RaymondLaptop1> > Personally, I have found it useful in doco I write to have a section on > "Common Tasks", with recommended/suggested examples of how to do them and > short rationale for the chosen method. It seems to me that if .pick() > is frequently desired and "None of the standard solutions are obvious > or easily discoverable" then they should be _made_ so with documentation. > > Comments? That's reasonable. It's in the same category as people who can't figure-out how to clear a list because they forgot about slice notation. Raymond From regebro at gmail.com Wed Nov 4 07:43:06 2009 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 4 Nov 2009 07:43:06 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <20091103185641.GA13969@pyropus.ca> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> <20091103174131.GA10645@pyropus.ca> <4AF0734A.3080705@v.loewis.de> <20091103185641.GA13969@pyropus.ca> Message-ID: <319e029f0911032243y48dfb71fpf399d9fd7881b434@mail.gmail.com> 2009/11/3 Charles Cazabon : > "Martin v. L?wis" wrote: > ?It's pretty easy to make Python source that works under 2.6 and 3.x. ?It's > ?basically impossible to make Python source that works under 2.4/2.5 and > ?3.x. Without using 2to3, yes. See http://code.google.com/p/python-incompatibility/ where I've attempted to document most 2.5/3.0 incompatibilities and explain how to get around them to run the same unmodified code under 2.6 and 3.0. (It's not complete, help appreciated, especially with the C-API part) > No. ?getmail v.4 runs fine on Python 2.3.4 through 2.6.x; getmail's code has > always been pretty forward-compatible. Then it's going to be dead easy to support both 2.x and 3.x by using 2to3. If you don't want to have separate packages youcan use distribute: http://packages.python.org/distribute/python3.html Also see my talk at PyCon: http://pycon.blip.tv/file/1949281/ -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From arcriley at gmail.com Wed Nov 4 08:05:42 2009 From: arcriley at gmail.com (Arc Riley) Date: Wed, 4 Nov 2009 02:05:42 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <3d2ce8cb0911031305y330b832fo58e6907011e69efb@mail.gmail.com> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <16F0B6B0-677F-464C-9C89-CF92D3713A65@gmail.com> <116885F3-819E-4B7C-86F7-8E5B246590EB@fuhm.net> <3FE75D51-199C-4221-9B38-94FD2FE9BC4E@gmail.com> <3d2ce8cb0911031305y330b832fo58e6907011e69efb@mail.gmail.com> Message-ID: I'm not aware of any currently active project that isn't in the process of adding Py3 support (or who has already done so). By "most maintainers" I'm referring to the long tail; the hundreds of 3rd party modules used in niche cases and can be easily replaced by those who need the functionality they provide. Free software is an evolutionary process. Evolution dictates that those who cannot adapt will die when faced with a changing environment. It's better for us all that they do. Thus, I am not attempting to persuade anyone to upgrade their package, rather, I'm explaining why it's best we ignore them and get on with doing productive work. On Tue, Nov 3, 2009 at 4:05 PM, Mike Klaas wrote: > > Be that as it may, the only way python 3 will be widely adopted if people > have motivation to (need to be compatible with other libs, pressure from > users, their own interest in fostering python 3.0, etc.). Deriding them as > "lazy" accomplishes nothing and obscures the fact that it is the python > maintainers responsibility to bring about this motivation if they want > python 3.0 to be adopted. No-one is going to convert to python 3.0 because > you called them lazy. > > -Mike > >> > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/arcriley%40gmail.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From regebro at gmail.com Wed Nov 4 08:06:50 2009 From: regebro at gmail.com (Lennart Regebro) Date: Wed, 4 Nov 2009 08:06:50 +0100 Subject: [Python-Dev] nonlocal keyword in 2.x? In-Reply-To: <4AF0816A.706@egenix.com> References: <4AEF4C53.7090902@v.loewis.de> <4AEFF3A3.3020103@egenix.com> <4AF06E20.7050307@v.loewis.de> <4AF0816A.706@egenix.com> Message-ID: <319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com> I should maybe point out that although I'm generally +1 on backporting, I'm not specifically anything on backporting the nonlocal keyword. There are probably things that would help more from a compatibility standpoint than that. For example, from __future__ import unicode_literals doesn't switch the types. So this code will rais an assertion error in 2.6. >>> from __future__ import unicode_literals >>> assert isinstance("fghjkl", str) While it of course will run fine in 3.0. Now this has a fairly trivial workaround or two, but still. I also would really like to see a real port of the bytes class to 2.6, but I have a vague memory that there was some reason that wouldn't work. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From p.f.moore at gmail.com Wed Nov 4 09:40:35 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 4 Nov 2009 08:40:35 +0000 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFC887.6020801@v.loewis.de> <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com> <79990c6b0911031416r36c6947ldd19984108d46f7e@mail.gmail.com> Message-ID: <79990c6b0911040040h4b7932fbla8b731c0c057d460@mail.gmail.com> 2009/11/4 Glyph Lefkowitz : > On Nov 3, 2009, at 5:16 PM, Paul Moore wrote: >> >> 2009/11/3 Brett Cannon : >>> >>> I'm afraid there is some FUD going around here, which is >>> understandable since no one wants to burn a ton of time on something >>> that will be difficult or take a lot of time. But I have not heard >>> anyone in this email thread (or anywhere for that matter) say that >>> they tried a port in earnest and it turned out to be difficult. > >> FWIW, I did a quick survey of some packages (a sampling of packages >> I've used or considered using in the past): >> >> Twisted - no plans yet for Python 3 > > > Speaking of FUD, we've had a plan for Python 3 support for some time: [...] Thanks, and my sincere apologies for spreading FUD - I did try to find details, and in fact I did spot a couple of the specific python 3 compatibility tickets you mentioned, but missed the link back to the master plan. Having said that, > ?http://stackoverflow.com/questions/172306/how-are-you-planning-on-handling-the-migration-to-python-3/214601#214601 seems pretty negative in terms of when Twisted will support Python 3. Based on my reading, the focus is more on when 2.x support will be dropped than on when there will be a version of Twisted which works with Python 3 (which I'd expect to be much sooner!) > If you're interested in helping, our core team has all not had much time for > Twisted lately and we need volunteers who are interested in doing code > reviews and becoming a committer to help shepherd these tickets through the > process. Personally, I don't *use* twisted. I occasionally play with it, and I sometimes end up using applications which rely on it, but I don't use it myself. So I couldn't be much direct help. (And yes, I know that means I'm not one of your users, so me asking for Python 3 support isn't an issue. No problem there). Paul. From doug.hellmann at gmail.com Wed Nov 4 11:05:34 2009 From: doug.hellmann at gmail.com (Doug Hellmann) Date: Wed, 4 Nov 2009 05:05:34 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <4AF095A1.6040406@voidspace.org.uk> References: <4AF095A1.6040406@voidspace.org.uk> Message-ID: <1B2F0471-AE7C-4AAC-9AD1-5C4A68014DFB@gmail.com> On Nov 3, 2009, at 3:42 PM, Michael Foord wrote: > Barry Warsaw wrote: >> On Nov 3, 2009, at 12:35 PM, Guido van Rossum wrote: >> >>> I've checked draft (!) PEP 3003, "Python Language Moratorium", into >>> SVN. As authors I've listed Jesse, Brett and myself. >>> >>> On python-ideas the moratorium idea got fairly positive responses >>> (more positive than I'd expected, in fact) but I'm bracing myself >>> for >>> fierce discussion here on python-dev. It's important to me that if >>> if >>> this is accepted it is a "rough consensus" decision (working code we >>> already have plenty of :-), not something enforced by a vocal >>> minority >>> or an influential individual such as myself. If there's too much >>> opposition I'll withdraw the PEP so as not to waste everybody's time >>> with a fruitless discussion. >>> >>> The PEP tries to spell out some gray areas but I'm sure there will >>> be >>> others; that's life. Do note that the PEP proposes to be >>> *retroactive* >>> back to the 3.1 release, i.e. the "frozen" version of the language >>> is >>> the state in which it was released as 3.1. >> >> I think this is a great idea. I'd love to see the energy normally >> put into evolving the language into making the stdlib really kick >> ass. >> > > +lots Ditto. Doug From steve at pearwood.info Wed Nov 4 12:00:49 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 4 Nov 2009 22:00:49 +1100 Subject: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it In-Reply-To: References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <200911040946.15805.steve@pearwood.info> Message-ID: <200911042200.50850.steve@pearwood.info> On Wed, 4 Nov 2009 10:10:30 am Guido van Rossum wrote: > On Tue, Nov 3, 2009 at 2:46 PM, Steven D'Aprano wrote: > > Since I was the person who decided that "arbitrary" meant "give a > > different result each time", I should answer that. > You're obviously talking about a *random* element. This is a separate > use case (though I agree many people don't know the difference). I'm aware of the difference between random and arbitrary, and in an earlier post I said that the One Obvious Way of getting a random element from a list would be to convert to a list and call random.choice(). Sorry for muddying the waters by linking to a page discussing such random selections. -- Steven D'Aprano From steve at pearwood.info Wed Nov 4 12:34:36 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 4 Nov 2009 22:34:36 +1100 Subject: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it In-Reply-To: <4AF0D0D7.3090403@canterbury.ac.nz> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <200911040946.15805.steve@pearwood.info> <4AF0D0D7.3090403@canterbury.ac.nz> Message-ID: <200911042234.36796.steve@pearwood.info> On Wed, 4 Nov 2009 11:54:47 am Greg Ewing wrote: > Steven D'Aprano wrote: > > I don't know how expensive it is to create a set iterator, > > Not expensive enough to justify burdening the set type with > extra functionality that will be extremely rarely used. As my previous posts on this topic tried to convey, this isn't primarily about efficiency, but about discoverability and obviousness. Anyway, given the level of opposition to the suggestion, I'm no longer willing to carry the flag for it. If anyone else -- perhaps the OP -- feels they want to take it any further, be my guest. -- Steven D'Aprano From ncoghlan at gmail.com Wed Nov 4 12:53:15 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 04 Nov 2009 21:53:15 +1000 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: <4AF16B2B.6010807@gmail.com> Jack Diederich wrote: > +1. There are no compelling language changes on the horizon ("yield > from" is nice but not necessary). Another +1 here. A note in the PEP that there are no changes to SVN that would need to be rolled back due to the moratorium would be a good addition (as per Antoine's review of the NEWS file). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Wed Nov 4 13:00:07 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 04 Nov 2009 22:00:07 +1000 Subject: [Python-Dev] Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?) In-Reply-To: <319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com> References: <4AEF4C53.7090902@v.loewis.de> <4AEFF3A3.3020103@egenix.com> <4AF06E20.7050307@v.loewis.de> <4AF0816A.706@egenix.com> <319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com> Message-ID: <4AF16CC7.6000705@gmail.com> Lennart Regebro wrote: > I also would really like to see a real port of the bytes class to 2.6, > but I have a vague memory that there was some reason that wouldn't > work. Not so much that it wouldn't work, but that the interfaces to support using it effectively really aren't there - lots of areas in the standard library needed to be tweaked to cope with bytes objects in 3.x. Generally speaking, the "bytes = str" trick represents a reasonable compromise as the APIs that you would pass a bytes object to in 3.x expect an 8-bit str instance in 2.x. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From mrts.pydev at gmail.com Wed Nov 4 13:20:27 2009 From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=) Date: Wed, 4 Nov 2009 14:20:27 +0200 Subject: [Python-Dev] A wordcode-based Python In-Reply-To: <4213.88.149.182.147.1242111241.squirrel@webmail5.pair.com> References: <49D4DA72.60401@v.loewis.de> <4A0590C4.1020904@v.loewis.de> <20090509143829.17F293A4080@sparrow.telecommunity.com> <4A059639.7040505@v.loewis.de> <20090509153716.D44633A4080@sparrow.telecommunity.com> <7FF9D9A9-211E-4E5D-BDD0-9C0315123975@zooko.com> <4c8bd6707712f01ccf3841c2c26169ef@preisshare.net> <1024.88.149.182.147.1242064816.squirrel@webmail5.pair.com> <43aa6ff70905111414n62d20099r9bb2b3ebd52a26ec@mail.gmail.com> <4213.88.149.182.147.1242111241.squirrel@webmail5.pair.com> Message-ID: On Tue, May 12, 2009 at 8:54 AM, Cesare Di Mauro wrote: >> Also, I checked out wpython at head to run Unladen Swallow's >> benchmarks against it, but it refuses to compile with either gcc 4.0.1 >> or 4.3.1 on Linux (fails in Python/ast.c). I can send you the build >> failures off-list, if you're interested. >> >> Thanks, >> Collin Winter > > I'm very interested, thanks. That's because I worked only on Windows > machines, so I definitely need to test and fix it to let it run on any other > platform. > > Cesare Re-animating an old discussion -- Cesare, any news on the wpython front? I did a checkout from http://wpython.googlecode.com/svn/trunk and was able to ./configure and make successfully on my 64-bit Linux box as well as to run the Unladen benchmarks. Given svn co http://svn.python.org/projects/python/tags/r261 in py261 and svn co http://wpython.googlecode.com/svn/trunk in wpy, $ python unladen-tests/perf.py -rm --benchmarks=-2to3,all py261/python wpy/python gives the following results: Report on Linux foo 2.6.31-14-generic #48-Ubuntu SMP Fri Oct 16 14:05:01 UTC 2009 x86_64 Total CPU cores: 2 ai: Min: 0.640516 -> 0.586532: 9.20% faster Avg: 0.677346 -> 0.632785: 7.04% faster Significant (t=4.336740, a=0.95) Stddev: 0.05839 -> 0.08455: 30.94% larger Mem max: 7412.000 -> 6768.000: 9.52% smaller Usage over time: http://tinyurl.com/ykwhmcc call_simple: Min: 1.880816 -> 1.701622: 10.53% faster Avg: 1.944320 -> 1.778701: 9.31% faster Significant (t=14.323045, a=0.95) Stddev: 0.09885 -> 0.06000: 64.74% smaller Mem max: 8100.000 -> 6636.000: 22.06% smaller Usage over time: http://tinyurl.com/yzsswgp django: Min: 1.287158 -> 1.315700: 2.17% slower Avg: 1.330423 -> 1.366978: 2.67% slower Significant (t=-4.475769, a=0.95) Stddev: 0.05663 -> 0.05885: 3.78% larger Mem max: 15508.000 -> 16228.000: 4.44% larger Usage over time: http://tinyurl.com/yfpbmjn iterative_count: Min: 0.211620 -> 0.124646: 69.78% faster Avg: 0.222778 -> 0.159868: 39.35% faster Significant (t=9.291635, a=0.95) Stddev: 0.04239 -> 0.05279: 19.69% larger Mem max: 7388.000 -> 6680.000: 10.60% smaller Usage over time: http://tinyurl.com/yj7s8h4 normal_startup: Min: 1.060017 -> 0.991366: 6.92% faster Avg: 1.189612 -> 1.170067: 1.67% faster Significant (t=2.002086, a=0.95) Stddev: 0.06942 -> 0.06864: 1.13% smaller Mem max: 3252.000 -> 4648.000: 30.03% larger Usage over time: http://tinyurl.com/ygo3bwt pickle: Min: 2.027566 -> 1.948784: 4.04% faster Avg: 2.051633 -> 2.043656: 0.39% faster Not significant Stddev: 0.03095 -> 0.07348: 57.88% larger Mem max: 8544.000 -> 7340.000: 16.40% smaller Usage over time: http://tinyurl.com/ykg9dn2 pickle_dict: Min: 1.658693 -> 1.656844: 0.11% faster Avg: 1.689483 -> 1.698176: 0.51% slower Not significant Stddev: 0.16945 -> 0.09403: 80.20% smaller Mem max: 6716.000 -> 7636.000: 12.05% larger Usage over time: http://tinyurl.com/yjhyame pickle_list: Min: 0.919083 -> 0.894758: 2.72% faster Avg: 0.956513 -> 0.921314: 3.82% faster Significant (t=2.131237, a=0.95) Stddev: 0.12744 -> 0.10506: 21.31% smaller Mem max: 6804.000 -> 8792.000: 22.61% larger Usage over time: http://tinyurl.com/ylc3ezf pybench: Min: 58781 -> 50836: 15.63% faster Avg: 60009 -> 51788: 15.87% faster regex_compile: Min: 0.934131 -> 0.862323: 8.33% faster Avg: 0.962159 -> 0.884848: 8.74% faster Significant (t=13.587168, a=0.95) Stddev: 0.04685 -> 0.03229: 45.11% smaller Mem max: 12584.000 -> 12740.000: 1.22% larger Usage over time: http://tinyurl.com/yjngu8j regex_effbot: Min: 0.130686 -> 0.122483: 6.70% faster Avg: 0.143453 -> 0.138078: 3.89% faster Not significant Stddev: 0.01864 -> 0.03177: 41.32% larger Mem max: 7652.000 -> 6660.000: 14.89% smaller Usage over time: http://tinyurl.com/ykcgntf regex_v8: Min: 0.135130 -> 0.150092: 9.97% slower Avg: 0.138027 -> 0.177309: 22.15% slower Significant (t=-8.197595, a=0.95) Stddev: 0.00258 -> 0.04785: 94.60% larger Mem max: 11124.000 -> 12236.000: 9.09% larger Usage over time: http://tinyurl.com/ykb5vzu rietveld: Min: 0.848245 -> 0.816473: 3.89% faster Avg: 1.033925 -> 1.019889: 1.38% faster Not significant Stddev: 0.11242 -> 0.13006: 13.56% larger Mem max: 23792.000 -> 24548.000: 3.08% larger Usage over time: http://tinyurl.com/yhdvz5v slowpickle: Min: 0.876736 -> 0.800203: 9.56% faster Avg: 0.932808 -> 0.870577: 7.15% faster Significant (t=5.020426, a=0.95) Stddev: 0.05600 -> 0.11059: 49.36% larger Mem max: 7200.000 -> 7276.000: 1.04% larger Usage over time: http://tinyurl.com/ykt2brq slowspitfire: Min: 1.029100 -> 0.948458: 8.50% faster Avg: 1.062486 -> 1.020777: 4.09% faster Significant (t=4.581669, a=0.95) Stddev: 0.05441 -> 0.07298: 25.44% larger Mem max: 139792.000 -> 129264.000: 8.14% smaller Usage over time: http://tinyurl.com/yh7vmlh slowunpickle: Min: 0.411744 -> 0.356784: 15.40% faster Avg: 0.444638 -> 0.393261: 13.06% faster Significant (t=7.009269, a=0.95) Stddev: 0.04147 -> 0.06044: 31.38% larger Mem max: 7132.000 -> 7848.000: 9.12% larger Usage over time: http://tinyurl.com/yfwvz3g startup_nosite: Min: 0.664456 -> 0.598770: 10.97% faster Avg: 0.933034 -> 0.761228: 22.57% faster Significant (t=20.660776, a=0.95) Stddev: 0.09645 -> 0.06728: 43.37% smaller Mem max: 1940.000 -> 1940.000: -0.00% smaller Usage over time: http://tinyurl.com/yzzxcmd threaded_count: Min: 0.220059 -> 0.138708: 58.65% faster Avg: 0.232347 -> 0.156120: 48.83% faster Significant (t=23.804797, a=0.95) Stddev: 0.01889 -> 0.02586: 26.96% larger Mem max: 6460.000 -> 7664.000: 15.71% larger Usage over time: http://tinyurl.com/yzm3awu unpack_sequence: Min: 0.000129 -> 0.000120: 7.57% faster Avg: 0.000218 -> 0.000194: 12.14% faster Significant (t=3.946194, a=0.95) Stddev: 0.00139 -> 0.00128: 8.13% smaller Mem max: 18948.000 -> 19056.000: 0.57% larger Usage over time: http://tinyurl.com/yf8es3f unpickle: Min: 1.191468 -> 1.206198: 1.22% slower Avg: 1.248471 -> 1.281957: 2.61% slower Significant (t=-2.658526, a=0.95) Stddev: 0.05513 -> 0.11325: 51.32% larger Mem max: 7776.000 -> 8676.000: 10.37% larger Usage over time: http://tinyurl.com/yz96gw2 unpickle_list: Min: 0.922200 -> 0.861167: 7.09% faster Avg: 0.955964 -> 0.976829: 2.14% slower Not significant Stddev: 0.04374 -> 0.21061: 79.23% larger Mem max: 6820.000 -> 8324.000: 18.07% larger Usage over time: http://tinyurl.com/yjbraxg --- The diff between the two trees is at http://dpaste.org/RpIv/ Best, Mart S?mermaa From mal at egenix.com Wed Nov 4 13:21:01 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 04 Nov 2009 13:21:01 +0100 Subject: [Python-Dev] Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?) In-Reply-To: <4AF16CC7.6000705@gmail.com> References: <4AEF4C53.7090902@v.loewis.de> <4AEFF3A3.3020103@egenix.com> <4AF06E20.7050307@v.loewis.de> <4AF0816A.706@egenix.com> <319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com> <4AF16CC7.6000705@gmail.com> Message-ID: <4AF171AD.1000703@egenix.com> Nick Coghlan wrote: > Lennart Regebro wrote: >> I also would really like to see a real port of the bytes class to 2.6, >> but I have a vague memory that there was some reason that wouldn't >> work. > > Not so much that it wouldn't work, but that the interfaces to support > using it effectively really aren't there - lots of areas in the standard > library needed to be tweaked to cope with bytes objects in 3.x. > > Generally speaking, the "bytes = str" trick represents a reasonable > compromise as the APIs that you would pass a bytes object to in 3.x > expect an 8-bit str instance in 2.x. Could you please add such hints, tricks and tips to the wiki page: http://wiki.python.org/moin/PortingToPy3k Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 04 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From ziade.tarek at gmail.com Wed Nov 4 13:29:35 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 4 Nov 2009 13:29:35 +0100 Subject: [Python-Dev] Refactoring installation schemes In-Reply-To: <94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com> References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com> <4AE8206A.9020108@egenix.com> <4AE824DA.4090701@voidspace.org.uk> <94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com> Message-ID: <94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com> On Wed, Oct 28, 2009 at 7:05 PM, Tarek Ziad? wrote: > Ok then I'll work on a patch for that change and start an issue about it soon. As I expected, being able to provide all those paths pulls a lot of other stuffs out of distutils. In fact, almost all the APIs that are located in distutils/sysconfig can be taken out of distutils, and cleaned up for stdlib's benefit. I've started to refactor the code in a module I have called "sysconfig", reusing distutils/sysconfig, distutils/command/install and site code. This "sysconfig" module should provide at the end very useful APIs to query the current Python environment. That's a work in progress but: I've started a branch at /python/branches/tarek_sysconfig so it's easier to get some feedbacks if anyone want to help on this. Regards Tarek From solipsis at pitrou.net Wed Nov 4 13:43:19 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 4 Nov 2009 12:43:19 +0000 (UTC) Subject: [Python-Dev] 2to3 interactive mode References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> Message-ID: Glyph Lefkowitz twistedmatrix.com> writes: > > Keep in mind also that the 2.x translation process is extremely slow > and results in a clunky development process. There's no '2to3 -- > interactive' commandline that lets me type python 2 at a >>> prompt > and get python 3 results out so that I can try experiments on the 3.x > interpreter; I have to actually put my experiments into my unit tests > and wait 10 minutes to see if it works. It's like writing C++. Please enter a feature request into the bug tracker. Regards Antoine. From ncoghlan at gmail.com Wed Nov 4 13:45:45 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 04 Nov 2009 22:45:45 +1000 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> <4AF06A5F.6090501@v.loewis.de> Message-ID: <4AF17779.40206@gmail.com> Terry Reedy wrote: > Guido van Rossum wrote: >> On Tue, Nov 3, 2009 at 9:37 AM, "Martin v. L?wis" >> wrote: >>> (and no, adding things like nonlocal to 2.7 doesn't making porting of >>> a real application or library any easier, since the existing application >>> or library simply doesn't use that keyword. >> >> Agreed. >> >>> In fact, no change to 2.x >>> can reasonably simplify porting - only changes to 3.x might - except >>> for changes to 2to3, which can simplify porting a lot. But 2to3 should >>> be run under 3.x, IMO.) >> >> Disagreed. Better "-3" warnings could make porting easier. (Not just >> more warnings -- "better" might mean fewer false positives for >> warnings already issued.) > > There is also Eric Smith's list to consider: PEP3118 new buffer > protocol, short float repr, and maybe io. The pure Python io module was already backported for the 2.6 release [1], as was the C API aspect of PEP 3118 [2]. Short float repr has since been backported for 2.7, as has the C accelerated io module implementation and the Python API (memoryview) aspect of PEP 3118. I believe those 3 features alone are more than enough justification to proceed with at least a 2.7 release (that is probably the point Eric was making in posting that list of features in the first place). As to how those backports can help with forward ports to Py3k, someone made the point elsewhere in the thread that testing/experimenting via 2to3 is a very C++ like development cycle - there is a long build time before you get to see the results of running a test. With features backported to 2.x, you can instead use more traditional version checks (or the interactive prompt) and get the usual quick feedback cycle via the 2.7 version, before submitting your code to the tender mercies of the 2to3 converter (or possibly avoid 2to3 altogether if the version checks turn out to suffice for a given use case). Cheers, Nick. [1] http://docs.python.org/whatsnew/2.6.html#pep-3116-new-i-o-library [2] http://docs.python.org/whatsnew/2.6.html#pep-3118-revised-buffer-protocol -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From list at qtrac.plus.com Wed Nov 4 14:18:24 2009 From: list at qtrac.plus.com (Mark Summerfield) Date: Wed, 4 Nov 2009 13:18:24 +0000 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF17779.40206@gmail.com> References: <4AF17779.40206@gmail.com> Message-ID: <200911041318.24648.list@qtrac.plus.com> Hi, I wanted to make some brief comments on this thread: - 2to3 encourages people to see Python 3 as exotic and other---and not to actually write in it. - 3to2 encourages people to use Python 3 and also provides a route to Python 2 compatibility. I hope that a point will be reached where people are encouraged to do a one off 2to3, hand fix, and once it passes their tests to keep a single Python 3 source and use 3to2 to support their users of older Pythons. - Unicode strings is the solution, not the problem, and is one of Python 3's most important advances. - Have any big ports been done? Yes, PyQt4. PyQt4 supports both Python 2 and Python 3---and the port was done by one person in his "spare" time over a period of months. PyQt4 wraps at least 700,000 lines of C++ code---and it isn't just GUI stuff, it has networking, threading, etc., and works on Linux, Mac, Windows, etc. - I do hope NumPy gets ported, since both on and off the lists it seems like a show-stopper for many people. - I hope the "ditch 3" calls are ignored. Python 3 is significantly better than (an already excellent) Python 2: eventually people will port---or those who start out with Python 3 will build their own libraries for what's missing, just as people did when Python 2 came out. - I think the developers have done a fantastic job with Python 3. I just wish more people realised how good it is! Regarding the Moratorium: +inf since I'd really love to see more time devoted to improving the standard library. My 2c:-) -- Mark Summerfield, Qtrac Ltd, www.qtrac.eu C++, Python, Qt, PyQt - training and consultancy "Advanced Qt Programming" - ISBN 0321635906 From ncoghlan at gmail.com Wed Nov 4 14:20:55 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 04 Nov 2009 23:20:55 +1000 Subject: [Python-Dev] Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?) In-Reply-To: <4AF171AD.1000703@egenix.com> References: <4AEF4C53.7090902@v.loewis.de> <4AEFF3A3.3020103@egenix.com> <4AF06E20.7050307@v.loewis.de> <4AF0816A.706@egenix.com> <319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com> <4AF16CC7.6000705@gmail.com> <4AF171AD.1000703@egenix.com> Message-ID: <4AF17FB7.5090708@gmail.com> M.-A. Lemburg wrote: > Nick Coghlan wrote: >> Lennart Regebro wrote: >>> I also would really like to see a real port of the bytes class to 2.6, >>> but I have a vague memory that there was some reason that wouldn't >>> work. >> Not so much that it wouldn't work, but that the interfaces to support >> using it effectively really aren't there - lots of areas in the standard >> library needed to be tweaked to cope with bytes objects in 3.x. >> >> Generally speaking, the "bytes = str" trick represents a reasonable >> compromise as the APIs that you would pass a bytes object to in 3.x >> expect an 8-bit str instance in 2.x. > > Could you please add such hints, tricks and tips to the wiki > page: > > http://wiki.python.org/moin/PortingToPy3k Done (although the task of figuring out how to get the wiki to display code properly defeated me... the normal Python documentation syntax for it seemed to give the wiki's ReST interpreter fits). I also mentioned the trick someone mentioned about "from __future__ import unicode_literals" not changing the meaning of 'str' since it only alters the parser but not the builtins. In writing it up, it occurred to me that having that kind of thing in a "py3_compat" compatibility module (to be used as, e.g., "from py3_compat import str, bytes") would not only make it easier to use in multiple modules, but also easier for 2to3 to remove when forward porting. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From cesare.di.mauro at gmail.com Wed Nov 4 14:30:01 2009 From: cesare.di.mauro at gmail.com (Cesare Di Mauro) Date: Wed, 4 Nov 2009 14:30:01 +0100 Subject: [Python-Dev] A wordcode-based Python In-Reply-To: References: <49D4DA72.60401@v.loewis.de> <20090509143829.17F293A4080@sparrow.telecommunity.com> <4A059639.7040505@v.loewis.de> <20090509153716.D44633A4080@sparrow.telecommunity.com> <7FF9D9A9-211E-4E5D-BDD0-9C0315123975@zooko.com> <4c8bd6707712f01ccf3841c2c26169ef@preisshare.net> <1024.88.149.182.147.1242064816.squirrel@webmail5.pair.com> <43aa6ff70905111414n62d20099r9bb2b3ebd52a26ec@mail.gmail.com> <4213.88.149.182.147.1242111241.squirrel@webmail5.pair.com> Message-ID: Hi Mart I had some problems and little time to dedicate to wpython in the last period, but I restarted again with it in the last month. Currently I'm working on changing and documenting the code so that almost every optimization can be selected. So you'll compile it enabling only the ones you are interested in. I've also investigated about some ideas which Antoine told me on grouping together FASTs and CONSTs in order to reduce bytecodes, but I've found that the suggested solution brings some problems with the current function call implementation that can hurt performance on some situations (mostly with recursive ones, because usually they need to create new frames, and constants references must be copied and INCREFed). Since it will require huge changes to the current code base, I don't know if it's worth the effort just to verify the idea. I'll think about it when the project will be "finalized". My plan is to finish the current work in a few days, and then remove the (may be ugly) hacks that I made to the Python object model that were needed to let tuples, lists and dictionaries be loaded as CONSTs. May be a the end of the month it'll be fixed (and the diffs against CPython will be reduced a lot, since a few files results changed). Next, I need to changed the trace code (in frameobject.c) to let the test_trace.py pass (at this time two tests are disabled because the VM crashes). Finally, I think to update the code base to 2.6.4. I think to release everything at the end of the year, but if someone is interested I can do a partial release at the end of November. Regarding your tests, they are very interesting, particularly for regex_v8 that showed an unexpected result for me. I'll investigate about it after I'll release wpython. I you have any questions, I'm at your disposal (thanks for your tests!) Cesare 2009/11/4 Mart S?mermaa > On Tue, May 12, 2009 at 8:54 AM, Cesare Di Mauro > wrote: > >> Also, I checked out wpython at head to run Unladen Swallow's > >> benchmarks against it, but it refuses to compile with either gcc 4.0.1 > >> or 4.3.1 on Linux (fails in Python/ast.c). I can send you the build > >> failures off-list, if you're interested. > >> > >> Thanks, > >> Collin Winter > > > > I'm very interested, thanks. That's because I worked only on Windows > > machines, so I definitely need to test and fix it to let it run on any > other > > platform. > > > > Cesare > > Re-animating an old discussion -- Cesare, any news on the wpython front? > > I did a checkout from http://wpython.googlecode.com/svn/trunk and > was able to ./configure and make successfully on my 64-bit Linux box > as well as to run the Unladen benchmarks. > > Given svn co http://svn.python.org/projects/python/tags/r261 in py261 > and svn co http://wpython.googlecode.com/svn/trunk in wpy, > > $ python unladen-tests/perf.py -rm --benchmarks=-2to3,all py261/python > wpy/python > > gives the following results: > > Report on Linux foo 2.6.31-14-generic #48-Ubuntu SMP Fri Oct 16 > 14:05:01 UTC 2009 x86_64 > Total CPU cores: 2 > > ai: > Min: 0.640516 -> 0.586532: 9.20% faster > Avg: 0.677346 -> 0.632785: 7.04% faster > Significant (t=4.336740, a=0.95) > Stddev: 0.05839 -> 0.08455: 30.94% larger > > Mem max: 7412.000 -> 6768.000: 9.52% smaller > Usage over time: http://tinyurl.com/ykwhmcc > > > call_simple: > Min: 1.880816 -> 1.701622: 10.53% faster > Avg: 1.944320 -> 1.778701: 9.31% faster > Significant (t=14.323045, a=0.95) > Stddev: 0.09885 -> 0.06000: 64.74% smaller > > Mem max: 8100.000 -> 6636.000: 22.06% smaller > Usage over time: http://tinyurl.com/yzsswgp > > > django: > Min: 1.287158 -> 1.315700: 2.17% slower > Avg: 1.330423 -> 1.366978: 2.67% slower > Significant (t=-4.475769, a=0.95) > Stddev: 0.05663 -> 0.05885: 3.78% larger > > Mem max: 15508.000 -> 16228.000: 4.44% larger > Usage over time: http://tinyurl.com/yfpbmjn > > > iterative_count: > Min: 0.211620 -> 0.124646: 69.78% faster > Avg: 0.222778 -> 0.159868: 39.35% faster > Significant (t=9.291635, a=0.95) > Stddev: 0.04239 -> 0.05279: 19.69% larger > > Mem max: 7388.000 -> 6680.000: 10.60% smaller > Usage over time: http://tinyurl.com/yj7s8h4 > > > normal_startup: > Min: 1.060017 -> 0.991366: 6.92% faster > Avg: 1.189612 -> 1.170067: 1.67% faster > Significant (t=2.002086, a=0.95) > Stddev: 0.06942 -> 0.06864: 1.13% smaller > > Mem max: 3252.000 -> 4648.000: 30.03% larger > Usage over time: http://tinyurl.com/ygo3bwt > > > pickle: > Min: 2.027566 -> 1.948784: 4.04% faster > Avg: 2.051633 -> 2.043656: 0.39% faster > Not significant > Stddev: 0.03095 -> 0.07348: 57.88% larger > > Mem max: 8544.000 -> 7340.000: 16.40% smaller > Usage over time: http://tinyurl.com/ykg9dn2 > > > pickle_dict: > Min: 1.658693 -> 1.656844: 0.11% faster > Avg: 1.689483 -> 1.698176: 0.51% slower > Not significant > Stddev: 0.16945 -> 0.09403: 80.20% smaller > > Mem max: 6716.000 -> 7636.000: 12.05% larger > Usage over time: http://tinyurl.com/yjhyame > > > pickle_list: > Min: 0.919083 -> 0.894758: 2.72% faster > Avg: 0.956513 -> 0.921314: 3.82% faster > Significant (t=2.131237, a=0.95) > Stddev: 0.12744 -> 0.10506: 21.31% smaller > > Mem max: 6804.000 -> 8792.000: 22.61% larger > Usage over time: http://tinyurl.com/ylc3ezf > > > pybench: > Min: 58781 -> 50836: 15.63% faster > Avg: 60009 -> 51788: 15.87% faster > > regex_compile: > Min: 0.934131 -> 0.862323: 8.33% faster > Avg: 0.962159 -> 0.884848: 8.74% faster > Significant (t=13.587168, a=0.95) > Stddev: 0.04685 -> 0.03229: 45.11% smaller > > Mem max: 12584.000 -> 12740.000: 1.22% larger > Usage over time: http://tinyurl.com/yjngu8j > > > regex_effbot: > Min: 0.130686 -> 0.122483: 6.70% faster > Avg: 0.143453 -> 0.138078: 3.89% faster > Not significant > Stddev: 0.01864 -> 0.03177: 41.32% larger > > Mem max: 7652.000 -> 6660.000: 14.89% smaller > Usage over time: http://tinyurl.com/ykcgntf > > > regex_v8: > Min: 0.135130 -> 0.150092: 9.97% slower > Avg: 0.138027 -> 0.177309: 22.15% slower > Significant (t=-8.197595, a=0.95) > Stddev: 0.00258 -> 0.04785: 94.60% larger > > Mem max: 11124.000 -> 12236.000: 9.09% larger > Usage over time: http://tinyurl.com/ykb5vzu > > > rietveld: > Min: 0.848245 -> 0.816473: 3.89% faster > Avg: 1.033925 -> 1.019889: 1.38% faster > Not significant > Stddev: 0.11242 -> 0.13006: 13.56% larger > > Mem max: 23792.000 -> 24548.000: 3.08% larger > Usage over time: http://tinyurl.com/yhdvz5v > > > slowpickle: > Min: 0.876736 -> 0.800203: 9.56% faster > Avg: 0.932808 -> 0.870577: 7.15% faster > Significant (t=5.020426, a=0.95) > Stddev: 0.05600 -> 0.11059: 49.36% larger > > Mem max: 7200.000 -> 7276.000: 1.04% larger > Usage over time: http://tinyurl.com/ykt2brq > > > slowspitfire: > Min: 1.029100 -> 0.948458: 8.50% faster > Avg: 1.062486 -> 1.020777: 4.09% faster > Significant (t=4.581669, a=0.95) > Stddev: 0.05441 -> 0.07298: 25.44% larger > > Mem max: 139792.000 -> 129264.000: 8.14% smaller > Usage over time: http://tinyurl.com/yh7vmlh > > > slowunpickle: > Min: 0.411744 -> 0.356784: 15.40% faster > Avg: 0.444638 -> 0.393261: 13.06% faster > Significant (t=7.009269, a=0.95) > Stddev: 0.04147 -> 0.06044: 31.38% larger > > Mem max: 7132.000 -> 7848.000: 9.12% larger > Usage over time: http://tinyurl.com/yfwvz3g > > > startup_nosite: > Min: 0.664456 -> 0.598770: 10.97% faster > Avg: 0.933034 -> 0.761228: 22.57% faster > Significant (t=20.660776, a=0.95) > Stddev: 0.09645 -> 0.06728: 43.37% smaller > > Mem max: 1940.000 -> 1940.000: -0.00% smaller > Usage over time: http://tinyurl.com/yzzxcmd > > > threaded_count: > Min: 0.220059 -> 0.138708: 58.65% faster > Avg: 0.232347 -> 0.156120: 48.83% faster > Significant (t=23.804797, a=0.95) > Stddev: 0.01889 -> 0.02586: 26.96% larger > > Mem max: 6460.000 -> 7664.000: 15.71% larger > Usage over time: http://tinyurl.com/yzm3awu > > > unpack_sequence: > Min: 0.000129 -> 0.000120: 7.57% faster > Avg: 0.000218 -> 0.000194: 12.14% faster > Significant (t=3.946194, a=0.95) > Stddev: 0.00139 -> 0.00128: 8.13% smaller > > Mem max: 18948.000 -> 19056.000: 0.57% larger > Usage over time: http://tinyurl.com/yf8es3f > > > unpickle: > Min: 1.191468 -> 1.206198: 1.22% slower > Avg: 1.248471 -> 1.281957: 2.61% slower > Significant (t=-2.658526, a=0.95) > Stddev: 0.05513 -> 0.11325: 51.32% larger > > Mem max: 7776.000 -> 8676.000: 10.37% larger > Usage over time: http://tinyurl.com/yz96gw2 > > > unpickle_list: > Min: 0.922200 -> 0.861167: 7.09% faster > Avg: 0.955964 -> 0.976829: 2.14% slower > Not significant > Stddev: 0.04374 -> 0.21061: 79.23% larger > > Mem max: 6820.000 -> 8324.000: 18.07% larger > Usage over time: http://tinyurl.com/yjbraxg > > --- > > The diff between the two trees is at > http://dpaste.org/RpIv/ > > Best, > Mart S?mermaa > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/cesare.di.mauro%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Wed Nov 4 16:06:55 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 4 Nov 2009 07:06:55 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> Message-ID: On Tue, Nov 3, 2009 at 9:51 PM, Glyph Lefkowitz wrote (amongst way too many words): > [...]?For example, 2.8 could emit a deprecation > warning for every old-style class that was defined, 2.9 could emit a > deprecation warning for every string constant declared without a 'b' or 'u' > prefix unless the module in question were in "3.x mode" (i.e. no-prefix == > 'u'). This proposal is hopelessly naive. It has been considered seriously from all possible sides before, and there just isn't a way to make this work. Not even with several releases as stepping points. -- --Guido van Rossum (python.org/~guido) From pje at telecommunity.com Wed Nov 4 16:37:26 2009 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 04 Nov 2009 10:37:26 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> Message-ID: <20091104153728.C2D043A408C@sparrow.telecommunity.com> At 12:51 AM 11/4/2009 -0500, Glyph Lefkowitz wrote: >With the 2.x series, users and operating systems seem to move on >fairly rapidly, because dependencies generally continue to work if you >upgrade just one version. This isn't quite as formal a requirement as >I would like (warnings get generated, unit tests fail, things do >break) but in practice, users can rely on it for most functionality. >If 3.x could be broken into a series of transitions like that, where >you can upgrade one version, fix some stuff, then upgrade another >version, even if you couldn't actually support more than 2 versions at >once, I think that we could pick up the migration pace to the point >where we might actually be using 3.x syntax in a few years. Having a >2.x series which goes to 2.9 and then stops isn't *quite* the same >thing as having one that moves over continuously to some 3.x version, >but it does seem to me that by that point the chasm between versions >will have narrowed to a crack, and the migration will be a little hop >over it rather than the currently-required great flying leap. +1 (I actually thought this was the original plan.) From collinw at gmail.com Wed Nov 4 16:54:13 2009 From: collinw at gmail.com (Collin Winter) Date: Wed, 4 Nov 2009 07:54:13 -0800 Subject: [Python-Dev] A wordcode-based Python In-Reply-To: References: <49D4DA72.60401@v.loewis.de> <20090509143829.17F293A4080@sparrow.telecommunity.com> <4A059639.7040505@v.loewis.de> <20090509153716.D44633A4080@sparrow.telecommunity.com> <7FF9D9A9-211E-4E5D-BDD0-9C0315123975@zooko.com> <4c8bd6707712f01ccf3841c2c26169ef@preisshare.net> <1024.88.149.182.147.1242064816.squirrel@webmail5.pair.com> <43aa6ff70905111414n62d20099r9bb2b3ebd52a26ec@mail.gmail.com> <4213.88.149.182.147.1242111241.squirrel@webmail5.pair.com> Message-ID: <43aa6ff70911040754k745015faocadb167e67664432@mail.gmail.com> On Wed, Nov 4, 2009 at 4:20 AM, Mart S?mermaa wrote: > On Tue, May 12, 2009 at 8:54 AM, Cesare Di Mauro > wrote: >>> Also, I checked out wpython at head to run Unladen Swallow's >>> benchmarks against it, but it refuses to compile with either gcc 4.0.1 >>> or 4.3.1 on Linux (fails in Python/ast.c). I can send you the build >>> failures off-list, if you're interested. >>> >>> Thanks, >>> Collin Winter >> >> I'm very interested, thanks. That's because I worked only on Windows >> machines, so I definitely need to test and fix it to let it run on any other >> platform. >> >> Cesare > > Re-animating an old discussion -- Cesare, any news on the wpython front? > > I did a checkout from http://wpython.googlecode.com/svn/trunk and > was able to ./configure and make successfully on my 64-bit Linux box > as well as to run the Unladen benchmarks. > > Given svn co http://svn.python.org/projects/python/tags/r261 in py261 > and svn co http://wpython.googlecode.com/svn/trunk in wpy, > > $ python unladen-tests/perf.py -rm --benchmarks=-2to3,all py261/python > wpy/python Do note that the --track_memory option to perf.py imposes some overhead that interferes with the performance figures. I'd recommend running the benchmarks again without --track_memory. That extra overhead is almost certainly what's causing some of the variability in the results. Collin Winter From guido at python.org Wed Nov 4 17:53:12 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 4 Nov 2009 08:53:12 -0800 Subject: [Python-Dev] nonlocal keyword in 2.x? In-Reply-To: <319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com> References: <4AEF4C53.7090902@v.loewis.de> <4AEFF3A3.3020103@egenix.com> <4AF06E20.7050307@v.loewis.de> <4AF0816A.706@egenix.com> <319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com> Message-ID: On Tue, Nov 3, 2009 at 11:06 PM, Lennart Regebro wrote: > I should maybe point out that although I'm generally +1 on > backporting, I'm not specifically anything on backporting the nonlocal > keyword. There are probably things that would help more from a > compatibility standpoint than that. > > For example, from __future__ import unicode_literals doesn't switch > the types. So this code will rais an assertion error in 2.6. > >>>> from __future__ import unicode_literals >>>> assert isinstance("fghjkl", str) > > While it of course will run fine in 3.0. Now this has a fairly trivial > workaround or two, but still. > > I also would really like to see a real port of the bytes class to 2.6, > but I have a vague memory that there was some reason that wouldn't > work. There are many interfaces that return a str instance which should become a bytes instance for code that is bytes-aware, but should stay a str instance (which after all has the same representation) for code that is not bytes-aware. In 3.x it's easy, such interfaces always return bytes instances. But in 2.x, it would have to be context-aware, and there are many reasons why that can't work. (E.g. an object could be shared between two modules, one of which is bytes-aware, while the other is not. Or a call could be made by a module that isn't bytes-aware and the return value passed to a module that is bytes-aware. Or vice versa. It's a mess unless *everyone* is bytes-aware, which is the 3.x world.) -- --Guido van Rossum (python.org/~guido) From mrts.pydev at gmail.com Wed Nov 4 18:32:07 2009 From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=) Date: Wed, 4 Nov 2009 19:32:07 +0200 Subject: [Python-Dev] A wordcode-based Python In-Reply-To: <43aa6ff70911040754k745015faocadb167e67664432@mail.gmail.com> References: <49D4DA72.60401@v.loewis.de> <4A059639.7040505@v.loewis.de> <20090509153716.D44633A4080@sparrow.telecommunity.com> <7FF9D9A9-211E-4E5D-BDD0-9C0315123975@zooko.com> <4c8bd6707712f01ccf3841c2c26169ef@preisshare.net> <1024.88.149.182.147.1242064816.squirrel@webmail5.pair.com> <43aa6ff70905111414n62d20099r9bb2b3ebd52a26ec@mail.gmail.com> <4213.88.149.182.147.1242111241.squirrel@webmail5.pair.com> <43aa6ff70911040754k745015faocadb167e67664432@mail.gmail.com> Message-ID: On Wed, Nov 4, 2009 at 5:54 PM, Collin Winter wrote: > Do note that the --track_memory option to perf.py imposes some > overhead that interferes with the performance figures. Thanks for the notice, without -m/--track_memory the deviation in results is indeed much smaller. > I'd recommend > running the benchmarks again without --track_memory. Done: $ python unladen-tests/perf.py -r --benchmarks=-2to3,all py261/python wpy/python Report on Linux zeus 2.6.31-14-generic #48-Ubuntu SMP Fri Oct 16 14:05:01 UTC 2009 x86_64 Total CPU cores: 2 ai: Min: 0.629343 -> 0.576259: 9.21% faster Avg: 0.634689 -> 0.581551: 9.14% faster Significant (t=39.404870, a=0.95) Stddev: 0.01259 -> 0.00484: 160.04% smaller call_simple: Min: 1.796710 -> 1.700046: 5.69% faster Avg: 1.801533 -> 1.716367: 4.96% faster Significant (t=137.452069, a=0.95) Stddev: 0.00522 -> 0.00333: 56.64% smaller django: Min: 1.280840 -> 1.275350: 0.43% faster Avg: 1.287179 -> 1.287233: 0.00% slower Not significant Stddev: 0.01055 -> 0.00581: 81.60% smaller iterative_count: Min: 0.211744 -> 0.123271: 71.77% faster Avg: 0.213148 -> 0.128596: 65.75% faster Significant (t=88.510311, a=0.95) Stddev: 0.00233 -> 0.00926: 74.80% larger normal_startup: Min: 0.520829 -> 0.516412: 0.86% faster Avg: 0.559170 -> 0.554678: 0.81% faster Not significant Stddev: 0.02031 -> 0.02093: 2.98% larger pickle: Min: 1.988127 -> 1.926643: 3.19% faster Avg: 2.000676 -> 1.936185: 3.33% faster Significant (t=36.712505, a=0.95) Stddev: 0.01650 -> 0.00603: 173.67% smaller pickle_dict: Min: 1.681116 -> 1.619192: 3.82% faster Avg: 1.701952 -> 1.629548: 4.44% faster Significant (t=34.513963, a=0.95) Stddev: 0.01721 -> 0.01200: 43.46% smaller pickle_list: Min: 0.918128 -> 0.884967: 3.75% faster Avg: 0.925534 -> 0.891200: 3.85% faster Significant (t=60.451407, a=0.95) Stddev: 0.00496 -> 0.00276: 80.00% smaller pybench: Min: 58692 -> 51128: 14.79% faster Avg: 59914 -> 52316: 14.52% faster regex_compile: Min: 0.894190 -> 0.816447: 9.52% faster Avg: 0.900353 -> 0.826003: 9.00% faster Significant (t=24.974080, a=0.95) Stddev: 0.00448 -> 0.02943: 84.78% larger regex_effbot: Min: 0.124442 -> 0.123750: 0.56% faster Avg: 0.134908 -> 0.126137: 6.95% faster Significant (t=5.496357, a=0.95) Stddev: 0.01581 -> 0.00218: 625.68% smaller regex_v8: Min: 0.132730 -> 0.143494: 7.50% slower Avg: 0.134287 -> 0.147387: 8.89% slower Significant (t=-40.654627, a=0.95) Stddev: 0.00108 -> 0.00304: 64.34% larger rietveld: Min: 0.754050 -> 0.737335: 2.27% faster Avg: 0.770227 -> 0.754642: 2.07% faster Significant (t=7.547765, a=0.95) Stddev: 0.01434 -> 0.01486: 3.49% larger slowpickle: Min: 0.858494 -> 0.795162: 7.96% faster Avg: 0.862350 -> 0.799479: 7.86% faster Significant (t=133.690989, a=0.95) Stddev: 0.00394 -> 0.00257: 52.92% smaller slowspitfire: Min: 0.955587 -> 0.909843: 5.03% faster Avg: 0.965960 -> 0.925845: 4.33% faster Significant (t=16.351067, a=0.95) Stddev: 0.01237 -> 0.02119: 41.63% larger slowunpickle: Min: 0.409312 -> 0.346982: 17.96% faster Avg: 0.412381 -> 0.349148: 18.11% faster Significant (t=242.889869, a=0.95) Stddev: 0.00198 -> 0.00169: 17.61% smaller startup_nosite: Min: 0.195620 -> 0.194328: 0.66% faster Avg: 0.230811 -> 0.238523: 3.23% slower Significant (t=-3.869944, a=0.95) Stddev: 0.01932 -> 0.02052: 5.87% larger threaded_count: Min: 0.222133 -> 0.133764: 66.06% faster Avg: 0.236670 -> 0.147750: 60.18% faster Significant (t=57.472693, a=0.95) Stddev: 0.01317 -> 0.00813: 61.98% smaller unpack_sequence: Min: 0.000129 -> 0.000119: 8.43% faster Avg: 0.000132 -> 0.000123: 7.22% faster Significant (t=24.614061, a=0.95) Stddev: 0.00003 -> 0.00011: 77.02% larger unpickle: Min: 1.191255 -> 1.149132: 3.67% faster Avg: 1.218023 -> 1.162351: 4.79% faster Significant (t=21.222711, a=0.95) Stddev: 0.02242 -> 0.01362: 64.54% smaller unpickle_list: Min: 0.880991 -> 0.965611: 8.76% slower Avg: 0.898949 -> 0.985231: 8.76% slower Significant (t=-17.387537, a=0.95) Stddev: 0.04838 -> 0.01103: 338.79% smaller From ssteinerx at gmail.com Wed Nov 4 19:20:42 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Wed, 4 Nov 2009 13:20:42 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com> References: <319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com> Message-ID: On Nov 4, 2009, at 1:06 AM, Lennart Regebro wrote: > 2009/11/3 ssteinerX at gmail.com : >> >> On Nov 2, 2009, at 7:26 PM, James Y Knight wrote: >>> >>> It really sounds like you're saying that switching to 3.x isn't >>> worth the >>> cost to you, but you want to force people (including yourself) to >>> do so >>> anyways, because ...? >> >> Because that's the future of Python > > Or not. Maybe it's a dead branch of Python? Maybe the 3.x line should just be put out of our misery, merged back to 2.7, 2.8, 2.9, and proceed as Glyph suggested in passing with increasing levels of deprecation until it just turns into 3.x on its own by running out of numbers. S From martin at v.loewis.de Wed Nov 4 19:37:51 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 04 Nov 2009 19:37:51 +0100 Subject: [Python-Dev] Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?) In-Reply-To: <4AF16CC7.6000705@gmail.com> References: <4AEF4C53.7090902@v.loewis.de> <4AEFF3A3.3020103@egenix.com> <4AF06E20.7050307@v.loewis.de> <4AF0816A.706@egenix.com> <319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com> <4AF16CC7.6000705@gmail.com> Message-ID: <4AF1C9FF.2060302@v.loewis.de> Nick Coghlan wrote: > Lennart Regebro wrote: >> I also would really like to see a real port of the bytes class to 2.6, >> but I have a vague memory that there was some reason that wouldn't >> work. > > Not so much that it wouldn't work, but that the interfaces to support > using it effectively really aren't there - lots of areas in the standard > library needed to be tweaked to cope with bytes objects in 3.x. I see the problem differently: if a bytes type was added, nothing would use it. In particular, IO wouldn't start returning bytes (although it could accept them); IO would continue to return str. Therefore, I'm skeptical that adding a *third* string type to 3.x would do any good. Regards, Martin From ctrachte at gmail.com Wed Nov 4 19:39:43 2009 From: ctrachte at gmail.com (Carl Trachte) Date: Wed, 4 Nov 2009 11:39:43 -0700 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com> Message-ID: <426ada670911041039m6df85994v321342231fbd3805@mail.gmail.com> On 11/4/09, ssteinerX at gmail.com wrote: > Maybe the 3.x line should just be put out of our misery, merged back > to 2.7, 2.8, 2.9, and proceed as Glyph suggested in passing with > increasing levels of deprecation until it just turns into 3.x on its > own by running out of numbers. As a user, I'm horrified. Granted, I'm not the most high powered user, but . . . my employer is already providing me with a 3.0 Python version on one of my work computers with the expectation that I'll be using it more and more. Sorry to butt in, but is this a joke? I thought all this was hashed out prior to inventing python 3.0. From arcriley at gmail.com Wed Nov 4 19:43:02 2009 From: arcriley at gmail.com (Arc Riley) Date: Wed, 4 Nov 2009 13:43:02 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com> Message-ID: That's not going to happen. Stop trolling the python-dev list. On Wed, Nov 4, 2009 at 1:20 PM, ssteinerX at gmail.com wrote: > Maybe the 3.x line should just be put out of our misery, merged back to > 2.7, 2.8, 2.9, and proceed as Glyph suggested in passing with increasing > levels of deprecation until it just turns into 3.x on its own by running out > of numbers. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Wed Nov 4 19:45:59 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 4 Nov 2009 10:45:59 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <426ada670911041039m6df85994v321342231fbd3805@mail.gmail.com> References: <319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com> <426ada670911041039m6df85994v321342231fbd3805@mail.gmail.com> Message-ID: On Wed, Nov 4, 2009 at 10:39 AM, Carl Trachte wrote: > On 11/4/09, ssteinerX at gmail.com wrote: > >> Maybe the 3.x line should just be put out of our misery, merged back >> to 2.7, 2.8, 2.9, and proceed as Glyph suggested in passing with >> increasing levels of deprecation until it just turns into 3.x on its >> own by running out of numbers. > > > As a user, I'm horrified. ?Granted, I'm not the most high powered > user, but . . . > my employer is already providing me with a 3.0 Python version on one > of my work computers with the expectation that I'll be using it more > and more. > > Sorry to butt in, but is this a joke? ?I thought all this was hashed > out prior to inventing python 3.0. > I have no idea who "ssteinerX" is. He certainly doesn't speak for the core developers. -- --Guido van Rossum (python.org/~guido) From martin at v.loewis.de Wed Nov 4 19:46:39 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 04 Nov 2009 19:46:39 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFC887.6020801@v.loewis.de> <3DE1D8FB-896E-441D-937D-AB63EB3A8E22@python.org> <5b8d13220911030540s3d700265ib00f4277b7ce4a2b@mail.gmail.com> <79990c6b0911031416r36c6947ldd19984108d46f7e@mail.gmail.com> Message-ID: <4AF1CC0F.8090709@v.loewis.de> Antoine Pitrou wrote: > Paul Moore gmail.com> writes: >> FWIW, I did a quick survey of some packages (a sampling of packages >> I've used or considered using in the past): >> >> Twisted - no plans yet for Python 3 > > Well Twisted depends on zope.interface which is not ported yet. That's not strictly true, see http://svn.zope.org/zope.interface/branches/regebro-python3/ While this isn't released yet, Lennart and myself have been working to make it work on 2.x and 3.x. So porting activities *could* start now (requiring 3.x user to use that branch). Regards, Martin From martin at v.loewis.de Wed Nov 4 20:02:05 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 04 Nov 2009 20:02:05 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> Message-ID: <4AF1CFAD.6000305@v.loewis.de> > For one thing, we have a very long row to hoe here. The migration to > 3.0 is a long, tedious process with little tangible benefit. I hope > that sometime in the next decade Twisted can accelerate the process of > dropping old 2.x versions, but I seriously doubt we could do a > feature-complete 3.1/2.6 version. I get the general impression that a > 3.2/2.7 port would be more feasible; hopefully a 3.3/2.8 would be even > moreso. Please understand that you will not need to drop 2.x versions in order to support 3.x. Just add 3.x support now and make sure it won't break 2.x support. > Also, the benefits of migrating to python 3.x are still negligible, as > far as I can tell. For Twisted, most definitely - you will need to support 2.x and 3.x simultaneously, so you can't really benefit from 3.x-only changes for a long time to come - perhaps until a 3to2 tool has a good quality, and probably not even then (since it will restrict you what you can do in 3.x code). > On the other hand, you've got NumPy, PyGTK, Unladen Swallow, > PyPy, Jython, IronPython, and so on and so forth. Since I started using > it, the strength of Python has been in its ecosystem, and the 3.x > ecosystem is not yet viable. Right - the advantages wouldn't be for Twisted itself, but for users of Twisted, which would see a larger ecosystem if Twisted was available. > The main reason I want a long 2.x series is that I believe it would more > easily allow us infrastructure folks to drop support for *older* > versions. With this big 2.x->3.x chasm, I can't really see an end in > sight for Twisted using Python 2.x as its _source_ language, translating > with 2to3. Well, 3to2 would then be an option for you: use Python 3 as the source language. > Some projects which depend on Twisted and want new versions > (and security fixes, etc) are going to want Python 2.x for a really long > time. Maybe they're just really conservative, maybe they don't have a > lot of maintenance energy, or maybe they have other dependencies which > haven't got a port; it doesn't really matter, empirically speaking > people want older versions of Python. But wouldn't these applications also break as Twisted drops support for old 2.x versions, and the applications fail to work on the newer 2.x version (say, 2.34)? > Keep in mind also that the 2.x translation process is extremely slow and > results in a clunky development process. There's no '2to3 > --interactive' commandline that lets me type python 2 at a >>> prompt > and get python 3 results out so that I can try experiments on the 3.x > interpreter; I have to actually put my experiments into my unit tests > and wait 10 minutes to see if it works. It's like writing C++. That's not my experience. I see a change in source (say, on Django) available for 3.x within 5 seconds. Regards, Martin From martin at v.loewis.de Wed Nov 4 20:08:58 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 04 Nov 2009 20:08:58 +0100 Subject: [Python-Dev] No buildbot to test wide unicode? In-Reply-To: References: Message-ID: <4AF1D14A.6040007@v.loewis.de> > It seems that there is no buildbot to test a wide unicode build. > > On http://www.python.org/dev/buildbot/3.x/ all outputs of the "configure" > steps show this message:: > checking what type to use for str... unsigned short > which looks like a ucs2 build to me. > > Since wide unicode is the standard chosen by some Linux distributions, > it would make sense to have at least one buildbot running with > --with-wide-unicode (3.x) or --enable-unicode=ucs4 (2.x). Can you propose some (one? two? more?) systems that might be best as candidates? I'd then setup two (sets of) builders; they would share the slave lock, so builds would run sequentially (unless the slave operator agrees to setup two slaves on one machine). Regards, Martin From ssteinerx at gmail.com Wed Nov 4 20:13:48 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Wed, 4 Nov 2009 14:13:48 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <426ada670911041039m6df85994v321342231fbd3805@mail.gmail.com> References: <319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com> <426ada670911041039m6df85994v321342231fbd3805@mail.gmail.com> Message-ID: <404F0418-E18D-4F44-9F8A-31D8EF38B601@gmail.com> On Nov 4, 2009, at 1:39 PM, Carl Trachte wrote: > On 11/4/09, ssteinerX at gmail.com wrote: > >> Maybe the 3.x line should just be put out of our misery, merged back >> to 2.7, 2.8, 2.9, and proceed as Glyph suggested in passing with >> increasing levels of deprecation until it just turns into 3.x on its >> own by running out of numbers. > > > As a user, I'm horrified. Granted, I'm not the most high powered > user, but . . . > my employer is already providing me with a 3.0 Python version on one > of my work computers with the expectation that I'll be using it more > and more. > > Sorry to butt in, but is this a joke? I thought all this was hashed > out prior to inventing python 3.0. Yes, of course it was a joke. 2.7 won't "turn into" Python 3.x any more that Perl will turn into Ruby. Oh, wait, maybe that was a bad example. The point was, that Python 3.x does not seem to be something that can be "evolved" into and, all along, I have been suggesting that, if Python 3.x is the future, let's let 2.7 be the last of the 2.x series, backport whatever will make it easiest to make 2to3 do as much of the work as possible, and just decide that 2.7 is the end of the line. I shudder to think how much time has been spent hacking things around to make them compatible with the 2.x series while trying to move to 3.x. If 2.x is over, let it be over and let's all focus on moving into Python 3.x with no more time doing other than bug-fixes on 2.x versions of things. S From cournape at gmail.com Wed Nov 4 20:15:59 2009 From: cournape at gmail.com (David Cournapeau) Date: Thu, 5 Nov 2009 04:15:59 +0900 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF1CFAD.6000305@v.loewis.de> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> <4AF1CFAD.6000305@v.loewis.de> Message-ID: <5b8d13220911041115y207a7625j65007a6e6fe7a9a7@mail.gmail.com> On Thu, Nov 5, 2009 at 4:02 AM, "Martin v. L?wis" wrote: > > That's not my experience. I see a change in source (say, on Django) > available for 3.x within 5 seconds. This is for which version of 2to3 ? I got similar experience (several minutes), but maybe I am using 2to3 the wrong way. On my machine, with 2to3 from 3.1.1, it takes ~ 1s to convert one single file of 200 lines, and converting a tiny subset of numpy takes > one minute. David From mbk.lists at gmail.com Wed Nov 4 20:30:47 2009 From: mbk.lists at gmail.com (Mike Krell) Date: Wed, 4 Nov 2009 12:30:47 -0700 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF1CFAD.6000305@v.loewis.de> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> <4AF1CFAD.6000305@v.loewis.de> Message-ID: On Wed, Nov 4, 2009 at 12:02 PM, "Martin v. L?wis" wrote: > > The main reason I want a long 2.x series is that I believe it would more > > easily allow us infrastructure folks to drop support for *older* > > versions. With this big 2.x->3.x chasm, I can't really see an end in > > sight for Twisted using Python 2.x as its _source_ language, translating > > with 2to3. > > Well, 3to2 would then be an option for you: use Python 3 as the source > language. > A migration path which would be made all the more compelling with the addition of the nonlocal keyword to 2.7 ;-) Mike -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Wed Nov 4 20:56:27 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Wed, 04 Nov 2009 20:56:27 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <5b8d13220911041115y207a7625j65007a6e6fe7a9a7@mail.gmail.com> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> <4AF1CFAD.6000305@v.loewis.de> <5b8d13220911041115y207a7625j65007a6e6fe7a9a7@mail.gmail.com> Message-ID: <4AF1DC6B.1000305@v.loewis.de> >> That's not my experience. I see a change in source (say, on Django) >> available for 3.x within 5 seconds. > > This is for which version of 2to3 ? I got similar experience (several > minutes), but maybe I am using 2to3 the wrong way. On my machine, with > 2to3 from 3.1.1, it takes ~ 1s to convert one single file of 200 > lines, and converting a tiny subset of numpy takes > one minute. The version released with 3.1. The trick is not to recompile all 2to3 code every time you make a source change. Instead, cache the 2to3 output in the build area, and have setup.py only invoke 2to3 for the files that got modified. So whenever I make a change, I do "python3 setup.py install". This checks all timestamps, finds the modified files (which will only be one), runs 2to3 on it, and then copies it into my 3.1 installation, where I can test the change. Recompiling a single file typically takes a few seconds, or less. It would be possible to also run out of the build area; you still would need to run "setup.py build" after every change. There is already support for this in both distutils (as released in 3.1), and distribute. Regards, Martin From alexandre at peadrop.com Wed Nov 4 21:19:30 2009 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Wed, 4 Nov 2009 15:19:30 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: On Tue, Nov 3, 2009 at 12:35 PM, Guido van Rossum wrote: > I've checked draft (!) PEP 3003, "Python Language Moratorium", into > SVN. As authors I've listed Jesse, Brett and myself. > +1 from me. -- Alexandre From debatem1 at gmail.com Wed Nov 4 22:32:39 2009 From: debatem1 at gmail.com (geremy condra) Date: Wed, 4 Nov 2009 16:32:39 -0500 Subject: [Python-Dev] Retrieve an arbitrary element from a set withoutremoving it In-Reply-To: <200911042234.36796.steve@pearwood.info> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <200911040946.15805.steve@pearwood.info> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> Message-ID: On Wed, Nov 4, 2009 at 6:34 AM, Steven D'Aprano wrote: > On Wed, 4 Nov 2009 11:54:47 am Greg Ewing wrote: >> Steven D'Aprano wrote: >> > I don't know how expensive it is to create a set iterator, >> >> Not expensive enough to justify burdening the set type with >> extra functionality that will be extremely rarely used. > > As my previous posts on this topic tried to convey, this isn't primarily > about efficiency, but about discoverability and obviousness. > > Anyway, given the level of opposition to the suggestion, I'm no longer > willing to carry the flag for it. If anyone else -- perhaps the OP -- > feels they want to take it any further, be my guest. > > > > -- > Steven D'Aprano I've said before that I'd like there to be one, standard way of doing this. A function call- set.pick() seems reasonably named to me- is probably the cleanest way to do that. Absent that, an example in the docs that illustrates the preferred idiom would be great. Is there any kind of consensus on either point? Geremy Condra From brett at python.org Wed Nov 4 23:13:41 2009 From: brett at python.org (Brett Cannon) Date: Wed, 4 Nov 2009 14:13:41 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com> Message-ID: On Wed, Nov 4, 2009 at 10:20, ssteinerX at gmail.com wrote: > > On Nov 4, 2009, at 1:06 AM, Lennart Regebro wrote: > >> 2009/11/3 ssteinerX at gmail.com : >>> >>> On Nov 2, 2009, at 7:26 PM, James Y Knight wrote: >>>> >>>> It really sounds like you're saying that switching to 3.x isn't worth >>>> the >>>> cost to you, but you want to force people (including yourself) to do so >>>> anyways, because ...? >>> >>> Because that's the future of Python >> >> Or not. Maybe it's a dead branch of Python? > > Maybe the 3.x line should just be put out of our misery, merged back to 2.7, > 2.8, 2.9, and proceed as Glyph suggested in passing with increasing levels > of deprecation until it just turns into 3.x on its own by running out of > numbers. > I am going to say this once: we are not killing off Python 3. First off, Python 3 is not even a year old! Considering people have not fully migrated to 2.6, should we kill it off as well? There is a certain lack of perspective on time scale. This is especially true when Guido himself has said on multiple occasions that moving the community to 3.x would be a mult-year process, as in 3-5 years process, not 11 months. Second, the people calling for us to potentially kill 3.x and just keep 2.x floating along have yet to say that they have tried porting their code and that it was difficult. Every person who has stepped forward stating they have done a port has said it was actually relatively straight-forward. Not only that, we have anecdotal evidence from multiple people that you can support code way back to whatever old version of Python RHEL is running. Third, the same people calling for the death of 3.x have not suggested they have used it extensively (if at all). I have yet to hear anyone say that 3.x is not at least a nice improvement, if not a huge one. I for one find 3.x more enjoyable to work in than 2.x, and that's saying a lot since I obviously loved Python 2.x enough to get involved in its development. I have also never heard anyone ever say, "I gave 3.x a fair shake and honestly, I wish I had not wasted the time." Wait until 3to2 gets to a good state (which will happen; it's my next project -- after I either get us moved to Hg or I simply give up on it -- and I know I am not the only core developer planning on making it happen). I realize that there is some fear that it will be time wasted if people port their code to 3.x if it somehow burns out. But do you honestly think that python-dev would leave you hanging like that? Let's take a worst-case scenario here and say that direct pick-up of 3.x after a couple years never happens. Fine, we then begin to backport features. But if you already ported your code then chances are you already support the new features. And you know what one of the first things we would back port would be? Unicode strings and bytes. And since that is the hardest thing to port to, you will have not wasted any time. In other words the calling for the death of 3.x is rather premature and honestly unfair until people have actually tried to port their code in earnest and it has been a couple of years for the community to catch up to what python-dev is pushing out the door (which always takes a while no matter what minor version has been released). -Brett From david.lyon at preisshare.net Wed Nov 4 23:16:22 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 04 Nov 2009 17:16:22 -0500 Subject: [Python-Dev] Refactoring installation schemes In-Reply-To: <94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com> References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com> <4AE8206A.9020108@egenix.com> <4AE824DA.4090701@voidspace.org.uk> <94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com> <94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com> Message-ID: On Wed, 4 Nov 2009 13:29:35 +0100, Tarek Ziad? wrote: > I've started to refactor the code in a module I have called > "sysconfig", reusing distutils/sysconfig, distutils/command/install > and site code. > > This "sysconfig" module should provide at the end very useful APIs to > query the current Python environment. > > That's a work in progress but: I've started a branch at > /python/branches/tarek_sysconfig so it's easier to get some feedbacks > if anyone want to help on this. Good job so far. Keep going. imho a refactoring of the installation and build schemes of distutils is due. It seems like you easily have the skills to do it. My advice would be to do it gradually, as you are. And focus on simplification, and filling in the functional gaps. Ask people what the functional gaps are on their platforms and try to mould it together in an unplatform specific way. Myself and others can assist with this, but best to do it on distutils-sig. I would even go so far as to use the python 3 as a carrot for the new work. imho The biggest and best thing that you could do for python packaging is to do a refactor of the .EGG format. What I mean here, is *take* the egg, and run with it. Modernise it and revamp it into a platform independent thing. People 'get' the egg idea, they just hate the current implementation. What we all *need* is for the egg to be what the source distribution now is. Not for it to be some proprietory out out of python object. Call it the new Tarek egg... Then refactor distutils build to focus on good egg creation.. Many people will help you.. including myself.. David From ben+python at benfinney.id.au Wed Nov 4 23:23:33 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 05 Nov 2009 09:23:33 +1100 Subject: [Python-Dev] 2to3, 3to2: official status (was: 2.7 Release? 2.7 == last of the 2.x line?) References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> <4AF1CFAD.6000305@v.loewis.de> Message-ID: <87skcuc4ka.fsf_-_@benfinney.id.au> "Martin v. L?wis" writes: > Well, 3to2 would then be an option for you: use Python 3 as the source > language. I was under the impression that 2to3 was officially supported as part of Python, but 3to2 was a third-party tool. What's the status of 3to2 now? Is it an official part of Python? -- \ ?A free society is one where it is safe to be unpopular.? | `\ ?Adlai Ewing Stevenson | _o__) | Ben Finney From brett at python.org Wed Nov 4 23:28:57 2009 From: brett at python.org (Brett Cannon) Date: Wed, 4 Nov 2009 14:28:57 -0800 Subject: [Python-Dev] 2to3, 3to2: official status (was: 2.7 Release? 2.7 == last of the 2.x line?) In-Reply-To: <87skcuc4ka.fsf_-_@benfinney.id.au> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> <4AF1CFAD.6000305@v.loewis.de> <87skcuc4ka.fsf_-_@benfinney.id.au> Message-ID: On Wed, Nov 4, 2009 at 14:23, Ben Finney wrote: > "Martin v. L?wis" writes: > >> Well, 3to2 would then be an option for you: use Python 3 as the source >> language. > > I was under the impression that 2to3 was officially supported as part of > Python, but 3to2 was a third-party tool. What's the status of 3to2 now? > Is it an official part of Python? Nope, third-party while it continues to mature. -Brett From fdrake at acm.org Wed Nov 4 23:40:35 2009 From: fdrake at acm.org (Fred Drake) Date: Wed, 4 Nov 2009 17:40:35 -0500 Subject: [Python-Dev] Refactoring installation schemes In-Reply-To: References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com> <4AE8206A.9020108@egenix.com> <4AE824DA.4090701@voidspace.org.uk> <94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com> <94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com> Message-ID: <9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com> On Wed, Nov 4, 2009 at 5:16 PM, David Lyon wrote: > I would even go so far as to use the python 3 as a carrot for > the new work. The packaging story is in such bad shape that it needs the work regardless, and keeping it to Python 3 would significantly reduce the set of potential volunteers. > imho The biggest and best thing that you could do for python > packaging is to do a refactor of the .EGG format. > > What I mean here, is *take* the egg, and run with it. Modernise > it and revamp it into a platform independent thing. People > 'get' the egg idea, they just hate the current implementation. There's certainly work on that, but... is it that people hate the format? Or working with setuptools? The fact that there's more than one egg format doesn't help, so you may be right. > Call it the new Tarek egg... The tegg? ;-) -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller From martin at v.loewis.de Wed Nov 4 23:41:49 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Wed, 04 Nov 2009 23:41:49 +0100 Subject: [Python-Dev] 2to3, 3to2: official status In-Reply-To: <87skcuc4ka.fsf_-_@benfinney.id.au> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> <4AF1CFAD.6000305@v.loewis.de> <87skcuc4ka.fsf_-_@benfinney.id.au> Message-ID: <4AF2032D.5050406@v.loewis.de> Ben Finney wrote: > "Martin v. L?wis" writes: > >> Well, 3to2 would then be an option for you: use Python 3 as the source >> language. > > I was under the impression that 2to3 was officially supported as part of > Python, but 3to2 was a third-party tool. What's the status of 3to2 now? > Is it an official part of Python? No, the status is exactly as you describe it. Regards, Martin From zookog at gmail.com Wed Nov 4 23:54:44 2009 From: zookog at gmail.com (Zooko O'Whielacronx) Date: Wed, 4 Nov 2009 15:54:44 -0700 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com> References: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com> <93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com> Message-ID: Folks: It occurred to me to wonder why I haven't investigated how hard it would be to make my Python packages Python-3-compatible. That's right -- I haven't even looked closely. I couldn't even tell you off the top of my head what is in Python 3 that I would have to think about except for the new unicode regime. I think the answer is that the payoff is just *so* low to me at this point that it doesn't even justify me taking 15 minutes to read "What's New In Python 3" or to execute 2to3 on my smallest package and see what it does. On the other hand, I'm totally committed to supporting Python 2.7, because my customers will demand it and because I expect that it will be easy. So, if you guys slip in your favorite new Python 3 feature into 2.7 and add a deprecation warning for your least favorite Python 2 misfeature, then probably within about 24 months I'll have fixed all code that uses the deprecated feature, and probably within about five years I'll consider dropping backwards-compatibility with Python 2.6 and starting to use that new feature that you added to Python 2.7. (I'm currently considering dropping Python 2.4 compatibility for the next releases of most of my code.) Regards, Zooko From p.f.moore at gmail.com Thu Nov 5 00:23:31 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 4 Nov 2009 23:23:31 +0000 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com> <93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com> Message-ID: <79990c6b0911041523q514dd227jc71eb7809ebbf06e@mail.gmail.com> 2009/11/4 Zooko O'Whielacronx : > On the other hand, I'm totally committed to supporting Python 2.7, > because my customers will demand it and because I expect that it will > be easy. Why do you think your customers will demand 2.7 support but not 3.1 support? If I were one of your customers, I'd want 3.1 support... Paul. From floris.bruynooghe at gmail.com Thu Nov 5 00:27:43 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Wed, 4 Nov 2009 23:27:43 +0000 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com> <93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com> Message-ID: <20091104232743.GD19537@laurie.devork> On Wed, Nov 04, 2009 at 03:54:44PM -0700, Zooko O'Whielacronx wrote: > It occurred to me to wonder why I haven't investigated how hard it > would be to make my Python packages Python-3-compatible. That's right > -- I haven't even looked closely. I couldn't even tell you off the > top of my head what is in Python 3 that I would have to think about > except for the new unicode regime. I think the answer is that the > payoff is just *so* low to me at this point that it doesn't even > justify me taking 15 minutes to read "What's New In Python 3" or to > execute 2to3 on my smallest package and see what it does. You have time to read this thread but no time to read "What's New In Python 3"? Personally I found porting to Python 3 a rather pleasant experience (include C extension module). I can't wait until I can drop support for Python 2.2-2.X. Regards Floris PS: I have to admit that the commerial code base I work on is still at Python 2.5, but that doesn't make me worry in any way. It'll get to Python 3 in time (it's running on 2.6 already in development). -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From aahz at pythoncraft.com Thu Nov 5 00:53:25 2009 From: aahz at pythoncraft.com (Aahz) Date: Wed, 4 Nov 2009 15:53:25 -0800 Subject: [Python-Dev] PyCon 2010: Poster sessions Message-ID: <20091104235325.GA936@panix.com> PyCon 2010: Poster sessions =============================================================== Due date: November 30, 2009 PyCon 2010 introduces a new type of presentation, the poster session. Poster sessions consist of two pieces: * A display space where you can put up information about a topic * Live Q&A during a plenary timeslot where people can get more information from you while you stand next to your display For more information and to submit a poster proposal, visit http://us.pycon.org/2010/conference/posters/ -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ [on old computer technologies and programmers] "Fancy tail fins on a brand new '59 Cadillac didn't mean throwing out a whole generation of mechanics who started with model As." --Andrew Dalke From david.lyon at preisshare.net Thu Nov 5 01:13:17 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 04 Nov 2009 19:13:17 -0500 Subject: [Python-Dev] Refactoring installation schemes In-Reply-To: <9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com> References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com> <4AE8206A.9020108@egenix.com> <4AE824DA.4090701@voidspace.org.uk> <94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com> <94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com> <9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com> Message-ID: <39deb0d1eef08419dad65d7354a8ffb1@preisshare.net> On Wed, 4 Nov 2009 17:40:35 -0500, Fred Drake wrote: > The packaging story is in such bad shape that it needs the work > regardless, and keeping it to Python 3 would significantly reduce the > set of potential volunteers. Well I guess that is a 'marketing decision'. Not a coding issue. Actually, I don't honestly think that there is a lack of volounteers or any lack of resources. The main challenge imho is to get peoples bikesheds (including my own bikeshed) to line up in such a way that people can ride together sharing some sort of parts. Tarek has been doing just fine at facilitating this. > There's certainly work on that, but... is it that people hate the > format? Or working with setuptools? In my experience, working with the setuptools implementation is the difficult part. There's a barrage of impossible to remember command line options and the documentation is imho more convoluted than it needs to be. Let me put the distutils documentation forward as some sort of reference as to where the setuptools documentation should be. Then, there are some relatively minor issues that just annoy users to no end. The simple one is that they live in your site-packages directory unpacked. People wouldn't have so many issues if they lived like every other 'normal' package. As for the format itself, there's nothing to hate that I can see. > The fact that there's more than one egg format doesn't help, so you > may be right. Yes. The implementation can be confusing. That's the only problem imo. >> Call it the new Tarek egg... > > The tegg? ;-) l'oeuf ? In summmary, it doesn't matter so much what an 'egg' is. All that is important is that it works on lots of python platforms, distutils can easily pop them out, they are well documented and relatively trouble free. Most importantly, credit for the original idea goes back to PJE and everything gets refactored to make it 'nice'. I don't think it's a massive refactoring operation myself, and I'm very sure well within Tareks skillset to take on. David From tjreedy at udel.edu Thu Nov 5 01:42:31 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 04 Nov 2009 19:42:31 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> Message-ID: Glyph Lefkowitz wrote: > For what it's worth, the official position of the Twisted project is not > that we have "no plan" to move to Python 3. It's that our plan is to do > exactly as Raymond suggests, and give the users a vote - in this case, > you vote with your patches :). You probably will not hear from potential users who skip Twisted because it is not available for 3.x. I suspect you do not hear much either from new users who only installed 2.x to use Twisted, but would prefer 3.x. There are regular questions on python-list about 'web programming with 3.0' or some such. > For one thing, we have a very long row to hoe here. The migration to > 3.0 is a long, tedious process with little tangible benefit. One group that benefits is new Python programmers. Python 3 is easier to learn, and is being used to teach Python in at least some schools and universities, and will be used more as more libraries are available. Hardly a week goes by on Python list without someone posting a problem using 2.x that has been solved in 3.x. Another group is existing programmers who were/are sufficiently annoyed by some of the things that got cleaned up. A third group is people who want to use non-ascii in identifiers, and who are delighted now that they can. Since you do not fall in these groups, I understand your impatience and reluctance with the change. I can also imagine that Twisted may be more affected by some of the changes than most other projects. [snip more] ... > There have been some other comments in this thread indicating that this > was not the case because some users indicated that they'd rather deal > with lots of changes "all at once". I wrote that based on both my reading of clp/pylist posts during the discussion of the int/int semantic change and Guido's report of private conversations he had had. > My understanding is that it was > done this way so that the *developers* of Python could make a clean > break, and design and implement a new version of Python without being > constrained by compatibility concerns. I do not believe that was ever intended. It certainly is not what happened; many changes were not made *because* of compatibility concerns and all went through the filter of 'is the benefit of this change worth the pain of breakage'. There is a big difference between not being straightjacketed and being unconstrained. > If you can show me an actual > application or library developer in Python who wanted this one-big-jump > migration, I will show you a crazy person. Be careful of labels. Once the prolonged and intense int/int debate shifted from the ontology of ints to the pragmatics of the proposed change, most people agreed that int/int 'should' have meant float(int)/float(int) from the beginning. But some were still strongly opposed to making the change because they (understandably) did not want to have to scan (by eye) possibly 10000s or even 100000s of lines for every a/b to determine whether any fix was needed. Some said that that would be such a major change that it should not be done until there was a new major release, a Python 3 off in the then distant future. Well, that future is now. I half-jokingly suggested that the change be made on Guido's original timetable, but that the '2.5' that completed the change simply be relabeled '3.0'. I personally would have preferred that it had been completed in 2.5. But that did not happen and more changes were made once they were made, and here we are. Terry Jan Reedy From tjreedy at udel.edu Thu Nov 5 01:42:56 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 04 Nov 2009 19:42:56 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com> <93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com> Message-ID: Zooko O'Whielacronx wrote: > Folks: > So, if you guys slip in your favorite new Python 3 feature into 2.7 > and add a deprecation warning for your least favorite Python 2 > misfeature, Just run with the -3 flag for warnings. Also see my response to Glyph. Terry Jan Reedy From python at rcn.com Thu Nov 5 02:07:23 2009 From: python at rcn.com (Raymond Hettinger) Date: Wed, 4 Nov 2009 17:07:23 -0800 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com><200911040946.15805.steve@pearwood.info><4AF0D0D7.3090403@canterbury.ac.nz><200911042234.36796.steve@pearwood.info> Message-ID: <99BB308202224D31AA5300223DB59912@RaymondLaptop1> [Steven D'Aprano] >> Anyway, given the level of opposition to the suggestion, I'm no longer >> willing to carry the flag for it. If anyone else -- perhaps the OP -- >> feels they want to take it any further, be my guest. [geremy condra] > I've said before that I'd like there to be one, standard way of > doing this. A function call- set.pick() seems reasonably named > to me- is probably the cleanest way to do that. Absent that, > an example in the docs that illustrates the preferred idiom > would be great. Summarizing my opposition to a new set method: 1) there already are at least two succinct ways to get the same effect 2) those ways work with any container, not just sets 3) set implementations in other languages show that this isn't needed. 4) there is value to keeping the API compact 5) isn't needed for optimization (selecting the same value in a loop makes no sense) 6) absence of real-world code examples that would be meaningfully improved I would be happy to add an example to the docs so that this thread can finally end. Raymond From eric at trueblade.com Thu Nov 5 02:12:13 2009 From: eric at trueblade.com (Eric Smith) Date: Wed, 04 Nov 2009 20:12:13 -0500 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it In-Reply-To: <99BB308202224D31AA5300223DB59912@RaymondLaptop1> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com><200911040946.15805.steve@pearwood.info><4AF0D0D7.3090403@canterbury.ac.nz><200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> Message-ID: <4AF2266D.2020706@trueblade.com> Raymond Hettinger wrote: > Summarizing my opposition to a new set method: > 1) there already are at least two succinct ways to get the same effect > 2) those ways work with any container, not just sets > 3) set implementations in other languages show that this isn't needed. > 4) there is value to keeping the API compact > 5) isn't needed for optimization (selecting the same value in a loop > makes no sense) > 6) absence of real-world code examples that would be meaningfully improved > > I would be happy to add an example to the docs so that this thread > can finally end. Please do! Eric. From tjreedy at udel.edu Thu Nov 5 02:48:25 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 04 Nov 2009 20:48:25 -0500 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it In-Reply-To: <4AF2266D.2020706@trueblade.com> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com><200911040946.15805.steve@pearwood.info><4AF0D0D7.3090403@canterbury.ac.nz><200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <4AF2266D.2020706@trueblade.com> Message-ID: Eric Smith wrote: > Raymond Hettinger wrote: >> Summarizing my opposition to a new set method: >> 1) there already are at least two succinct ways to get the same effect >> 2) those ways work with any container, not just sets >> 3) set implementations in other languages show that this isn't needed. >> 4) there is value to keeping the API compact >> 5) isn't needed for optimization (selecting the same value in a loop >> makes no sense) >> 6) absence of real-world code examples that would be meaningfully >> improved Agreed >> I would be happy to add an example to the docs so that this thread >> can finally end. > > Please do! Yes! From ben+python at benfinney.id.au Thu Nov 5 03:49:22 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 05 Nov 2009 13:49:22 +1100 Subject: [Python-Dev] 2to3, 3to2: official status References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> <4AF1CFAD.6000305@v.loewis.de> <87skcuc4ka.fsf_-_@benfinney.id.au> <4AF2032D.5050406@v.loewis.de> Message-ID: <87bpjhd6tp.fsf@benfinney.id.au> "Martin v. L?wis" writes: > Ben Finney wrote: > > "Martin v. L?wis" writes: > > > >> Well, 3to2 would then be an option for you: use Python 3 as the > >> source language. > > > > I was under the impression that 2to3 was officially supported as > > part of Python, but 3to2 was a third-party tool. [?] Is it an > > official part of Python? > > No, the status is exactly as you describe it. Okay. It's probably best for anyone with their Python developer hat on (which, in this forum, is all the time for any Python developer) to make the status of 3to2 clear when recommending it to people concerned about future plans. -- \ ?Odious ideas are not entitled to hide from criticism behind | `\ the human shield of their believers' feelings.? ?Richard | _o__) Stallman | Ben Finney From collinw at gmail.com Thu Nov 5 04:09:09 2009 From: collinw at gmail.com (Collin Winter) Date: Wed, 4 Nov 2009 19:09:09 -0800 Subject: [Python-Dev] 2to3, 3to2: official status In-Reply-To: <87bpjhd6tp.fsf@benfinney.id.au> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> <4AF1CFAD.6000305@v.loewis.de> <87skcuc4ka.fsf_-_@benfinney.id.au> <4AF2032D.5050406@v.loewis.de> <87bpjhd6tp.fsf@benfinney.id.au> Message-ID: <43aa6ff70911041909w2a8051e6r3201ed77617b6743@mail.gmail.com> Hi Ben, On Wed, Nov 4, 2009 at 6:49 PM, Ben Finney wrote: > "Martin v. L?wis" writes: > >> Ben Finney wrote: >> > "Martin v. L?wis" writes: >> > >> >> Well, 3to2 would then be an option for you: use Python 3 as the >> >> source language. >> > >> > I was under the impression that 2to3 was officially supported as >> > part of Python, but 3to2 was a third-party tool. [?] Is it an >> > official part of Python? >> >> No, the status is exactly as you describe it. > > Okay. It's probably best for anyone with their Python developer hat on > (which, in this forum, is all the time for any Python developer) to make > the status of 3to2 clear when recommending it to people concerned about > future plans. Are you implying that we shouldn't recommend 3to2 to people wanting to develop in Py3k and back-translate to 2.x? Thanks, Collin Winter From ben+python at benfinney.id.au Thu Nov 5 04:20:41 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 05 Nov 2009 14:20:41 +1100 Subject: [Python-Dev] 2to3, 3to2: official status References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> <4AF1CFAD.6000305@v.loewis.de> <87skcuc4ka.fsf_-_@benfinney.id.au> <4AF2032D.5050406@v.loewis.de> <87bpjhd6tp.fsf@benfinney.id.au> <43aa6ff70911041909w2a8051e6r3201ed77617b6743@mail.gmail.com> Message-ID: <87pr7xbqt2.fsf@benfinney.id.au> Collin Winter writes: > On Wed, Nov 4, 2009 at 6:49 PM, Ben Finney wrote: > > Okay. It's probably best for anyone with their Python developer hat > > on (which, in this forum, is all the time for any Python developer) > > to make the status of 3to2 clear when recommending it to people > > concerned about future plans. > > Are you implying that we shouldn't recommend 3to2 to people wanting to > develop in Py3k and back-translate to 2.x? No, I'm implying that mentioning Python 3, 3to2, and 2to3 all together in a discussion about how to migrate can easily give the false impression that they're all equally supported by the Python developers. That 3to2 is *not* supported officially is certainly something I'd want to know if a Python core developer was recommending it to me to reassure me about the migration path to Python 3, and I didn't get that impression from the way it's been casually referenced in this discussion. -- \ ?Never express yourself more clearly than you are able to | `\ think.? ?Niels Bohr | _o__) | Ben Finney From brett at python.org Thu Nov 5 04:52:28 2009 From: brett at python.org (Brett Cannon) Date: Wed, 4 Nov 2009 19:52:28 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com> <93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com> Message-ID: On Wed, Nov 4, 2009 at 14:54, Zooko O'Whielacronx wrote: > Folks: > > It occurred to me to wonder why I haven't investigated how hard it > would be to make my Python packages Python-3-compatible. ?That's right > -- I haven't even looked closely. ?I couldn't even tell you off the > top of my head what is in Python 3 that I would have to think about > except for the new unicode regime. ?I think the answer is that the > payoff is just *so* low to me at this point that it doesn't even > justify me taking 15 minutes to read "What's New In Python 3" or to > execute 2to3 on my smallest package and see what it does. > But the payoff is low for you because you want an object-capabilities system and Python 3 doesn't support that kind of use (nor was Python designed to support objcap in the first place, so it's already a strained use in 2.x). -Brett From stefan_ml at behnel.de Thu Nov 5 08:28:26 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 05 Nov 2009 08:28:26 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: Guido van Rossum, 03.11.2009 18:35: > Python, as a language is more than the core implementation -- > CPython -- with a rich, mature and vibrant community of implementations, > such as Jython [2]_, IronPython [3]_ and PyPy [4]_ that are a benefit > not only to the community, but to the language itself. I noticed that this says "such as", but if it's intended to name the major implementations, I wonder why Cython is missing from that list. Stefan From fuzzyman at voidspace.org.uk Thu Nov 5 10:13:55 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 05 Nov 2009 10:13:55 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: <4AF29753.3040801@voidspace.org.uk> Stefan Behnel wrote: > Guido van Rossum, 03.11.2009 18:35: > >> Python, as a language is more than the core implementation -- >> CPython -- with a rich, mature and vibrant community of implementations, >> such as Jython [2]_, IronPython [3]_ and PyPy [4]_ that are a benefit >> not only to the community, but to the language itself. >> > > I noticed that this says "such as", but if it's intended to name the major > implementations, I wonder why Cython is missing from that list. > Is Cython an implementation of Python? It's a grey area, but it seems like Cython is a "Python inspired" language (similar to Shedskin) rather than an implementation of Python: The Cython language is very close to the Python language, but Cython additionally supports calling C functions and declaring C types on variables and class attributes. This allows the compiler to generate very efficient C code from Cython code. (From http://www.cython.org/ ) There are several partial implementations, including Python inspired languages, but if we are looking at 'major complete implementations' then the current list seems to be: CPython, Jython, IronPython and PyPy. Even Unladen Swallow is a fork (sorry - I mean branch) of CPython rather than a separate implementation. All the best, Michael Foord > Stefan > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ From stefan_ml at behnel.de Thu Nov 5 11:11:41 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 05 Nov 2009 11:11:41 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <4AF29753.3040801@voidspace.org.uk> References: <4AF29753.3040801@voidspace.org.uk> Message-ID: Michael Foord, 05.11.2009 10:13: > Stefan Behnel wrote: >> if it's intended to name the major >> implementations, I wonder why Cython is missing from that list. > > Is Cython an implementation of Python? We certainly aim for Python compatibility. That's a proclaimed 1.0 goal. Not sure if that means Py3 or Py2 syntax, but that's just a parser detail, after all. > It's a grey area, but it seems like Cython is a "Python inspired" > language (similar to Shedskin) rather than an implementation of Python: > > The Cython language is very close to the Python language, but Cython > additionally supports calling C functions and declaring C types on > variables and class attributes. This allows the compiler to generate > very efficient C code from Cython code. > > (From http://www.cython.org/ ) Cython is different from Shedskin because Shedskin /requires/ the 'Python' code to be static. Cython does not. It /optimises/ static code, but it will not complain about dynamic code. It /does/ make some static assumptions in that it considers builtins true builtins. However, it does not prevent you from replacing them in your code, as long as you do it inside the module. Certainly a restriction compared to Python, where you can import a module into a changed dict environment that redefines 'object', but not a major restriction IMO, and certainly not one that impacts much code. > There are several partial implementations, including Python inspired > languages, but if we are looking at 'major complete implementations' > then the current list seems to be: CPython, Jython, IronPython and PyPy. Cython certainly is not a complete implementation, sure. It still lacks support for local classes, for example, metaclasses aren't handled, and generators are still in design stage. But if one of the goals of the moratorium is to let implementations catch up with the syntax, then Cython is certainly in line with that. I doubt that there will be much left on the TODO list two years from now. > Even Unladen Swallow is a fork (sorry - I mean branch) of CPython rather > than a separate implementation. Cython is somewhere in the middle. It's not a fork or branch. Rather, it depends on the normal CPython runtime for certain features, such as the standard library, the type/object implementation, garbage collection, and a couple of dynamic features. But that doesn't prevent it from compiling your module into an executable application. Stefan From fuzzyman at voidspace.org.uk Thu Nov 5 11:18:17 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 05 Nov 2009 11:18:17 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4AF29753.3040801@voidspace.org.uk> Message-ID: <4AF2A669.4010708@voidspace.org.uk> Stefan Behnel wrote: > Michael Foord, 05.11.2009 10:13: > >> Stefan Behnel wrote: >> >>> if it's intended to name the major >>> implementations, I wonder why Cython is missing from that list. >>> >> Is Cython an implementation of Python? >> > > We certainly aim for Python compatibility. That's a proclaimed 1.0 goal. > Not sure if that means Py3 or Py2 syntax, but that's just a parser detail, > after all. > > Right, but I think the main point is that Cython is not an implementation of Python. It uses Python syntax, calling into the Python runtime for many of its features, and *extending* Python syntax for compilation to C. (Right?) Obviously a grey area - but if you take Cython code and try to run it on ClassicPython then you are likely to have syntax errors. The same is not true of the other full implementations (although where they use native platform features you may not have the same libraries available - but the language is unchanged). All the best, Michael > >> It's a grey area, but it seems like Cython is a "Python inspired" >> language (similar to Shedskin) rather than an implementation of Python: >> >> The Cython language is very close to the Python language, but Cython >> additionally supports calling C functions and declaring C types on >> variables and class attributes. This allows the compiler to generate >> very efficient C code from Cython code. >> >> (From http://www.cython.org/ ) >> > > Cython is different from Shedskin because Shedskin /requires/ the 'Python' > code to be static. Cython does not. It /optimises/ static code, but it will > not complain about dynamic code. > > It /does/ make some static assumptions in that it considers builtins true > builtins. However, it does not prevent you from replacing them in your > code, as long as you do it inside the module. Certainly a restriction > compared to Python, where you can import a module into a changed dict > environment that redefines 'object', but not a major restriction IMO, and > certainly not one that impacts much code. > > > >> There are several partial implementations, including Python inspired >> languages, but if we are looking at 'major complete implementations' >> then the current list seems to be: CPython, Jython, IronPython and PyPy. >> > > Cython certainly is not a complete implementation, sure. It still lacks > support for local classes, for example, metaclasses aren't handled, and > generators are still in design stage. But if one of the goals of the > moratorium is to let implementations catch up with the syntax, then Cython > is certainly in line with that. I doubt that there will be much left on the > TODO list two years from now. > > > >> Even Unladen Swallow is a fork (sorry - I mean branch) of CPython rather >> than a separate implementation. >> > > Cython is somewhere in the middle. It's not a fork or branch. Rather, it > depends on the normal CPython runtime for certain features, such as the > standard library, the type/object implementation, garbage collection, and a > couple of dynamic features. But that doesn't prevent it from compiling your > module into an executable application. > > Stefan > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ From ziade.tarek at gmail.com Thu Nov 5 11:35:41 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 5 Nov 2009 11:35:41 +0100 Subject: [Python-Dev] Refactoring installation schemes In-Reply-To: <39deb0d1eef08419dad65d7354a8ffb1@preisshare.net> References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com> <4AE8206A.9020108@egenix.com> <4AE824DA.4090701@voidspace.org.uk> <94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com> <94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com> <9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com> <39deb0d1eef08419dad65d7354a8ffb1@preisshare.net> Message-ID: <94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com> 2009/11/5 David Lyon : >>> Call it the new Tarek egg... >> >> The tegg? ?;-) > > l'oeuf ? While I like 'tegg' a lot ;) , I would like to make it clear that "eggs" are not part of Distutils. That's the Setuptools format. Although there's a distinction to make between the *installation* format, and the *distribution* format (sdist, bdist, etc) Right now we have several installation formats which leads to problems I won't mention here. PEP 376 is working on a default, unified, *installation* format, that tries to gather the good ideas of Pip, Setuptools etc.. and propose a unified format for our site-packages. This new standard will come with APIs in pkgutil to be able to query installed distribution etc. This work is also linked to PEP 345 work where we are modifying the Metadata, and to PEP 386 that proposes a standard version comparison scheme. But there's no plan to include a new *distribution* format in Distutils. In any case those PEPs are not finished yet, so everyone can help at distutils-SIG Regards Tarek From stefan_ml at behnel.de Thu Nov 5 11:47:46 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 05 Nov 2009 11:47:46 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <4AF2A669.4010708@voidspace.org.uk> References: <4AF29753.3040801@voidspace.org.uk> <4AF2A669.4010708@voidspace.org.uk> Message-ID: Michael Foord, 05.11.2009 11:18: > Stefan Behnel wrote: >> We certainly aim for Python compatibility. That's a proclaimed 1.0 goal. > > Right, but I think the main point is that Cython is not an > implementation of Python. It uses Python syntax, calling into the Python > runtime for many of its features It is a Python implementation in the sense that it implements syntax and semantics of the Python language, be it through generated C code or by calling into the CPython runtime. You could theoretically drop the entire interpreter from CPython, and Cython would still work with the remaining parts - not that it would be worth the effort on either side... > and *extending* Python syntax for compilation to C. Programmers can decide themselves if they want to statically type variables, and they can do so without breaking with Python syntax. It is certainly not required to use non-Python syntax to compile Python code, not even if you want to get a fast binary module as a result. > Obviously a grey area - but if you take Cython code and try to run it on > ClassicPython then you are likely to have syntax errors. The same is not > true of the other full implementations (although where they use native > platform features you may not have the same libraries available - but > the language is unchanged). To me, the term "Python implementation" just means something that implements the Python language. That shouldn't prevent it from providing additional features. Jython can interface with Java code, IronPython can interface with .NET code, Cython can interface with C code. Not sure what PyPy allows in addition to being a Python implementation, but there surely are language features also in PyPy that no other Python implementation provides. Stefan From mal at egenix.com Thu Nov 5 11:56:28 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 05 Nov 2009 11:56:28 +0100 Subject: [Python-Dev] Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?) In-Reply-To: <4AF17FB7.5090708@gmail.com> References: <4AEF4C53.7090902@v.loewis.de> <4AEFF3A3.3020103@egenix.com> <4AF06E20.7050307@v.loewis.de> <4AF0816A.706@egenix.com> <319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com> <4AF16CC7.6000705@gmail.com> <4AF171AD.1000703@egenix.com> <4AF17FB7.5090708@gmail.com> Message-ID: <4AF2AF5C.4050401@egenix.com> Nick Coghlan wrote: > M.-A. Lemburg wrote: >> Nick Coghlan wrote: >>> Lennart Regebro wrote: >>>> I also would really like to see a real port of the bytes class to 2.6, >>>> but I have a vague memory that there was some reason that wouldn't >>>> work. >>> Not so much that it wouldn't work, but that the interfaces to support >>> using it effectively really aren't there - lots of areas in the standard >>> library needed to be tweaked to cope with bytes objects in 3.x. >>> >>> Generally speaking, the "bytes = str" trick represents a reasonable >>> compromise as the APIs that you would pass a bytes object to in 3.x >>> expect an 8-bit str instance in 2.x. >> >> Could you please add such hints, tricks and tips to the wiki >> page: >> >> http://wiki.python.org/moin/PortingToPy3k > > Done (although the task of figuring out how to get the wiki to display > code properly defeated me... the normal Python documentation syntax for > it seemed to give the wiki's ReST interpreter fits). Thanks. > I also mentioned the trick someone mentioned about "from __future__ > import unicode_literals" not changing the meaning of 'str' since it only > alters the parser but not the builtins. > > In writing it up, it occurred to me that having that kind of thing in a > "py3_compat" compatibility module (to be used as, e.g., "from py3_compat > import str, bytes") would not only make it easier to use in multiple > modules, but also easier for 2to3 to remove when forward porting. That would certainly be useful to have, yes, since all developers wanting to support both 2.7 and 3.2 using the same code base will have a need for these things. It would also help to have a package that contains proxy modules using the Python 3.x module/package names which map the functionality back to the Python 2.x modules. This avoids problems with freezing tools that scan the source code for imports as well as pickles which store the full dotted path name to classes. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 05 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From fijall at gmail.com Thu Nov 5 11:58:40 2009 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 5 Nov 2009 11:58:40 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4AF29753.3040801@voidspace.org.uk> <4AF2A669.4010708@voidspace.org.uk> Message-ID: <693bc9ab0911050258p56cd81dcvca46e4fc087effa3@mail.gmail.com> > > To me, the term "Python implementation" just means something that > implements the Python language. That shouldn't prevent it from providing > additional features. Jython can interface with Java code, IronPython can > interface with .NET code, Cython can interface with C code. Not sure what > PyPy allows in addition to being a Python implementation, but there surely > are language features also in PyPy that no other Python implementation > provides. > > Stefan > Hi. Features available in PyPy (or Java or IronPython) are not language features. These are features are available through normal python syntax, just some libraries are available extra. Python is something that passes most of CPython's test suite, modulo stuff that is considered implementation detail (like reference counting). In fact, some of extra PyPy features can be achieved via extension modules like greenlets. Cython might as well at some point become one, but so far it does not even import the whole test suite (correct me if I'm wrong), hence it's not yet python. Even if the ultimate goal is to support whole Python, it's not achieved yet, so it should not be considered implementation of Python (as of today, it might as well be one tomorrow). Cheers, fijal From fuzzyman at voidspace.org.uk Thu Nov 5 12:38:49 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 05 Nov 2009 12:38:49 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4AF29753.3040801@voidspace.org.uk> <4AF2A669.4010708@voidspace.org.uk> Message-ID: <4AF2B949.9030706@voidspace.org.uk> Stefan Behnel wrote: > Michael Foord, 05.11.2009 11:18: > >> Stefan Behnel wrote: >> >>> We certainly aim for Python compatibility. That's a proclaimed 1.0 goal. >>> >> Right, but I think the main point is that Cython is not an >> implementation of Python. It uses Python syntax, calling into the Python >> runtime for many of its features >> > > It is a Python implementation in the sense that it implements syntax and > semantics of the Python language, be it through generated C code or by > calling into the CPython runtime. Implementing Python by calling into Python does not seem to really be a 'new implementation'... In that sense Cython is a wrapper not an implementation. > You could theoretically drop the entire > interpreter from CPython, and Cython would still work with the remaining > parts - not that it would be worth the effort on either side... > > > >> and *extending* Python syntax for compilation to C. >> > > Programmers can decide themselves if they want to statically type > variables, and they can do so without breaking with Python syntax. It is > certainly not required to use non-Python syntax to compile Python code, not > even if you want to get a fast binary module as a result. > > But you extend the language. Coders can write Python or 'not-Python' (or perhaps 'Python plus'). Where you extend the language you are no longer implementing Python but a language oriented tool with Python inspired syntax intended for use with Python. > >> Obviously a grey area - but if you take Cython code and try to run it on >> ClassicPython then you are likely to have syntax errors. The same is not >> true of the other full implementations (although where they use native >> platform features you may not have the same libraries available - but >> the language is unchanged). >> > > To me, the term "Python implementation" just means something that > implements the Python language. Which Cython doesn't - it can parse Python and it calls into Python. It doesn't implement runtime features (from the sounds of it). > That shouldn't prevent it from providing > additional features. Heh. Imagine if Microsoft had taken IronPython and added new syntax, that wasn't valid Python, and still tried to claim that IronPython was an implementation of Python. Few people would agree... > Jython can interface with Java code, IronPython can > interface with .NET code, Cython can interface with C code. Not sure what > PyPy allows in addition to being a Python implementation, but there surely > are language features also in PyPy that no other Python implementation > provides. > No. All of these implementation provide their integration through existing language features of Python, not by adding new *language* level features. All the best, Michael > Stefan > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ From ncoghlan at gmail.com Thu Nov 5 13:08:18 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 05 Nov 2009 22:08:18 +1000 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> <4AF1CFAD.6000305@v.loewis.de> Message-ID: <4AF2C032.4040605@gmail.com> Mike Krell wrote: > Well, 3to2 would then be an option for you: use Python 3 as the source > language. Making life easier for 3to2 is an *excellent* rationale for backports. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Thu Nov 5 13:10:27 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 05 Nov 2009 22:10:27 +1000 Subject: [Python-Dev] 2to3, 3to2: official status In-Reply-To: <87pr7xbqt2.fsf@benfinney.id.au> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> <4AF1CFAD.6000305@v.loewis.de> <87skcuc4ka.fsf_-_@benfinney.id.au> <4AF2032D.5050406@v.loewis.de> <87bpjhd6tp.fsf@benfinney.id.au> <43aa6ff70911041909w2a8051e6r3201ed77617b6743@mail.gmail.com> <87pr7xbqt2.fsf@benfinney.id.au> Message-ID: <4AF2C0B3.3010601@gmail.com> Ben Finney wrote: > That 3to2 is *not* supported officially is certainly something I'd want > to know if a Python core developer was recommending it to me to reassure > me about the migration path to Python 3, and I didn't get that > impression from the way it's been casually referenced in this > discussion. I would stick a "yet" in that sentence (i.e. "not supported officially yet"). It's certainly an addition we should be considering for 3.2 if it is sufficiently mature by then (and if not 3.2, then we can take another look for 3.3). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From stefan_ml at behnel.de Thu Nov 5 13:13:13 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 05 Nov 2009 13:13:13 +0100 Subject: [Python-Dev] Cython as a Python implementation (was: PEP 3003 - Python Language Moratorium) In-Reply-To: <4AF2B949.9030706@voidspace.org.uk> References: <4AF29753.3040801@voidspace.org.uk> <4AF2A669.4010708@voidspace.org.uk> <4AF2B949.9030706@voidspace.org.uk> Message-ID: Michael Foord, 05.11.2009 12:38: > Stefan Behnel wrote: >> Michael Foord, 05.11.2009 11:18: >>> I think the main point is that Cython is not an >>> implementation of Python. It uses Python syntax, calling into the Python >>> runtime for many of its features >> >> It is a Python implementation in the sense that it implements syntax and >> semantics of the Python language, be it through generated C code or by >> calling into the CPython runtime. > > Implementing Python by calling into Python does not seem to really be a > 'new implementation'... In that sense Cython is a wrapper not an > implementation. Would it help anyone if we started forking CPython, just so we can say "look, we implement Python"? (note that this will never happen, I'm just trying to find out where we can draw the line here) To clarify this statement: >> You could theoretically drop the entire interpreter from CPython, and >> Cython would still work with the remaining parts I think the main parts of CPython that we (re-)use are: - all built-in types - ref-counting and garbage collection - frames for traceback reporting - stdlib There may be other parts, but that's what comes to my mind right now. You can additionally use the CPython interpreter through the 'exec' statement or by calling into uncompiled code. For a Python implementation, I don't see any use in reimplementing the built-in types, for example. They are well done and pretty fast, especially when used from C code, i.e. outside of the interpreter. The frame/traceback handling code is only for interaction with uncompiled Python code, so that you get nice looking tracebacks (which I personally wouldn't consider a required part of the language itself). Most Python implementations do not reimplement the stdlib, or at most a minor part of it, so that's right out of the discussion. I'm not sure how to value the ref-counting/GC reuse, but I doubt that it is worth being considered an argument for Cython being a CPython wrapper. >> Programmers can decide themselves if they want to statically type >> variables, and they can do so without breaking with Python syntax. It is >> certainly not required to use non-Python syntax to compile Python >> code, not even if you want to get a fast binary module as a result. > > But you extend the language. Coders can write Python or 'not-Python' (or > perhaps 'Python plus'). Where you extend the language you are no longer > implementing Python but a language oriented tool with Python inspired > syntax intended for use with Python. But that's just a detail of the parser. It wouldn't be hard at all to switch off the support for extended features when compiling .py files (as opposed to .pyx files). It's just not currently done. Would you consider Cython a Python implementation if we implemented this? (which I guess we will do anyway at some point, simply because the extensions are actually invalid code in the given context) Stefan From dirkjan at ochtman.nl Thu Nov 5 13:18:23 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Thu, 5 Nov 2009 13:18:23 +0100 Subject: [Python-Dev] Cython as a Python implementation (was: PEP 3003 - Python Language Moratorium) In-Reply-To: References: <4AF29753.3040801@voidspace.org.uk> <4AF2A669.4010708@voidspace.org.uk> <4AF2B949.9030706@voidspace.org.uk> Message-ID: On Thu, Nov 5, 2009 at 13:13, Stefan Behnel wrote: > Would you consider Cython a Python implementation if we implemented this? > (which I guess we will do anyway at some point, simply because the > extensions are actually invalid code in the given context) Why do you want to be recognized as a Python implementation, anyway? I don't really understand why this seems so important to you. It's not like python-dev doesn't want to listen to you, right? Cheers, Dirkjan From ncoghlan at gmail.com Thu Nov 5 13:19:45 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 05 Nov 2009 22:19:45 +1000 Subject: [Python-Dev] Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?) In-Reply-To: <4AF1C9FF.2060302@v.loewis.de> References: <4AEF4C53.7090902@v.loewis.de> <4AEFF3A3.3020103@egenix.com> <4AF06E20.7050307@v.loewis.de> <4AF0816A.706@egenix.com> <319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com> <4AF16CC7.6000705@gmail.com> <4AF1C9FF.2060302@v.loewis.de> Message-ID: <4AF2C2E1.3050601@gmail.com> Martin v. L?wis wrote: > Nick Coghlan wrote: >> Lennart Regebro wrote: >>> I also would really like to see a real port of the bytes class to 2.6, >>> but I have a vague memory that there was some reason that wouldn't >>> work. >> Not so much that it wouldn't work, but that the interfaces to support >> using it effectively really aren't there - lots of areas in the standard >> library needed to be tweaked to cope with bytes objects in 3.x. > > I see the problem differently: if a bytes type was added, nothing would > use it. In particular, IO wouldn't start returning bytes (although it > could accept them); IO would continue to return str. Therefore, I'm > skeptical that adding a *third* string type to 3.x would do any good. I'm guessing you meant '2.x' in that last sentence, in which case we agree (just emphasising different parts of the "binary data" ecosystem that would be necessary to make a backported bytes type effective). Although I think Guido did a better job than either of us in explaining why backporting the full bytes type to 2.x really wouldn't help all that much. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Thu Nov 5 13:31:52 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 05 Nov 2009 22:31:52 +1000 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <426ada670911041039m6df85994v321342231fbd3805@mail.gmail.com> References: <319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com> <426ada670911041039m6df85994v321342231fbd3805@mail.gmail.com> Message-ID: <4AF2C5B8.8070204@gmail.com> Carl Trachte wrote: > On 11/4/09, ssteinerX at gmail.com wrote: > >> Maybe the 3.x line should just be put out of our misery, merged back >> to 2.7, 2.8, 2.9, and proceed as Glyph suggested in passing with >> increasing levels of deprecation until it just turns into 3.x on its >> own by running out of numbers. > > > As a user, I'm horrified. Granted, I'm not the most high powered > user, but . . . > my employer is already providing me with a 3.0 Python version on one > of my work computers with the expectation that I'll be using it more > and more. > > Sorry to butt in, but is this a joke? I thought all this was hashed > out prior to inventing python 3.0. > Don't worry, 3.x is still the future of the Python language. Some of the interested onlookers are just rehashing discussions that happened years ago *before* the 3.x branch was created. It boils down to the fact that the real beneficiaries of the 2.x to 3.x transition are the people that aren't using Python yet, so existing users (especially maintainers of large libraries and frameworks) bear a disproportionate amount of the cost of the transition while gaining little of the benefit. They're understandably irritated by that and the situation is likely to take a couple more years to sort itself out. While it may be hard to tell without knowing who is and isn't a core developer, the only point seriously under discussion is whether there is going to be a 2.8 after 2.7, and the current answer to that is looking to be "probably not". Planning on that basis probably isn't a bad idea. Even if we do decide to create a 2.8 after 3k has already been merged back to the trunk, a new 2.8 development branch could easily be created based on the 2.7 maintenance branch instead of the trunk. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From theller at ctypes.org Thu Nov 5 13:53:46 2009 From: theller at ctypes.org (Thomas Heller) Date: Thu, 05 Nov 2009 13:53:46 +0100 Subject: [Python-Dev] Cython as a Python implementation In-Reply-To: References: <4AF29753.3040801@voidspace.org.uk> <4AF2A669.4010708@voidspace.org.uk> <4AF2B949.9030706@voidspace.org.uk> Message-ID: Dirkjan Ochtman schrieb: > On Thu, Nov 5, 2009 at 13:13, Stefan Behnel wrote: >> Would you consider Cython a Python implementation if we implemented this? >> (which I guess we will do anyway at some point, simply because the >> extensions are actually invalid code in the given context) > > Why do you want to be recognized as a Python implementation, anyway? I > don't really understand why this seems so important to you. Advertising, I assume. That's what Stefan often does (at least it feels this way, for me). -- Thanks, Thomas From solipsis at pitrou.net Thu Nov 5 14:02:35 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 5 Nov 2009 13:02:35 +0000 (UTC) Subject: [Python-Dev] Cython as a Python implementation References: <4AF29753.3040801@voidspace.org.uk> <4AF2A669.4010708@voidspace.org.uk> <4AF2B949.9030706@voidspace.org.uk> Message-ID: Thomas Heller ctypes.org> writes: > > > Why do you want to be recognized as a Python implementation, anyway? I > > don't really understand why this seems so important to you. > > Advertising, I assume. That's what Stefan often does (at least it feels > this way, for me). Well, it's true that their work often goes unnoticed in favour of more pie-in-the-sky things such as PyPy and unladen-swallow. From abpillai at gmail.com Thu Nov 5 15:05:49 2009 From: abpillai at gmail.com (Anand Balachandran Pillai) Date: Thu, 5 Nov 2009 19:35:49 +0530 Subject: [Python-Dev] Cython as a Python implementation In-Reply-To: References: <4AF29753.3040801@voidspace.org.uk> <4AF2A669.4010708@voidspace.org.uk> <4AF2B949.9030706@voidspace.org.uk> Message-ID: <8548c5f30911050605o14982697i6a7cc4be68243959@mail.gmail.com> On Thu, Nov 5, 2009 at 6:32 PM, Antoine Pitrou wrote: > Thomas Heller ctypes.org> writes: > > > > > Why do you want to be recognized as a Python implementation, anyway? I > > > don't really understand why this seems so important to you. > > > > Advertising, I assume. That's what Stefan often does (at least it feels > > this way, for me). > > Well, it's true that their work often goes unnoticed in favour of more > pie-in-the-sky things such as PyPy and unladen-swallow. > Perhaps something to do with naming. "PyPy" and "unladen-swallow" are obviously more eye-catching than "Cython". Just my $0.02. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/abpillai%40gmail.com > Regards, -- --Anand -------------- next part -------------- An HTML attachment was scrubbed... URL: From metawilm at gmail.com Thu Nov 5 16:21:27 2009 From: metawilm at gmail.com (Willem Broekema) Date: Thu, 5 Nov 2009 16:21:27 +0100 Subject: [Python-Dev] Cython as a Python implementation In-Reply-To: <8548c5f30911050605o14982697i6a7cc4be68243959@mail.gmail.com> References: <4AF2A669.4010708@voidspace.org.uk> <4AF2B949.9030706@voidspace.org.uk> <8548c5f30911050605o14982697i6a7cc4be68243959@mail.gmail.com> Message-ID: On Thu, Nov 5, 2009 at 3:05 PM, Anand Balachandran Pillai wrote: > ?Perhaps something to do with naming. "PyPy" and "unladen-swallow" > ?are obviously more eye-catching than "Cython". Heh, CLPython is not very catchy either. Maybe that's why it's still under the radar here. - Willem From stefan_ml at behnel.de Thu Nov 5 16:45:39 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 05 Nov 2009 16:45:39 +0100 Subject: [Python-Dev] Cython as a Python implementation In-Reply-To: References: <4AF29753.3040801@voidspace.org.uk> <4AF2A669.4010708@voidspace.org.uk> <4AF2B949.9030706@voidspace.org.uk> Message-ID: Thomas Heller, 05.11.2009 13:53: > Advertising, I assume. That's what Stefan often does (at least it feels > this way, for me). Well, there's nothing inherently wrong with telling people about the right tool for the job, especially when it's free. And don't forget that postings are self-selecting by their topic, so you may simply not have come across the non-advertising posts of mine, and I may have skipped worthy non-advertising-affine topics that flew below my minimum level of conscious reading. Stefan From guido at python.org Thu Nov 5 16:57:19 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 5 Nov 2009 07:57:19 -0800 Subject: [Python-Dev] Cython as a Python implementation In-Reply-To: References: <4AF29753.3040801@voidspace.org.uk> <4AF2A669.4010708@voidspace.org.uk> <4AF2B949.9030706@voidspace.org.uk> Message-ID: On Thu, Nov 5, 2009 at 4:53 AM, Thomas Heller wrote: > Dirkjan Ochtman schrieb: >> On Thu, Nov 5, 2009 at 13:13, Stefan Behnel wrote: >>> Would you consider Cython a Python implementation if we implemented this? >>> (which I guess we will do anyway at some point, simply because the >>> extensions are actually invalid code in the given context) >> >> Why do you want to be recognized as a Python implementation, anyway? I >> don't really understand why this seems so important to you. > > Advertising, I assume. ?That's what Stefan often does (at least it feels > this way, for me). I'm skeptical about Cython as a Python implementation. AFAIK it requires CPython to work (for the runtime functionality) and while Cython's syntax is very Python-like, it is probably best characterized as an extended subset (:-). I guess when it runs the entire standard test suite without errors it could claim to be a Python implementation; until then, I would rather consider it a useful tool to go with CPython. (The name is actually a bit confusing, but I don't have a desire to change it.) -- --Guido van Rossum (python.org/~guido) From solipsis at pitrou.net Thu Nov 5 17:06:03 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 5 Nov 2009 16:06:03 +0000 (UTC) Subject: [Python-Dev] No buildbot to test wide unicode? References: <4AF1D14A.6040007@v.loewis.de> Message-ID: Le Wed, 04 Nov 2009 20:08:58 +0100, Martin v. L?wis a ?crit?: >> >> Since wide unicode is the standard chosen by some Linux distributions, >> it would make sense to have at least one buildbot running with >> --with-wide-unicode (3.x) or --enable-unicode=ucs4 (2.x). > > Can you propose some (one? two? more?) systems that might be best as > candidates? I'd then setup two (sets of) builders; they would share the > slave lock, so builds would run sequentially (unless the slave operator > agrees to setup two slaves on one machine). I would certainly agree to setup two slaves on mine. There are ample resources available. Regards Antoine. From stefan_ml at behnel.de Thu Nov 5 17:08:52 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 05 Nov 2009 17:08:52 +0100 Subject: [Python-Dev] Cython as a Python implementation In-Reply-To: References: <4AF29753.3040801@voidspace.org.uk> <4AF2A669.4010708@voidspace.org.uk> <4AF2B949.9030706@voidspace.org.uk> Message-ID: Antoine Pitrou, 05.11.2009 14:02: > Thomas Heller writes: >>> Why do you want to be recognized as a Python implementation, anyway? I >>> don't really understand why this seems so important to you. >> >> Advertising, I assume. That's what Stefan often does (at least it feels >> this way, for me). > > Well, it's true that their work often goes unnoticed in favour of more > pie-in-the-sky things such as PyPy and unladen-swallow. That's likely got to do with it. It's not the first time I read about the "four" Python implementations. It may be related to the lack of Cython developers at the language summits, but, well, it's not like those were cheap to join either, if you count travel time/costs etc. I'm well aware, and I openly admit that Cython is laging behind in terms of Python language features when compared with other Python implementations. A reason for this is that the goal of the Cython project to design a /superset/ of the Python language is simply more ambitious (and certainly more time consuming) than just writing "yet another Python implementation". We put a lot of thought work into language design between Python and C, and also into code performance optimisations, where others can just go straight for CPython compatibility. But this doesn't mean we are not writing a Python implementation. It just means that we do more than that, and that this 'more' takes time. It has often been argued that it's easy to implement Python faster if you actually don't implement Python, but that doesn't apply to the current Cython anymore. What it lacks in terms of features is nothing that would impact the performance of existing features in any way. Like most of the other Python implementations, Cython is ready to use now. Unlike most others, it has all batteries included. And unlike some others, it's fast and even scales extremely well with various performance requirements. Personally, I consider Cython one of the killer arguments against performance prejudices about the CPython platform, if not about the Python language itself. Stefan From rdmurray at bitdance.com Thu Nov 5 17:29:26 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 5 Nov 2009 11:29:26 -0500 (EST) Subject: [Python-Dev] No buildbot to test wide unicode? In-Reply-To: References: <4AF1D14A.6040007@v.loewis.de> Message-ID: On Thu, 5 Nov 2009 at 16:06, Antoine Pitrou wrote: > Le Wed, 04 Nov 2009 20:08:58 +0100, Martin v. L??wis a ??crit??: >>> Since wide unicode is the standard chosen by some Linux distributions, >>> it would make sense to have at least one buildbot running with >>> --with-wide-unicode (3.x) or --enable-unicode=ucs4 (2.x). >> >> Can you propose some (one? two? more?) systems that might be best as >> candidates? I'd then setup two (sets of) builders; they would share the >> slave lock, so builds would run sequentially (unless the slave operator >> agrees to setup two slaves on one machine). > > I would certainly agree to setup two slaves on mine. There are ample > resources available. I could do so as well. Gentoo is one of the distributions that uses the wide build by default, so that would make it a good test candidate as well. --David From stefan_ml at behnel.de Thu Nov 5 17:40:11 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 05 Nov 2009 17:40:11 +0100 Subject: [Python-Dev] Cython as a Python implementation In-Reply-To: <693bc9ab0911050258p56cd81dcvca46e4fc087effa3@mail.gmail.com> References: <4AF29753.3040801@voidspace.org.uk> <4AF2A669.4010708@voidspace.org.uk> <693bc9ab0911050258p56cd81dcvca46e4fc087effa3@mail.gmail.com> Message-ID: Maciej Fijalkowski, 05.11.2009 11:58: > Python is something that passes most of CPython's test > suite, modulo stuff that is considered implementation detail (like > reference counting). Certainly not an implementation detail to Cython. ;) > Cython might as well at some point become one, but so far it does not even > import the whole test suite (correct me if I'm wrong), hence it's not > yet python. The test suite is a problem anyway, given that Cython doesn't implement a specific level of the Python language. It aims for 2.6 compatibility, but it already implements many of the Py3k PEPs, for example. But, yes, it doesn't currently import the test suite (without cheating). Many of the tests fail to compile due to the use of lambda statements (which won't take long to support on top of the existing closures) and conditionally defined classes (which will likely take longer, unless we cheat a bit by special casing the simple cases). BTW, other implementations lack some standard library modules. I find it a lot easier to work around missing syntax than to work around missing dependencies. But maybe that's just me. Stefan From ubershmekel at gmail.com Thu Nov 5 18:34:29 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Thu, 5 Nov 2009 19:34:29 +0200 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF2C5B8.8070204@gmail.com> References: <319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com> <426ada670911041039m6df85994v321342231fbd3805@mail.gmail.com> <4AF2C5B8.8070204@gmail.com> Message-ID: <9d153b7c0911050934k714aca73q62dd78cf9033584e@mail.gmail.com> On Thu, Nov 5, 2009 at 2:31 PM, Nick Coghlan wrote: > While it may be hard to tell without knowing who is and isn't a core > developer > Maybe there should be badges or something, hmmm, sounds like making an svn-commits-hall-of-fame for python could be a nice project. --yuv -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Thu Nov 5 19:01:56 2009 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 5 Nov 2009 19:01:56 +0100 Subject: [Python-Dev] Cython as a Python implementation (was: PEP 3003 - Python Language Moratorium) In-Reply-To: References: <4AF29753.3040801@voidspace.org.uk> <4AF2A669.4010708@voidspace.org.uk> <4AF2B949.9030706@voidspace.org.uk> Message-ID: <693bc9ab0911051001x495f3305v92e8312fcc143d04@mail.gmail.com> > Most Python implementations do not reimplement the stdlib, or at most a > minor part of it, so that's right out of the discussion. Did you actually check? From dinov at microsoft.com Thu Nov 5 19:35:08 2009 From: dinov at microsoft.com (Dino Viehland) Date: Thu, 5 Nov 2009 18:35:08 +0000 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4AF29753.3040801@voidspace.org.uk> Message-ID: <1A472770E042064698CB5ADC83A12ACD04C4A6C6@TK5EX14MBXC118.redmond.corp.microsoft.com> Stefan wrote: > It /does/ make some static assumptions in that it considers builtins > true > builtins. However, it does not prevent you from replacing them in your > code, as long as you do it inside the module. Certainly a restriction > compared to Python, where you can import a module into a changed dict > environment that redefines 'object', but not a major restriction IMO, > and certainly not one that impacts much code. To me this is a deal breaker which prevents Cython from being a Python implementation. From a talk given by Colin Winter at the LLVM dev meeting (http://llvm.org/devmtg/2009-10/) it seems like Unladen Swallow wanted to do something like this as well and Guido said no. In this case the breaking change is so subtle that I'd personally hate to run into something like this porting code to Cython and having to figure out why it's not working. More generally if all the implementations pick and choose what features of Python we want to ignore for performance reasons then we'll just end up w/ a bunch of incompatible things that look very similar. From collinw at gmail.com Thu Nov 5 19:40:48 2009 From: collinw at gmail.com (Collin Winter) Date: Thu, 5 Nov 2009 10:40:48 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <1A472770E042064698CB5ADC83A12ACD04C4A6C6@TK5EX14MBXC118.redmond.corp.microsoft.com> References: <4AF29753.3040801@voidspace.org.uk> <1A472770E042064698CB5ADC83A12ACD04C4A6C6@TK5EX14MBXC118.redmond.corp.microsoft.com> Message-ID: <43aa6ff70911051040t3a915217o7ffec1f432e9def7@mail.gmail.com> On Thu, Nov 5, 2009 at 10:35 AM, Dino Viehland wrote: > Stefan wrote: >> It /does/ make some static assumptions in that it considers builtins >> true >> builtins. However, it does not prevent you from replacing them in your >> code, as long as you do it inside the module. Certainly a restriction >> compared to Python, where you can import a module into a changed dict >> environment that redefines 'object', but not a major restriction IMO, >> and certainly not one that impacts much code. > > To me this is a deal breaker which prevents Cython from being a Python > implementation. ?From a talk given by Colin Winter at the LLVM dev meeting > (http://llvm.org/devmtg/2009-10/) it seems like Unladen Swallow wanted to > do something like this as well and Guido said no. ?In this case the breaking > change is so subtle that I'd personally hate to run into something like > this porting code to Cython and having to figure out why it's not working. To clarify, I was joking when I told that story (or at least I was joking with Guido when I asked him if we could break that). It clearly *would* be easier if we could just ignore this point of Python compatibility, but that's not an option, so we've had to optimize around it. It's not that hard to do, but it's still extra work. Collin Winter From stefan_ml at behnel.de Thu Nov 5 19:55:17 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 05 Nov 2009 19:55:17 +0100 Subject: [Python-Dev] Cython as a Python implementation In-Reply-To: <693bc9ab0911051001x495f3305v92e8312fcc143d04@mail.gmail.com> References: <4AF29753.3040801@voidspace.org.uk> <4AF2A669.4010708@voidspace.org.uk> <4AF2B949.9030706@voidspace.org.uk> <693bc9ab0911051001x495f3305v92e8312fcc143d04@mail.gmail.com> Message-ID: Maciej Fijalkowski, 05.11.2009 19:01: >> Most Python implementations do not reimplement the stdlib, or at most a >> minor part of it, so that's right out of the discussion. > > Did you actually check? Well, I know that Jython uses the original stdlib modules through svn:externals in the build, last thing I heard about IronPython was that they are allowed to use other people's code now, so I imagine they do the same thing, and I wouldn't expect PyPy to rewrite the existing Python code that exists in the stdlib. So the only remaining problem are the C extensions in CPython, and that's the minor part I mentioned above. Stefan From pje at telecommunity.com Thu Nov 5 20:18:22 2009 From: pje at telecommunity.com (P.J. Eby) Date: Thu, 05 Nov 2009 14:18:22 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <20091104232743.GD19537@laurie.devork> References: <261064E43172924BBADB00DA14BB4B672A14855D01@XCH-NW-14V.nw.nos.boeing.com> <93A0C3CE-3388-49B9-B994-8AB84E6F48B8@seanet.com> <20091104232743.GD19537@laurie.devork> Message-ID: <20091105191827.896D13A40A8@sparrow.telecommunity.com> At 11:27 PM 11/4/2009 +0000, Floris Bruynooghe wrote: > > >You have time to read this thread but no time to read "What's New In >Python 3"? > > Given that the average developer using tons of existing libraries on 2.x is unlikely to see any killer benefits in moving to 3.x, it's a natural attitude to have. I fought this same battle with setuptools for a long time before it sank in that people who don't perceive a need aren't going to RTFM, no matter how much time said RTFMing would save them in the long run, vs. sitting around complaining. IOW, once someone has become annoyed by the mere appearance of a necessity to deal with something that appears to be being foisted on them (whether it's setuptools or Python 3), the natural tendency is to minimize any actual work that would move in the direction of the thing they feel forced to deal with. For me, the closest thing to a killer feature in 3.x is argument type declarations, and it'd be a mild convenience at best. From a distance, many of the other changes appear like anti-features, if only because they're changing what I've been used to for twelve-plus years. (A few, like the removal of __metaclass__-in-locals support, are an active hindrance to porting.) So no, I haven't actually tried to port anything, nor have I done more than lightly skim the porting docs... looking for some reason why I'd *want* to move to Python 3. Heck, I have yet to use 2.6 to run any production code, and find some of *its* changes a bit annoying from a porting perspective. (E.g. dropping the "sets" module.) To make Py3 migration worthwhile to developers with heavy investment in the 2.x lines (and especially those supporting all the way back to 2.3 and 2.4), it'd have to have some *really* killer features. That is, be more like a Python 3000, and less like a Python 2.6 with a few bells and whistles, hampered by having to relearn some of the basic types and a soon-to-be-rebuilt standard library. Even if, in truth, the cost-benefit ratio right now *is* good for migrating to 3.x, nobody's doing a good job at promoting what those benefits are. (And it being easy to port to is NOT a benefit: nobody cares how easy it is to do something they don't see a reason to do in the first place.) From guido at python.org Thu Nov 5 20:36:23 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 5 Nov 2009 11:36:23 -0800 Subject: [Python-Dev] Cython as a Python implementation In-Reply-To: References: <4AF2A669.4010708@voidspace.org.uk> <4AF2B949.9030706@voidspace.org.uk> Message-ID: Stefan, I think your attempts to see Cython accepted as one of the major Python implementations is misguided. It is not self-contained, it is an add-on tool for CPython (like its ancestor PyRex). I think Cython is incredibly useful (and I spoke to many very happy users yesterday at UCB) but trying to present it as a separate Python implementation is not useful. Please stop the discussion about this topic, they are distracting. -- --Guido van Rossum (python.org/~guido) From martin at v.loewis.de Thu Nov 5 21:08:07 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 05 Nov 2009 21:08:07 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF2C032.4040605@gmail.com> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> <4AF1CFAD.6000305@v.loewis.de> <4AF2C032.4040605@gmail.com> Message-ID: <4AF330A7.5000809@v.loewis.de> > Mike Krell wrote: >> Well, 3to2 would then be an option for you: use Python 3 as the source >> language. > > Making life easier for 3to2 is an *excellent* rationale for backports. > I'm skeptical. If new features get added to 2.7: why would that simplify 3to2? It couldn't *use* these features, since it surely would have to support 2.6 and earlier as well. Not sure what 3to2 would do about difficult-to-convert 3.x feature (as, perhaps, the nonlocal keyword). If it currently gives up, it then may offer you to restrict your target versions to 2.7+. Not sure whether users would use that option, though - perhaps they rather stop using nonlocal in 3.x if 3to2 cannot support it for all 2.x versions they are interested in. Perhaps 3to2 has a work-around that still provides a good backport in most cases. Then, the backport would not make the tool any simpler: if 3to2 would start using the backport, it would actually get more complicated (not easier), as it now needs to support two cases. Regards, Martin From regebro at gmail.com Thu Nov 5 21:14:03 2009 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 5 Nov 2009 21:14:03 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF1CFAD.6000305@v.loewis.de> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> <4AF1CFAD.6000305@v.loewis.de> Message-ID: <319e029f0911051214k7e5e69bfnddff9b0572765bfe@mail.gmail.com> 2009/11/4 "Martin v. L?wis" : >> Keep in mind also that the 2.x translation process is extremely slow and >> results in a clunky development process. ?There's no '2to3 >> --interactive' commandline that lets me type python 2 at a >>> prompt >> and get python 3 results out so that I can try experiments on the 3.x >> interpreter; I have to actually put my experiments into my unit tests >> and wait 10 minutes to see if it works. ?It's like writing C++. > > That's not my experience. I see a change in source (say, on Django) > available for 3.x within 5 seconds. True, but you need to set up a process that will convert only the changed files, and before Distribute came with 3.0 support, that was tedious. Now it's easy, if you want to use distribute. (Except that there is some bug I promised to look at this week, but haven't....) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From fuzzyman at voidspace.org.uk Thu Nov 5 21:22:43 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 05 Nov 2009 21:22:43 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF330A7.5000809@v.loewis.de> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> <4AF1CFAD.6000305@v.loewis.de> <4AF2C032.4040605@gmail.com> <4AF330A7.5000809@v.loewis.de> Message-ID: <4AF33413.9060408@voidspace.org.uk> Martin v. L?wis wrote: >> Mike Krell wrote: >> >>> Well, 3to2 would then be an option for you: use Python 3 as the source >>> language. >>> >> Making life easier for 3to2 is an *excellent* rationale for backports. >> >> > > I'm skeptical. If new features get added to 2.7: why would that simplify > 3to2? It couldn't *use* these features, since it surely would have to > support 2.6 and earlier as well. > > Not sure what 3to2 would do about difficult-to-convert 3.x feature (as, > perhaps, the nonlocal keyword). If it currently gives up, it then may > offer you to restrict your target versions to 2.7+. Not sure whether > users would use that option, though - perhaps they rather stop using > nonlocal in 3.x if 3to2 cannot support it for all 2.x versions they are > interested in. > I would have thought you could translate nonlocal with the following: Python 3: def scope(): name = value do_something_with(name) def inner(): nonlocal name name = new_value do_something_else(name) Python 2 def scope(): name = [value] do_something_with(name[0]) def inner(): name[0] = new_value do_something_else(name[0]) I would love to see nonlocal backported to 2.7 as it cleans up a simple pattern that I use relatively often for testing. Suppose you have an class and you want to test that method a calls method b, in Python 2 you might write something like this: def test_method_a_calls_method_b(): instance = SomeClass() was_called = [] def method_b(): was_called.append(True) instance.method_b = method_b instance.method_a() assert was_called == [True] in Python 3 you can replace this with the slightly nicer: def test_method_a_calls_method_b(): instance = SomeClass() was_called = False def method_b(): nonlocal was_called was_called = True instance.method_b = method_b instance.method_a() assert was_called As to the argument that releasing 2.7 is pointless as few people would use it for several years, the success of Python 2.6 shows that although *many* people don't / can't use new versions of Python for several years many other people are able to and do use new versions of Python. All the best, Michael Foord > Perhaps 3to2 has a work-around that still provides a good backport in > most cases. Then, the backport would not make the tool any simpler: if > 3to2 would start using the backport, it would actually get more > complicated (not easier), as it now needs to support two cases. > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ From eric at trueblade.com Thu Nov 5 21:22:42 2009 From: eric at trueblade.com (Eric Smith) Date: Thu, 05 Nov 2009 15:22:42 -0500 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF330A7.5000809@v.loewis.de> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> <4AF1CFAD.6000305@v.loewis.de> <4AF2C032.4040605@gmail.com> <4AF330A7.5000809@v.loewis.de> Message-ID: <4AF33412.3070104@trueblade.com> Martin v. L?wis wrote: >> Mike Krell wrote: >>> Well, 3to2 would then be an option for you: use Python 3 as the source >>> language. >> Making life easier for 3to2 is an *excellent* rationale for backports. >> > > I'm skeptical. If new features get added to 2.7: why would that simplify > 3to2? It couldn't *use* these features, since it surely would have to > support 2.6 and earlier as well. > > Not sure what 3to2 would do about difficult-to-convert 3.x feature (as, > perhaps, the nonlocal keyword). If it currently gives up, it then may > offer you to restrict your target versions to 2.7+. Not sure whether > users would use that option, though - perhaps they rather stop using > nonlocal in 3.x if 3to2 cannot support it for all 2.x versions they are > interested in. But surely someday 2.7 will be the oldest targetted 2.x version of Python for 3to2 and other tools (regardless of whether there's a 2.8). When that day comes, 3to2 can be made simpler, or can increase the amount of Python 3.x it can convert (or both) if we add 3.x features to 2.7. Of course, planning for a time so far in the future is difficult, and possibly pointless. From regebro at gmail.com Thu Nov 5 21:31:54 2009 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 5 Nov 2009 21:31:54 +0100 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: References: <319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com> Message-ID: <319e029f0911051231i44775539p407e52eb872b9119@mail.gmail.com> 2009/11/4 ssteinerX at gmail.com : > Maybe the 3.x line should just be put out of our misery, merged back to 2.7, > 2.8, 2.9, and proceed as Glyph suggested in passing with increasing levels > of deprecation until it just turns into 3.x on its own by running out of > numbers. Yeah, maybe. If people haven't moved over to Python 3 in 2015 I think we should start considering it. Let's discuss this again then. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From regebro at gmail.com Thu Nov 5 21:36:09 2009 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 5 Nov 2009 21:36:09 +0100 Subject: [Python-Dev] Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?) In-Reply-To: <4AF17FB7.5090708@gmail.com> References: <4AEFF3A3.3020103@egenix.com> <4AF06E20.7050307@v.loewis.de> <4AF0816A.706@egenix.com> <319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com> <4AF16CC7.6000705@gmail.com> <4AF171AD.1000703@egenix.com> <4AF17FB7.5090708@gmail.com> Message-ID: <319e029f0911051236q7219e7bcp68f3892c24081a7d@mail.gmail.com> 2009/11/4 Nick Coghlan : > In writing it up, it occurred to me that having that kind of thing in a > "py3_compat" compatibility module (to be used as, e.g., "from py3_compat > import str, bytes") would not only make it easier to use in multiple > modules, but also easier for 2to3 to remove when forward porting. Well, when using 2to3 it already handles that stuff. But a module like that would be very handy if you want to support both 2.6 and 3.x without 2to3. With such a module it would be quite simple. In fact, I think the module should be called "timemachine". ;-) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From regebro at gmail.com Thu Nov 5 21:38:32 2009 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 5 Nov 2009 21:38:32 +0100 Subject: [Python-Dev] Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?) In-Reply-To: <4AF16CC7.6000705@gmail.com> References: <4AEF4C53.7090902@v.loewis.de> <4AEFF3A3.3020103@egenix.com> <4AF06E20.7050307@v.loewis.de> <4AF0816A.706@egenix.com> <319e029f0911032306j746bc263jcf6d0dfeaba9f24d@mail.gmail.com> <4AF16CC7.6000705@gmail.com> Message-ID: <319e029f0911051238v232dffffred78247f5011d1a9@mail.gmail.com> 2009/11/4 Nick Coghlan : > Lennart Regebro wrote: >> I also would really like to see a real port of the bytes class to 2.6, >> but I have a vague memory that there was some reason that wouldn't >> work. > > Not so much that it wouldn't work, but that the interfaces to support > using it effectively really aren't there - lots of areas in the standard > library needed to be tweaked to cope with bytes objects in 3.x. Ah, right, that was the problem, the standard library. I knew I heard a good reason against it. > Generally speaking, the "bytes = str" trick represents a reasonable > compromise as the APIs that you would pass a bytes object to in 3.x > expect an 8-bit str instance in 2.x. Yeah, the problem being that bytes and str has quite different API's. Ah well. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From brett at python.org Thu Nov 5 21:39:08 2009 From: brett at python.org (Brett Cannon) Date: Thu, 5 Nov 2009 12:39:08 -0800 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <9d153b7c0911050934k714aca73q62dd78cf9033584e@mail.gmail.com> References: <319e029f0911032206y64386499hf4564839ec9acb1a@mail.gmail.com> <426ada670911041039m6df85994v321342231fbd3805@mail.gmail.com> <4AF2C5B8.8070204@gmail.com> <9d153b7c0911050934k714aca73q62dd78cf9033584e@mail.gmail.com> Message-ID: On Thu, Nov 5, 2009 at 09:34, Yuvgoog Greenle wrote: > On Thu, Nov 5, 2009 at 2:31 PM, Nick Coghlan wrote: >> >> While it may be hard to tell without knowing who is and isn't a core >> developer > > Maybe there should be badges or something, hmmm, sounds like making an > svn-commits-hall-of-fame for python could be a nice project. > You might be interested in http://www.ohloh.net/p/python/contributors . -Brett > --yuv > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > > From stefan_ml at behnel.de Thu Nov 5 21:41:07 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 05 Nov 2009 21:41:07 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <1A472770E042064698CB5ADC83A12ACD04C4A6C6@TK5EX14MBXC118.redmond.corp.microsoft.com> References: <4AF29753.3040801@voidspace.org.uk> <1A472770E042064698CB5ADC83A12ACD04C4A6C6@TK5EX14MBXC118.redmond.corp.microsoft.com> Message-ID: Dino Viehland, 05.11.2009 19:35: > Stefan wrote: >> It /does/ make some static assumptions in that it considers builtins >> true >> builtins. However, it does not prevent you from replacing them in your >> code, as long as you do it inside the module. Certainly a restriction >> compared to Python, where you can import a module into a changed dict >> environment that redefines 'object', but not a major restriction IMO, >> and certainly not one that impacts much code. > > To me this is a deal breaker which prevents Cython from being a Python > implementation. From a talk given by Colin Winter at the LLVM dev meeting > (http://llvm.org/devmtg/2009-10/) it seems like Unladen Swallow wanted to > do something like this as well and Guido said no. In this case the breaking > change is so subtle that I'd personally hate to run into something like > this porting code to Cython and having to figure out why it's not working. I assume that this is artificially exaggerated to make a point, as this behaviour is obviously not a technical requirement but an optimisation, which could potentially be disabled. Stefan From chris at subtlety.com Thu Nov 5 21:43:32 2009 From: chris at subtlety.com (Chris Bergstresser) Date: Thu, 5 Nov 2009 14:43:32 -0600 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it In-Reply-To: <99BB308202224D31AA5300223DB59912@RaymondLaptop1> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <200911040946.15805.steve@pearwood.info> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> Message-ID: <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> On Wed, Nov 4, 2009 at 7:07 PM, Raymond Hettinger wrote: > [Steven D'Aprano] >>> Anyway, given the level of opposition to the suggestion, I'm no longer >>> willing to carry the flag for it. If anyone else -- perhaps the OP -- >>> feels they want to take it any further, be my guest. I feel pretty strongly that it's a wart in the language, and a sufficiently strong one that it should be remedied. I'm happy to champion it, but haven't the faintest idea what that entails. > Summarizing my opposition to a new set method: > 1) there already are at least two succinct ways to get the same effect > 2) those ways work with any container, not just sets > 3) set implementations in other languages show that this isn't needed. > 4) there is value to keeping the API compact > 5) isn't needed for optimization (selecting the same value in a loop makes > no sense) > 6) absence of real-world code examples that would be meaningfully improved > > I would be happy to add an example to the docs so that this thread > can finally end. Adding an example to the docs does not solve the problem, which is if you come across the following code: for x in s: break ... it really looks like it does nothing. It's only because of the slightly idiosyncratic way Python handles variable scoping that it has an effect at all, and that effect isn't overtly related to what the code says, which is "Iterate over all the elements in this set, then immediately stop after the first one". s.get() or s.pick() are both more succinct and more clear, saying "Get me an arbitrary element from this set". To make matters worse, "for x in s: break" fails silently when s is empty, and "x = iter(s).next()" raises a StopIteration exception. Neither is clear. The obvious way, for newcomers, of achieving the effect is: x = s.pop() s.add(x) ... and that's simply horrible in terms of efficiency. So the "obvious" way of doing it in Python is wrong(TM), and the "correct" way of doing it is obscure and raises misleading exceptions. I suppose, mulling things over, the method should be called .pick(), which avoids any confusion with .get(). And, as I've stated, I think it should return a member of the set, with no guarantees what member of the set is returned. It could be the same one every time, or a random one, or the last one placed in the set. For cases where people want to cycle through the members of the set in a predictable order, they can either copy the contents into a list (sorted or unsorted) *or* subclass set and override the .pick() method to place stronger guarantees on the API. So, summarizing my responses: 1) the two succinct ways are unclear and not immediately obvious 2) the existing methods aren't needed for other objects 3) set implementations in other languages are irrelevant 4) this is a small, targeted change which not make the API disordered or unruly 5) could very well be needed for optimization, in cases where constructing an iterator is expensive 6) there have been several real-world examples posted which would be improved by this change -- Chris From alexander.belopolsky at gmail.com Thu Nov 5 22:09:11 2009 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 5 Nov 2009 16:09:11 -0500 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it In-Reply-To: <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <200911040946.15805.steve@pearwood.info> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> Message-ID: On Thu, Nov 5, 2009 at 3:43 PM, Chris Bergstresser wrote: > .. and "x = iter(s).next()" raises a StopIteration > exception. And that's why the documented recipe should probably recommend next(iter(s), default) instead. Especially because iter(s).next() is not even valid code in 3.0. From martin at v.loewis.de Thu Nov 5 22:21:01 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 05 Nov 2009 22:21:01 +0100 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it In-Reply-To: <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <200911040946.15805.steve@pearwood.info> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> Message-ID: <4AF341BD.2010203@v.loewis.de> > I feel pretty strongly that it's a wart in the language, and a > sufficiently strong one that it should be remedied. I'm happy to > champion it, but haven't the faintest idea what that entails. There are two ways a) write a library that provides what you want, publish it on PyPI, and report back in a few years of how many users your library has, what they use it for, and why it should become builtin b) write a PEP, wait a few years for the language moratorium to be lifted, provide an implementation, and put the PEP for pronouncement. Careful reading of the Moratorium PEP may allow shortening of the wait. In any case, it seems that this specific change will see some opposition. So you will need to convince the opposition, one way or the other. > The obvious way, for newcomers, of achieving the effect is: > > x = s.pop() > s.add(x) > > ... and that's simply horrible in terms of efficiency. So the > "obvious" way of doing it in Python is wrong(TM), and the "correct" > way of doing it is obscure and raises misleading exceptions. If you chose to write a PEP, include a proof that this approach is indeed horrible in terms of efficiency; I question that. Regards, Martin From solipsis at pitrou.net Thu Nov 5 22:46:56 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 5 Nov 2009 21:46:56 +0000 (UTC) Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> <4AF1CFAD.6000305@v.loewis.de> <4AF2C032.4040605@gmail.com> <4AF330A7.5000809@v.loewis.de> <4AF33413.9060408@voidspace.org.uk> Message-ID: Michael Foord voidspace.org.uk> writes: > > I would love to see nonlocal backported to 2.7 as it cleans up a simple > pattern that I use relatively often for testing. Well you know I'm sure that if someone proposes a proper patch it will eventually get accepted ;) Regards Antoine. From martin at v.loewis.de Thu Nov 5 22:51:31 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 05 Nov 2009 22:51:31 +0100 Subject: [Python-Dev] No buildbot to test wide unicode? In-Reply-To: References: <4AF1D14A.6040007@v.loewis.de> Message-ID: <4AF348E3.1080200@v.loewis.de> >> I would certainly agree to setup two slaves on mine. There are ample >> resources available. > > I could do so as well. Gentoo is one of the distributions that uses > the wide build by default, so that would make it a good test candidate > as well. I have now setup two slaves, murray-gentoo-wide and pitrou-ubuntu-wide, same passwords. They configure with the option "--with-wide-unicode" (if I did that correctly). I think this means that the 2.x branches will have to grow this option also (although configure should ignore the option, so it should still build). Regards, Martin From dinov at microsoft.com Thu Nov 5 22:55:27 2009 From: dinov at microsoft.com (Dino Viehland) Date: Thu, 5 Nov 2009 21:55:27 +0000 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4AF29753.3040801@voidspace.org.uk> <1A472770E042064698CB5ADC83A12ACD04C4A6C6@TK5EX14MBXC118.redmond.corp.microsoft.com> Message-ID: <1A472770E042064698CB5ADC83A12ACD04C4BACB@TK5EX14MBXC118.redmond.corp.microsoft.com> Stefan wrote: > I assume that this is artificially exaggerated to make a point, as this > behaviour is obviously not a technical requirement but an optimisation, > which could potentially be disabled. > If there's a way to disable this then that's fine and IMO when it was disabled you'd still be Python. But changing behavior in the name of optimization is no longer just an optimization. For what it's worth in IronPython we're going to look at making this faster in the future as well. We already cache built-in values locally in a module and invalidate the cache when the builtins changes. In the future I'd like to even combine loading a global and invoking it into a single call site for even better optimizations. And you could even imagine an extreme scenario where when built-ins changed all the code in the system that depended upon them is re-compiled punishing users who actually do this but I doubt we'll go that far. But the important thing is that the optimizations don't change the language behavior otherwise I don't think you're still Python. But of course that's what makes it so challenging and fun to try and optimize Python :) From guido at python.org Thu Nov 5 23:05:17 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 5 Nov 2009 14:05:17 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: On Tue, Nov 3, 2009 at 9:35 AM, Guido van Rossum wrote: > I've checked draft (!) PEP 3003, "Python Language Moratorium", into > SVN. As authors I've listed Jesse, Brett and myself. I haven't seen substantial opposition against the PEP -- in fact I can't recall any, and many people have explicitly posted in support of it. So unless opposition suddenly appears in the next few days, I'll move it to the Accepted state next Monday. -- --Guido van Rossum (python.org/~guido) From python at rcn.com Thu Nov 5 23:11:20 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 5 Nov 2009 14:11:20 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium References: Message-ID: [GvR] > I haven't seen substantial opposition against the PEP -- in fact I > can't recall any, and many people have explicitly posted in support of > it. So unless opposition suddenly appears in the next few days, I'll > move it to the Accepted state next Monday. But it would have been so much fun to have a never ending python-ideas thread on BEGIN/END blocks ;-) C'est la vie, Raymond From arfrever.fta at gmail.com Thu Nov 5 23:20:29 2009 From: arfrever.fta at gmail.com (Arfrever Frehtes Taifersar Arahesis) Date: Thu, 5 Nov 2009 23:20:29 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: <200911052320.30872.Arfrever.FTA@gmail.com> 2009-11-03 18:35:10 Guido van Rossum napisa?(a): > I've checked draft (!) PEP 3003, "Python Language Moratorium", into > SVN. As authors I've listed Jesse, Brett and myself. > > On python-ideas the moratorium idea got fairly positive responses > (more positive than I'd expected, in fact) but I'm bracing myself for > fierce discussion here on python-dev. It's important to me that if if > this is accepted it is a "rough consensus" decision (working code we > already have plenty of :-), not something enforced by a vocal minority > or an influential individual such as myself. If there's too much > opposition I'll withdraw the PEP so as not to waste everybody's time > with a fruitless discussion. > > The PEP tries to spell out some gray areas but I'm sure there will be > others; that's life. Do note that the PEP proposes to be *retroactive* > back to the 3.1 release, i.e. the "frozen" version of the language is > the state in which it was released as 3.1. Does moratorium allow to add support for e.g. 'from __future__ import yield_from' in Python 3.2? -- Arfrever Frehtes Taifersar Arahesis From brett at python.org Thu Nov 5 23:18:46 2009 From: brett at python.org (Brett Cannon) Date: Thu, 5 Nov 2009 14:18:46 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <200911052320.30872.Arfrever.FTA@gmail.com> References: <200911052320.30872.Arfrever.FTA@gmail.com> Message-ID: On Thu, Nov 5, 2009 at 14:20, Arfrever Frehtes Taifersar Arahesis wrote: > 2009-11-03 18:35:10 Guido van Rossum napisa?(a): >> I've checked draft (!) PEP 3003, "Python Language Moratorium", into >> SVN. As authors I've listed Jesse, Brett and myself. >> >> On python-ideas the moratorium idea got fairly positive responses >> (more positive than I'd expected, in fact) but I'm bracing myself for >> fierce discussion here on python-dev. It's important to me that if if >> this is accepted it is a "rough consensus" decision (working code we >> already have plenty of :-), not something enforced by a vocal minority >> or an influential individual such as myself. If there's too much >> opposition I'll withdraw the PEP so as not to waste everybody's time >> with a fruitless discussion. >> >> The PEP tries to spell out some gray areas but I'm sure there will be >> others; that's life. Do note that the PEP proposes to be *retroactive* >> back to the 3.1 release, i.e. the "frozen" version of the language is >> the state in which it was released as 3.1. > > Does moratorium allow to add support for e.g. 'from __future__ import yield_from' > in Python 3.2? No. -Brett From chris at subtlety.com Thu Nov 5 23:19:32 2009 From: chris at subtlety.com (Chris Bergstresser) Date: Thu, 5 Nov 2009 16:19:32 -0600 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it In-Reply-To: <4AF341BD.2010203@v.loewis.de> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <200911040946.15805.steve@pearwood.info> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <4AF341BD.2010203@v.loewis.de> Message-ID: <7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> On Thu, Nov 5, 2009 at 3:21 PM, "Martin v. L?wis" wrote: > There are two ways > > a) write a library that provides what you want, publish it on PyPI, > ? and report back in a few years of how many users your library has, > ? what they use it for, and why it should become builtin This clearly isn't called for in this case. We're talking about a single function on a collection. In this case, importing an alternative set API (and maintaining the dependency) is more work than just writing your own workaround. The purpose of adding a method is to prevent the need of everyone writing their own workaround. > b) write a PEP, wait a few years for the language moratorium to be > ? lifted, provide an implementation, and put the PEP for pronouncement. > ? Careful reading of the Moratorium PEP may allow shortening of the > ? wait. Clearly, I'll need to write up the PEP. > In any case, it seems that this specific change will see some > opposition. So you will need to convince the opposition, one way or > the other. I doubt some of the people on either side are going to be convinced. I'd settle for convincing most of the fence-sitters, along with a few of the loyal opposition. -- Chris From python at rcn.com Thu Nov 5 23:30:56 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 5 Nov 2009 14:30:56 -0800 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <200911040946.15805.steve@pearwood.info> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <4AF341BD.2010203@v.loewis.de> <7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> Message-ID: [Chris Bergstresser] > Clearly, I'll need to write up the PEP. Why not write a short, fast get_first() function for your utils directory and be done with it? That could work with sets, mappings, generators, and other containers and iterators. No need to fatten the set/frozenset API for something so trivial and so rarely needed. Raymond From david.lyon at preisshare.net Thu Nov 5 23:29:22 2009 From: david.lyon at preisshare.net (David Lyon) Date: Thu, 05 Nov 2009 17:29:22 -0500 Subject: [Python-Dev] Refactoring installation schemes In-Reply-To: <94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com> References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com> <4AE8206A.9020108@egenix.com> <4AE824DA.4090701@voidspace.org.uk> <94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com> <94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com> <9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com> <39deb0d1eef08419dad65d7354a8ffb1@preisshare.net> <94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com> Message-ID: <0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net> On Thu, 5 Nov 2009 11:35:41 +0100, Tarek Ziad? wrote: > PEP 376 is working on a default, unified, *installation* format, that > tries to gather the good ideas of Pip, Setuptools etc.. and propose a > unified format for our site-packages. This new standard will come with > APIs in pkgutil to be able to query installed distribution etc. This > work is also linked to PEP 345 work where we are modifying the > Metadata, and to PEP 386 that proposes a standard version comparison > scheme. Perphaps.. But if you put all these PEPs together, implementing all the new features can't come to more than 300 lines of code... Since we hardly got anywhere on them in 2009, it will be interesting to see how much of it gets done in 2010. > But there's no plan to include a new *distribution* format in Distutils. I wasn't suggesting that - at all. And saying that 'eggs' are a *new* python package format isn't really really helpful because to my understanding they've been around for some number of years. No, i won't raise why we have EGG_INFO directories and a whole lot of half working egg stuff in standard python... I'm just asking why it can't be more consistant? while we're on the refactoring topic. Be fair... I'm saying finish what is already there.. or take out the crap .. It isn't fair to suggest that I am somehow asking for some big change when I am simply pointing out all the junk that's in there that's already half built. > In any case those PEPs are not finished yet, so everyone can help at > distutils-SIG True - and False. But I've been on the list for some twelve months asking for work to help out with, and haven't been assigned a single task to do yet. Seriously, if you won't allocate work out to people then how can it get done? Whilst I personally think a lot of the stuff in those PEPs is not high on quality, why don't we just get them implemented anyway? I'm a fairly proficient develper, but I can't get assigned a single work item.. And to me, it doesn't seem any harder than just selecting 'djlyon' on the python tracker for some work items... Surely those PEPs all amount to 300+ lines of code. With two people working on it, that's only surely 150+ lines of code each... That shouldn't be such a big challenge for 2010.. David From dirkjan at ochtman.nl Thu Nov 5 23:53:37 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Thu, 5 Nov 2009 23:53:37 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: On Thu, Nov 5, 2009 at 23:05, Guido van Rossum wrote: > I haven't seen substantial opposition against the PEP -- in fact I > can't recall any, and many people have explicitly posted in support of > it. So unless opposition suddenly appears in the next few days, I'll > move it to the Accepted state next Monday. Let me state first, I think the PEP is great, and I have no objection to its current form. I do have one qualm, where I wonder if the PEP shouldn't be a little stricter. As a gentoo developer and Mercurial maintainer, most of the pain in the recent migration towards 2.6 has not been in language changes, but in the standard library. Unfortunately, it's exempt from the moratorium in the PEP. Which makes me wonder, why are we not adding another moratorium, on deprecations in the standard library? In other words, let's not deprecate things like md5 or sha or the popen family of functions, but keep all of that as it is, for both 2.x and 3.x, so people can direct their energy towards other things (hopefully porting their 2.x codebase to 3.x). The standard library has already been through a lot of evolution in the 2.x to 3.x transition, so one might assume there's not a lot of stuff in the 3.x stdlib that would need deprecation in the short term. And for 2.x, well, I'd certainly hope we don't need to deprecate much more there before it finally gets EOL'ed, so it should be a relatively light maintenance load to bear. Is this just crazy talk? Cheers, Dirkjan From robert.kern at gmail.com Thu Nov 5 23:57:40 2009 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 05 Nov 2009 16:57:40 -0600 Subject: [Python-Dev] Refactoring installation schemes In-Reply-To: <0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net> References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com> <4AE8206A.9020108@egenix.com> <4AE824DA.4090701@voidspace.org.uk> <94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com> <94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com> <9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com> <39deb0d1eef08419dad65d7354a8ffb1@preisshare.net> <94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com> <0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net> Message-ID: On 2009-11-05 16:29 PM, David Lyon wrote: > But I've been on the list for some twelve months asking for work > to help out with, and haven't been assigned a single task to do > yet. > > Seriously, if you won't allocate work out to people then how can > it get done? With rare exceptions, open source projects don't really "assign" work. If you want to help, help. Don't wait for someone to tell you exactly what to do. No one will. Go to the tracker, find something interesting, and do it. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From python at rcn.com Fri Nov 6 00:02:01 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 5 Nov 2009 15:02:01 -0800 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com><200911040946.15805.steve@pearwood.info><4AF0D0D7.3090403@canterbury.ac.nz><200911042234.36796.steve@pearwood.info><99BB308202224D31AA5300223DB59912@RaymondLaptop1><7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com><4AF341BD.2010203@v.loewis.de><7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> Message-ID: <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> [me] > Why not write a short, fast get_first() function for your utils directory and be done with it? > That could work with sets, mappings, generators, and other containers and iterators. > No need to fatten the set/frozenset API for something so trivial and so rarely needed. Forgot to post the code. It is short, fast, and easy. It is explicit about handing the case with an empty input. And it is specific about which value it returns (always the first iterated value; not an arbitrary one). There's no guessing about what it does. It gets the job done. def first(iterable): 'Return the first value from a container or iterable. If empty, raises LookupError' for value in iterable: return value raise LookupError('no first value; iterable is empty') If desired, it is not hard to change to the last time to return a default value or some other exception. Raymond From ziade.tarek at gmail.com Fri Nov 6 00:03:43 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 6 Nov 2009 00:03:43 +0100 Subject: [Python-Dev] Refactoring installation schemes In-Reply-To: <0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net> References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com> <4AE824DA.4090701@voidspace.org.uk> <94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com> <94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com> <9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com> <39deb0d1eef08419dad65d7354a8ffb1@preisshare.net> <94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com> <0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net> Message-ID: <94bdd2610911051503t68ef0931p2f6200f3e59c07b4@mail.gmail.com> On Thu, Nov 5, 2009 at 11:29 PM, David Lyon wrote: [...] > > But if you put all these PEPs together, implementing all the new features > can't come to more than 300 lines of code... > > Since we hardly got anywhere on them in 2009, it will be interesting to > see how much of it gets done in 2010. The work that was done during the last year was brainstorming on PEPs. You can't count that work in SLOCs, but in trying to find a consensus on packaging problems. And I can fairly say that we are very close to something that can get accepted by the community, then turned into code in Distutils/pkgutil. > >> But there's no plan to include a new *distribution* format in Distutils. > > I wasn't suggesting that - at all. > > And saying that 'eggs' are a *new* python package format isn't really > really helpful because to my understanding they've been around for > some number of years. They are new to Distutils. They are not new to the Python packaging eco-system of course, and PEP 376 rely on many ideas created in Setuptools. > No, i won't raise why we have EGG_INFO directories and a whole lot > of half working egg stuff in standard python... I'm just asking > why it can't be more consistant? while we're on the refactoring > topic. The consistency will come through the standard we are building in PEP 376. [..] > But I've been on the list for some twelve months asking for work > to help out with, and haven't been assigned a single task to do > yet. > > Seriously, if you won't allocate work out to people then how can > it get done? I am sorry that you feel that way. We don't allocate coding work to people in this process. That's not the way it works. Rather, people help in building the PEPs by providing their own feedback/experience. At the end, we are trying to have PEPs that adresses the wider range of cases. > > Whilst I personally think a lot of the stuff in those PEPs is not > high on quality, why don't we just get them implemented anyway? > > I'm a fairly proficient develper, but I can't get assigned a single > work item.. We have already prototypes for each PEP so people can try them out, enhance them while the PEPs are being built. If you want to help in their coding, you are more than welcome. They are on a DVCS at bitbucket. > > And to me, it doesn't seem any harder than just selecting 'djlyon' > on the python tracker for some work items... > > Surely those PEPs all amount to 300+ lines of code. With two people > working on it, that's only surely 150+ lines of code each... That > shouldn't be such a big challenge for 2010.. Again, the big challenge is not on the coding part. If it was, Distutils would have them already. The challenge is on the PEPs, and on making sure we collect all PoVs and feedbacks, before we change Distutils in Python 2.7/3.2 Regards Tarek From debatem1 at gmail.com Fri Nov 6 00:04:54 2009 From: debatem1 at gmail.com (geremy condra) Date: Thu, 5 Nov 2009 18:04:54 -0500 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it In-Reply-To: References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <200911040946.15805.steve@pearwood.info> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> Message-ID: On Thu, Nov 5, 2009 at 4:09 PM, Alexander Belopolsky wrote: > On Thu, Nov 5, 2009 at 3:43 PM, Chris Bergstresser wrote: >> .. and "x = iter(s).next()" raises a StopIteration >> exception. > > And that's why the documented recipe should probably recommend > next(iter(s), default) instead. ?Especially because iter(s).next() is > not even valid code in 3.0. This seems reasonably legible to you? Strikes me as coding by incantation. Also, while I've heard people say that the naive approach is slower, I'm not getting that result. Here's my test: >>> smrt = timeit.Timer("next(iter(s))", "s=set(range(100))") >>> smrt.repeat(10) [1.2845709323883057, 0.60247397422790527, 0.59621405601501465, 0.59133195877075195, 0.58387589454650879, 0.56839084625244141, 0.56839680671691895, 0.56877803802490234, 0.56905913352966309, 0.56846404075622559] >>> naive = timeit.Timer("x=s.pop();s.add(x)", "s=set(range(100))") >>> naive.repeat(10) [0.93139314651489258, 0.53566789627075195, 0.53674602508544922, 0.53608798980712891, 0.53634309768676758, 0.53557991981506348, 0.53578495979309082, 0.53666114807128906, 0.53576493263244629, 0.53491711616516113] Perhaps my test is flawed in some way? Geremy Condra From foom at fuhm.net Fri Nov 6 00:17:19 2009 From: foom at fuhm.net (James Y Knight) Date: Thu, 5 Nov 2009 18:17:19 -0500 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it In-Reply-To: References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <200911040946.15805.steve@pearwood.info> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> Message-ID: <9CAD2251-C51D-472B-98A4-104E1AFEEDE1@fuhm.net> On Nov 5, 2009, at 6:04 PM, geremy condra wrote: > Perhaps my test is flawed in some way? Yes: you're testing the speed of something that makes absolutely no sense to do in a tight loop, so *who the heck cares how fast any way of doing it is*! Is this thread over yet? James From guido at python.org Fri Nov 6 00:18:12 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 5 Nov 2009 15:18:12 -0800 Subject: [Python-Dev] Refactoring installation schemes In-Reply-To: <0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net> References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com> <4AE824DA.4090701@voidspace.org.uk> <94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com> <94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com> <9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com> <39deb0d1eef08419dad65d7354a8ffb1@preisshare.net> <94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com> <0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net> Message-ID: David, you have an attitude problem. Your contributions (the post below is just an example) don't sound healthy to me -- you just complain and whine and denigrate Tarek's work. In a previous post you claimed to have had a particular idea first (it doesn't matter which idea) and you managed to make it sound bad that your idea was eventually accepted. This is not a productive attitude. Surely the problem isn't writing 300 lines of code. The problem is getting everyone to agree on which 300 lines of code should be written. That is the problem at hand, and claiming that nothing happened because no code was written and all that was agreed on amounts to 300 lines of code is outright demeaning. Stop it. You are wearing out your welcome. --Guido PS. Submitting a counter-PEP to the peps editors that hasn't been discussed on the SIG list at all is also a bad move. You really need to change the way you try to interact with the SIG. On Thu, Nov 5, 2009 at 2:29 PM, David Lyon wrote: > On Thu, 5 Nov 2009 11:35:41 +0100, Tarek Ziad? > wrote: >> PEP 376 is working on a default, unified, *installation* format, that >> tries to gather the good ideas of Pip, Setuptools etc.. and propose a >> unified format for our site-packages. This new standard will come with >> APIs in pkgutil to be able to query installed distribution etc. This >> work is also linked to PEP 345 work where we are modifying the >> Metadata, and to PEP 386 that proposes a standard version comparison >> scheme. > > Perphaps.. > > But if you put all these PEPs together, implementing all the new features > can't come to more than 300 lines of code... > > Since we hardly got anywhere on them in 2009, it will be interesting to > see how much of it gets done in 2010. > >> But there's no plan to include a new *distribution* format in Distutils. > > I wasn't suggesting that - at all. > > And saying that 'eggs' are a *new* python package format isn't really > really helpful because to my understanding they've been around for > some number of years. > > No, i won't raise why we have EGG_INFO directories and a whole lot > of half working egg stuff in standard python... I'm just asking > why it can't be more consistant? while we're on the refactoring > topic. > > Be fair... > > I'm saying finish what is already there.. or take out the crap .. > > It isn't fair to suggest that I am somehow asking for some big > change when I am simply pointing out all the junk that's in > there that's already half built. > >> In any case those PEPs are not finished yet, so everyone can help at >> distutils-SIG > > True - and False. > > But I've been on the list for some twelve months asking for work > to help out with, and haven't been assigned a single task to do > yet. > > Seriously, if you won't allocate work out to people then how can > it get done? > > Whilst I personally think a lot of the stuff in those PEPs is not > high on quality, why don't we just get them implemented anyway? > > I'm a fairly proficient develper, but I can't get assigned a single > work item.. > > And to me, it doesn't seem any harder than just selecting 'djlyon' > on the python tracker for some work items... > > Surely those PEPs all amount to 300+ lines of code. With two people > working on it, that's only surely 150+ lines of code each... That > shouldn't be such a big challenge for 2010.. > > David > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) From jnoller at gmail.com Fri Nov 6 00:21:42 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 5 Nov 2009 18:21:42 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> On Thu, Nov 5, 2009 at 5:53 PM, Dirkjan Ochtman wrote: > On Thu, Nov 5, 2009 at 23:05, Guido van Rossum wrote: >> I haven't seen substantial opposition against the PEP -- in fact I >> can't recall any, and many people have explicitly posted in support of >> it. So unless opposition suddenly appears in the next few days, I'll >> move it to the Accepted state next Monday. > > Let me state first, I think the PEP is great, and I have no objection > to its current form. > > I do have one qualm, where I wonder if the PEP shouldn't be a little > stricter. As a gentoo developer and Mercurial maintainer, most of the > pain in the recent migration towards 2.6 has not been in language > changes, but in the standard library. Unfortunately, it's exempt from > the moratorium in the PEP. > > Which makes me wonder, why are we not adding another moratorium, on > deprecations in the standard library? In other words, let's not > deprecate things like md5 or sha or the popen family of functions, but > keep all of that as it is, for both 2.x and 3.x, so people can direct > their energy towards other things (hopefully porting their 2.x > codebase to 3.x). > > The standard library has already been through a lot of evolution in > the 2.x to 3.x transition, so one might assume there's not a lot of > stuff in the 3.x stdlib that would need deprecation in the short term. > And for 2.x, well, I'd certainly hope we don't need to deprecate much > more there before it finally gets EOL'ed, so it should be a relatively > light maintenance load to bear. > > Is this just crazy talk? > > Cheers, > > Dirkjan > I'm against restricting deprecation warnings within the stdlib as part of this. I actually want more things cleaned up and possibly deprecated. That being said, a deprecation warning just means we will remove it One Day - anything being deprecated will need a PEP and follow the long path to actual removal. So, -1 from me ;) jesse From brett at python.org Fri Nov 6 00:24:11 2009 From: brett at python.org (Brett Cannon) Date: Thu, 5 Nov 2009 15:24:11 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: On Thu, Nov 5, 2009 at 14:53, Dirkjan Ochtman wrote: > On Thu, Nov 5, 2009 at 23:05, Guido van Rossum wrote: >> I haven't seen substantial opposition against the PEP -- in fact I >> can't recall any, and many people have explicitly posted in support of >> it. So unless opposition suddenly appears in the next few days, I'll >> move it to the Accepted state next Monday. > > Let me state first, I think the PEP is great, and I have no objection > to its current form. > > I do have one qualm, where I wonder if the PEP shouldn't be a little > stricter. As a gentoo developer and Mercurial maintainer, most of the > pain in the recent migration towards 2.6 has not been in language > changes, but in the standard library. Unfortunately, it's exempt from > the moratorium in the PEP. > > Which makes me wonder, why are we not adding another moratorium, on > deprecations in the standard library? In other words, let's not > deprecate things like md5 or sha or the popen family of functions, but > keep all of that as it is, for both 2.x and 3.x, so people can direct > their energy towards other things (hopefully porting their 2.x > codebase to 3.x). > > The standard library has already been through a lot of evolution in > the 2.x to 3.x transition, so one might assume there's not a lot of > stuff in the 3.x stdlib that would need deprecation in the short term. > And for 2.x, well, I'd certainly hope we don't need to deprecate much > more there before it finally gets EOL'ed, so it should be a relatively > light maintenance load to bear. > > Is this just crazy talk? So you are basically asking for a moratorium on stdlib deprecations. Considering how much was removed in Python 3 from 2.6 this is a really minor worry. And the only deprecation that I can see potentially coming down the pipeline is optparse for argparse and cProfile to help finish PEP 3108. And, as Jesse said in another reply, it would be nice to take this time to reflect upon where we want the stdlib to go and to potentially clear other things out. So I am -0 on this as leaving stuff in for an extra release isn't going to kill us, but I would prefer to not put it off. -Brett From guido at python.org Fri Nov 6 00:26:16 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 5 Nov 2009 15:26:16 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> Message-ID: On Thu, Nov 5, 2009 at 3:21 PM, Jesse Noller wrote: > On Thu, Nov 5, 2009 at 5:53 PM, Dirkjan Ochtman wrote: >> On Thu, Nov 5, 2009 at 23:05, Guido van Rossum wrote: >>> I haven't seen substantial opposition against the PEP -- in fact I >>> can't recall any, and many people have explicitly posted in support of >>> it. So unless opposition suddenly appears in the next few days, I'll >>> move it to the Accepted state next Monday. >> >> Let me state first, I think the PEP is great, and I have no objection >> to its current form. >> >> I do have one qualm, where I wonder if the PEP shouldn't be a little >> stricter. As a gentoo developer and Mercurial maintainer, most of the >> pain in the recent migration towards 2.6 has not been in language >> changes, but in the standard library. Unfortunately, it's exempt from >> the moratorium in the PEP. >> >> Which makes me wonder, why are we not adding another moratorium, on >> deprecations in the standard library? In other words, let's not >> deprecate things like md5 or sha or the popen family of functions, but >> keep all of that as it is, for both 2.x and 3.x, so people can direct >> their energy towards other things (hopefully porting their 2.x >> codebase to 3.x). >> >> The standard library has already been through a lot of evolution in >> the 2.x to 3.x transition, so one might assume there's not a lot of >> stuff in the 3.x stdlib that would need deprecation in the short term. >> And for 2.x, well, I'd certainly hope we don't need to deprecate much >> more there before it finally gets EOL'ed, so it should be a relatively >> light maintenance load to bear. >> >> Is this just crazy talk? >> >> Cheers, >> >> Dirkjan >> > > I'm against restricting deprecation warnings within the stdlib as part > of this. I actually want more things cleaned up and possibly > deprecated. That being said, a deprecation warning just means we will > remove it One Day - anything being deprecated will need a PEP and > follow the long path to actual removal. > > So, -1 from me ;) > > jesse Actually, I think Dirkjan has a point. I'm not sure that we need another moratorium (that's a rather dramatic kind of decision which should be very rare indeed) but I do agree that deprecations are often more of a pain than they're worth. For example, take the deprecation of the md5 and sha modules in Python 2.6. They make it a bit of a pain to write code that *cleanly* supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when importing md5 instead of hashlib). You can silence the warning, but that is in itself not particularly clean, and users really hate having the warnings. I have come to the conclusion that there are better ways to pre-announce that a module is going to disappear instead of deprecation warnings. -- --Guido van Rossum (python.org/~guido) From jnoller at gmail.com Fri Nov 6 00:34:37 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 5 Nov 2009 18:34:37 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> Message-ID: <4222a8490911051534n504d2a02qd53177662e0c4c70@mail.gmail.com> On Thu, Nov 5, 2009 at 6:26 PM, Guido van Rossum wrote: >> >> I'm against restricting deprecation warnings within the stdlib as part >> of this. I actually want more things cleaned up and possibly >> deprecated. That being said, a deprecation warning just means we will >> remove it One Day - anything being deprecated will need a PEP and >> follow the long path to actual removal. >> >> So, -1 from me ;) >> >> jesse > > Actually, I think Dirkjan has a point. I'm not sure that we need > another moratorium (that's a rather dramatic kind of decision which > should be very rare indeed) but I do agree that deprecations are often > more of a pain than they're worth. > > For example, take the deprecation of the md5 and sha modules in Python > 2.6. They make it a bit of a pain to write code that *cleanly* > supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when > importing md5 instead of hashlib). You can silence the warning, but > that is in itself not particularly clean, and users really hate having > the warnings. > > I have come to the conclusion that there are better ways to > pre-announce that a module is going to disappear instead of > deprecation warnings. > I'm interested in hearing how to handle "pending removals" other than deprecation warnings - I guess I'm against the idea that we shouldn't remove/plan to remove things from the stdlib and signal those intentions to users during the moratorium. The mechanics of that can be something other than deprecation warnings, we can add a line to the current moratorium that puts the nix on deprecation warnings (because yes, Dirkjan is right - they can be a pain) but we should outline the alternative process. jesse From brett at python.org Fri Nov 6 00:35:28 2009 From: brett at python.org (Brett Cannon) Date: Thu, 5 Nov 2009 15:35:28 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> Message-ID: On Thu, Nov 5, 2009 at 15:26, Guido van Rossum wrote: > On Thu, Nov 5, 2009 at 3:21 PM, Jesse Noller wrote: >> On Thu, Nov 5, 2009 at 5:53 PM, Dirkjan Ochtman wrote: >>> On Thu, Nov 5, 2009 at 23:05, Guido van Rossum wrote: >>>> I haven't seen substantial opposition against the PEP -- in fact I >>>> can't recall any, and many people have explicitly posted in support of >>>> it. So unless opposition suddenly appears in the next few days, I'll >>>> move it to the Accepted state next Monday. >>> >>> Let me state first, I think the PEP is great, and I have no objection >>> to its current form. >>> >>> I do have one qualm, where I wonder if the PEP shouldn't be a little >>> stricter. As a gentoo developer and Mercurial maintainer, most of the >>> pain in the recent migration towards 2.6 has not been in language >>> changes, but in the standard library. Unfortunately, it's exempt from >>> the moratorium in the PEP. >>> >>> Which makes me wonder, why are we not adding another moratorium, on >>> deprecations in the standard library? In other words, let's not >>> deprecate things like md5 or sha or the popen family of functions, but >>> keep all of that as it is, for both 2.x and 3.x, so people can direct >>> their energy towards other things (hopefully porting their 2.x >>> codebase to 3.x). >>> >>> The standard library has already been through a lot of evolution in >>> the 2.x to 3.x transition, so one might assume there's not a lot of >>> stuff in the 3.x stdlib that would need deprecation in the short term. >>> And for 2.x, well, I'd certainly hope we don't need to deprecate much >>> more there before it finally gets EOL'ed, so it should be a relatively >>> light maintenance load to bear. >>> >>> Is this just crazy talk? >>> >>> Cheers, >>> >>> Dirkjan >>> >> >> I'm against restricting deprecation warnings within the stdlib as part >> of this. I actually want more things cleaned up and possibly >> deprecated. That being said, a deprecation warning just means we will >> remove it One Day - anything being deprecated will need a PEP and >> follow the long path to actual removal. >> >> So, -1 from me ;) >> >> jesse > > Actually, I think Dirkjan has a point. I'm not sure that we need > another moratorium (that's a rather dramatic kind of decision which > should be very rare indeed) but I do agree that deprecations are often > more of a pain than they're worth. > > For example, take the deprecation of the md5 and sha modules in Python > 2.6. They make it a bit of a pain to write code that *cleanly* > supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when > importing md5 instead of hashlib). You can silence the warning, but > that is in itself not particularly clean, and users really hate having > the warnings. > > I have come to the conclusion that there are better ways to > pre-announce that a module is going to disappear instead of > deprecation warnings. > What exactly are those better ways? Document as deprecated only? -Brett From ubershmekel at gmail.com Fri Nov 6 00:52:54 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Fri, 6 Nov 2009 01:52:54 +0200 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it In-Reply-To: <9CAD2251-C51D-472B-98A4-104E1AFEEDE1@fuhm.net> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <200911040946.15805.steve@pearwood.info> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <9CAD2251-C51D-472B-98A4-104E1AFEEDE1@fuhm.net> Message-ID: <9d153b7c0911051552v7cfabf61ifede37a72ec239d8@mail.gmail.com> On Fri, Nov 6, 2009 at 1:17 AM, James Y Knight wrote: > > Is this thread over yet? > > Sorry, I just had to point out that pop/add has a side effect that would be apparent on a set that multiple threads access - it loses an item and then gets it back. Sounds like a sleeper race condition that's going to be rare but extremely hard to find if it does occur. Crooked as a gil. --yuv -------------- next part -------------- An HTML attachment was scrubbed... URL: From bobbyrward at gmail.com Fri Nov 6 00:55:34 2009 From: bobbyrward at gmail.com (Bobby R. Ward) Date: Thu, 5 Nov 2009 17:55:34 -0600 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> Message-ID: <48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com> > What exactly are those better ways? Document as deprecated only? > > -Brett A switch to ENABLE those warnings? Lord knows I'm sick of filtering them out of logs. A switch to enable deprecation warnings would give developers a chance to see them when migrating to a new version of python. And it would prevent users from having to deal with them. -- Bobby R. Ward ------------------------------ bobbyrward at gmail.com http://github.com/bobbyrward http://launchpad.net/~bobbyrward "While many languages can be used to encrypt data, PERL has something built-in that gives you encryption. Perl calls it `syntax`." -- http://uncyclopedia.wikia.com/wiki/Perl From david.lyon at preisshare.net Fri Nov 6 00:55:01 2009 From: david.lyon at preisshare.net (David Lyon) Date: Thu, 05 Nov 2009 18:55:01 -0500 Subject: [Python-Dev] Refactoring installation schemes In-Reply-To: References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com> <4AE824DA.4090701@voidspace.org.uk> <94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com> <94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com> <9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com> <39deb0d1eef08419dad65d7354a8ffb1@preisshare.net> <94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com> <0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net> Message-ID: <6d81ac30c331f8b9ca71a75d1049ffd4@preisshare.net> Tarek, Guido, Forgive my grumpy tone.. Looking positive, if we now have a rough consensus that the PEPS might represent some 300+ lines of code... then good - lets get started, that's all that I meant. I'm glad above all, that you noticed that part foremost. If it's a simple case that alternative's to the solutions cannot be considered - because I am grumpy - whatever, then that's something I can live with. I didn't know that proposing a different solution wasn't a good idea. > PS. Submitting a counter-PEP to the peps editors that hasn't been > discussed on the SIG list at all is also a bad move. You really need > to change the way you try to interact with the SIG. Here are the references to the discussions from distutils-sig. In these references you'll see that my interaction with the list was not grumpy but was entirely positive. I feel that I got a fair hearing on distutils list and I had a number of supporters to my proposal. The proposal is in the final stages of being wrapped up. Here are the discussion References that have occurred over the year: http://mail.python.org/pipermail/distutils-sig/2009-November/thread.html November 2009 Archives by thread * Messages sorted by: [ subject ] [ author ] [ date ] * More info on this list... Starting: Sun Nov 1 09:31:46 CET 2009 Ending: Fri Nov 6 00:09:22 CET 2009 Messages: 65 * [Distutils] Alternate static metadata PEP submission... Georg Brandl o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Floris Bruynooghe + [Distutils] Alternate static metadata PEP submission... Georg Brandl + [Distutils] Alternate static metadata PEP submission... David Lyon # [Distutils] Alternate static metadata PEP submission... Georg Brandl # [Distutils] Alternate static metadata PEP submission... David Lyon http://mail.python.org/pipermail/distutils-sig/2009-October/thread.html # [Distutils] Alternate static metadata PEP submission... David Lyon * [Distutils] Alternate static metadata PEP submission... Tarek Ziad? o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Tarek Ziad? o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Eric Smith o [Distutils] Alternate static metadata PEP submission... Tarek Ziad? o [Distutils] Alternate static metadata PEP submission... David Lyon * [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau * [Distutils] Alternate static metadata PEP submission... Floris Bruynooghe o [Distutils] Alternate static metadata PEP submission... Tarek Ziad? o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Tarek Ziad? o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Floris Bruynooghe o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Tarek Ziad? o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Ian Bicking o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Fred Drake o [Distutils] Alternate static metadata PEP submission... Kevin Teague o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... Fred Drake o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Chris Withers o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Chris Withers o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Chris Withers o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Chris Withers o [Distutils] Alternate static metadata PEP submission... Piotr Ozarowski o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Piotr Ozarowski o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Chris Withers o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Chris Withers o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... Chris Withers o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Fred Drake o [Distutils] Alternate static metadata PEP submission... Chris Withers o [Distutils] Alternate static metadata PEP submission... Fred Drake o [Distutils] Alternate static metadata PEP submission... Chris Withers o [Distutils] Alternate static metadata PEP submission... Eric Smith o [Distutils] Alternate static metadata PEP submission... Fred Drake o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon http://mail.python.org/pipermail/distutils-sig/2009-August/thread.html [Distutils] Static metadata using setup.cfg Tarek Ziad? * [Distutils] Static metadata using setup.cfg Hanno Schlichting * [Distutils] Static metadata using setup.cfg P.J. Eby o [Distutils] Static metadata using setup.cfg Tarek Ziad? + [Distutils] Static metadata using setup.cfg Tarek Ziad? + [Distutils] Static metadata using setup.cfg Tarek Ziad? * [Distutils] Static metadata using setup.cfg David Lyon o [Distutils] Static metadata using setup.cfg Jeff Rush + [Distutils] Static metadata using setup.cfg Ben Finney + [Distutils] Static metadata using setup.cfg Tarek Ziad? + [Distutils] Static metadata using setup.cfg David Lyon + [Distutils] Static metadata using setup.cfg Eric Smith + [Distutils] Static metadata using setup.cfg Tarek Ziad? + [Distutils] Static metadata using setup.cfg Tarek Ziad? + [Distutils] Static metadata using setup.cfg Chris Withers + [Distutils] Static metadata using setup.cfg Tarek Ziad? + [Distutils] Static metadata using setup.cfg Chris Withers * [Distutils] Static metadata using setup.cfg David Lyon o [Distutils] Static metadata using setup.cfg Sridhar Ratnakumar + [Distutils] Static metadata using setup.cfg David Lyon o [Distutils] Static metadata using setup.cfg Alex Gr?nholm + [Distutils] Static metadata using setup.cfg David Lyon + [Distutils] Static metadata using setup.cfg Floris Bruynooghe + [Distutils] Static metadata using setup.cfg Ronald Oussoren + [Distutils] Static metadata using setup.cfg P.J. Eby + [Distutils] Static metadata using setup.cfg Chris Withers o [Distutils] Static metadata using setup.cfg Chris Withers + [Distutils] Static metadata using setup.cfg Tarek Ziad? + [Distutils] Static metadata using setup.cfg Chris Withers + [Distutils] Static metadata using setup.cfg Tarek Ziad? + [Distutils] Static metadata using setup.cfg David Lyon + [Distutils] Static metadata using setup.cfg Tarek Ziad? http://mail.python.org/pipermail/distutils-sig/2009-July/thread.html # [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) David Lyon # [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) P.J. Eby * [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) Eric Smith * [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) David Cournapeau o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) Jim Fulton o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) David Cournapeau o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) Jim Fulton o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) David Cournapeau o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) Jim Fulton o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) Tres Seaver o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) David Cournapeau o [Distutils] Questions about virtual-python on ubuntu David Lyon o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) Paul Moore o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) Jim Fulton o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) David Lyon o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) Jim Fulton o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) David Lyon o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) Jim Fulton o [Distutils] "setup.py needs to go away" (was [PEP 376] - Open questions on python-dev) David Lyon # [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a Procedure David Lyon # [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a Procedure Jeff Rush # [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a Procedure David Lyon # [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a Procedure P.J. Eby # [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a Procedure David Lyon # [Distutils] Removing the hack from setup.py - Setup.py as a Class, not a Procedure Jean-Paul Calderone # [Distutils] Setup.py as a Class, not a Procedure David Lyon # [Distutils] Setup.py as a Class, not a Procedure Jean-Paul Calderone # [Distutils] Setup.py as a Class, not a Procedure David Lyon [Distutils] introspecting package data Chris Withers http://mail.python.org/pipermail/distutils-sig/2009-June/thread.html * [Distutils] introspecting package data Tarek Ziad? o [Distutils] introspecting package data Chris Withers + [Distutils] introspecting package data Jim Fulton + [Distutils] introspecting package data Chris Withers + [Distutils] introspecting package data Tarek Ziad? + [Distutils] introspecting package data David Lyon On Thu, 5 Nov 2009 15:18:12 -0800, Guido van Rossum wrote: > David, you have an attitude problem. Your contributions (the post > below is just an example) don't sound healthy to me -- you just > complain and whine and denigrate Tarek's work. In a previous post you > claimed to have had a particular idea first (it doesn't matter which > idea) and you managed to make it sound bad that your idea was > eventually accepted. This is not a productive attitude. > > Surely the problem isn't writing 300 lines of code. The problem is > getting everyone to agree on which 300 lines of code should be > written. That is the problem at hand, and claiming that nothing > happened because no code was written and all that was agreed on > amounts to 300 lines of code is outright demeaning. > > Stop it. You are wearing out your welcome. > > --Guido > > PS. Submitting a counter-PEP to the peps editors that hasn't been > discussed on the SIG list at all is also a bad move. You really need > to change the way you try to interact with the SIG. > > On Thu, Nov 5, 2009 at 2:29 PM, David Lyon > wrote: >> On Thu, 5 Nov 2009 11:35:41 +0100, Tarek Ziad? >> wrote: >>> PEP 376 is working on a default, unified, *installation* format, that >>> tries to gather the good ideas of Pip, Setuptools etc.. and propose a >>> unified format for our site-packages. This new standard will come with >>> APIs in pkgutil to be able to query installed distribution etc. This >>> work is also linked to PEP 345 work where we are modifying the >>> Metadata, and to PEP 386 that proposes a standard version comparison >>> scheme. >> >> Perphaps.. >> >> But if you put all these PEPs together, implementing all the new features >> can't come to more than 300 lines of code... >> >> Since we hardly got anywhere on them in 2009, it will be interesting to >> see how much of it gets done in 2010. >> >>> But there's no plan to include a new *distribution* format in Distutils. >> >> I wasn't suggesting that - at all. >> >> And saying that 'eggs' are a *new* python package format isn't really >> really helpful because to my understanding they've been around for >> some number of years. >> >> No, i won't raise why we have EGG_INFO directories and a whole lot >> of half working egg stuff in standard python... I'm just asking >> why it can't be more consistant? while we're on the refactoring >> topic. >> >> Be fair... >> >> I'm saying finish what is already there.. or take out the crap .. >> >> It isn't fair to suggest that I am somehow asking for some big >> change when I am simply pointing out all the junk that's in >> there that's already half built. >> >>> In any case those PEPs are not finished yet, so everyone can help at >>> distutils-SIG >> >> True - and False. >> >> But I've been on the list for some twelve months asking for work >> to help out with, and haven't been assigned a single task to do >> yet. >> >> Seriously, if you won't allocate work out to people then how can >> it get done? >> >> Whilst I personally think a lot of the stuff in those PEPs is not >> high on quality, why don't we just get them implemented anyway? >> >> I'm a fairly proficient develper, but I can't get assigned a single >> work item.. >> >> And to me, it doesn't seem any harder than just selecting 'djlyon' >> on the python tracker for some work items... >> >> Surely those PEPs all amount to 300+ lines of code. With two people >> working on it, that's only surely 150+ lines of code each... That >> shouldn't be such a big challenge for 2010.. >> >> David >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/guido%40python.org >> From ubershmekel at gmail.com Fri Nov 6 01:13:29 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Fri, 6 Nov 2009 02:13:29 +0200 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com> References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com> Message-ID: <9d153b7c0911051613o62be5dc9l52102f7805cd4a0e@mail.gmail.com> On Fri, Nov 6, 2009 at 1:55 AM, Bobby R. Ward wrote: > > A switch to ENABLE those warnings? > > I think a few people I know would still be raising strings like exceptions if not for the deprecation warnings. Most people won't turn on the switch with the extra warnings. --yuv -------------- next part -------------- An HTML attachment was scrubbed... URL: From exarkun at twistedmatrix.com Fri Nov 6 01:14:30 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Fri, 06 Nov 2009 00:14:30 -0000 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com> References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com> Message-ID: <20091106001430.3229.796306500.divmod.xquotient.180@localhost.localdomain> On 5 Nov, 11:55 pm, bobbyrward at gmail.com wrote: >>What exactly are those better ways? Document as deprecated only? >> >>-Brett > >A switch to ENABLE those warnings? > > >Lord knows I'm sick of filtering them out of logs. > >A switch to enable deprecation warnings would give developers a >chance to see them when migrating to a new version of python. And it >would prevent users from having to deal with them. PendingDeprecationWarning is silent by default and requires a switch to be enabled. Jean-Paul From debatem1 at gmail.com Fri Nov 6 01:30:05 2009 From: debatem1 at gmail.com (geremy condra) Date: Thu, 5 Nov 2009 19:30:05 -0500 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it In-Reply-To: <9CAD2251-C51D-472B-98A4-104E1AFEEDE1@fuhm.net> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <200911040946.15805.steve@pearwood.info> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <9CAD2251-C51D-472B-98A4-104E1AFEEDE1@fuhm.net> Message-ID: On Thu, Nov 5, 2009 at 6:17 PM, James Y Knight wrote: > On Nov 5, 2009, at 6:04 PM, geremy condra wrote: >> >> Perhaps my test is flawed in some way? > > Yes: you're testing the speed of something that makes absolutely no sense to > do in a tight loop, so *who the heck cares how fast any way of doing it is*! > > Is this thread over yet? > > James I'm testing the speed because the claim was made that the pop/add approach was inefficient. Here's the full quote: > The obvious way, for newcomers, of achieving the effect is: > > x = s.pop() > s.add(x) > > ... and that's simply horrible in terms of efficiency. So the > "obvious" way of doing it in Python is wrong(TM), and the "correct" > way of doing it is obscure and raises misleading exceptions. Since I use this in a graph theory library that I am currently trying to scale to handle 300 million node simulations, and this is used in several relevant algorithms, I was concerned. Geremy Condra From python at mrabarnett.plus.com Fri Nov 6 01:33:54 2009 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 06 Nov 2009 00:33:54 +0000 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: <4AF36EF2.3050706@mrabarnett.plus.com> Guido van Rossum wrote: > On Tue, Nov 3, 2009 at 9:35 AM, Guido van Rossum wrote: >> I've checked draft (!) PEP 3003, "Python Language Moratorium", into >> SVN. As authors I've listed Jesse, Brett and myself. > > I haven't seen substantial opposition against the PEP -- in fact I > can't recall any, and many people have explicitly posted in support of > it. So unless opposition suddenly appears in the next few days, I'll > move it to the Accepted state next Monday. > My view is that instead of there being new language features appearing in the next few years, what we have in Python 3 _are_ the new language features. We just need time to digest them before looking for more. :-) +1 From ziade.tarek at gmail.com Fri Nov 6 02:09:58 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 6 Nov 2009 02:09:58 +0100 Subject: [Python-Dev] Refactoring installation schemes In-Reply-To: <6d81ac30c331f8b9ca71a75d1049ffd4@preisshare.net> References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com> <94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com> <94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com> <9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com> <39deb0d1eef08419dad65d7354a8ffb1@preisshare.net> <94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com> <0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net> <6d81ac30c331f8b9ca71a75d1049ffd4@preisshare.net> Message-ID: <94bdd2610911051709y6050bca1w8e10c44d3598d145@mail.gmail.com> 2009/11/6 David Lyon : [..] > Here are the references to the discussions from distutils-sig. In these > references you'll see that my interaction with the list was not grumpy > but was entirely positive. I feel that I got a fair hearing on distutils > list and I had a number of supporters to my proposal. > > The proposal is in the final stages of being wrapped up. There are three different topics : 1. a standard for the installation format: PEP 376 2. changes in the metadata format : PEP 345 (-- work in progress here, some parts are still missing) 3. ways to built the metadata that are shipped into a distribution statically, using Distutils : PEP 390 was started for that And as a matter of fact, PEP 390 is not so important, because at the end, the changes that are made in PEP 345 are the most important ones. They allow describing metadata fields that can vary depending on the target platform. PEP 390 is just the simplest way to make it possible to describe static metadata using a setup.cfg ConfigParser section, rather than some code in setup.py. And when we built it, it helped us understand what we really wanted on PEP 345 side (with a big insights from people like Marc-Andr? on this) But PEP 390 limits its scope to the Metadata fields and now waits for PEP 345 changes. Your proposal is a partial alternative to PEP 390, and is a quite interesting work to dig in, and the SIG is discussing it, but a very important point about it is that it does a lot more. It's a new *building* system on its own, and does not limit itself to describing metadata fields. That's fine, and that's quite interesting, but I doubt we will be able to push it in Distutils itself any sooner, because it a whole different system ala scons (which is a great tool don't get me wrong). I think (as I told you before IIRC) that you need to create a project to demonstrate it and I think it can be great to continue getting some help from Distutils-SIG for this. Furthermore, building such a tool on the top of Distutils can improve Distutils itself, because it would be yet another project that would provide valuable feedback on the APIs and standards we provide. That's for example what we are getting from projects like PyPM or Enthought Enstaller. As a matter of fact, there's already great feedback from Active State on PEP 376 because they want to be early adopters of the upcoming standard and make sure we do the right choices in there. Enthought also started to use the prototype we have written for PEP 386 (version comparisons) IIRC. But in the existing PEPs discussions, we need to focus on their respective scopes. And they are all targeted to improve the existing stdlib tool : Distutils. (and a wanted side effect is to make Distutils smaller if we can, and a robust basis for all third party tools) That's why you can't just drop a counter-PEP to any existing PEP we have started for Distutils. Imho help us build those PEPs, and / or create your own build system, wether its based or not on Distutils. Regards Tarek From brett at python.org Fri Nov 6 02:25:20 2009 From: brett at python.org (Brett Cannon) Date: Thu, 5 Nov 2009 17:25:20 -0800 Subject: [Python-Dev] Refactoring installation schemes In-Reply-To: <6d81ac30c331f8b9ca71a75d1049ffd4@preisshare.net> References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com> <94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com> <94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com> <9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com> <39deb0d1eef08419dad65d7354a8ffb1@preisshare.net> <94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com> <0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net> <6d81ac30c331f8b9ca71a75d1049ffd4@preisshare.net> Message-ID: On Thu, Nov 5, 2009 at 15:55, David Lyon wrote: > > Tarek, Guido, > > Forgive my grumpy tone.. > > Looking positive, if we now have a rough consensus that the PEPS might > represent some 300+ lines of code... then good - lets get started, that's > all that I meant. I'm glad above all, that you noticed that part foremost. > > If it's a simple case that alternative's to the solutions cannot > be considered - because I am grumpy - whatever, then that's something > I can live with. I didn't know that proposing a different solution > wasn't a good idea. That is passive-aggressive and uncalled for. Proposing an opposing view is fine when done respectfully which you have not done here on python-dev. > >> PS. Submitting a counter-PEP to the peps editors that hasn't been >> discussed on the SIG list at all is also a bad move. You really need >> to change the way you try to interact with the SIG. > > Here are the references to the discussions from distutils-sig. In these > references you'll see that my interaction with the list was not grumpy > but was entirely positive. I feel that I got a fair hearing on distutils > list and I had a number of supporters to my proposal. > That's fine, but it's Tarek's call and as a group we reached the conclusion that Tarek is doing a fine job and is being more than fair. It has been stated as such and yet you keep pushing us like we have not heard you. We did hear you the first time, but you just seem to not get the message. Adding to the fact that your tone has been an issue and that we don't want to listen to it anymore has pushed Guido to the brink and me over the edge. I honestly don't know how Tarek keeps his patience over on the distutils SIG if he has to deal with kind of crap constantly. Personally, I am now sending your emails to the trash since you can't seem to even reply to Guido's email w/o some nasty undertone. Hopefully someday you can learn to communicate in a better, friendlier manner. -Brett From david.lyon at preisshare.net Fri Nov 6 03:51:09 2009 From: david.lyon at preisshare.net (David Lyon) Date: Thu, 05 Nov 2009 21:51:09 -0500 Subject: [Python-Dev] Refactoring installation schemes In-Reply-To: <94bdd2610911051709y6050bca1w8e10c44d3598d145@mail.gmail.com> References: <94bdd2610910271455s7854e6a9u94e3aeeeacb69c02@mail.gmail.com> <94bdd2610910281105q328bc3a5qaed020ef3afe28df@mail.gmail.com> <94bdd2610911040429j50a6df87m51aac2464dd13bd7@mail.gmail.com> <9cee7ab80911041440j43e51e39qc77a426547d02bcb@mail.gmail.com> <39deb0d1eef08419dad65d7354a8ffb1@preisshare.net> <94bdd2610911050235l4e851612i46e0392d352eaa20@mail.gmail.com> <0d223447fa9aebaeb78f5be82eb3aaab@preisshare.net> <6d81ac30c331f8b9ca71a75d1049ffd4@preisshare.net> <94bdd2610911051709y6050bca1w8e10c44d3598d145@mail.gmail.com> Message-ID: <4257d43e76eca63ae99c2638a0dc5e54@preisshare.net> Tarek, > Your proposal is a partial alternative to PEP 390, and is a quite > interesting work to dig in, and the SIG is discussing it, > but a very important point about it is that it does a lot more. It's a > new *building* system on its own, and does not limit itself to > describing metadata fields. Actually no, the proposal is only for a metadata installation of packages that addresses the security problem of requiring users to run untrusted and unverified setup.py's. That was where this discussion originated and that is what I am addressing. PEP-390 doesn't go there at all... My proposal uses the static metadata contained within existing package formats (PKG_INFO + sources.txt) and therefore doesn't require any changes to the build system. There isn't a new build system. I never proposed that. However PEP-390 does imply changing the build system. > That's why you can't just drop a counter-PEP to any existing PEP we > have started for Distutils. Imho help us build those PEPs, and / or > create your own build system, wether its based or not on Distutils. Once again, I can't see why all the fuss. All I did was have a discussion on distutils-ml about a different way of specifying package dependencies that might work across different python platforms. It was suggested to me that I do a PEP.. So I took the documentation on face value that developers could submit a PEP-proposal. > Imho help us build those PEPs, This could best apply to PEP-345: (PKG_INFO) Requires: sys Requires(python <= 2.4): lxml Requires(windows): win32com Requires(linux): pyodbc Requires(linux ubuntu gnome python <= 3.4): gix Requires(windows xp python < 2.2): win32prn Thank's for reading my alternate-metadata installation proposal and I accept your feedback. What I can do is clarify that my proposal is not for a build system but for a metadata installation scheme based on a static setup.py, existing metadata and use of "python -m setup install" for invocation. I'm just trying to make this stuff no more complicated than it needs to be... David From chris at subtlety.com Fri Nov 6 03:59:24 2009 From: chris at subtlety.com (Chris Bergstresser) Date: Thu, 5 Nov 2009 20:59:24 -0600 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it In-Reply-To: <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <4AF341BD.2010203@v.loewis.de> <7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> Message-ID: <7b0045ed0911051859y13f8abf3u7b5b4549aa2a41fc@mail.gmail.com> On Thu, Nov 5, 2009 at 5:02 PM, Raymond Hettinger wrote: > Forgot to post the code. ?It is short, fast, and easy. ?It is explicit about > handing the case with an empty input. ?And it is specific about which value > it returns (always the first iterated value; not an arbitrary one). ?There's > no guessing about what it does. ?It gets the job done. I'm trying to take this suggestion in the best possible light, which is that you honestly think I didn't read past Chapter 3 of the Python Tutorial, and I am therefore in fact unfamiliar with function definitions. -- Chris From chris at subtlety.com Fri Nov 6 04:12:45 2009 From: chris at subtlety.com (Chris Bergstresser) Date: Thu, 5 Nov 2009 21:12:45 -0600 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it In-Reply-To: References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <9CAD2251-C51D-472B-98A4-104E1AFEEDE1@fuhm.net> Message-ID: <7b0045ed0911051912x6cb1bd19r7677a10f47d36a45@mail.gmail.com> On Thu, Nov 5, 2009 at 6:30 PM, geremy condra wrote: > I'm testing the speed because the claim was made that the pop/add > approach was inefficient. Here's the full quote: > >> ? ?The obvious way, for newcomers, of achieving the effect is: >> >> ?x = s.pop() >> ?s.add(x) >> >> ... and that's simply horrible in terms of efficiency. ?So the >> "obvious" way of doing it in Python is wrong(TM), and the "correct" >> way of doing it is obscure and raises misleading exceptions. I was talking mainly from a theoretical standpoint, and because the library I'm working on is designed to work seamlessly over the network. In those cases, where the set the user is working with is actually a proxy object across the wire, the time to acquire the locks, remove the object, release the locks, reacquire the locks, add the object, then rerelease the locks is *significantly* more expensive than just noting the set hasn't changed and returning a cached object from it. -- Chris From guido at python.org Fri Nov 6 04:23:26 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 5 Nov 2009 19:23:26 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> Message-ID: On Thu, Nov 5, 2009 at 3:35 PM, Brett Cannon wrote: > On Thu, Nov 5, 2009 at 15:26, Guido van Rossum wrote: >> I have come to the conclusion that there are better ways to >> pre-announce that a module is going to disappear instead of >> deprecation warnings. > > What exactly are those better ways? Document as deprecated only? Sorry, I have an existence proof, but no construction. :-) Ideas welcome. Silent deprecations, loud documentation, biweekly home visits, whatever, as long as it doesn't log a message by default. The thing is, in practice, people will be testing with a new release anyway, and that's the earliest time they are likely to take action (other than silencing the warnings). -- --Guido van Rossum (python.org/~guido) From rdmurray at bitdance.com Fri Nov 6 04:53:27 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 5 Nov 2009 22:53:27 -0500 (EST) Subject: [Python-Dev] Status of the Buildbot fleet and related bugs Message-ID: The buildbot waterfall is much greener now. Thanks to all who have contributed to making it so (and it hasn't just been Mark and Antoine and I, though we've been the most directly active (and yes, Mark, you did contribute several fixes!)). The 'stable builders' fleet is green now except for: (1) issue 7269: occasional 2.6/trunk bsddb3 failures on windows (2) issue 6748: random 3.1/3.x failures on most buidbots. (3) the klose-debian-ppc builder being offline Of these, (2) is by _far_ the biggest issue, and the one that causes the most flap (success-failure-success). And flap is the thing that most harms the buildbot usefulness. Anyone who wants to debug this on a platform where it is consistently reproducible please email me your public key and I'll give you a shell account on my buildbot (Antoine already has one). In the 'unstable' builder fleet, Antoine's new builder seems to be stable across the board, while mine fails consistently on 3.1 and 3.x because of the test_telnetlib bug. Thomas Heller's OS X buildbot is consistently having lots of test failures (the same ones each time I've checked). The master claims the klose-debian-alpha buildbot doesn't know about branches, which is odd since it was working not too long ago. The remaining buildslaves appear to have been offline for some time. Open issues here are: (1) issue 3864: FreeBSD testing hangs consistently. According to the ticket this is a FreeBSD bug fixed in 6.4, so an OS upgrade on the buildslave would probably solve it. (2) issue 4970: consistent signal 32 error on the norwitz-x86 Gentoo buildslave in 3.1 and 3.x. This may be due to the box running an old threading library, but it does make one wonder what changed in 3.x that exposed it. Another issue that I've seen on the buildbots but that doesn't seem to be showing up right now (is it fixed?) is issue 7251, which Mark is working on. So, overall I think the buildbot fleet is in good shape, and if we can nail issue 6748 I think it will be back to being an important resource for sanity checking our checkins. By the way, Georg set up the IRC interface on the #python-dev channel, so you can hang out there if you want to get realtime reports of which buildbots have going from success to failure and vice versa. --David From mbk.lists at gmail.com Fri Nov 6 05:12:25 2009 From: mbk.lists at gmail.com (Mike Krell) Date: Thu, 5 Nov 2009 21:12:25 -0700 Subject: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line? In-Reply-To: <4AF330A7.5000809@v.loewis.de> References: <4AEFD9AE.7090104@molden.no> <4AEFF450.9060007@voidspace.org.uk> <79EADA413EFB44F6864EF71D5B830088@RaymondLaptop1> <79990c6b0911030323x47a9f008ue7256c56e88c0ee8@mail.gmail.com> <5CB59A21-D8C9-4BC1-A1F6-4438B86A8BC5@twistedmatrix.com> <4AF1CFAD.6000305@v.loewis.de> <4AF2C032.4040605@gmail.com> <4AF330A7.5000809@v.loewis.de> Message-ID: On Thu, Nov 5, 2009 at 1:08 PM, "Martin v. L?wis" wrote: > > Mike Krell wrote: > >> Well, 3to2 would then be an option for you: use Python 3 as the > source > >> language. > > > > Making life easier for 3to2 is an *excellent* rationale for backports. > Clarifying a bit of potentially misleading editing: I wrote neither of the statements quoted above. M v L wrote the first and Nick C. wrote the second. > I'm skeptical. If new features get added to 2.7: why would that simplify > 3to2? It couldn't *use* these features, since it surely would have to > support 2.6 and earlier as well. > > Not sure what 3to2 would do about difficult-to-convert 3.x feature (as, > perhaps, the nonlocal keyword). If it currently gives up, it then may > offer you to restrict your target versions to 2.7+. Not sure whether > users would use that option, though - perhaps they rather stop using > nonlocal in 3.x if 3to2 cannot support it for all 2.x versions they are > interested in. > > Perhaps 3to2 has a work-around that still provides a good backport in > most cases. Then, the backport would not make the tool any simpler: if > 3to2 would start using the backport, it would actually get more > complicated (not easier), as it now needs to support two cases. > I basically agree with you here, except perhaps for the likely definition of "versions they are interested in". You have suggested on several occasions that a 2.7 (or 2.8) containing new syntax such as nonlocal would be of limited value because the vast majority of developers interested in supporting 2.x would have to support 2.6 as well. I respectfully suggest that may not necessarily be the case. I suspect that there are lots of small fish out there like me who have the luxury of being able to hop onto whatever version of the language suits them the best. Not to mention all of the new users who will be drawn to Python over the next several years while the 3.x standard library and third party library situation becomes more stable and comprehensive. Why not make the 2.x feature set the best it can be given the likelihood that 2.x will be the most compelling alternative to many users until the 3.x libraries mature? Of course, it's easy for me to ask other people to the hard work. It might be fun to take a crack at implementing nonlocal myself, but I know next to nothing about the implementation of CPython. By the time I pestered y'all with enough questions to get up to speed, you'd probably wish you'd just implemented it yourself -- less work :-) Mike -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.arbash.meinel at gmail.com Fri Nov 6 05:21:05 2009 From: john.arbash.meinel at gmail.com (John Arbash Meinel) Date: Thu, 05 Nov 2009 22:21:05 -0600 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it In-Reply-To: References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <200911040946.15805.steve@pearwood.info> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> Message-ID: <4AF3A431.2020607@gmail.com> geremy condra wrote: > On Thu, Nov 5, 2009 at 4:09 PM, Alexander Belopolsky > wrote: >> On Thu, Nov 5, 2009 at 3:43 PM, Chris Bergstresser wrote: >>> .. and "x = iter(s).next()" raises a StopIteration >>> exception. >> And that's why the documented recipe should probably recommend >> next(iter(s), default) instead. Especially because iter(s).next() is >> not even valid code in 3.0. > > This seems reasonably legible to you? Strikes me as coding by > incantation. Also, while I've heard people say that the naive > approach is slower, I'm not getting that result. Here's my test: > > >>>> smrt = timeit.Timer("next(iter(s))", "s=set(range(100))") >>>> smrt.repeat(10) > [1.2845709323883057, 0.60247397422790527, 0.59621405601501465, > 0.59133195877075195, 0.58387589454650879, 0.56839084625244141, > 0.56839680671691895, 0.56877803802490234, 0.56905913352966309, > 0.56846404075622559] > >>>> naive = timeit.Timer("x=s.pop();s.add(x)", "s=set(range(100))") >>>> naive.repeat(10) > [0.93139314651489258, 0.53566789627075195, 0.53674602508544922, > 0.53608798980712891, 0.53634309768676758, 0.53557991981506348, > 0.53578495979309082, 0.53666114807128906, 0.53576493263244629, > 0.53491711616516113] > > > Perhaps my test is flawed in some way? > > Geremy Condra Well, it also uses a fairly trivial set. 'set(range(100))' is going to generally have 0 collisions and everything will hash to a unique bucket. As such, pop ing an item out of the hash is a single "val = table[int & mask]; table[int & mask] = _dummy", and then looking it up again requires 2 table lookups (one finds the _dummy, the next finds that the chain is broken and can rewrite the _dummy.) However, if a set is more full, or has more collisions, or ... then pop() and add() become relatively more expensive. Surprising to me, is that "next(iter(s))" was actually slower than .pop() + .add() for 100 node set in my testing: $ alias TIMEIT="python -m timeit -s 's = set(range(100)'" $ TIMEIT "x = next(iter(s))" 1000000 loops, best of 3: 0.263 usec per loop $ TIMEIT "x = s.pop(); s.add(x)" 1000000 loops, best of 3: 0.217 usec per loop though both are much slower than the fastest we've found: $ TIMEIT "for x in s: break" 10000000 loops, best of 3: 0.0943 usec per loop now, what about a set with *lots* of collisions. Create 100 keys that all hash to the same bucket: aliase TIMEIT="python -m timeit -s 's = set([x*1024*1024 for x in range(100)])'" $ TIMEIT "x = next(iter(s))" 1000000 loops, best of 3: 0.257 usec per loop $ TIMEIT "x = s.pop(); s.add(x)" 1000000 loops, best of 3: 0.218 usec per loop I tried a few different ways, and I got the same results, until I did: $ python -m timeit -s "s = set(range(100000, 1000100))" "next(iter(s))" 1000 loops, best of 3: 255 usec per loop ^^^^ Now something seems terribly wrong here. next(iter(s)) suddenly jumps up from being < 0.3 us, to being more than 200us. Or ~1000x slower. I realize the above has 900k keys, which is big. But 'next(iter(s))' should only touch 1, and, in fact, should always return the *first* entry. My best guess is just that the *first* entry in the internal set table is no longer close to offset 0, which means that 'next(iter(s))' has to evaluate a bunch of table slots before it finds a non-empty entry. Anyway, none of the proposals will really ever be faster than: for x in s: break It is a bit ugly of a construct, but you don't have an attribute lookup, etc. As long as you don't do: for x in s: pass Then it stays nice and fast. John =:-> From debatem1 at gmail.com Fri Nov 6 06:04:30 2009 From: debatem1 at gmail.com (geremy condra) Date: Fri, 6 Nov 2009 00:04:30 -0500 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it In-Reply-To: <4AF3A431.2020607@gmail.com> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <200911040946.15805.steve@pearwood.info> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <4AF3A431.2020607@gmail.com> Message-ID: On Thu, Nov 5, 2009 at 11:21 PM, John Arbash Meinel wrote: > geremy condra wrote: >> On Thu, Nov 5, 2009 at 4:09 PM, Alexander Belopolsky >> wrote: >>> On Thu, Nov 5, 2009 at 3:43 PM, Chris Bergstresser wrote: >>>> .. and "x = iter(s).next()" raises a StopIteration >>>> exception. >>> And that's why the documented recipe should probably recommend >>> next(iter(s), default) instead. ?Especially because iter(s).next() is >>> not even valid code in 3.0. >> >> This seems reasonably legible to you? Strikes me as coding by >> incantation. Also, while I've heard people say that the naive >> approach is slower, I'm not getting that result. Here's my test: >> >> >>>>> smrt = timeit.Timer("next(iter(s))", "s=set(range(100))") >>>>> smrt.repeat(10) >> [1.2845709323883057, 0.60247397422790527, 0.59621405601501465, >> 0.59133195877075195, 0.58387589454650879, 0.56839084625244141, >> 0.56839680671691895, 0.56877803802490234, 0.56905913352966309, >> 0.56846404075622559] >> >>>>> naive = timeit.Timer("x=s.pop();s.add(x)", "s=set(range(100))") >>>>> naive.repeat(10) >> [0.93139314651489258, 0.53566789627075195, 0.53674602508544922, >> 0.53608798980712891, 0.53634309768676758, 0.53557991981506348, >> 0.53578495979309082, 0.53666114807128906, 0.53576493263244629, >> 0.53491711616516113] >> >> >> Perhaps my test is flawed in some way? >> >> Geremy Condra > > Well, it also uses a fairly trivial set. 'set(range(100))' is going to > generally have 0 collisions and everything will hash to a unique bucket. > ?As such, pop ing an item out of the hash is a single "val = table[int & > mask]; table[int & mask] = _dummy", and then looking it up again > requires 2 table lookups (one finds the _dummy, the next finds that the > chain is broken and can rewrite the _dummy.) > > However, if a set is more full, or has more collisions, or ... then > pop() and add() become relatively more expensive. > > Surprising to me, is that "next(iter(s))" was actually slower than > .pop() + .add() for 100 node set in my testing: > > $ alias TIMEIT="python -m timeit -s 's = set(range(100)'" > $ TIMEIT "x = next(iter(s))" > 1000000 loops, best of 3: 0.263 usec per loop > > $ TIMEIT "x = s.pop(); s.add(x)" > 1000000 loops, best of 3: 0.217 usec per loop > > though both are much slower than the fastest we've found: > $ TIMEIT "for x in s: break" > 10000000 loops, best of 3: 0.0943 usec per loop > > > now, what about a set with *lots* of collisions. Create 100 keys that > all hash to the same bucket: > aliase TIMEIT="python -m timeit -s 's = set([x*1024*1024 for x in > range(100)])'" > $ TIMEIT "x = next(iter(s))" > 1000000 loops, best of 3: 0.257 usec per loop > > $ TIMEIT "x = s.pop(); s.add(x)" > 1000000 loops, best of 3: 0.218 usec per loop > > I tried a few different ways, and I got the same results, until I did: > > $ python -m timeit -s "s = set(range(100000, 1000100))" "next(iter(s))" > 1000 loops, best of 3: 255 usec per loop > > ^^^^ Now something seems terribly wrong here. next(iter(s)) suddenly > jumps up from being < 0.3 us, to being more than 200us. Or ~1000x slower. > > I realize the above has 900k keys, which is big. But 'next(iter(s))' > should only touch 1, and, in fact, should always return the *first* > entry. My best guess is just that the *first* entry in the internal set > table is no longer close to offset 0, which means that 'next(iter(s))' > has to evaluate a bunch of table slots before it finds a non-empty entry. > > > Anyway, none of the proposals will really ever be faster than: > ?for x in s: break > > It is a bit ugly of a construct, but you don't have an attribute lookup, > etc. As long as you don't do: > ?for x in s: pass > > Then it stays nice and fast. > > John > =:-> Thanks for the info. Doing a bit more digging it appears that taking the lookup out of the picture puts the pop/add more or less on par with the for x in s: break solution, with the former behaving more predictably and the latter averaging marginally faster times. Either way, the loss in readability isn't worth it to me to get the minor improvement in performance. Given that adding a set.pick method would incur half the lookup cost that pop/add does, I think its reasonable to say that even using the fastest method proposed to implement it won't differ all that greatly from the least efficient proposal, and that its therefore pointless to consider set.pick from an optimisation standpoint. Aesthetics, of course, are another thing altogether. Geremy Condra From steve at pearwood.info Fri Nov 6 06:18:38 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 6 Nov 2009 16:18:38 +1100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <4AF29753.3040801@voidspace.org.uk> References: <4AF29753.3040801@voidspace.org.uk> Message-ID: <200911061618.38790.steve@pearwood.info> On Thu, 5 Nov 2009 08:13:55 pm Michael Foord wrote: > There are several partial implementations, including Python inspired > languages, but if we are looking at 'major complete implementations' > then the current list seems to be: CPython, Jython, IronPython and > PyPy. Even Unladen Swallow is a fork (sorry - I mean branch) of > CPython rather than a separate implementation. I don't know how mature or active it is, so it may not count as either major or complete, but there's also CLPython: http://common-lisp.net/project/clpython/ -- Steven D'Aprano From martin at v.loewis.de Fri Nov 6 06:43:06 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 06 Nov 2009 06:43:06 +0100 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it In-Reply-To: <7b0045ed0911051859y13f8abf3u7b5b4549aa2a41fc@mail.gmail.com> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <4AF341BD.2010203@v.loewis.de> <7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> <7b0045ed0911051859y13f8abf3u7b5b4549aa2a41fc@mail.gmail.com> Message-ID: <4AF3B76A.1050006@v.loewis.de> > On Thu, Nov 5, 2009 at 5:02 PM, Raymond Hettinger wrote: >> Forgot to post the code. It is short, fast, and easy. It is explicit about >> handing the case with an empty input. And it is specific about which value >> it returns (always the first iterated value; not an arbitrary one). There's >> no guessing about what it does. It gets the job done. > > I'm trying to take this suggestion in the best possible light, > which is that you honestly think I didn't read past Chapter 3 of the > Python Tutorial, and I am therefore in fact unfamiliar with function > definitions. I read Raymond's suggestion rather as a question: why bother with a tedious, multi-year process, when a three-line function will achieve exactly the same? Regards, Martin From brett at python.org Fri Nov 6 06:55:51 2009 From: brett at python.org (Brett Cannon) Date: Thu, 5 Nov 2009 21:55:51 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> Message-ID: On Thu, Nov 5, 2009 at 19:23, Guido van Rossum wrote: > On Thu, Nov 5, 2009 at 3:35 PM, Brett Cannon wrote: >> On Thu, Nov 5, 2009 at 15:26, Guido van Rossum wrote: >>> I have come to the conclusion that there are better ways to >>> pre-announce that a module is going to disappear instead of >>> deprecation warnings. >> >> What exactly are those better ways? Document as deprecated only? > > Sorry, I have an existence proof, but no construction. :-) > > Ideas welcome. Silent deprecations, loud documentation, biweekly home > visits, whatever, as long as it doesn't log a message by default. Well, one option is to come up with the equivalent of -3, but for all warnings; the antithesis of -W. And obviously glaring deprecation warnings in the docs (removal has been discussed but always shot down as someone who comes across old code might still need docs for it). The clarification I need is will this in any way influence when modules are removed. If they stay in for the life of a major version then I want it made clear that bug fixes for the code take lower priority over all other code in the standard library. -Brett From steve at pearwood.info Fri Nov 6 07:01:01 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 6 Nov 2009 17:01:01 +1100 Subject: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it In-Reply-To: <9d153b7c0911051552v7cfabf61ifede37a72ec239d8@mail.gmail.com> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <9CAD2251-C51D-472B-98A4-104E1AFEEDE1@fuhm.net> <9d153b7c0911051552v7cfabf61ifede37a72ec239d8@mail.gmail.com> Message-ID: <200911061701.02292.steve@pearwood.info> On Fri, 6 Nov 2009 10:52:54 am Yuvgoog Greenle wrote: > On Fri, Nov 6, 2009 at 1:17 AM, James Y Knight wrote: > > Is this thread over yet? > > Sorry, I just had to point out that pop/add has a side effect that > would be apparent on a set that multiple threads access - it loses an > item and then gets it back. Sounds like a sleeper race condition > that's going to be rare but extremely hard to find if it does occur. > Crooked as a gil. Surely Raymond's suggestion also suffers from a similar race condition? for x in set: return x creates a set_iterator. If another thread modifies the original set after the set_iterator is created but before the return, you would get a mysterious and almost impossible to debug RuntimeError. -- Steven D'Aprano From chris at subtlety.com Fri Nov 6 07:19:27 2009 From: chris at subtlety.com (Chris Bergstresser) Date: Fri, 6 Nov 2009 00:19:27 -0600 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it In-Reply-To: <4AF3B76A.1050006@v.loewis.de> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <4AF341BD.2010203@v.loewis.de> <7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> <7b0045ed0911051859y13f8abf3u7b5b4549aa2a41fc@mail.gmail.com> <4AF3B76A.1050006@v.loewis.de> Message-ID: <7b0045ed0911052219u1893bd62gebf71bf46721014a@mail.gmail.com> On Thu, Nov 5, 2009 at 11:43 PM, "Martin v. L?wis" wrote: > I read Raymond's suggestion rather as a question: why bother with a > tedious, multi-year process, when a three-line function will achieve > exactly the same? Because it doesn't achieve exactly the same. What I want is a sane, rational way to describe what I'm doing in the core API, so other programmers learning the language don't spend the amount of time I did perplexed that there was a .pop() and a .remove() and a .discard(), but there wasn't a .pick(). I don't want to have to write the same little helper function in every project to fill a deficiency in the library. I don't want to have to argue about the flaws in solutions with race conditions, or the fact that cheap functions become expensive functions when performed over the network, or that there's a real value in having an atomic operation which throws a sane exception when it fails, or how it's disturbing to my OCD core to have an API which believes: if x in s: s.remove(x) ... is too confusing, so there should be a .discard() method, but ... for x in s: break ... is self-evident and obvious, so there's no need for a .pick(). -- Chris From dickinsm at gmail.com Fri Nov 6 09:27:48 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 6 Nov 2009 08:27:48 +0000 Subject: [Python-Dev] Status of the Buildbot fleet and related bugs In-Reply-To: References: Message-ID: <5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com> On Fri, Nov 6, 2009 at 3:53 AM, R. David Murray wrote: > ? ?(1) issue 3864: FreeBSD testing hangs consistently. ?According to the > ? ? ? ?ticket this is a FreeBSD bug fixed in 6.4, so an OS upgrade > ? ? ? ?on the buildslave would probably solve it. I think the particular issue mentioned in 3864 is fixed, in some sense: test_signal used to hang, but now just plain fails in a reasonable amount of time (~15 seconds) instead. So at least the test_signal failure isn't preventing us from seeing the results of other tests. The big problem now on the FreeBSD buildbot is that test_multiprocessing reliably causes the whole test run to abort with 'Signal 12'. Solving this may be as simple as just getting someone to install a copy of FreeBSD 6.2 on an ssh-accessible machine so that the source of the error can be tracked down. > ? ?(2) issue 4970: consistent signal 32 error on the norwitz-x86 Gentoo > ? ? ? ?buildslave in 3.1 and 3.x. ?This may be due to the box > ? ? ? ?running an old threading library, but it does make one wonder what > ? ? ? ?changed in 3.x that exposed it. This error has been happening since well before 3.0 was released. Asking for access to Neal's machine is probably the only sensible way to diagnose it. (A less invasive but slower way to debug would be to create a branch especially for this bug and do repeated runs to figure out which part of test_os is causing the failure.) > Another issue that I've seen on the buildbots but that doesn't > seem to be showing up right now (is it fixed?) is issue 7251, which > Mark is working on. It's not fixed, but I hope to have time to fix it this weekend. It's just not showing up on some runs because test_multiprocessing kills the buildslave first. :-) > So, overall I think the buildbot fleet is in good shape, and if > we can nail issue 6748 I think it will be back to being an > important resource for sanity checking our checkins. Wholeheartedly agreed! Sorting out the FreeBSD test_multiprocessing Signal 12 would be nice, too. I'll open an issue for this, unless someone else gets there first. Mark From theller at ctypes.org Fri Nov 6 10:02:24 2009 From: theller at ctypes.org (Thomas Heller) Date: Fri, 06 Nov 2009 10:02:24 +0100 Subject: [Python-Dev] Status of the Buildbot fleet and related bugs In-Reply-To: References: Message-ID: R. David Murray schrieb: > The buildbot waterfall is much greener now. Thanks to all who have > contributed to making it so (and it hasn't just been Mark and Antoine > and I, though we've been the most directly active (and yes, Mark, you > did contribute several fixes!)). [...] > In the 'unstable' builder fleet, Antoine's new builder seems to be > stable across the board, while mine fails consistently on 3.1 and 3.x > because of the test_telnetlib bug. Thomas Heller's OS X buildbot is > consistently having lots of test failures (the same ones each time > I've checked). My buildbot is behind our companys firewall. Well, I was able to fix the test_smtpnet test by additionally opening port 465 in the firewall; however I'm not really sure I should do that. I had to open another port already although that one is probably less critical. For the other test failures, I have no idea where they come from. -- Thanks, Thomas From ncoghlan at gmail.com Fri Nov 6 10:35:46 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 06 Nov 2009 19:35:46 +1000 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> Message-ID: <4AF3EDF2.2010406@gmail.com> Brett Cannon wrote: > On Thu, Nov 5, 2009 at 19:23, Guido van Rossum wrote: >> On Thu, Nov 5, 2009 at 3:35 PM, Brett Cannon wrote: >>> On Thu, Nov 5, 2009 at 15:26, Guido van Rossum wrote: >>>> I have come to the conclusion that there are better ways to >>>> pre-announce that a module is going to disappear instead of >>>> deprecation warnings. >>> What exactly are those better ways? Document as deprecated only? >> Sorry, I have an existence proof, but no construction. :-) >> >> Ideas welcome. Silent deprecations, loud documentation, biweekly home >> visits, whatever, as long as it doesn't log a message by default. > > Well, one option is to come up with the equivalent of -3, but for all > warnings; the antithesis of -W. And obviously glaring deprecation > warnings in the docs (removal has been discussed but always shot down > as someone who comes across old code might still need docs for it). For a PEP 3003-specific solution, we could adopt a policy that we will add only Pending Deprecation warnings during the moratorium period. The actually deprecation clock would then only start in the first release after the moratorium ends (i.e. full Deprecation Warning in that release, removal in the following release). No doubt some third party developers would end up being shocked by a raft of warnings at that point, but those that are paying more attention would get a chance to clean up the warnings without alarming their users. Longer term, a solution may be to extend the standard deprecation period one release and make pending deprecation warnings required rather than optional. That way, on the ball developers would have a full release to quash deprecation warnings before their users encountered them by default. That is: Release X.Y: deprecated in docs, pending deprecation in code Release X.Y+1: deprecated in code Release X.Y+2: removed in code and docs (Or we could just make that the policy now and not do anything specific in relation to the moratorium and the standard library) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From metawilm at gmail.com Fri Nov 6 10:46:00 2009 From: metawilm at gmail.com (Willem Broekema) Date: Fri, 6 Nov 2009 10:46:00 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <200911061618.38790.steve@pearwood.info> References: <4AF29753.3040801@voidspace.org.uk> <200911061618.38790.steve@pearwood.info> Message-ID: On Fri, Nov 6, 2009 at 6:18 AM, Steven D'Aprano wrote: > I don't know how mature or active it is, so it may not count as either > major or complete, but there's also CLPython: > > http://common-lisp.net/project/clpython/ CLPython is in steady development, quite complete and stable on the language level (somewhere between 2.5 and 2.6), but missing most built-in library functionality. (It reuses the pure-Python parts of the stdlib.) As its developer, I don't care much about being mentioned in discussions or presentations, as CLPython is not as widely used as IronPython and Jython. But in a PEP like this, which is directly about alternative implementations, it deserves to be mentioned, IMHO. - Willem From dirkjan at ochtman.nl Fri Nov 6 10:47:14 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Fri, 6 Nov 2009 10:47:14 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <4AF3EDF2.2010406@gmail.com> References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <4AF3EDF2.2010406@gmail.com> Message-ID: On Fri, Nov 6, 2009 at 10:35, Nick Coghlan wrote: > Longer term, a solution may be to extend the standard deprecation period > one release and make pending deprecation warnings required rather than > optional. That way, on the ball developers would have a full release to > quash deprecation warnings before their users encountered them by default. > > That is: > > ?Release X.Y: deprecated in docs, pending deprecation in code > ?Release X.Y+1: deprecated in code > ?Release X.Y+2: removed in code and docs > > (Or we could just make that the policy now and not do anything specific > in relation to the moratorium and the standard library) This sounds like an improvement for things like Mercurial, at least. We did support 2.3-2.6 until relatively recently, and I think that's hard to get around for software that actually has to work on the user's box. This is a bit different from webapps, I suspect, where you "only" have to support the servers, which you often have more control over. But supporting 4 releases in a row has been a bit of a PITA. Luckily, we finally felt it was time to drop 2.3, so now we can make use of luxuries such as subprocess... Still, supporting 3 releases seems relatively common in the Python world (after all, parts of Zope still require 2.4, I think), and so it would be nice if the stdlib moved a little bit slower. Cheers, Dirkjan From ncoghlan at gmail.com Fri Nov 6 11:20:24 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 06 Nov 2009 20:20:24 +1000 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <4AF3EDF2.2010406@gmail.com> Message-ID: <4AF3F868.5040403@gmail.com> Dirkjan Ochtman wrote: > On Fri, Nov 6, 2009 at 10:35, Nick Coghlan wrote: >> Longer term, a solution may be to extend the standard deprecation period >> one release and make pending deprecation warnings required rather than >> optional. That way, on the ball developers would have a full release to >> quash deprecation warnings before their users encountered them by default. >> >> That is: >> >> Release X.Y: deprecated in docs, pending deprecation in code >> Release X.Y+1: deprecated in code >> Release X.Y+2: removed in code and docs >> >> (Or we could just make that the policy now and not do anything specific >> in relation to the moratorium and the standard library) > > This sounds like an improvement for things like Mercurial, at least. > We did support 2.3-2.6 until relatively recently, and I think that's > hard to get around for software that actually has to work on the > user's box. This is a bit different from webapps, I suspect, where you > "only" have to support the servers, which you often have more control > over. > > But supporting 4 releases in a row has been a bit of a PITA. Luckily, > we finally felt it was time to drop 2.3, so now we can make use of > luxuries such as subprocess... Still, supporting 3 releases seems > relatively common in the Python world (after all, parts of Zope still > require 2.4, I think), and so it would be nice if the stdlib moved a > little bit slower. I would personally be open to the open to the idea of requiring 2 releases worth of Pending Deprecation warnings. Then you would have the following situation for warnings-free operation on up to 3 versions of Python: Release X.Y-2: old approach only Release X.Y-1: no change Release X.Y: new approach added, old approach deprecated in docs, pending deprecation warning in code Release X.Y+1: no change Release X.Y+2: deprecation warning in code Release X.Y+3: old approach removed from docs and code Libraries and applications supporting up to 3 Python versions simultaneously would then have a clear path to follow: use the old mechanism up to Release X.Y+1, then switch to the new mechanism in Release X.Y+2 We would be looking at around 5 years to completely kill off a feature at that point, which actually aligns fairly well with the time period for which we provide source-only security patches for really old branches. Making such a system practical would probably require some additional infrastructure in the warnings module to handle the details though. Specifically, rather than having to update the code to raise the correct kind of warning each release, it would be better to instead be able to write something like "warnings.warn_deprecated(msg, (3, 4))" and have it generate PendingDeprecationWarning messages in 3.2 and 3.3, and then DeprecationWarning messages in 3.4 and later. In code, something like: def warn_deprecated(msg, version, ref=sys.version, stacklevel=2): deprecated = version >= ref if deprecated: category = DeprecationWarning else: category = PendingDeprecationWarning warn(msg, category, stacklevel) return deprecated Would people be interested in such a helper method (along with a corresponding C API, naturally)? With the reference version passed in as an optional argument, it would even be applicable to more than just the Python core and standard library. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From stefan_ml at behnel.de Fri Nov 6 11:42:43 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 06 Nov 2009 11:42:43 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: Raymond Hettinger, 05.11.2009 23:11: > [GvR] >> I haven't seen substantial opposition against the PEP -- in fact I >> can't recall any, and many people have explicitly posted in support of >> it. So unless opposition suddenly appears in the next few days, I'll >> move it to the Accepted state next Monday. > > But it would have been so much fun to have a never ending python-ideas > thread on BEGIN/END blocks ;-) According to the PEP, you can still have that. It just won't lead to an implementation being merged (or at least becoming part of a release) within the next two years. Stefan From db3l.net at gmail.com Fri Nov 6 11:54:51 2009 From: db3l.net at gmail.com (David Bolen) Date: Fri, 06 Nov 2009 05:54:51 -0500 Subject: [Python-Dev] Status of the Buildbot fleet and related bugs References: <5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com> Message-ID: Mark Dickinson writes: > On Fri, Nov 6, 2009 at 3:53 AM, R. David Murray wrote: > >> ? ?(1) issue 3864: FreeBSD testing hangs consistently. ?According to the >> ? ? ? ?ticket this is a FreeBSD bug fixed in 6.4, so an OS upgrade >> ? ? ? ?on the buildslave would probably solve it. > > I think the particular issue mentioned in 3864 is fixed, in some sense: > test_signal used to hang, but now just plain fails in a reasonable amount of > time (~15 seconds) instead. So at least the test_signal failure isn't > preventing us from seeing the results of other tests. > > The big problem now on the FreeBSD buildbot is that test_multiprocessing > reliably causes the whole test run to abort with 'Signal 12'. Solving this > may be as simple as just getting someone to install a copy of FreeBSD 6.2 > on an ssh-accessible machine so that the source of the error can be tracked > down. I could arrange ssh access to the build slave if that would help anyone who wants to look into that. Just contact me directly. In terms of the overall release, I'm also fine with upgrading the build slave to 6.4. Or I could jump up to 7.2 instead. When I first brought the build slave up, 7.x wasn't finalized yet - not sure now which release is more prevalent in use. -- David From hfuerstenau at gmx.net Fri Nov 6 08:25:29 2009 From: hfuerstenau at gmx.net (=?ISO-8859-1?Q?Hagen_F=FCrstenau?=) Date: Fri, 06 Nov 2009 08:25:29 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <20091106001430.3229.796306500.divmod.xquotient.180@localhost.localdomain> References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com> <20091106001430.3229.796306500.divmod.xquotient.180@localhost.localdomain> Message-ID: <4AF3CF69.6090709@gmx.net> >> A switch to enable deprecation warnings would give developers a >> chance to see them when migrating to a new version of python. And it >> would prevent users from having to deal with them. > > PendingDeprecationWarning is silent by default and requires a switch to > be enabled. So what we need is perhaps not a stdlib moratorium, but rather strict adherence to a sequence of PendingDeprecationWarning -> DeprecationWarning -> Removal? (The first one seems to have been underused so far.) That way nothing could get removed without having had at least two warnings in subsequent minor versions, one silent and one loud. People who want to avoid loud warnings can check for silent ones before upgrading, and everyone is guaranteed to have seen a warning before final removal. - Hagen From exarkun at twistedmatrix.com Fri Nov 6 15:04:12 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Fri, 06 Nov 2009 14:04:12 -0000 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <20091106135058.GA13578@vidar.dreamhost.com> References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com> <20091106001430.3229.796306500.divmod.xquotient.180@localhost.localdomain> <4AF3CF69.6090709@gmx.net> <20091106135058.GA13578@vidar.dreamhost.com> Message-ID: <20091106140412.3229.80002680.divmod.xquotient.193@localhost.localdomain> On 01:50 pm, dripton at ripton.net wrote: >On 2009.11.06 08:25:29 +0100, Hagen F?rstenau wrote: >> >> A switch to enable deprecation warnings would give developers a >> >> chance to see them when migrating to a new version of python. And >>it >> >> would prevent users from having to deal with them. >> > >> > PendingDeprecationWarning is silent by default and requires a switch >>to >> > be enabled. >> >>So what we need is perhaps not a stdlib moratorium, but rather strict >>adherence to a sequence of PendingDeprecationWarning -> >>DeprecationWarning -> Removal? (The first one seems to have been >>underused so far.) > >+1 > >I really don't like the loud warnings for sha and md5 in 2.6, >considering that hashlib wasn't even added until 2.5. It should be >possible to write code that works across all the versions of Python 2 >that are currently shipping with mainstream Linux distrobutions without >deprecation warnings or conditional version-dependent code. > >To go a step further, maybe follow javac's example and emit just one >summary deprecation warning per program by default. Something >like "this program uses deprecated API calls. Run it with >python -deprecation to see the details." End users don't even need to be told this much, though (and if it's javac that does this for Java, then indeed Java end users probably aren't seeing this either). I think it would be great for deprecation warnings to be completely silent for a while *and* for the policy to be prominently documented so that developers, the people who really need this information, know how to get it. Jean-Paul From jnoller at gmail.com Fri Nov 6 15:05:48 2009 From: jnoller at gmail.com (Jesse Noller) Date: Fri, 6 Nov 2009 09:05:48 -0500 Subject: [Python-Dev] Status of the Buildbot fleet and related bugs In-Reply-To: <5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com> References: <5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com> Message-ID: <4222a8490911060605k17bcd575if927dc6317dabfb0@mail.gmail.com> On Fri, Nov 6, 2009 at 3:27 AM, Mark Dickinson wrote: > On Fri, Nov 6, 2009 at 3:53 AM, R. David Murray wrote: > >> ? ?(1) issue 3864: FreeBSD testing hangs consistently. ?According to the >> ? ? ? ?ticket this is a FreeBSD bug fixed in 6.4, so an OS upgrade >> ? ? ? ?on the buildslave would probably solve it. > > I think the particular issue mentioned in 3864 ?is fixed, in some sense: > test_signal used to hang, but now just plain fails in a reasonable amount of > time (~15 seconds) instead. ?So at least the test_signal failure isn't > preventing us from seeing the results of other tests. > > The big problem now on the FreeBSD buildbot is that test_multiprocessing > reliably causes the whole test run to abort with 'Signal 12'. ?Solving this > may be as simple as just getting someone to install a copy of FreeBSD 6.2 > on an ssh-accessible machine so that the source of the error can be tracked > down. Sorry I haven't been watching the bots; the signal 12 is new, I don't have direct access to a fbsd box to poke at it. I'm fairly alarmed that it's triggering that. Let me know what I can do to help - fbsd support of certain lower level shared semaphore stuff has been spotty up until the most recent versions. > Wholeheartedly agreed! ?Sorting out the FreeBSD test_multiprocessing > Signal 12 would be nice, too. ?I'll open an issue for this, unless > someone else gets there first. If/when you do, please add me on the noisy list. I remember there being some patche(s) pending for FBSD around improved shared semaphore support, but I can't find the bug report right now. jesse From daniel at stutzbachenterprises.com Fri Nov 6 15:17:59 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Fri, 6 Nov 2009 08:17:59 -0600 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it In-Reply-To: <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <4AF341BD.2010203@v.loewis.de> <7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> Message-ID: On Thu, Nov 5, 2009 at 5:02 PM, Raymond Hettinger wrote: > def first(iterable): > 'Return the first value from a container or iterable. If empty, raises > LookupError' > for value in iterable: > return value > raise LookupError('no first value; iterable is empty') > (This email should not be construed to mean that I support adding a .pick method; I do not.) I realized this morning that the "for i in s: break" idiom can be O(n), even assuming no hash collisions. Observe this subtly O(n**2) algorithm: Cashew:~$ cat > test.py while s: for i in s: break # Imagine a complex algorithm here that depends on i still being in s s.remove(i) Cashew:~$ python2.5 /usr/lib/python2.5/timeit.py -s 's=set(range(20000))' "`cat test.py`" 10 loops, best of 3: 31.2 msec per loop Cashew:~$ python2.5 /usr/lib/python2.5/timeit.py -s 's=set(range(40000))' "`cat test.py`" 10 loops, best of 3: 124 msec per loop Cashew:~$ python2.5 /usr/lib/python2.5/timeit.py -s 's=set(range(80000))' "`cat test.py`" 10 loops, best of 3: 491 msec per loop Cashew:~$ python2.5 /usr/lib/python2.5/timeit.py -s 's=set(range(160000))' "`cat test.py`" 10 loops, best of 3: 1.96 sec per loop Doubling n quadruples the run-time, i.e., O(n**2) behavior. I wonder if those clamoring for a pick() method and complaining about efficiency are inadvertently running into this? The problem arises because we're systematically unbalancing the hash table. The iterator returns the first valid entry in the hash table, which we remove. Repeat several times and pretty soon the iterator has to spend O(n) time scanning through empty entries to find the first remaining valid entry. On the other hand, the pop/add idiom is O(1), since the .pop implementation contains some voodoo to remember where it left off. Consequently, the following algorithm is O(n) instead of O(n**2): Cashew:~$ cat > test.py while s: i = s.pop() s.add(i) # Imagine a complex algorithm here that depends on i still being in s s.remove(i) Cashew:~$ python2.5 /usr/lib/python2.5/timeit.py -s 's=set(range(400000))' "`cat test.py`" 10 loops, best of 3: 28 msec per loop Cashew:~$ python2.5 /usr/lib/python2.5/timeit.py -s 's=set(range(800000))' "`cat test.py`" 10 loops, best of 3: 56.2 msec per loop Cashew:~$ python2.5 /usr/lib/python2.5/timeit.py -s 's=set(range(1600000))' "`cat test.py`" 10 loops, best of 3: 113 msec per loop Cashew:~$ python2.5 /usr/lib/python2.5/timeit.py -s 's=set(range(3200000))' "`cat test.py`" 10 loops, best of 3: 227 msec per loop Doubling n doubles the run-time, i.e., O(n) behavior. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From dickinsm at gmail.com Fri Nov 6 16:40:10 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 6 Nov 2009 15:40:10 +0000 Subject: [Python-Dev] Status of the Buildbot fleet and related bugs In-Reply-To: References: <5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com> Message-ID: <5c6f2a5d0911060740g3178b2d0p48e8ef483d6e9c7d@mail.gmail.com> On Fri, Nov 6, 2009 at 10:54 AM, David Bolen wrote: > Mark Dickinson writes: >> The big problem now on the FreeBSD buildbot is that test_multiprocessing >> reliably causes the whole test run to abort with 'Signal 12'. ?Solving this >> may be as simple as just getting someone to install a copy of FreeBSD 6.2 >> on an ssh-accessible machine so that the source of the error can be tracked >> down. > > I could arrange ssh access to the build slave if that would help > anyone who wants to look into that. I suspect it would help a lot! Thanks for the offer. Jesse would likely be able to pin down the cause of the failure faster than I would, but I don't know how many cycles he has available. > Just contact me directly. I'll do that, if/when I find time. > In terms of the overall release, I'm also fine with upgrading the > build slave to 6.4. ?Or I could jump up to 7.2 instead. ?When I first > brought the build slave up, 7.x wasn't finalized yet - not sure now > which release is more prevalent in use. Not sure either, but it's certainly useful to be able to test on FreeBSD 6.x. Ideally, we'd have buildslaves for both... Mark From guido at python.org Fri Nov 6 17:02:47 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 6 Nov 2009 08:02:47 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> Message-ID: On Thu, Nov 5, 2009 at 9:55 PM, Brett Cannon wrote: > The clarification I need is will this in any way influence when > modules are removed. If they stay in for the life of a major version > then I want it made clear that bug fixes for the code take lower > priority over all other code in the standard library. I think we should be as cautious as ever with removing modules. We've had our chance for clean-up without abandon with Py3k; now we should stick with our commitment to backwards compatibility. In fact, we should probably be *more* conservative than we already were given that the amount of code written in Python is always increasing and hence the cumulative pain caused by incompatible changes will increase too. I'm fine with silent deprecations or requiring a flag to turn on deprecation warnings (like Java does). We're not yet at the point where C is, but who wouldn't be next to C on the TIOBE index? :-) -- --Guido van Rossum (python.org/~guido) From ubershmekel at gmail.com Fri Nov 6 17:25:31 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Fri, 6 Nov 2009 18:25:31 +0200 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> Message-ID: <9d153b7c0911060825n2ab937catbd576a058e8b5543@mail.gmail.com> On Fri, Nov 6, 2009 at 6:02 PM, Guido van Rossum wrote: > I'm fine with silent deprecations or requiring a flag to turn on > deprecation warnings (like Java does). > > It's easier for java because they have compile time for the deprecation warnings. Maybe we should somehow silence all warnings when running eggs? --yuv -------------- next part -------------- An HTML attachment was scrubbed... URL: From debatem1 at gmail.com Fri Nov 6 17:25:59 2009 From: debatem1 at gmail.com (geremy condra) Date: Fri, 6 Nov 2009 11:25:59 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> Message-ID: On Fri, Nov 6, 2009 at 11:02 AM, Guido van Rossum wrote: > On Thu, Nov 5, 2009 at 9:55 PM, Brett Cannon wrote: >> The clarification I need is will this in any way influence when >> modules are removed. If they stay in for the life of a major version >> then I want it made clear that bug fixes for the code take lower >> priority over all other code in the standard library. > > I think we should be as cautious as ever with removing modules. We've > had our chance for clean-up without abandon with Py3k; now we should > stick with our commitment to backwards compatibility. In fact, we > should probably be *more* conservative than we already were given that > the amount of code written in Python is always increasing and hence > the cumulative pain caused by incompatible changes will increase too. > > I'm fine with silent deprecations or requiring a flag to turn on > deprecation warnings (like Java does). > > We're not yet at the point where C is, but who wouldn't be next to C > on the TIOBE index? :-) I'd take being next to lisp if it meant that we didn't have to become any more like C ;). And speaking of TIOBE, my impression- and apparently yours- was that Python was on its way up, but TIOBE lists us as being down from a little over a year ago. Anybody know anything about their methodology? Geremy Condra From status at bugs.python.org Fri Nov 6 18:07:13 2009 From: status at bugs.python.org (Python tracker) Date: Fri, 6 Nov 2009 18:07:13 +0100 (CET) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20091106170713.E492C782A4@psf.upfronthosting.co.za> ACTIVITY SUMMARY (10/30/09 - 11/06/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2479 open (+15) / 16616 closed (+15) / 19095 total (+30) Open issues with patches: 991 Average duration of open issues: 683 days. Median duration of open issues: 437 days. Open Issues Breakdown open 2444 (+15) pending 34 ( +0) Issues Created Or Reopened (31) _______________________________ mac binary download link for 2.6.4 broken 10/30/09 CLOSED http://bugs.python.org/issue7243 reopened beard itertools.zip_longest behaves strangely with an iterable class 10/31/09 CLOSED http://bugs.python.org/issue7244 created durban patch better Ctrl-C support in pdb (program can be resumed) 10/31/09 http://bugs.python.org/issue7245 created isandler getpass raises IOError when several returns are in stdin before 11/01/09 CLOSED http://bugs.python.org/issue7246 created avihu test_fcntl_64_bit from test_fcntl.py fails in Python 2.6.4 11/01/09 http://bugs.python.org/issue7247 created Chainsaw test_importlib uses a fixed name in /tmp 11/01/09 CLOSED http://bugs.python.org/issue7248 created pitrou Consider allowing io.BytesIO sizes to be passed as 'long' in 2.6 11/02/09 CLOSED http://bugs.python.org/issue7249 created ryles wsgiref.handlers.CGIHandler caches os.environ, leaking info betw 11/02/09 http://bugs.python.org/issue7250 created snprbob86 patch, easy, 26backport Mark expected failures of test_math, test_cmath and test_round a 11/02/09 http://bugs.python.org/issue7251 created mark.dickinson buildbot list().index() should provide better error reporting 11/02/09 CLOSED http://bugs.python.org/issue7252 created djc AssertionError Tree is insane with *args and reduce 11/02/09 CLOSED http://bugs.python.org/issue7253 created jaraco Class members not properly initialized if declared outside of fu 11/02/09 CLOSED http://bugs.python.org/issue7254 created vladc6 "Default" word boundaries for Unicode data? 11/03/09 http://bugs.python.org/issue7255 created RegEx4All parse_qs and parse_qsl in urlparse are not documented as New in 11/03/09 CLOSED http://bugs.python.org/issue7256 created VeXocide Improve documentation of list.sort and sorted() 11/03/09 http://bugs.python.org/issue7257 created olau test_multiprocessing leaves a temporary directory 11/04/09 CLOSED http://bugs.python.org/issue7258 created pitrou Documentation update for operator module 11/04/09 CLOSED http://bugs.python.org/issue7259 created gagenellina patch SyntaxError with a not-existing offset for unicode code 11/04/09 CLOSED http://bugs.python.org/issue7260 created noam Document 2.x -> 3.x round changes in "What's New" documents. 11/04/09 http://bugs.python.org/issue7261 created mark.dickinson codecs.open() + eol (windows) 11/04/09 http://bugs.python.org/issue7262 created shamilbi Fix set.intersection docstring 11/04/09 http://bugs.python.org/issue7263 created belopolsky patch test_threading_local sometimes hangs when run with -R 11/04/09 CLOSED http://bugs.python.org/issue7264 created pitrou patch IDLE not working 11/05/09 CLOSED http://bugs.python.org/issue7265 created Carey test_lib2to3 failure under Windows 11/05/09 http://bugs.python.org/issue7266 created pitrou easy format method: c presentation type broken 11/05/09 http://bugs.python.org/issue7267 created doerwalter 2to3 execfile conversion changes behavior 11/06/09 CLOSED http://bugs.python.org/issue7268 created flashk Windows buildbot occasional DBFileExistsError failures in test_b 11/06/09 http://bugs.python.org/issue7269 created r.david.murray buildbot Tests for synchronization primitives 11/06/09 http://bugs.python.org/issue7270 created pitrou patch Typo in turtle module documentation. 11/06/09 http://bugs.python.org/issue7271 created mark.dickinson test_multiprocessing fails consistently with 'signal 12' on Free 11/06/09 http://bugs.python.org/issue7272 created mark.dickinson buildbot struct.pack adding extra '\x00' character in very specific case 11/06/09 CLOSED http://bugs.python.org/issue7273 created cmadrigal Issues Now Closed (37) ______________________ Minor Change For Better cross compile 788 days http://bugs.python.org/issue1115 gregory.p.smith patch IDLE fails to launch 669 days http://bugs.python.org/issue1743 richjtd IDLE won't start in custom directory. 414 days http://bugs.python.org/issue3881 srid patch Add "key" argument to "bisect" module functions 348 days http://bugs.python.org/issue4356 rhettinger patch xml.etree does not escape CR, LF and TAB characters within attri 110 days http://bugs.python.org/issue6492 ezio.melotti Compilation error if configuref --with-computed-gotos 93 days http://bugs.python.org/issue6603 mark.dickinson patch, easy fnmatch fails on filenames containing \n character 86 days http://bugs.python.org/issue6665 gregory.p.smith Intermittent failures in test_mailbox 51 days http://bugs.python.org/issue6896 pitrou patch test_signal fails on os x 10.6 29 days http://bugs.python.org/issue7042 mark.dickinson patch SysLogHandler can't handle Unicode 29 days http://bugs.python.org/issue7077 vinay.sajip Value error 'path is on drive c: start on drive d:' in os.path.r 8 days http://bugs.python.org/issue7195 ncoghlan Getpass echo's password to screen on 2.6, but not on 2.5 or 3.1 6 days http://bugs.python.org/issue7208 gregory.p.smith patch Proposed Syntax Checks in Test Suite 5 days http://bugs.python.org/issue7210 ChuckRhode patch select module - kevent ident field 64 bit issue 9 days http://bugs.python.org/issue7211 pitrou patch IDLE Subprocess Startup Error {Solved} 8 days http://bugs.python.org/issue7217 amaury.forgeotdarc thread reaping is imperfect 3 days http://bugs.python.org/issue7222 pitrou patch One obvious way to do interning 9 days http://bugs.python.org/issue7224 rhettinger patch test_hotshot fails on solaris 2 days http://bugs.python.org/issue7230 pitrou subprocess.Popen.stdout.flush fails os OS-X 10.6.1 5 days http://bugs.python.org/issue7240 pjenvey mac binary download link for 2.6.4 broken 0 days http://bugs.python.org/issue7243 beard itertools.zip_longest behaves strangely with an iterable class 1 days http://bugs.python.org/issue7244 rhettinger patch getpass raises IOError when several returns are in stdin before 1 days http://bugs.python.org/issue7246 avihu test_importlib uses a fixed name in /tmp 3 days http://bugs.python.org/issue7248 brett.cannon Consider allowing io.BytesIO sizes to be passed as 'long' in 2.6 1 days http://bugs.python.org/issue7249 pitrou list().index() should provide better error reporting 0 days http://bugs.python.org/issue7252 benjamin.peterson AssertionError Tree is insane with *args and reduce 0 days http://bugs.python.org/issue7253 benjamin.peterson Class members not properly initialized if declared outside of fu 0 days http://bugs.python.org/issue7254 r.david.murray parse_qs and parse_qsl in urlparse are not documented as New in 0 days http://bugs.python.org/issue7256 georg.brandl test_multiprocessing leaves a temporary directory 0 days http://bugs.python.org/issue7258 pitrou Documentation update for operator module 0 days http://bugs.python.org/issue7259 georg.brandl patch SyntaxError with a not-existing offset for unicode code 0 days http://bugs.python.org/issue7260 ezio.melotti test_threading_local sometimes hangs when run with -R 1 days http://bugs.python.org/issue7264 pitrou patch IDLE not working 1 days http://bugs.python.org/issue7265 loewis 2to3 execfile conversion changes behavior 0 days http://bugs.python.org/issue7268 flashk struct.pack adding extra '\x00' character in very specific case 0 days http://bugs.python.org/issue7273 mark.dickinson socket.inet_aton returns 8 bytes on LP64 platforms 265 days http://bugs.python.org/issue1008086 gregory.p.smith 64bit, easy Investigated ref leak report related to thread regrtest.py 866 days http://bugs.python.org/issue1739118 ocean-city patch Top Issues Most Discussed (10) ______________________________ 14 itertools.zip_longest behaves strangely with an iterable class 1 days closed http://bugs.python.org/issue7244 14 64 bit python fails on Windows 7 12 days open http://bugs.python.org/issue7206 14 Python 2.5 64 bit compile fails on Solaris 10/gcc 4.1.1 1036 days open http://bugs.python.org/issue1628484 11 2to3 execfile conversion changes behavior 0 days closed http://bugs.python.org/issue7268 11 test_debuglevel from test_telnetlib.py fails 78 days open http://bugs.python.org/issue6748 10 Intermittent failures in test_mailbox 51 days closed http://bugs.python.org/issue6896 9 cross compile patch 1915 days open http://bugs.python.org/issue1006238 8 One obvious way to do interning 9 days closed http://bugs.python.org/issue7224 7 wsgiref.handlers.CGIHandler caches os.environ, leaking info bet 4 days open http://bugs.python.org/issue7250 7 Documentation add note about SystemRandom 30 days open http://bugs.python.org/issue7076 From brett at python.org Fri Nov 6 18:37:59 2009 From: brett at python.org (Brett Cannon) Date: Fri, 6 Nov 2009 09:37:59 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> Message-ID: On Fri, Nov 6, 2009 at 08:02, Guido van Rossum wrote: > On Thu, Nov 5, 2009 at 9:55 PM, Brett Cannon wrote: >> The clarification I need is will this in any way influence when >> modules are removed. If they stay in for the life of a major version >> then I want it made clear that bug fixes for the code take lower >> priority over all other code in the standard library. > > I think we should be as cautious as ever with removing modules. We've > had our chance for clean-up without abandon with Py3k; now we should > stick with our commitment to backwards compatibility. In fact, we > should probably be *more* conservative than we already were given that > the amount of code written in Python is always increasing and hence > the cumulative pain caused by incompatible changes will increase too. > > I'm fine with silent deprecations or requiring a flag to turn on > deprecation warnings (like Java does). > We can discuss this on the stdlib-sig and come back with a proposal on how to update PEP 4 with an explicit policy on how to handle deprecations (either Nick's proposal or some flag, and then how long to let the deprecation last, i.e. three versions or the life of a major version of Python). > We're not yet at the point where C is, but who wouldn't be next to C > on the TIOBE index? :-) > > -- > --Guido van Rossum (python.org/~guido) > From stechert at gmail.com Fri Nov 6 20:31:09 2009 From: stechert at gmail.com (Andre Stechert) Date: Fri, 6 Nov 2009 11:31:09 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> Message-ID: <12a697af0911061131yaa62530n16d11c78317d8f28@mail.gmail.com> On Fri, Nov 6, 2009 at 8:25 AM, geremy condra wrote: > And speaking of TIOBE, my impression- and apparently yours- was > that Python was on its way up, but TIOBE lists us as being down > from a little over a year ago. Anybody know anything about their > methodology? There's a detailed explanation at: http://www.tiobe.com/index.php/content/paperinfo/tpci/tpci_definition.htm Andre From tseaver at palladion.com Fri Nov 6 20:37:44 2009 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 06 Nov 2009 14:37:44 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <4AF3EDF2.2010406@gmail.com> Message-ID: <4AF47B08.4020009@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dirkjan Ochtman wrote: > But supporting 4 releases in a row has been a bit of a PITA. Luckily, > we finally felt it was time to drop 2.3, so now we can make use of > luxuries such as subprocess... Still, supporting 3 releases seems > relatively common in the Python world (after all, parts of Zope still > require 2.4, I think), and so it would be nice if the stdlib moved a > little bit slower. The current release of Zope (finally) requires 2.5 or 2.6: we have a "don't intentionally break 2.4" policy in place, however, FBO those users whose system Python is still 2.4, and who can't / won't build their own Python. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkr0ewgACgkQ+gerLs4ltQ7W2ACeOXPwhpg8Hi4/XZom/B59/7GD 4mwAnRkixQm0A3ZkNRQMPW1JIbEOy1iC =gTbW -----END PGP SIGNATURE----- From guido at python.org Fri Nov 6 21:22:38 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 6 Nov 2009 12:22:38 -0800 Subject: [Python-Dev] OpenSSL vulnerability Message-ID: Now that a new SSL vulnerability is out (http://extendedsubset.com/?p=8) should we regenerate binary distributions that include copies of openssl (I think only the Windows MSIs) ? Does it affect any of our ssl APIs? -- --Guido van Rossum (python.org/~guido) From db3l.net at gmail.com Fri Nov 6 21:34:36 2009 From: db3l.net at gmail.com (David Bolen) Date: Fri, 06 Nov 2009 15:34:36 -0500 Subject: [Python-Dev] Status of the Buildbot fleet and related bugs References: <5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com> <5c6f2a5d0911060740g3178b2d0p48e8ef483d6e9c7d@mail.gmail.com> Message-ID: Mark Dickinson writes: > Not sure either, but it's certainly useful to be able to test on > FreeBSD 6.x. Ideally, we'd have buildslaves for both... Well, let me plan on leaving the current slave at 6.x (but working on getting it to 6.4). I could probably see about providing an additional 7.x slave - the only real issue is that at the moment they'd both be VMs on the same physical host, which is already shared with my Windows slave. I think a recent posting by Martin mentioned being able to interlock one or more slaves so they didn't execute in parallel, so that could help clean up the contention, though I must admit to not knowing for sure if serial execution would yield a faster overall result among all the slaves, though it feels likely. -- David From glyph at twistedmatrix.com Fri Nov 6 21:48:23 2009 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Fri, 6 Nov 2009 15:48:23 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <20091106140412.3229.80002680.divmod.xquotient.193@localhost.localdomain> References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com> <20091106001430.3229.796306500.divmod.xquotient.180@localhost.localdomain> <4AF3CF69.6090709@gmx.net> <20091106135058.GA13578@vidar.dreamhost.com> <20091106140412.3229.80002680.divmod.xquotient.193@localhost.localdomain> Message-ID: <68272E43-D993-4833-8DC9-DB45CB40F368@twistedmatrix.com> On Nov 6, 2009, at 9:04 AM, exarkun at twistedmatrix.com wrote: > End users don't even need to be told this much, though (and if it's > javac that does this for Java, then indeed Java end users probably > aren't seeing this either). I think it would be great for > deprecation warnings to be completely silent for a while *and* for > the policy to be prominently documented so that developers, the > people who really need this information, know how to get it. Documentation would be great, but then you have to get people to read the documentation and that's kind of tricky. Better would be for every project on PyPI to have a score which listed warnings emitted with each version of Python. People love optimizing for stuff like that and comparing it. I suspect that even if all warnings were completely silent by default, developers would suddenly become keenly interested in fixing them if there were a metric like that publicly posted somewhere :). From debatem1 at gmail.com Fri Nov 6 22:15:52 2009 From: debatem1 at gmail.com (geremy condra) Date: Fri, 6 Nov 2009 16:15:52 -0500 Subject: [Python-Dev] OpenSSL vulnerability In-Reply-To: References: Message-ID: On Fri, Nov 6, 2009 at 3:22 PM, Guido van Rossum wrote: > Now that a new SSL vulnerability is out > (http://extendedsubset.com/?p=8) should we regenerate binary > distributions that include copies of openssl (I think only the Windows > MSIs) ? > > Does it affect any of our ssl APIs? > > -- > --Guido van Rossum (python.org/~guido) The proposal on the table is to add a TLS extension that takes care of the problem, leave clients unchanged, and to stop servers from rehandshaking with clients that don't support the extension. AFAICS, that's all supposed to be handled by openssl. Certainly the EVP stuff won't need to be modified. The version of openssl being distributed should definitely be brought up to 0.9.8l though. Geremy Condra From tjreedy at udel.edu Fri Nov 6 22:45:39 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 06 Nov 2009 16:45:39 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> Message-ID: Brett Cannon wrote: > On Fri, Nov 6, 2009 at 08:02, Guido van Rossum wrote: >> On Thu, Nov 5, 2009 at 9:55 PM, Brett Cannon wrote: >>> The clarification I need is will this in any way influence when >>> modules are removed. If they stay in for the life of a major version >>> then I want it made clear that bug fixes for the code take lower >>> priority over all other code in the standard library. >> I think we should be as cautious as ever with removing modules. We've >> had our chance for clean-up without abandon with Py3k; now we should >> stick with our commitment to backwards compatibility. In fact, we >> should probably be *more* conservative than we already were given that >> the amount of code written in Python is always increasing and hence >> the cumulative pain caused by incompatible changes will increase too. >> >> I'm fine with silent deprecations or requiring a flag to turn on >> deprecation warnings (like Java does). >> > > We can discuss this on the stdlib-sig and come back with a proposal on > how to update PEP 4 with an explicit policy on how to handle > deprecations (either Nick's proposal or some flag, and then how long > to let the deprecation last, i.e. three versions or the life of a > major version of Python). Complete removal of modules during the seems to have caused problems especially for people supporting a range of versions. On the other hand, you want to be able to replace and cease support of modules. So rather that complete removal, perhaps move them to a new package called, for instance, 'old'. And move their doc sections to a new chapter call 'Old Modules' which would begin with something like "The following modules have been replaced but are still present in the *old* package for back compatibility. They are no longer maintained and will possibly disappear in the future, such as if there is ever a Python4." This suggestion based on the "3. Non-essential Built-in Functions" of the 2.x doc, whose contents disappeared in 3.0 (even though that seems not to have been specified in the doc). Terry Jan Reedy From rdmurray at bitdance.com Fri Nov 6 22:48:37 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 6 Nov 2009 16:48:37 -0500 (EST) Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <68272E43-D993-4833-8DC9-DB45CB40F368@twistedmatrix.com> References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com> <20091106001430.3229.796306500.divmod.xquotient.180@localhost.localdomain> <4AF3CF69.6090709@gmx.net> <20091106135058.GA13578@vidar.dreamhost.com> <20091106140412.3229.80002680.divmod.xquotient.193@localhost.localdomain> <68272E43-D993-4833-8DC9-DB45CB40F368@twistedmatrix.com> Message-ID: On Fri, 6 Nov 2009 at 15:48, Glyph Lefkowitz wrote: > On Nov 6, 2009, at 9:04 AM, exarkun at twistedmatrix.com wrote: >> End users don't even need to be told this much, though (and if it's javac >> that does this for Java, then indeed Java end users probably aren't seeing >> this either). I think it would be great for deprecation warnings to be >> completely silent for a while *and* for the policy to be prominently >> documented so that developers, the people who really need this information, >> know how to get it. > > Documentation would be great, but then you have to get people to read the > documentation and that's kind of tricky. Better would be for every project > on PyPI to have a score which listed warnings emitted with each version of > Python. People love optimizing for stuff like that and comparing it. > > I suspect that even if all warnings were completely silent by default, > developers would suddenly become keenly interested in fixing them if there > were a metric like that publicly posted somewhere :). +1, but somebody needs to write the code... --David (RDM) From exarkun at twistedmatrix.com Fri Nov 6 22:52:10 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Fri, 06 Nov 2009 21:52:10 -0000 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com> <20091106001430.3229.796306500.divmod.xquotient.180@localhost.localdomain> <4AF3CF69.6090709@gmx.net> <20091106135058.GA13578@vidar.dreamhost.com> <20091106140412.3229.80002680.divmod.xquotient.193@localhost.localdomain> <68272E43-D993-4833-8DC9-DB45CB40F368@twistedmatrix.com> Message-ID: <20091106215210.3229.1142452245.divmod.xquotient.212@localhost.localdomain> On 09:48 pm, rdmurray at bitdance.com wrote: >On Fri, 6 Nov 2009 at 15:48, Glyph Lefkowitz wrote: >>On Nov 6, 2009, at 9:04 AM, exarkun at twistedmatrix.com wrote: >>>End users don't even need to be told this much, though (and if it's >>>javac that does this for Java, then indeed Java end users probably >>>aren't seeing this either). I think it would be great for >>>deprecation warnings to be completely silent for a while *and* for >>>the policy to be prominently documented so that developers, the >>>people who really need this information, know how to get it. >> >>Documentation would be great, but then you have to get people to read >>the documentation and that's kind of tricky. Better would be for >>every project on PyPI to have a score which listed warnings emitted >>with each version of Python. People love optimizing for stuff like >>that and comparing it. >> >>I suspect that even if all warnings were completely silent by default, >>developers would suddenly become keenly interested in fixing them if >>there were a metric like that publicly posted somewhere :). > >+1, but somebody needs to write the code... How would you collect this information? Would you run the test suite for each project? This would reward projects with small or absent test suites. ;) Jean-Paul From janssen at parc.com Fri Nov 6 23:18:22 2009 From: janssen at parc.com (Bill Janssen) Date: Fri, 6 Nov 2009 14:18:22 PST Subject: [Python-Dev] OpenSSL vulnerability In-Reply-To: References: Message-ID: <40373.1257545902@parc.com> Guido, I'm working from . I believe geremy is right. The current SSL module does not expose much of the SSL API, so servers implemented in Python, using it, should (fortuituously) be immune to the some of the attacks outlined, simply because there's no way to do an application-initiated renegotiation, which the first two scenarios presuppose. On the other hand, there's no way to do application-directed session resumption, either, which might be a good add to support new or updated application protocols which address this problem. So I think there's not much we can do in Python source code to address this, unless there's a switch we can throw in the existing OpenSSL API to turn off renegotiation completely. I'll look, and I'll talk this over with our security group. Building binaries with newer versions of OpenSSL is pretty much always a good idea, it seems to me. More generally, this is a nice description of how simply layering TLS onto existing application protocols like HTTP doesn't always work very well. Bill From g.brandl at gmx.net Sat Nov 7 00:24:51 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 07 Nov 2009 00:24:51 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> Message-ID: Guido van Rossum schrieb: > Actually, I think Dirkjan has a point. I'm not sure that we need > another moratorium (that's a rather dramatic kind of decision which > should be very rare indeed) but I do agree that deprecations are often > more of a pain than they're worth. > > For example, take the deprecation of the md5 and sha modules in Python > 2.6. They make it a bit of a pain to write code that *cleanly* > supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when > importing md5 instead of hashlib). You can silence the warning, but > that is in itself not particularly clean, and users really hate having > the warnings. Trying to import hashlib and importing md5 on ImportError isn't *too* unclean, is it? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From exarkun at twistedmatrix.com Fri Nov 6 23:36:34 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Fri, 06 Nov 2009 22:36:34 -0000 Subject: [Python-Dev] OpenSSL vulnerability In-Reply-To: <40373.1257545902@parc.com> References: <40373.1257545902@parc.com> Message-ID: <20091106223634.3229.2049231320.divmod.xquotient.232@localhost.localdomain> On 10:18 pm, janssen at parc.com wrote: >Guido, > >I'm working from . > >I believe geremy is right. The current SSL module does not expose much >of the SSL API, so servers implemented in Python, using it, should >(fortuituously) be immune to the some of the attacks outlined, simply >because there's no way to do an application-initiated renegotiation, >which the first two scenarios presuppose. On the other hand, there's >no >way to do application-directed session resumption, either, which might >be a good add to support new or updated application protocols which >address this problem. Also, for Python 2.5 and earlier, any SSL-based code is vulnerable to a MitM anyway, so this can only be an issue for code using the new APIs in Python 2.6. Jean-Paul From guido at python.org Fri Nov 6 23:37:10 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 6 Nov 2009 14:37:10 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> Message-ID: On Fri, Nov 6, 2009 at 3:24 PM, Georg Brandl wrote: > Guido van Rossum schrieb: > >> Actually, I think Dirkjan has a point. I'm not sure that we need >> another moratorium (that's a rather dramatic kind of decision which >> should be very rare indeed) but I do agree that deprecations are often >> more of a pain than they're worth. >> >> For example, take the deprecation of the md5 and sha modules in Python >> 2.6. They make it a bit of a pain to write code that *cleanly* >> supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when >> importing md5 instead of hashlib). You can silence the warning, but >> that is in itself not particularly clean, and users really hate having >> the warnings. > > Trying to import hashlib and importing md5 on ImportError isn't *too* unclean, > is it? That's not all -- you also have to modify the code that uses the module, unless you use "import as", which has problems of its own. Plus, I may not care (yet) about supporting 2.7, and yet I am forced to change my code to cleanly support 2.6. I really don't like it. -- --Guido van Rossum (python.org/~guido) From g.brandl at gmx.net Sat Nov 7 00:34:25 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 07 Nov 2009 00:34:25 +0100 Subject: [Python-Dev] Status of the Buildbot fleet and related bugs In-Reply-To: References: Message-ID: R. David Murray schrieb: > So, overall I think the buildbot fleet is in good shape, and if > we can nail issue 6748 I think it will be back to being an > important resource for sanity checking our checkins. Yay! Thanks to all of you! > By the way, Georg set up the IRC interface on the #python-dev channel, > so you can hang out there if you want to get realtime reports of which > buildbots have going from success to failure and vice versa. JFTR, I didn't set up the IRC bot (I assume that credit goes to Martin, even if it's only one line in the buildbot config :). I just tried to get it to say something :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Sat Nov 7 00:59:10 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 07 Nov 2009 00:59:10 +0100 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it In-Reply-To: <7b0045ed0911052219u1893bd62gebf71bf46721014a@mail.gmail.com> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <4AF341BD.2010203@v.loewis.de> <7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> <7b0045ed0911051859y13f8abf3u7b5b4549aa2a41fc@mail.gmail.com> <4AF3B76A.1050006@v.loewis.de> <7b0045ed0911052219u1893bd62gebf71bf46721014a@mail.gmail.com> Message-ID: Chris Bergstresser schrieb: > On Thu, Nov 5, 2009 at 11:43 PM, "Martin v. L?wis" wrote: >> I read Raymond's suggestion rather as a question: why bother with a >> tedious, multi-year process, when a three-line function will achieve >> exactly the same? > > Because it doesn't achieve exactly the same. What I want is a > sane, rational way to describe what I'm doing in the core API, so > other programmers learning the language don't spend the amount of time > I did perplexed that there was a .pop() and a .remove() and a > .discard(), but there wasn't a .pick(). I don't want to have to write > the same little helper function in every project to fill a deficiency > in the library. Why don't you write that little helper function only in those projects that actually need it? I suspect that will be a small fraction... > I don't want to have to argue about the flaws in > solutions with race conditions, or the fact that cheap functions > become expensive functions when performed over the network, or that > there's a real value in having an atomic operation which throws a sane > exception when it fails, or how it's disturbing to my OCD core to have > an API which believes: > > if x in s: > s.remove(x) > > ... is too confusing, so there should be a .discard() method, but ... > > for x in s: > break > > ... is self-evident and obvious, so there's no need for a .pick(). It's not just the self-evidence of the required "workaround" that determines whether an API function is added to a builtin type. It's whether you need it. Imagine someone asking for a function on lists that removes duplicates, but keeps the order of all first occurrences. This is not a completely unimaginable method, and writing it by hand is more than a few lines (see e.g. http://code.activestate.com/recipes/52560/#c1). Still, it's not needed frequently enough to require being a list method. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From guido at python.org Sat Nov 7 00:12:28 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 6 Nov 2009 15:12:28 -0800 Subject: [Python-Dev] OpenSSL vulnerability In-Reply-To: <20091106223634.3229.2049231320.divmod.xquotient.232@localhost.localdomain> References: <40373.1257545902@parc.com> <20091106223634.3229.2049231320.divmod.xquotient.232@localhost.localdomain> Message-ID: On Fri, Nov 6, 2009 at 2:36 PM, wrote: > Also, for Python 2.5 and earlier, any SSL-based code is vulnerable to a MitM > anyway, so this can only be an issue for code using the new APIs in Python > 2.6. That's not going to stop the wannabe-self-proclaimed-so-called-vulnerability-"experts" from whining about Python not releasing updated binary distributions though. :-( -- --Guido van Rossum (python.org/~guido) From andymac at bullseye.apana.org.au Sat Nov 7 01:39:20 2009 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Sat, 07 Nov 2009 11:39:20 +1100 Subject: [Python-Dev] Status of the Buildbot fleet and related bugs In-Reply-To: <5c6f2a5d0911060740g3178b2d0p48e8ef483d6e9c7d@mail.gmail.com> References: <5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com> <5c6f2a5d0911060740g3178b2d0p48e8ef483d6e9c7d@mail.gmail.com> Message-ID: <4AF4C1B8.1090007@bullseye.andymac.org> Mark Dickinson wrote: > On Fri, Nov 6, 2009 at 10:54 AM, David Bolen wrote: >> >> In terms of the overall release, I'm also fine with upgrading the >> build slave to 6.4. Or I could jump up to 7.2 instead. When I first >> brought the build slave up, 7.x wasn't finalized yet - not sure now >> which release is more prevalent in use. > > Not sure either, but it's certainly useful to be able to test on > FreeBSD 6.x. Ideally, we'd have buildslaves for both... http://security.freebsd.org/#sup lists the releases currently with active security updates and their EOL dates. Upgrading the 6.2 buildbot to 6.4 would seem a good idea to me. Unfortunately FreeBSD's binary update capability only became available with 6.3... :-( A separate 7.2 buildbot (which could then be binary updated to 7.3 on release) would also seem a good idea. 8.0 is at RC3 and could be expected to be finalised fairly soon. My understanding is there won't be another 6.x release, and I'm inferring that there will be one more 7.x release (7.3). Andrew. -- ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac at bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac at pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From ncoghlan at gmail.com Sat Nov 7 03:42:35 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 07 Nov 2009 12:42:35 +1000 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> Message-ID: <4AF4DE9B.6080703@gmail.com> Georg Brandl wrote: > Guido van Rossum schrieb: > >> Actually, I think Dirkjan has a point. I'm not sure that we need >> another moratorium (that's a rather dramatic kind of decision which >> should be very rare indeed) but I do agree that deprecations are often >> more of a pain than they're worth. >> >> For example, take the deprecation of the md5 and sha modules in Python >> 2.6. They make it a bit of a pain to write code that *cleanly* >> supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when >> importing md5 instead of hashlib). You can silence the warning, but >> that is in itself not particularly clean, and users really hate having >> the warnings. > > Trying to import hashlib and importing md5 on ImportError isn't *too* unclean, > is it? Having had to do this myself a few times, I was always a little surprised the "import-or" idea [1] didn't get more support the last time it was proposed. Ah well, don't need to worry about that idea again until 3.4 or so :) Cheers, Nick P.S. For anyone unfamiliar with it, "import-or" was a suggestion made around the time xml.etree was added to the standard library to provide a "import x or y as z" shorthand for: try: import x as z except ImportError: import y as z (allowing "from a or b import c" was also part of the suggestion, as was allowing multiple "or" elements in a single import request) Support for the idea was lukewarm at best, hostile at worst (hence the lack of PEP). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From db3l.net at gmail.com Sat Nov 7 03:44:44 2009 From: db3l.net at gmail.com (David Bolen) Date: Fri, 06 Nov 2009 21:44:44 -0500 Subject: [Python-Dev] Status of the Buildbot fleet and related bugs References: <5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com> <5c6f2a5d0911060740g3178b2d0p48e8ef483d6e9c7d@mail.gmail.com> <4AF4C1B8.1090007@bullseye.andymac.org> Message-ID: Andrew MacIntyre writes: > Upgrading the 6.2 buildbot to 6.4 would seem a good idea to me. > Unfortunately FreeBSD's binary update capability only became available > with 6.3... :-( No biggie - I was just planning on installing a new VM from scratch anyway, and then just cutting over VMs to switch over. -- David From guido at python.org Sat Nov 7 04:16:10 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 6 Nov 2009 19:16:10 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <4AF4DE9B.6080703@gmail.com> References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <4AF4DE9B.6080703@gmail.com> Message-ID: On Fri, Nov 6, 2009 at 6:42 PM, Nick Coghlan wrote: > Georg Brandl wrote: >> Guido van Rossum schrieb: >> >>> Actually, I think Dirkjan has a point. I'm not sure that we need >>> another moratorium (that's a rather dramatic kind of decision which >>> should be very rare indeed) but I do agree that deprecations are often >>> more of a pain than they're worth. >>> >>> For example, take the deprecation of the md5 and sha modules in Python >>> 2.6. They make it a bit of a pain to write code that *cleanly* >>> supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when >>> importing md5 instead of hashlib). You can silence the warning, but >>> that is in itself not particularly clean, and users really hate having >>> the warnings. >> >> Trying to import hashlib and importing md5 on ImportError isn't *too* unclean, >> is it? > > Having had to do this myself a few times, I was always a little > surprised the "import-or" idea [1] didn't get more support the last time > it was proposed. > > Ah well, don't need to worry about that idea again until 3.4 or so :) > > Cheers, > Nick > > P.S. For anyone unfamiliar with it, "import-or" was a suggestion made > around the time xml.etree was added to the standard library to provide a > "import x or y as z" shorthand for: > ?try: > ? ?import x as z > ?except ImportError: > ? ?import y as z > > (allowing "from a or b import c" was also part of the suggestion, as was > allowing multiple "or" elements in a single import request) > > Support for the idea was lukewarm at best, hostile at worst (hence the > lack of PEP). I'm still -1 on that -- while it may sound like a good shorthand if you have to deal with this, it's IMO pretty mysterious if you encounter this and don't already happen to know what it means. Also, it makes no sense without the 'as' clause which makes it a bit awkward. (I could probably come up with a few more arguments against it, just trying to hint that I'm personally less than lukewarm. :-) -- --Guido van Rossum (python.org/~guido) From stephen at xemacs.org Sat Nov 7 09:35:22 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 7 Nov 2009 17:35:22 +0900 (JST) Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <20091106215210.3229.1142452245.divmod.xquotient.212@localhost.localdomain> References: Message-ID: <20091107083522.7AC4C1A266C@uwakimon.sk.tsukuba.ac.jp> ~b <4222a8490911051521r1b9c8165id6e0f12d62da0613 at mail.gmail.com> <48c184130911051555r9a5b78cs3a13cb1345d3cbf7 at mail.gmail.com> <20091106001430.3229.796306500.divmod.xquotient.180 at localhost.localdomain> <4AF3CF69.6090709 at gmx.net> <20091106135058.GA13578 at vidar.dreamhost.com> <20091106140412.3229.80002680.divmod.xquotient.193 at localhost.localdomain> <68272E43-D993-4833-8DC9-DB45CB40F368 at twistedmatrix.com> <20091106215210.3229.1142452245.divmod.xquotient.212 at localhost.localdomain> X-Mailer: VM 8.0.12-devo-585 under 21.5 (beta29) "garbanzo" d20e0a45a4b2 XEmacs Lucid (x86_64-unknown-linux) Date: Sat, 07 Nov 2009 17:35:21 +0900 Message-ID: <87aayyspfa.fsf at uwakimon.sk.tsukuba.ac.jp> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii exarkun at twistedmatrix.com writes: > How would you collect this information? Would you run the test suite > for each project? This would reward projects with small or absent test > suites. ;) Multiply by the number of lines of code and divide by the number of tests run. From glyph at twistedmatrix.com Sat Nov 7 09:34:21 2009 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Sat, 7 Nov 2009 03:34:21 -0500 Subject: [Python-Dev] Status of the Buildbot fleet and related bugs In-Reply-To: References: Message-ID: <2D95753F-B50F-4F1B-9A8A-B8CA789AB148@twistedmatrix.com> On Nov 6, 2009, at 6:34 PM, Georg Brandl wrote: > R. David Murray schrieb: > >> So, overall I think the buildbot fleet is in good shape, and if >> we can nail issue 6748 I think it will be back to being an >> important resource for sanity checking our checkins. > > Yay! Thanks to all of you! Indeed! It's great to see so much work going into build and test maintenance. Thanks a lot! From glyph at twistedmatrix.com Sat Nov 7 09:36:18 2009 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Sat, 7 Nov 2009 03:36:18 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <20091106215210.3229.1142452245.divmod.xquotient.212@localhost.localdomain> References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com> <20091106001430.3229.796306500.divmod.xquotient.180@localhost.localdomain> <4AF3CF69.6090709@gmx.net> <20091106135058.GA13578@vidar.dreamhost.com> <20091106140412.3229.80002680.divmod.xquotient.193@localhost.localdomain> <68272E43-D993-4833-8DC9-DB45CB40F368@twistedmatrix.com> <20091106215210.3229.1142452245.divmod.xquotient.212@localhost.localdomain> Message-ID: <4A329419-0784-4026-BF77-C13D8DB2A515@twistedmatrix.com> On Nov 6, 2009, at 4:52 PM, exarkun at twistedmatrix.com wrote: > On 09:48 pm, rdmurray at bitdance.com wrote: >> On Fri, 6 Nov 2009 at 15:48, Glyph Lefkowitz wrote: >> >>> Documentation would be great, but then you have to get people to >>> read the documentation and that's kind of tricky. Better would be >>> for every project on PyPI to have a score which listed warnings >>> emitted with each version of Python. People love optimizing for >>> stuff like that and comparing it. >>> >>> I suspect that even if all warnings were completely silent by >>> default, developers would suddenly become keenly interested in >>> fixing them if there were a metric like that publicly posted >>> somewhere :). >> >> +1, but somebody needs to write the code... > > How would you collect this information? Would you run the test > suite for each project? This would reward projects with small or > absent test suites. ;) *I* would not collect this information, as I am far enough behind on other projects ;-) but I if I were to advise someone *else* as to how to do it, I'd probably add a feature to the 'warnings' module where users could opt-in (sort of like popcon.debian.org) to report warnings encountered during normal invocations of any of their Python programs. I would also advise such a hypothetical data-gathering project to start with a buildbot doing coverage runs; any warning during the test suite would be 1 demerit, any warning during an actual end-user run of the application *not* caught by the test suite would be 1000 demerits :). And actually it would make more sense if this were part of an overall quality metric, like http://pycheesecake.org/ proposes (although I think that cheesecake's current metric is not really that great, the idea is wonderful). From baptiste.lepilleur at gmail.com Sat Nov 7 11:51:58 2009 From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur) Date: Sat, 7 Nov 2009 11:51:58 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: References: <1256502140.5621.163.camel@localhost> Message-ID: Hi Antoine, I was finally able to compile py3k and run the benchmark (my compilation issue was caused by checking out on Windows and compiling on Unix. Some Makefile templates are missing correct EOL properties in SVN I think). The benchmark results can be obtained from: http://gaiacrtn.free.fr/py/benchmark-newgil/benchmark-newgil.tar.bz2 and viewed from: http://gaiacrtn.free.fr/py/benchmark-newgil/ I ran the benchmark on two platforms: - Solaris X86, 16 cores: some python extension are likely missing (see config.log) - Windows XP SP3, 4 cores: all python extensions but TCL (I didn't bother checking why it failed as it is not used in the benchmark). It is a release build. The results look promising but I let you share your conclusion (some latency results seem a bit strange from my understanding). Side-note: PCBuild requires nasmw.exe but it no longer exists in the latest version. I had to rename nasm.exe to nasmw.exe. Would be nice to add this to the readme to avoid confusion... Baptiste. 2009/11/1 Antoine Pitrou > > Hello again, > > Brett Cannon python.org> writes: > > > > I think it's worth it. Removal of the GIL is a totally open-ended problem > > with no solution in sight. This, on the other hand, is a performance > benefit > > now. I say move forward with this. If it happens to be short-lived > because > > some actually figures out how to remove the GIL then great, but is that > > really going to happen between now and Python 3.2? I doubt it. > > Based on this whole discussion, I think I am going to merge the new GIL > work > into the py3k branch, with priority requests disabled. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From skip at pobox.com Sat Nov 7 13:10:57 2009 From: skip at pobox.com (skip at pobox.com) Date: Sat, 7 Nov 2009 06:10:57 -0600 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <4AF4DE9B.6080703@gmail.com> Message-ID: <19189.25553.111384.537110@montanaro.dyndns.org> Guido> ... it's IMO pretty mysterious if you encounter this and don't Guido> already happen to know what it means. If you require parens maybe it parses better: import (a or b or c) as mod Given that the or operator shortcuts I think that (a or b or c) terminates once a module is found isn't too hard to grasp. Skip From exarkun at twistedmatrix.com Sat Nov 7 14:44:46 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Sat, 07 Nov 2009 13:44:46 -0000 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <19189.25553.111384.537110@montanaro.dyndns.org> References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <4AF4DE9B.6080703@gmail.com> <19189.25553.111384.537110@montanaro.dyndns.org> Message-ID: <20091107134446.3229.1411095315.divmod.xquotient.283@localhost.localdomain> On 12:10 pm, skip at pobox.com wrote: > Guido> ... it's IMO pretty mysterious if you encounter this and >don't > Guido> already happen to know what it means. > >If you require parens maybe it parses better: > > import (a or b or c) as mod > >Given that the or operator shortcuts I think that (a or b or c) >terminates >once a module is found isn't too hard to grasp. Did everyone forget what the subject of this thread is? Jean-Paul From ssteinerx at gmail.com Sat Nov 7 15:30:45 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Sat, 7 Nov 2009 09:30:45 -0500 Subject: [Python-Dev] People want CPAN :-) In-Reply-To: <87iqdm7nkp.fsf@benfinney.id.au> References: <87iqdm7nkp.fsf@benfinney.id.au> Message-ID: On Nov 7, 2009, at 3:20 AM, Ben Finney wrote: > Guido van Rossum writes: > >> On Fri, Nov 6, 2009 at 2:52 PM, David Lyon >> wrote: >>> > I think buildbot-style test runs for PyPI packages would raise average > package quality on PyPI. Please excuse the cross-post but I wanted to make sure that all these "CPAN for Python" discussions got this message and I've lost track of which list which part of what discussion had occurred on. We are currently extending our distutils/Distribute test system to include installation of a broad range of packages as part of the pre- release process for a future release of Distribute and as part of our "smoke" test for distutils/Distribute. Eventually, the goal is to integrate this with our buildbot system but that's a ways off. Our goal is to install a range of packages and, where practicable, actually run and record any errors with the packages' individual test suites. Right now, our "smoke" test only does Twisted and numpy. We've discussed how to collect test results from Twisted trial and we'll be working on similar things for other test runners (nose et al.). For Twisted, we're going to install and test both the current release version and an svn checkout from trunk. It would be an extension of that concept to install and test *all* packages from PyPI but would, obviously, take considerable horsepower (and time) to run such an exhaustive test (especially if we're talking about 2.4?, 2.5, 2.6, 2.7, and 3.1+. Right now I'm extending the configuration file for our smoke test to allow for various test runners (e.g. nose, twisted trial, etc.) so we can "smoke out" more installation problems and/or failed tests after installation. For the first pass, I'm just focusing on Twisted and trial, then numpy, then finding packages that support nose so that I can collect the data on what ran, what passed, and what didn't. I'm planning on collecting this all in a database and making some simple API so that it can be mined by very simple apps later. At the point where that infrastructure is in place, we could pretty easily mine the data to do all kinds of crazy things people have mentioned like: * A ranking system of test coverage * Complexity analysis * Test coverage * Run pylint, pyflakes, 2to3, whatever automated measurement tools over the code * Send test failure messages to maintainers (maybe with opt-in in the new meta-data). * Whatever! We're actively working on this right now; anyone who wants to lend a hand is welcome to contact me off-list and we can talk about what types of things we are needing and where we could use a hand. All in all, I think this could be a big leap forward for the Python distribution ecosystem whether or not we eventually write the PyPan I wished for as a new Perl refugee. Thanks, S From solipsis at pitrou.net Sat Nov 7 16:06:28 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 7 Nov 2009 15:06:28 +0000 (UTC) Subject: [Python-Dev] Reworking the GIL References: <1256502140.5621.163.camel@localhost> Message-ID: Hello, > Solaris X86, 16 cores: some python extension are likely missing (see config.log) > Windows XP SP3, 4 cores: all python extensions but TCL (I didn't bother checking why it failed as it is not used in the benchmark). It is a release build. > > The results look promising but I let you share your conclusion (some latency results seem a bit strange from my understanding). Thank you! The latency results don't look that strange to me. If you're surprised that py3k shows better latency than newgil on the "pi calculation" workload, it's easy to understand why: py3k speculatively releases the GIL so often on that workload (every 100 opcodes) that the latencies are indeed very good, but if you look at the corresponding throughput numbers they are dismal (your Solaris box shows it falling to less than 20% with two threads running compared to the baseline number for single-thread execution, and on your Windows box the number is hardly better with 45%). So, to sum it up, the way the current GIL manages to have good latencies is by issueing an unreasonable number of system calls on a contended lock, and potentially killing throughput performance (this depends on the OS too, because numbers under Linux are not so catastrophic). The new GIL, on the other hand, is much more balanced in that it achieves rather predictable latencies (especially when you don't overcommit the OS by issueing more computational threads than you have CPU cores) while preserving throughput performance. > Side-note: PCBuild requires nasmw.exe but it no longer exists in the latest > version. I had to rename nasm.exe to nasmw.exe. Would be nice to add this to > the readme to avoid confusion... You should file a bug on http://bugs.python.org Regards Antoine. From jnoller at gmail.com Sat Nov 7 16:08:55 2009 From: jnoller at gmail.com (Jesse Noller) Date: Sat, 7 Nov 2009 10:08:55 -0500 Subject: [Python-Dev] People want CPAN :-) In-Reply-To: References: <87iqdm7nkp.fsf@benfinney.id.au> Message-ID: <4222a8490911070708j70d06be0j19a598fdddcee459@mail.gmail.com> On Sat, Nov 7, 2009 at 9:30 AM, ssteinerX at gmail.com wrote: > On Nov 7, 2009, at 3:20 AM, Ben Finney wrote: > >> Guido van Rossum writes: >> >>> On Fri, Nov 6, 2009 at 2:52 PM, David Lyon >>> wrote: >>>> >> I think buildbot-style test runs for PyPI packages would raise average >> package quality on PyPI. > > Please excuse the cross-post but I wanted to make sure that all these "CPAN > for Python" discussions got this message and I've lost track of which list > which part of what discussion had occurred on. > > We are currently extending our distutils/Distribute test system to include > installation of a broad range of packages as part of the pre-release process > for a future release of Distribute and as part of our "smoke" test for > distutils/Distribute. ? ?Eventually, the goal is to integrate this with our > buildbot system but that's a ways off. Who is "we"? From ssteinerx at gmail.com Sat Nov 7 16:10:59 2009 From: ssteinerx at gmail.com (ssteinerX@gmail.com) Date: Sat, 7 Nov 2009 10:10:59 -0500 Subject: [Python-Dev] People want CPAN :-) In-Reply-To: <4222a8490911070708j70d06be0j19a598fdddcee459@mail.gmail.com> References: <87iqdm7nkp.fsf@benfinney.id.au> <4222a8490911070708j70d06be0j19a598fdddcee459@mail.gmail.com> Message-ID: <1293335C-B422-472E-ACD8-4E46A756A9C8@gmail.com> On Nov 7, 2009, at 10:08 AM, Jesse Noller wrote: > On Sat, Nov 7, 2009 at 9:30 AM, ssteinerX at gmail.com > wrote: >> On Nov 7, 2009, at 3:20 AM, Ben Finney wrote: >> >>> Guido van Rossum writes: >>> >>>> On Fri, Nov 6, 2009 at 2:52 PM, David Lyon >>> > >>>> wrote: >>>>> >>> I think buildbot-style test runs for PyPI packages would raise >>> average >>> package quality on PyPI. >> >> Please excuse the cross-post but I wanted to make sure that all >> these "CPAN >> for Python" discussions got this message and I've lost track of >> which list >> which part of what discussion had occurred on. >> >> We are currently extending our distutils/Distribute test system to >> include >> installation of a broad range of packages as part of the pre- >> release process >> for a future release of Distribute and as part of our "smoke" test >> for >> distutils/Distribute. Eventually, the goal is to integrate this >> with our >> buildbot system but that's a ways off. > > Who is "we"? We is the people working on Distribute/distutils. S From guido at python.org Sat Nov 7 17:06:37 2009 From: guido at python.org (Guido van Rossum) Date: Sat, 7 Nov 2009 08:06:37 -0800 Subject: [Python-Dev] Reworking the GIL In-Reply-To: References: <1256502140.5621.163.camel@localhost> Message-ID: Antoine, How close are you to merging this into the Py3k branch? It looks like a solid piece of work, that can only get better in the period between now and the release of 3.2. But I don't want to rush you, and I only have had a brief look at your code. (I whipped up a small Dave Beazley example and was impressed by the performance of your code compared to the original py3k branch -- from 150% to 100% CPU usage according to top with only 2 threads.) My only suggestion so far: the description could use more explicit documentation on the various variables and macros and how they combine. I also expect that priority requests aren't important; it somehow seems strange that if multiple threads are all doing I/O each new thread whose I/O completes would get to preempt whoever else is active immediately. (Also the choice of *not* making a priority request when a read returns no bytes seems strange 00 if I read the code correctly.) Anyway, thanks for this work! -- --Guido van Rossum (python.org/~guido) From g.brandl at gmx.net Sat Nov 7 18:30:26 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 07 Nov 2009 18:30:26 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <20091107134446.3229.1411095315.divmod.xquotient.283@localhost.localdomain> References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <4AF4DE9B.6080703@gmail.com> <19189.25553.111384.537110@montanaro.dyndns.org> <20091107134446.3229.1411095315.divmod.xquotient.283@localhost.localdomain> Message-ID: exarkun at twistedmatrix.com schrieb: > On 12:10 pm, skip at pobox.com wrote: >> Guido> ... it's IMO pretty mysterious if you encounter this and >>don't >> Guido> already happen to know what it means. >> >>If you require parens maybe it parses better: >> >> import (a or b or c) as mod >> >>Given that the or operator shortcuts I think that (a or b or c) >>terminates >>once a module is found isn't too hard to grasp. > > Did everyone forget what the subject of this thread is? Why? The subject of this thread is "Importing module alternatives cleanly". We just happened to forget adapting the "Subject:" line. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From solipsis at pitrou.net Sat Nov 7 18:01:57 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 07 Nov 2009 18:01:57 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: References: <1256502140.5621.163.camel@localhost> Message-ID: <1257613317.3437.63.camel@localhost> Hello Guido, > How close are you to merging this into the Py3k branch? It looks like > a solid piece of work, that can only get better in the period between > now and the release of 3.2. But I don't want to rush you, and I only > have had a brief look at your code. The code is ready. Priority requests are already disabled, I'm just wondering whether to remove them from the code, or leave them there in case someone thinks they're useful. I suppose removing them is ok. > My only suggestion so far: the description could use more explicit > documentation on the various variables and macros and how they > combine. Is it before or after http://mail.python.org/pipermail/python-checkins/2009-November/087482.html ? Regards Antoine. From guido at python.org Sat Nov 7 20:33:13 2009 From: guido at python.org (Guido van Rossum) Date: Sat, 7 Nov 2009 11:33:13 -0800 Subject: [Python-Dev] Reworking the GIL In-Reply-To: <1257613317.3437.63.camel@localhost> References: <1256502140.5621.163.camel@localhost> <1257613317.3437.63.camel@localhost> Message-ID: On Sat, Nov 7, 2009 at 9:01 AM, Antoine Pitrou wrote: > > Hello Guido, > >> How close are you to merging this into the Py3k branch? It looks like >> a solid piece of work, that can only get better in the period between >> now and the release of 3.2. But I don't want to rush you, and I only >> have had a brief look at your code. > > The code is ready. Priority requests are already disabled, I'm just > wondering whether to remove them from the code, or leave them there in > case someone thinks they're useful. I suppose removing them is ok. I would remove them -- if and when we find there's a need for something like them I suspect it won't look like what you currently have, so it's better not to complexificate your current patch with them. (I would remove all traces, including the conditions passed into the end macros.) >> My only suggestion so far: the description could use more explicit >> documentation on the various variables and macros and how they >> combine. > > Is it before or after > http://mail.python.org/pipermail/python-checkins/2009-November/087482.html ? After. While that is already really helpful, not all the code is easily linked back to paragraphs in that comment block, and some variables are not mentioned by name in the block. -- --Guido van Rossum (python.org/~guido) From baptiste.lepilleur at gmail.com Sat Nov 7 20:37:30 2009 From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur) Date: Sat, 7 Nov 2009 20:37:30 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: References: <1256502140.5621.163.camel@localhost> Message-ID: 2009/11/7 Antoine Pitrou > > [...] > So, to sum it up, the way the current GIL manages to have good latencies is > by > issueing an unreasonable number of system calls on a contended lock, and > potentially killing throughput performance (this depends on the OS too, > because > numbers under Linux are not so catastrophic). > Ah, I remember reading this in the analysis that was published now! I made another benchmark using one of my script I ported to python 3 (and simplified a bit). I only test the total execution time. Tests done on Windows XP SP3. Processor is an Intel Core 2 Quad Q9300 (4 cores). You can get the script from: http://gaiacrtn.free.fr/py/benchmark-kwd-newgil/purekeyword-py26-3k.py Script + test doc (940KB): http://gaiacrtn.free.fr/py/benchmark-kwd-newgil/benchmark-kwd-newgil.tar.bz2 The threaded loop is: for match in self.punctuation_pattern.finditer( document ): word = document[last_start_index:match.start()] if len(word) > 1 and len(word) < MAX_KEYWORD_LENGTH: words[word] = words.get(word, 0) + 1 last_start_index = match.end() if word: word_count += 1 Here are the results: -j0 (main thread only) 2.5.2 : 17.991s, 17.947s, 17.780s 2.6.2 : 19.071s, 19.023s, 19.054s 3.1.1 : 46.384s, 46.321s, 46.425s newgil: 47.483s, 47.605s, 47.512s -j4 (4 consumer threads, main thread producing/waiting) 2.5.2 : 31.105s, 30.568s 2.6.2 : 31.550s, 30.599s 3.1.1 : 85.114s, 85.185s newgil: 48.428s, 49.217s It shows that, on my platform for this specific benchmark: - newgil manage to leverage a significant amount of parallelism (1.7) where python 3.1 does not (3.1 is 80% slower) - newgil as a small impact on non multi-threaded execution (~1-2%) [may be worth investigating] - 3.1 is more than 2 times slower than python 2.6 on this benchmark - 2.6 is "only" 65% slower when run with multiple threads compared to the 80% slower of 3.1. Newgil is a vast improvement as it manages to leverage the short time the GIL is released by finditer [if I understood correctly in 3.x regex release the GIL]. What's worry me is the single threaded performance degradation between 2.6 and 3.1 on this test. Could the added GIL release/acquire on each finditer call explain this? -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Sat Nov 7 20:56:00 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 07 Nov 2009 20:56:00 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: References: <1256502140.5621.163.camel@localhost> Message-ID: <1257623760.3437.72.camel@localhost> Hello again, > It shows that, on my platform for this specific benchmark: > * newgil manage to leverage a significant amount of parallelism > (1.7) where python 3.1 does not (3.1 is 80% slower) I think you are mistaken: -j0 (main thread only) newgil: 47.483s, 47.605s, 47.512s -j4 (4 consumer threads, main thread producing/waiting) newgil: 48.428s, 49.217s The runtimes are actually the same, so newgil doesn't leverage anything. However, it doesn't degrade performance like 2.x/3.1 does :-) > * newgil as a small impact on non multi-threaded execution > (~1-2%) [may be worth investigating] It goes from very slightly slower to very slightly faster and it is likely to be caused by variations in generated output from the compiler. > * 3.1 is more than 2 times slower than python 2.6 on this > benchmark That's the most worrying outcome I'd say. Are you sure the benchmark really does the same thing? Under 2.6, you should add re.UNICODE to the regular expression flags so as to match the 3.x semantics. > [if I understood correctly in 3.x regex release the GIL]. Unless I've missed something it doesn't, no. This could be a separate opportunity for optimization, if someone wants to take a look at it. Regards Antoine. From steve at pearwood.info Sun Nov 8 01:14:59 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 8 Nov 2009 11:14:59 +1100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: Message-ID: <200911081115.01803.steve@pearwood.info> On Fri, 6 Nov 2009 09:05:17 am Guido van Rossum wrote: > On Tue, Nov 3, 2009 at 9:35 AM, Guido van Rossum wrote: > > I've checked draft (!) PEP 3003, "Python Language Moratorium", into > > SVN. As authors I've listed Jesse, Brett and myself. > > I haven't seen substantial opposition against the PEP -- in fact I > can't recall any, and many people have explicitly posted in support > of it. So unless opposition suddenly appears in the next few days, > I'll move it to the Accepted state next Monday. I'm not a core developer, but I have made some objections on the python-ideas list. For what it is worth, I don't think a moratorium is needed, and it is not cost free. Whether the cost is worth it is a matter of opinion, but I've read posts suggesting that there's no downside to a moratorium. I think that's naive. Firstly, the core developers are very conservative in the features they add to the language. Nobody suggests that will change -- there are (as far as I know) a grand total of *one* proposed language changes under consideration for 3.2, namely "yield from". It isn't like the moratorium is needed to stem a flood of vast numbers of incompatible language changes, and this is not likely to change. The given reason for the moratorium is to give the developers of other Python implementations a chance to catch up. But as far as I know -- and I welcome correction -- only one such developer has spoken up, and he says that new language features generally aren't a problem for IronPython, but some standard library features are: http://mail.python.org/pipermail/python-ideas/2009-October/006328.html (In fairness, I should point out that Dino did not vote against the moratorium, he gave a conditional "in favour" vote but asked for some wiggle-room. A moratorium with wiggle-room is not a moratorium.) A moratorium isn't cost-free. With the back-end free to change, patches will go stale over 2+ years. People will lose interest or otherwise move on. Those with good ideas but little patience will be discouraged. I fully expect that, human nature being as it is, those proposing a change, good or bad, will be told not to bother wasting their time, there's a moratorium on at least as often as they'll be encouraged to bide their time while the moratorium is on. A moratorium turns Python's conservativeness up to 11. If Python already has a reputation for being conservative in the features it accepts -- and I think it does -- then a moratorium risks giving the impression that Python has become the language of choice for old guys sitting on their porch yelling at the damn kids to get off the lawn. That's a plus for Cobol. I don't think it is a plus for Python. What one person sees as "stable", another may see as "moribund". Perception does matter. Nick Coglan spoke of Jython having "fell by the wayside" when CPython introduced new style classes: http://mail.python.org/pipermail/python-ideas/2009-October/006431.html I think that speaks to the opposite conclusion that Nick intended. If developers craved stability, and were put off by the pace of changes to CPython, why weren't they migrating to Jython, which had one new production release in six years? http://www.jython.org/archive/22/news.html Nick also described C as changing much more slowly over its life that Python has. This is misleading: it is true that the C standard has been stable, but that doesn't mean that C compilers have been. Changes to the standard were driven by changes introduced by the implementations, not visa versa. Likewise for Cobol, where the first implementation to introduce OOP was released in 1997, but the standard wasn't updated until 2002. Python is nothing like that. Python doesn't have an official standard, and Guido has dismissed the need for an ISO standard. The other implementations have historically let CPython lead as far as language evolution goes. If CPython stops, likely Python stops. Who is going to risk adding language features that break compatibility with the most popular implementation? It won't be Python anymore. The PEP says the moratorium will last "at least" two years from the release of 3.1. Given the normal slow pace of new language features, two years isn't bad -- that's effectively just 3.2. But Guido has suggested it could last to 3.3. Could it last beyond that? 3.4? Until some distant future Python-4000? At the very least, I believe, any moratorium should have a clear end date. A clear end date will be a powerful counter to the impression that Python the language is moribund. It says, this is an exceptional pause, not a permanent halt. -- Steven D'Aprano From steve at pearwood.info Sun Nov 8 01:16:47 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 8 Nov 2009 11:16:47 +1100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <200911061618.38790.steve@pearwood.info> Message-ID: <200911081116.47419.steve@pearwood.info> On Fri, 6 Nov 2009 08:46:00 pm Willem Broekema wrote: > CLPython is in steady development, quite complete and stable on the > language level (somewhere between 2.5 and 2.6), but missing most > built-in library functionality. (It reuses the pure-Python parts of > the stdlib.) > > As its developer, I don't care much about being mentioned in > discussions or presentations, as CLPython is not as widely used as > IronPython and Jython. But in a PEP like this, which is directly > about alternative implementations, it deserves to be mentioned, IMHO. Willem, the rationale for this PEP is to give alternative implementations the chance to catch up with CPython. Given your statement that CLPython is quite complete on the language level, but missing standard library features, how do you think the moratorium will help CLPython? -- Steven D'Aprano From john.arbash.meinel at gmail.com Sun Nov 8 01:39:18 2009 From: john.arbash.meinel at gmail.com (John Arbash Meinel) Date: Sat, 07 Nov 2009 18:39:18 -0600 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <200911081115.01803.steve@pearwood.info> References: <200911081115.01803.steve@pearwood.info> Message-ID: <4AF61336.5010004@gmail.com> ... > A moratorium isn't cost-free. With the back-end free to change, patches > will go stale over 2+ years. People will lose interest or otherwise > move on. Those with good ideas but little patience will be discouraged. > I fully expect that, human nature being as it is, those proposing a > change, good or bad, will be told not to bother wasting their time, > there's a moratorium on at least as often as they'll be encouraged to > bide their time while the moratorium is on. > I believe if you go back to the very beginning of this thread, Guido considers this a *feature* not a *bug*. He wanted to introduce a moratorium at least partially because he was tired of endless threads about anonymous code blocks, etc. Which aren't going to be included in the language anyway, so he may as well make a point to say "and neither will anything else for a while". I don't mean to put words into his mouth, so please correct me if I'm wrong. John =:-> From metawilm at gmail.com Sun Nov 8 02:03:50 2009 From: metawilm at gmail.com (Willem Broekema) Date: Sun, 8 Nov 2009 02:03:50 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <200911081116.47419.steve@pearwood.info> References: <200911061618.38790.steve@pearwood.info> <200911081116.47419.steve@pearwood.info> Message-ID: On Sun, Nov 8, 2009 at 1:16 AM, Steven D'Aprano wrote: > Willem, the rationale for this PEP is to give alternative > implementations the chance to catch up with CPython. > > Given your statement that CLPython is quite complete on the language > level, but missing standard library features, how do you think the > moratorium will help CLPython? It would be great to have a pause in Python-the-language, so that CLPython might at the end of the moratorium finally be up to date with the latest 2.x and 3.x features. Supporting standard libraries is a bigger challenge, as the ones written in C need rewriting. Whether or not there is a pause on that front does not matter too much to that situation. - Willem From baptiste.lepilleur at gmail.com Sun Nov 8 15:07:22 2009 From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur) Date: Sun, 8 Nov 2009 15:07:22 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: <1257623760.3437.72.camel@localhost> References: <1256502140.5621.163.camel@localhost> <1257623760.3437.72.camel@localhost> Message-ID: 2009/11/7 Antoine Pitrou > > Hello again, > > > It shows that, on my platform for this specific benchmark: > > * newgil manage to leverage a significant amount of parallelism > > (1.7) where python 3.1 does not (3.1 is 80% slower) > > I think you are mistaken: > > -j0 (main thread only) > newgil: 47.483s, 47.605s, 47.512s > -j4 (4 consumer threads, main thread producing/waiting) > newgil: 48.428s, 49.217s > > The runtimes are actually the same, so newgil doesn't leverage anything. > However, it doesn't degrade performance like 2.x/3.1 does :-) > Ooops, I was comparing to 3.1 -j4 times which make no sense. One would think I wanted to see that result since I though the GIL was released :/. This greatly reduce the interest of this benchmark... > > * 3.1 is more than 2 times slower than python 2.6 on this > > benchmark > > That's the most worrying outcome I'd say. Are you sure the benchmark > really does the same thing? Under 2.6, you should add re.UNICODE to the > regular expression flags so as to match the 3.x semantics. > I've tried, but there is no change in result (the regexp does not use \w & co but specify a lot unicode ranges). All strings are already of unicode type in 2.6. > > [if I understood correctly in 3.x regex release the GIL]. > > Unless I've missed something it doesn't, no. > Hmmm, I was confusing with other modules (bzip2 & hashlib?). Looking back at the result of your benchmark it's obvious. Is there a place where the list of functions releasing the GIL is available? I did not see anything in bz2.compress documentation. -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Sun Nov 8 15:17:48 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 8 Nov 2009 14:17:48 +0000 (UTC) Subject: [Python-Dev] Reworking the GIL References: <1256502140.5621.163.camel@localhost> <1257623760.3437.72.camel@localhost> Message-ID: Baptiste Lepilleur gmail.com> writes: > > I've tried, but there is no change in result (the regexp does not use \w & > co but specify a lot unicode ranges). All strings are already of unicode > type in 2.6. No they aren't. You should add "from __future__ import unicode_literals" at the start of your script and run it again. > Hmmm, I was confusing with other modules (bzip2 & hashlib?). Looking back at > the result of your benchmark it's obvious. Is there a place where the list of > functions releasing the GIL is available? I did not see anything in > bz2.compress documentation. No, there isn't. You'd have to test, or read the source code. But bz2 and zlib, for example, do release the GIL. Regards Antoine. From stefan-usenet at bytereef.org Sun Nov 8 17:26:48 2009 From: stefan-usenet at bytereef.org (Stefan Krah) Date: Sun, 8 Nov 2009 17:26:48 +0100 Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs In-Reply-To: <1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za> References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za> <1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za> Message-ID: <20091108162648.GA4099@mail.bytereef.org> Hi, in a (misguided) bugreport (http://bugs.python.org/issue7279) I was questioning the reasons for allowing NaN comparisons with == and != rather than raising InvalidOperation. I think two main issues emerge from the brief discussion: 1. Should the comparison operators follow the 'compare' function from the standard? The standard says: "An implementation may use this operation [compare] under the covers to implement a closed set of comparison operations (greater than, equal, etc.) if desired. It need not, in this case, expose the compare operation itself." So, I'd say that this supports following 'compare' as closely as possible. In fact the operators <, <=, >, >= already follow 'compare-signal' in their behavior (they raise for any NaN operand). 2. What is the use of == and != outside the decimal scope? Mark mentions that Python uses == to test for set and dict memberships, but that you cannot put decimal NaNs into sets: 'TypeError: Cannot hash a NaN value' I want to add that Decimal('NaN') == Decimal('NaN') gives False, which should somewhat limit the uses of == for NaNs outside the decimal realm anyway. Are there cases where == and != are actually needed to give a result for NaNs? Stefan Krah From martin at v.loewis.de Sun Nov 8 19:44:42 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 08 Nov 2009 19:44:42 +0100 Subject: [Python-Dev] Status of the Buildbot fleet and related bugs In-Reply-To: References: Message-ID: <4AF7119A.2040201@v.loewis.de> > JFTR, I didn't set up the IRC bot (I assume that credit goes to Martin, > even if it's only one line in the buildbot config :). I just tried to > get it to say something :) Yes, it was always "on". I don't use IRC regularly, so I don't know whether it's useful. Regards, Martin From python at rcn.com Sun Nov 8 19:53:10 2009 From: python at rcn.com (Raymond Hettinger) Date: Sun, 8 Nov 2009 10:53:10 -0800 Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za><1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za> <20091108162648.GA4099@mail.bytereef.org> Message-ID: <4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1> [Stefan Krah] > in a (misguided) bugreport (http://bugs.python.org/issue7279) I was > questioning the reasons for allowing NaN comparisons with == and != > rather than raising InvalidOperation. Do you have any actual use case issues or are these theoretical musings? I ask only because a good use case might suggest the best way to adapt the standard to the regular python api for equality/inequality operators. NaNs are odd ducks. They are unique in violating our basic notions of equality (any relation that is reflexsive, symmetric, and transitive). Once you use them in a context that goes beyond the decimal spec, it is no surprise that you run into difficulties where NaNs don't fit very well (because they violate basic assumptions and invariants in other code). > Are there cases where == and != are actually needed to give a result > for NaNs? I would say that anywhere someone needs the full behaviors specified by the standard, they need to use the actual compare() method which allows for a decimal context to be specified and allows for more than just a true/false return value (i.e. a NaN is a possible result). Raymond From martin at v.loewis.de Sun Nov 8 19:56:23 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 08 Nov 2009 19:56:23 +0100 Subject: [Python-Dev] OpenSSL vulnerability In-Reply-To: References: <40373.1257545902@parc.com> <20091106223634.3229.2049231320.divmod.xquotient.232@localhost.localdomain> Message-ID: <4AF71457.3050705@v.loewis.de> >> Also, for Python 2.5 and earlier, any SSL-based code is vulnerable to a MitM >> anyway, so this can only be an issue for code using the new APIs in Python >> 2.6. > > That's not going to stop the > wannabe-self-proclaimed-so-called-vulnerability-"experts" from whining > about Python not releasing updated binary distributions though. :-( The Windows binaries currently build with 0.9.8g. Since changing that would be a source code change (even though just a single line), I think a full source release would be necessary (most likely then for both 2.6 and 3.1). Regards, Martin From solipsis at pitrou.net Sun Nov 8 20:01:01 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 8 Nov 2009 19:01:01 +0000 (UTC) Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za> <1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za> <20091108162648.GA4099@mail.bytereef.org> Message-ID: Stefan Krah bytereef.org> writes: > > Are there cases where == and != are actually needed to give a result > for NaNs? It is a common expectation that == and != always succeed. They return True or False, but don't raise an exception even on unrelated operands: >>> b"a" == "a" False >>> "5" == 5 False >>> {} == 0.0 False >>> None == (lambda x: 1) False >>> int == max False The only place I know of where this expectation isn't met is when comparing "naive" and "timezone-aware" datetime objects, which raises a TypeError (IIRC). From tjreedy at udel.edu Sun Nov 8 20:33:24 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 08 Nov 2009 14:33:24 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <4AF61336.5010004@gmail.com> References: <200911081115.01803.steve@pearwood.info> <4AF61336.5010004@gmail.com> Message-ID: John Arbash Meinel wrote: > He wanted to introduce a moratorium at least partially because he was > tired of endless threads about anonymous code blocks, etc. Which aren't > going to be included in the language anyway, so he may as well make a > point to say "and neither will anything else for a while". I have not said anything on this topic either way. But I think a general moratorium is a false solution to that particular problem. Guido should just make a clear pronouncement (again) and let people quote that in response to initial posts, before they *appear* to gain steam. Each time Python doubles in popularity, the number of people wanting such things will approximately double. That is the price of success. Terry Jan Reedy From rdmurray at bitdance.com Sun Nov 8 20:59:05 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Sun, 8 Nov 2009 14:59:05 -0500 (EST) Subject: [Python-Dev] Status of the Buildbot fleet and related bugs In-Reply-To: <4AF7119A.2040201@v.loewis.de> References: <4AF7119A.2040201@v.loewis.de> Message-ID: On Sun, 8 Nov 2009 at 19:44, "Martin v. L?wis" wrote: >> JFTR, I didn't set up the IRC bot (I assume that credit goes to Martin, >> even if it's only one line in the buildbot config :). I just tried to >> get it to say something :) > > Yes, it was always "on". I don't use IRC regularly, so I don't know > whether it's useful. I think it is. --David (RDM) From greg at krypto.org Sun Nov 8 21:56:34 2009 From: greg at krypto.org (Gregory P. Smith) Date: Sun, 8 Nov 2009 12:56:34 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <9d153b7c0911051613o62be5dc9l52102f7805cd4a0e@mail.gmail.com> References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com> <9d153b7c0911051613o62be5dc9l52102f7805cd4a0e@mail.gmail.com> Message-ID: <52dc1c820911081256q7fc3f7fdk87b2f12932dce11d@mail.gmail.com> On Thu, Nov 5, 2009 at 4:13 PM, Yuvgoog Greenle wrote: > On Fri, Nov 6, 2009 at 1:55 AM, Bobby R. Ward wrote: >> >> A switch to ENABLE those warnings? >> > > I think a few people I know would still be raising strings like exceptions > if not for the deprecation warnings. Most people won't turn on the switch > with the extra warnings. > --yuv Yep and their code will break in the future as a result. Thats their problem (or whoever employed them and neglected to do code reviews and beat them with a stick for using a Python 1.x "feature" in 2009). SUGGESTION: Our Whats New document currently has a single "new, improved and deprecated" section in it. We should make Deprecations its own section so that they're easy to find all in one concise place when upgrading code between releases. Having seen the fallout from adding deprecating warnings to code: many annoyed developers whos workflow you've interrupted who will all eventually turn the warning off by default at all times. I have to agree. The DeprecationWarning being on by default is a failed experiement. We should treat it and the rarely noticed PendingDeprecationWarning as the same thing by default. -gps From g.brandl at gmx.net Sun Nov 8 22:25:38 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 08 Nov 2009 22:25:38 +0100 Subject: [Python-Dev] OpenSSL vulnerability In-Reply-To: References: <40373.1257545902@parc.com> <20091106223634.3229.2049231320.divmod.xquotient.232@localhost.localdomain> Message-ID: Guido van Rossum schrieb: > On Fri, Nov 6, 2009 at 2:36 PM, wrote: >> Also, for Python 2.5 and earlier, any SSL-based code is vulnerable to a MitM >> anyway, so this can only be an issue for code using the new APIs in Python >> 2.6. > > That's not going to stop the > wannabe-self-proclaimed-so-called-vulnerability-"experts" from whining > about Python not releasing updated binary distributions though. :-( Yet it has been quiet on the Finnish front so far :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From greg.ewing at canterbury.ac.nz Sun Nov 8 22:28:35 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 09 Nov 2009 10:28:35 +1300 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <4AF61336.5010004@gmail.com> References: <200911081115.01803.steve@pearwood.info> <4AF61336.5010004@gmail.com> Message-ID: <4AF73803.9000303@canterbury.ac.nz> John Arbash Meinel wrote: > He wanted to introduce a moratorium at least partially because he was > tired of endless threads about anonymous code blocks, etc. Which aren't > going to be included in the language anyway, so he may as well make a > point to say "and neither will anything else for a while". If anonymous code blocks still get discussed even when they have no chance of being accepted, this suggests that a moratorium is *not* going to stop discussion of new features. I'm a bit confused about Guido's reasons for wanting a moratorium. He started out by saying that the intention wasn't to stop people discussing ideas, then later he seemed to say that he was finding all these discussions were distracting, then he explicitly said he wanted to "crush" people's hopes of getting new features into the language. That all sounds like he actually *does* want to discourage such discussions. -- Greg From g.brandl at gmx.net Sun Nov 8 22:27:46 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 08 Nov 2009 22:27:46 +0100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <52dc1c820911081256q7fc3f7fdk87b2f12932dce11d@mail.gmail.com> References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com> <9d153b7c0911051613o62be5dc9l52102f7805cd4a0e@mail.gmail.com> <52dc1c820911081256q7fc3f7fdk87b2f12932dce11d@mail.gmail.com> Message-ID: Gregory P. Smith schrieb: > On Thu, Nov 5, 2009 at 4:13 PM, Yuvgoog Greenle wrote: >> On Fri, Nov 6, 2009 at 1:55 AM, Bobby R. Ward wrote: >>> >>> A switch to ENABLE those warnings? >>> >> >> I think a few people I know would still be raising strings like exceptions >> if not for the deprecation warnings. Most people won't turn on the switch >> with the extra warnings. >> --yuv > > Yep and their code will break in the future as a result. Thats their > problem (or whoever employed them and neglected to do code reviews and > beat them with a stick for using a Python 1.x "feature" in 2009). > > SUGGESTION: Our Whats New document currently has a single "new, > improved and deprecated" section in it. We should make Deprecations > its own section so that they're easy to find all in one concise place > when upgrading code between releases. Good point, I'll make that change if AMK agrees. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From solipsis at pitrou.net Sun Nov 8 22:38:04 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 8 Nov 2009 21:38:04 +0000 (UTC) Subject: [Python-Dev] PEP 3003 - Python Language Moratorium References: <200911081115.01803.steve@pearwood.info> <4AF61336.5010004@gmail.com> <4AF73803.9000303@canterbury.ac.nz> Message-ID: Greg Ewing canterbury.ac.nz> writes: > > If anonymous code blocks still get discussed even when > they have no chance of being accepted, this suggests that > a moratorium is *not* going to stop discussion of new > features. Well, if they get discussed, it's probably that some people can't help participating in the discussions, even though they know the idea is doomed to failure ;) From steve at pearwood.info Mon Nov 9 00:06:25 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 9 Nov 2009 10:06:25 +1100 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <200911081115.01803.steve@pearwood.info> References: <200911081115.01803.steve@pearwood.info> Message-ID: <200911091006.26433.steve@pearwood.info> On Sun, 8 Nov 2009 11:14:59 am Steven D'Aprano wrote: > At the very least, I believe, any moratorium should have a clear end > date. A clear end date will be a powerful counter to the impression > that Python the language is moribund. It says, this is an exceptional > pause, not a permanent halt. Proposal: No new language features in odd-numbered point releases (3.3, 3.5, ...). Even-numbered point releases (3.4, 3.6, ...) may include new language features provided they meet the usual standards for new features. 3.2 is a special case: as an even-numbered release, it would normally allow new features, but in recognition of the special nature of the 2.x to 3.1/3.2 migration, no new language features will be allowed. Advantages: * It slows down changes to the language while still allowing sufficiently high-standard new features. * Alternate implementations have a stable language version to aim for. Assuming point releases come ever 12-18 months, that stable language version will last 2-3 years. * It doesn't have the psychological baggage of an unconditional ban on new features for the indefinite future. It gives a fixed, known schedule for when new features will be permitted, without the uncertainty of "at the BDFL's pleasure". -- Steven D'Aprano From debatem1 at gmail.com Mon Nov 9 01:01:26 2009 From: debatem1 at gmail.com (geremy condra) Date: Sun, 8 Nov 2009 19:01:26 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <200911091006.26433.steve@pearwood.info> References: <200911081115.01803.steve@pearwood.info> <200911091006.26433.steve@pearwood.info> Message-ID: On Sun, Nov 8, 2009 at 6:06 PM, Steven D'Aprano wrote: > On Sun, 8 Nov 2009 11:14:59 am Steven D'Aprano wrote: >> At the very least, I believe, any moratorium should have a clear end >> date. A clear end date will be a powerful counter to the impression >> that Python the language is moribund. It says, this is an exceptional >> pause, not a permanent halt. > > Proposal: > > No new language features in odd-numbered point releases (3.3, 3.5, ...). > Even-numbered point releases (3.4, 3.6, ...) may include new language > features provided they meet the usual standards for new features. > > 3.2 is a special case: as an even-numbered release, it would normally > allow new features, but in recognition of the special nature of the 2.x > to 3.1/3.2 migration, no new language features will be allowed. > > Advantages: > > * It slows down changes to the language while still allowing > sufficiently high-standard new features. > > * Alternate implementations have a stable language version to aim for. > Assuming point releases come ever 12-18 months, that stable language > version will last 2-3 years. > > * It doesn't have the psychological baggage of an unconditional ban on > new features for the indefinite future. It gives a fixed, known > schedule for when new features will be permitted, without the > uncertainty of "at the BDFL's pleasure". > > > -- > Steven D'Aprano FWIW, I view a definite end point as a definite plus. Geremy Condra From jnoller at gmail.com Mon Nov 9 02:22:52 2009 From: jnoller at gmail.com (Jesse Noller) Date: Sun, 8 Nov 2009 20:22:52 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <200911081115.01803.steve@pearwood.info> <200911091006.26433.steve@pearwood.info> Message-ID: On Nov 8, 2009, at 7:01 PM, geremy condra wrote: > On Sun, Nov 8, 2009 at 6:06 PM, Steven D'Aprano > wrote: >> On Sun, 8 Nov 2009 11:14:59 am Steven D'Aprano wrote: >>> At the very least, I believe, any moratorium should have a clear end >>> date. A clear end date will be a powerful counter to the impression >>> that Python the language is moribund. It says, this is an >>> exceptional >>> pause, not a permanent halt. >> >> Proposal: >> >> No new language features in odd-numbered point releases (3.3, >> 3.5, ...). >> Even-numbered point releases (3.4, 3.6, ...) may include new language >> features provided they meet the usual standards for new features. >> >> 3.2 is a special case: as an even-numbered release, it would normally >> allow new features, but in recognition of the special nature of the >> 2.x >> to 3.1/3.2 migration, no new language features will be allowed. >> >> Advantages: >> >> * It slows down changes to the language while still allowing >> sufficiently high-standard new features. >> >> * Alternate implementations have a stable language version to aim >> for. >> Assuming point releases come ever 12-18 months, that stable language >> version will last 2-3 years. >> >> * It doesn't have the psychological baggage of an unconditional ban >> on >> new features for the indefinite future. It gives a fixed, known >> schedule for when new features will be permitted, without the >> uncertainty of "at the BDFL's pleasure". >> >> >> -- >> Steven D'Aprano > > > FWIW, I view a definite end point as a definite plus. > > Geremy Condra > There is a time outlined in the pep. From skip at pobox.com Mon Nov 9 02:41:24 2009 From: skip at pobox.com (skip at pobox.com) Date: Sun, 08 Nov 2009 19:41:24 -0600 Subject: [Python-Dev] raw binary data and 2to3 Message-ID: SpamBayes has several files which contain raw 8-bit data embedded in string literals. Before I do manual work to make them parseable by 2to3 I thought I would ask if there was either a fixer available which I'm not getting by default or if there is an opportunity to add a new fixer to 2to3. The usage is pretty straightforward. For example, a string literal might contain the bytes for a GIF image: data = "GIF89a(..." Is there a potentially automated path from where the code is today to something Python 3 (and 2to3) will like? Skip From debatem1 at gmail.com Mon Nov 9 02:45:56 2009 From: debatem1 at gmail.com (geremy condra) Date: Sun, 8 Nov 2009 20:45:56 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <200911081115.01803.steve@pearwood.info> <200911091006.26433.steve@pearwood.info> Message-ID: On Sun, Nov 8, 2009 at 8:22 PM, Jesse Noller wrote: > > > On Nov 8, 2009, at 7:01 PM, geremy condra wrote: > >> On Sun, Nov 8, 2009 at 6:06 PM, Steven D'Aprano >> wrote: >>> >>> On Sun, 8 Nov 2009 11:14:59 am Steven D'Aprano wrote: >>>> >>>> At the very least, I believe, any moratorium should have a clear end >>>> date. A clear end date will be a powerful counter to the impression >>>> that Python the language is moribund. It says, this is an exceptional >>>> pause, not a permanent halt. >>> >>> Proposal: >>> >>> No new language features in odd-numbered point releases (3.3, 3.5, ...). >>> Even-numbered point releases (3.4, 3.6, ...) may include new language >>> features provided they meet the usual standards for new features. >>> >>> 3.2 is a special case: as an even-numbered release, it would normally >>> allow new features, but in recognition of the special nature of the 2.x >>> to 3.1/3.2 migration, no new language features will be allowed. >>> >>> Advantages: >>> >>> * It slows down changes to the language while still allowing >>> sufficiently high-standard new features. >>> >>> * Alternate implementations have a stable language version to aim for. >>> Assuming point releases come ever 12-18 months, that stable language >>> version will last 2-3 years. >>> >>> * It doesn't have the psychological baggage of an unconditional ban on >>> new features for the indefinite future. It gives a fixed, known >>> schedule for when new features will be permitted, without the >>> uncertainty of "at the BDFL's pleasure". >>> >>> >>> -- >>> Steven D'Aprano >> >> >> FWIW, I view a definite end point as a definite plus. >> >> Geremy Condra >> > > There is a time outlined in the pep. > I quote: "This PEP proposes a temporary moratorium (suspension) of all changes to the Python language syntax, semantics, and built-ins for a period of *at least two years* from the release of Python 3.1." Emphasis mine. Like I say, a definite end point would be much preferred to n > 2. If possible, I'd also like to hear some of Steven's other points addressed. Geremy Condra From benjamin at python.org Mon Nov 9 02:48:46 2009 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 8 Nov 2009 19:48:46 -0600 Subject: [Python-Dev] raw binary data and 2to3 In-Reply-To: References: Message-ID: <1afaf6160911081748u3f47f5dche74b1f0171c5bcff@mail.gmail.com> 2009/11/8 : > > SpamBayes has several files which contain raw 8-bit data embedded in > string literals. ?Before I do manual work to make them parseable by 2to3 > I thought I would ask if there was either a fixer available which I'm > not getting by default or if there is an opportunity to add a new fixer > to 2to3. > > The usage is pretty straightforward. ?For example, a string literal > might contain the bytes for a GIF image: > > ? ?data = "GIF89a(..." > > Is there a potentially automated path from where the code is today to > something Python 3 (and 2to3) will like? Not to my knowledge. I would prefer to not add a fixer for this directly to 2to3 because it is not correct for most programs. However, I think 2to3 should grow some sort of plugin system, so custom fixers can easily be written and used. -- Regards, Benjamin From guido at python.org Mon Nov 9 03:35:53 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 8 Nov 2009 18:35:53 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <200911091006.26433.steve@pearwood.info> References: <200911081115.01803.steve@pearwood.info> <200911091006.26433.steve@pearwood.info> Message-ID: On Sun, Nov 8, 2009 at 3:06 PM, Steven D'Aprano wrote: > No new language features in odd-numbered point releases (3.3, 3.5, ...). > Even-numbered point releases (3.4, 3.6, ...) may include new language > features provided they meet the usual standards for new features. Oh no, not the eve/odd numbering scheme. Nobody will be able to remember which is which. -- --Guido van Rossum (python.org/~guido) From guido at python.org Mon Nov 9 03:41:32 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 8 Nov 2009 18:41:32 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <200911081115.01803.steve@pearwood.info> <200911091006.26433.steve@pearwood.info> Message-ID: On Sun, Nov 8, 2009 at 5:45 PM, geremy condra wrote: > I quote: > > "This PEP proposes a temporary moratorium (suspension) of all changes > to the Python language syntax, semantics, and built-ins for a period > of *at least two years* from the release of Python 3.1." > > Emphasis mine. I since added this: """In particular, the moratorium would include Python 3.2 (to be released 18-24 months after 3.1) but (unless explicitly extended) allow Python 3.3 to once again include language changes.""" > Like I say, a definite end point would be much preferred to n > 2. My time machine doesn't work very well in the future. So I can't tell what we'll find necessary 2 years from now. But I would be fine with defining the time limit to be max(time(3.1) + 2 years, time(3.2)). I.e. the moratorium (unless explicitly extended) ends as soon as 3.2 has been released *and* at least 2 years have passed since 3.1. > If possible, I'd also like to hear some of Steven's other points addressed. They haven't changed my mind. -- --Guido van Rossum (python.org/~guido) From guido at python.org Mon Nov 9 03:42:47 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 8 Nov 2009 18:42:47 -0800 Subject: [Python-Dev] raw binary data and 2to3 In-Reply-To: <1afaf6160911081748u3f47f5dche74b1f0171c5bcff@mail.gmail.com> References: <1afaf6160911081748u3f47f5dche74b1f0171c5bcff@mail.gmail.com> Message-ID: On Sun, Nov 8, 2009 at 5:48 PM, Benjamin Peterson wrote: > 2009/11/8 ?: >> >> SpamBayes has several files which contain raw 8-bit data embedded in >> string literals. ?Before I do manual work to make them parseable by 2to3 >> I thought I would ask if there was either a fixer available which I'm >> not getting by default or if there is an opportunity to add a new fixer >> to 2to3. >> >> The usage is pretty straightforward. ?For example, a string literal >> might contain the bytes for a GIF image: >> >> ? ?data = "GIF89a(..." >> >> Is there a potentially automated path from where the code is today to >> something Python 3 (and 2to3) will like? > > Not to my knowledge. I would prefer to not add a fixer for this > directly to 2to3 because it is not correct for most programs. However, > I think 2to3 should grow some sort of plugin system, so custom fixers > can easily be written and used. But if you're happy with only supporting 2.6, you can use b"..." and the right thing will happen. -- --Guido van Rossum (python.org/~guido) From skip at pobox.com Mon Nov 9 03:55:25 2009 From: skip at pobox.com (skip at pobox.com) Date: Sun, 8 Nov 2009 20:55:25 -0600 Subject: [Python-Dev] raw binary data and 2to3 In-Reply-To: References: <1afaf6160911081748u3f47f5dche74b1f0171c5bcff@mail.gmail.com> Message-ID: <19191.33949.443509.601813@montanaro.dyndns.org> Guido> But if you're happy with only supporting 2.6, you can use b"..." and Guido> the right thing will happen. SpamBayes still supports 2.4... Thanks for the feedback. I'll update the source manually, then run 2to3. S From scott+python-dev at scottdial.com Mon Nov 9 03:57:28 2009 From: scott+python-dev at scottdial.com (Scott Dial) Date: Sun, 08 Nov 2009 21:57:28 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <200911081115.01803.steve@pearwood.info> <200911091006.26433.steve@pearwood.info> Message-ID: <4AF78518.1040709@scottdial.com> Guido van Rossum wrote: > On Sun, Nov 8, 2009 at 3:06 PM, Steven D'Aprano wrote: >> No new language features in odd-numbered point releases (3.3, 3.5, ...). >> Even-numbered point releases (3.4, 3.6, ...) may include new language >> features provided they meet the usual standards for new features. > > Oh no, not the eve/odd numbering scheme. Nobody will be able to > remember which is which. > In this case, does it really matter? Usually the confusion is with stable/unstable being even/odd or odd/even, but in this case the options are stable/stable (it just happens that certain pairwise versions are related). -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From debatem1 at gmail.com Mon Nov 9 04:50:12 2009 From: debatem1 at gmail.com (geremy condra) Date: Sun, 8 Nov 2009 22:50:12 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <200911081115.01803.steve@pearwood.info> <200911091006.26433.steve@pearwood.info> Message-ID: On Sun, Nov 8, 2009 at 9:41 PM, Guido van Rossum wrote: > On Sun, Nov 8, 2009 at 5:45 PM, geremy condra wrote: >> I quote: >> >> "This PEP proposes a temporary moratorium (suspension) of all changes >> to the Python language syntax, semantics, and built-ins for a period >> of *at least two years* from the release of Python 3.1." >> >> Emphasis mine. > > I since added this: > > """In particular, > the moratorium would include Python 3.2 (to be released 18-24 months > after 3.1) but (unless explicitly extended) allow Python 3.3 to once > again include language changes.""" > >> Like I say, a definite end point would be much preferred to n > 2. > > My time machine doesn't work very well in the future. So I can't tell > what we'll find necessary 2 years from now. But I would be fine with > defining the time limit to be max(time(3.1) + 2 years, time(3.2)). > I.e. the moratorium (unless explicitly extended) ends as soon as 3.2 > has been released *and* at least 2 years have passed since 3.1. Ok, thanks for the clarification. Could you spell out what you would consider grounds for a future extension? >> If possible, I'd also like to hear some of Steven's other points addressed. > > They haven't changed my mind. Ok, but the fact that you (or Steven) hold a particular set of beliefs is a singularly unconvincing argument. Could you explain why you don't agree, if only for the record? Geremy Condra From robert.kern at gmail.com Mon Nov 9 05:20:35 2009 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 08 Nov 2009 22:20:35 -0600 Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs In-Reply-To: References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za> <1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za> <20091108162648.GA4099@mail.bytereef.org> Message-ID: Antoine Pitrou wrote: > Stefan Krah bytereef.org> writes: >> Are there cases where == and != are actually needed to give a result >> for NaNs? > > It is a common expectation that == and != always succeed. They return True or > False, but don't raise an exception even on unrelated operands: It is a common expectation, but a false one. __eq__ and __ne__ are explicitly allowed to return anything, not just bools. http://www.python.org/dev/peps/pep-0207/ -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From nnorwitz at gmail.com Mon Nov 9 06:53:40 2009 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 8 Nov 2009 21:53:40 -0800 Subject: [Python-Dev] Status of the Buildbot fleet and related bugs In-Reply-To: <5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com> References: <5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com> Message-ID: On Fri, Nov 6, 2009 at 12:27 AM, Mark Dickinson wrote: > On Fri, Nov 6, 2009 at 3:53 AM, R. David Murray wrote: > >> ? ?(2) issue 4970: consistent signal 32 error on the norwitz-x86 Gentoo >> ? ? ? ?buildslave in 3.1 and 3.x. ?This may be due to the box >> ? ? ? ?running an old threading library, but it does make one wonder what >> ? ? ? ?changed in 3.x that exposed it. > > This error has been happening since well before 3.0 was released. ?Asking > for access to Neal's machine is probably the only sensible way to diagnose > it. (A less invasive but slower way to debug would be to create a branch > especially for this bug and do repeated runs to figure out which part of test_os > is causing the failure.) IIRC, I spent quite a bit of time trying to nail this down. I don't remember finding any useful information on the cause (beyond narrowing it to some tests). As Mark said, this has been happening for a long time. I'm reticent to provide access to the machine, as it's not really mine. I'm not even sure I have access, I haven't logged in for a long time. I'd just like to say thanks again to everyone for making the buildbots more green and also improving the general testing infrastructure for Python. n From techtonik at gmail.com Mon Nov 9 07:45:30 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Mon, 9 Nov 2009 08:45:30 +0200 Subject: [Python-Dev] IDLE as default Python editor (Was: [pygame] Python IDE for windoz) Message-ID: Hello, Quite an interesting question recently popped up in pygame community that I'd like to ask to Python developers. How many of you use IDLE? What's wrong with it? >From my side I like the idea of having default Python editor that is small, fast, convenient and extensible/embeddable. IDLE is small and fast, but I feel really uncomfortable with its. The worst thing - I can't change it. Does anybody else feel the same to think if we could replace IDLE with something more maintainable by providing Scintilla bindings, for example? Regards, -- anatoly t. On Sat, Oct 10, 2009 at 8:15 PM, RB[0] wrote: > Most? people just don't like it - personally it's all I use, Windows or > otherwise... > > On Sat, Oct 10, 2009 at 12:23 PM, Kris Schnee wrote: >> >> What's wrong with the built-in editor, IDLE? > > From ben+python at benfinney.id.au Mon Nov 9 08:04:40 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 09 Nov 2009 18:04:40 +1100 Subject: [Python-Dev] IDLE as default Python editor References: Message-ID: <87iqdk2n7b.fsf@benfinney.id.au> anatoly techtonik writes: > Quite an interesting question recently popped up in pygame community > that I'd like to ask to Python developers. This forum is specifically about development *of* Python. You would do better to ask on the discussion forum for Python users . Or, if you're interested in gauging interest in a new idea for changing Python, you could raise it on the ?python-ideas? forum . -- \ ?The generation of random numbers is too important to be left | `\ to chance.? ?Robert R. Coveyou | _o__) | Ben Finney From db3l.net at gmail.com Mon Nov 9 08:29:54 2009 From: db3l.net at gmail.com (David Bolen) Date: Mon, 09 Nov 2009 02:29:54 -0500 Subject: [Python-Dev] Updates to XP-4 and FreeBSD build slaves Message-ID: I think in a prior discussion, it was suggested that build slave updates were ok for this list - I apologize to those who may not be interested. I've just completed some updates to my two build slaves. XP-4 is now running XP Pro SP3 (was SP2) and is using the full version of VS 2008 (was previously using Express). FreeBSD is now running FreeBSD 6.4-RELEASE (was 6.2-RELEASE). Both are new VMs, and also now use the latest buildbot (0.7.11p3) release. For at least the next few days they are being run on a different host while I do some long-awaited maintenance to the old host. I've done manual builds on both, as well as reissuing a few builds through the master, but if anyone notices anything strange please feel free to drop me a note. -- David PS: I did discover one interesting bug by starting fresh on Windows. It turns out that one of the distutils tests assumes Notepad entries in the registry that aren't actually present unless you run Notepad at least once. I hadn't needed to run it, but then noticed the build failures so ran it just to get the registry populated. From martin at v.loewis.de Mon Nov 9 08:46:06 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 09 Nov 2009 08:46:06 +0100 Subject: [Python-Dev] raw binary data and 2to3 In-Reply-To: References: Message-ID: <4AF7C8BE.3090400@v.loewis.de> > data = "GIF89a(..." > > Is there a potentially automated path from where the code is today to > something Python 3 (and 2to3) will like? Not sure how you'll fix these; I personally always provided a b() function that will do the right thing in both 2.x and 3.x. This can get eventually replaced by a b prefix. Regards, Martin From martin at v.loewis.de Mon Nov 9 08:48:13 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 09 Nov 2009 08:48:13 +0100 Subject: [Python-Dev] IDLE as default Python editor (Was: [pygame] Python IDE for windoz) In-Reply-To: References: Message-ID: <4AF7C93D.2090505@v.loewis.de> > Quite an interesting question recently popped up in pygame community > that I'd like to ask to Python developers. > > How many of you use IDLE? I do, sometimes. > What's wrong with it? AFAICT, nothing. >>From my side I like the idea of having default Python editor that is > small, fast, convenient and extensible/embeddable. IDLE is small and > fast, but I feel really uncomfortable with its. The worst thing - I > can't change it. This I cannot understand. I have changed IDLE many times. Regards, Martin From tjreedy at udel.edu Mon Nov 9 09:25:42 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 09 Nov 2009 03:25:42 -0500 Subject: [Python-Dev] IDLE as default Python editor (Was: [pygame] Python IDE for windoz) In-Reply-To: References: Message-ID: anatoly techtonik wrote: > Hello, > > Quite an interesting question recently popped up in pygame community > that I'd like to ask to Python developers. > > How many of you use IDLE? I do, both shell and editor (for Python code). > What's wrong with it? See tracker. I agree that discussion of alternatives belongs elsewhere. tjr From martin at v.loewis.de Mon Nov 9 09:42:15 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 09 Nov 2009 09:42:15 +0100 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it In-Reply-To: References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <4AF341BD.2010203@v.loewis.de> <7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> Message-ID: <4AF7D5E7.4080901@v.loewis.de> > The problem arises because we're systematically unbalancing the hash > table. The iterator returns the first valid entry in the hash table, > which we remove. Repeat several times and pretty soon the iterator has > to spend O(n) time scanning through empty entries to find the first > remaining valid entry. Interesting. Something goes wrong, it seems: if items get removed over and over again, I think the set should shrink (not sure whether it actually does). Then, I think you should end up with an amortized O(1) for selecting an element (assuming that the underlying hashes don't collide). Regards, Martin From stefan-usenet at bytereef.org Mon Nov 9 11:42:49 2009 From: stefan-usenet at bytereef.org (Stefan Krah) Date: Mon, 9 Nov 2009 11:42:49 +0100 Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs In-Reply-To: <4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1> References: <20091108162648.GA4099@mail.bytereef.org> <4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1> Message-ID: <20091109104249.GA3377@mail.bytereef.org> Raymond Hettinger wrote: > [Stefan Krah] > >in a (misguided) bugreport (http://bugs.python.org/issue7279) I was > >questioning the reasons for allowing NaN comparisons with == and != > >rather than raising InvalidOperation. > > Do you have any actual use case issues or are these theoretical musings? > I ask only because a good use case might suggest the best way to adapt > the standard to the regular python api for equality/inequality operators. I think my reasoning goes the opposite way: The current behavior (raising InvalidOperation) of <, <=, >=, > is sensible and as close to the standard as one can get. This behavior was not chosen for the equality/inequality operators because they _might_ be used for other purposes. But since Decimal("NaN") == Decimal("NaN") gives False, these non-decimal use cases don't work: >>> d = {0:Decimal("NaN")} >>> Decimal("NaN") in d.values() False So, since non-decimal use cases are limited at best, the equality/inequality operators might as well have the behavior of the other comparison operators, which is safer for the user. I can also give a decimal use case where the current behavior is problematic A variable initialized to a signaling NaN should always cause an exception. But this doesn't: salary = Decimal("sNaN") minimum_wage = 1000 if (salary == minimum_wage): print "do stuff" else: print "do other stuff" Stefan Krah From solipsis at pitrou.net Mon Nov 9 12:21:27 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 9 Nov 2009 11:21:27 +0000 (UTC) Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs References: <20091108162648.GA4099@mail.bytereef.org> <4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1> <20091109104249.GA3377@mail.bytereef.org> Message-ID: Stefan Krah bytereef.org> writes: > > >>> d = {0:Decimal("NaN")} > >>> Decimal("NaN") in d.values() > False > > So, since non-decimal use cases are limited at best, the equality/inequality > operators might as well have the behavior of the other comparison operators, > which is safer for the user. The problem is when searching for /another/ object which hashes the same as Decimal("NaN"). Here is a made-up situation to show you the problem: >>> class H(object): ... def __hash__(self): return hash(1) ... def __eq__(self, other): raise ValueError ... >>> h = H() >>> d = {h: ""} >>> d[1] Traceback (most recent call last): File "", line 1, in File "", line 3, in __eq__ ValueError >>> d[1] = 2 Traceback (most recent call last): File "", line 1, in File "", line 3, in __eq__ ValueError From stefan-usenet at bytereef.org Mon Nov 9 13:21:59 2009 From: stefan-usenet at bytereef.org (Stefan Krah) Date: Mon, 9 Nov 2009 13:21:59 +0100 Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs In-Reply-To: References: <20091108162648.GA4099@mail.bytereef.org> <4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1> <20091109104249.GA3377@mail.bytereef.org> Message-ID: <20091109122159.GA4383@mail.bytereef.org> Antoine Pitrou wrote: > Stefan Krah bytereef.org> writes: > > > > >>> d = {0:Decimal("NaN")} > > >>> Decimal("NaN") in d.values() > > False > > > > So, since non-decimal use cases are limited at best, the equality/inequality > > operators might as well have the behavior of the other comparison operators, > > which is safer for the user. > > The problem is when searching for /another/ object which hashes the same as > Decimal("NaN"). Here is a made-up situation to show you the problem: > > >>> class H(object): > ... def __hash__(self): return hash(1) > ... def __eq__(self, other): raise ValueError > ... > >>> h = H() > >>> d = {h: ""} > >>> d[1] > Traceback (most recent call last): > File "", line 1, in > File "", line 3, in __eq__ > ValueError > >>> d[1] = 2 > Traceback (most recent call last): > File "", line 1, in > File "", line 3, in __eq__ > ValueError I see the point, but Decimal("NaN") does not hash: >>> hash(Decimal("NaN")) Traceback (most recent call last): File "", line 1, in File "/usr/lib/python2.7/decimal.py", line 937, in __hash__ raise TypeError('Cannot hash a NaN value.') TypeError: Cannot hash a NaN value. Also, NaNs cause problems in non-decimal comparisons virtually everywhere: >>> L = [1, 2, Decimal("NaN")] >>> L.sort() Traceback (most recent call last): File "", line 1, in File "/usr/lib/python2.7/decimal.py", line 877, in __lt__ ans = self._compare_check_nans(other, context) File "/usr/lib/python2.7/decimal.py", line 782, in _compare_check_nans self) File "/usr/lib/python2.7/decimal.py", line 3755, in _raise_error raise error(explanation) decimal.InvalidOperation: comparison involving NaN >>> I think problems like these would best be avoided by having a separate __totalorder__ or __lexorder__ method instead of using __eq__, __lt__, etc., but this is of course outside the scope of this discussion (and I have no idea how difficult it would be to implement that). Stefan Krah From solipsis at pitrou.net Mon Nov 9 13:34:54 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 9 Nov 2009 12:34:54 +0000 (UTC) Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs References: <20091108162648.GA4099@mail.bytereef.org> <4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1> <20091109104249.GA3377@mail.bytereef.org> <20091109122159.GA4383@mail.bytereef.org> Message-ID: Stefan Krah bytereef.org> writes: > > I see the point, but Decimal("NaN") does not hash: Ok but witness again: >>> L = [1, 2, Decimal("NaN"), 3] >>> 3 in L True >>> class H(object): ... def __eq__(self, other): raise ValueError ... >>> L = [1, 2, H(), 3] >>> 3 in L Traceback (most recent call last): File "", line 1, in File "", line 2, in __eq__ ValueError (NB: interestingly, float("nan") does hash) Regards Antoine. From ncoghlan at gmail.com Mon Nov 9 13:39:28 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 09 Nov 2009 22:39:28 +1000 Subject: [Python-Dev] Status of the Buildbot fleet and related bugs In-Reply-To: References: <5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com> Message-ID: <4AF80D80.9030506@gmail.com> Neal Norwitz wrote: > I'd just like to say thanks again to everyone for making the buildbots > more green and also improving the general testing infrastructure for > Python. I'm *really* liking the new assertions in unittest. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From dickinsm at gmail.com Mon Nov 9 13:40:06 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 9 Nov 2009 12:40:06 +0000 Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs In-Reply-To: <20091109122159.GA4383@mail.bytereef.org> References: <20091108162648.GA4099@mail.bytereef.org> <4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1> <20091109104249.GA3377@mail.bytereef.org> <20091109122159.GA4383@mail.bytereef.org> Message-ID: <5c6f2a5d0911090440nc9e0dedsdd46769ec8fca15b@mail.gmail.com> On Mon, Nov 9, 2009 at 12:21 PM, Stefan Krah wrote: > I see the point, but Decimal("NaN") does not hash: > >>>> hash(Decimal("NaN")) > Traceback (most recent call last): > ?File "", line 1, in > ?File "/usr/lib/python2.7/decimal.py", line 937, in __hash__ > ? ?raise TypeError('Cannot hash a NaN value.') > TypeError: Cannot hash a NaN value. I think that may represent an excess of caution. float nans do hash quite happily, and I can't see a good reason for preventing Decimal nans from having a hash. Mark From ncoghlan at gmail.com Mon Nov 9 13:43:31 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 09 Nov 2009 22:43:31 +1000 Subject: [Python-Dev] IDLE as default Python editor In-Reply-To: <87iqdk2n7b.fsf@benfinney.id.au> References: <87iqdk2n7b.fsf@benfinney.id.au> Message-ID: <4AF80E73.5090104@gmail.com> Ben Finney wrote: > anatoly techtonik writes: > >> Quite an interesting question recently popped up in pygame community >> that I'd like to ask to Python developers. > > This forum is specifically about development *of* Python. Anatoly's question is actually a fair one for python-dev - we're the ones that *ship* Idle, so it is legitimate to ask our reasons for continuing to do so. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Mon Nov 9 13:51:09 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 09 Nov 2009 22:51:09 +1000 Subject: [Python-Dev] IDLE as default Python editor (Was: [pygame] Python IDE for windoz) In-Reply-To: References: Message-ID: <4AF8103D.5050809@gmail.com> anatoly techtonik wrote: > Hello, > > Quite an interesting question recently popped up in pygame community > that I'd like to ask to Python developers. > > How many of you use IDLE? I use it if it's the only syntax highlighting Python editor on the box (since machines in the lab usually aren't set up with full dev environments). Otherwise I'm more likely to use a general syntax highlighting text editor (e.g. Kate on KDE) >>From my side I like the idea of having default Python editor that is > small, fast, convenient and extensible/embeddable. IDLE is small and > fast, but I feel really uncomfortable with its. The worst thing - I > can't change it. Does anybody else feel the same to think if we could > replace IDLE with something more maintainable by providing Scintilla > bindings, for example? I don't particularly *like* IDLE, but that's largely because I don't like Tcl/Tk. However, as far as cross-platform widget sets that are reasonable to bundle with Python go (both from a licensing and code size perspective), choices are pretty limited (plus Tcl/Tk has the advantage of incumbency). So while I can't claim to like Idle in and of itself, I do like the fact that the language comes with its own syntax highlighting editor. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From dickinsm at gmail.com Mon Nov 9 14:01:10 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 9 Nov 2009 13:01:10 +0000 Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs In-Reply-To: <20091108162648.GA4099@mail.bytereef.org> References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za> <1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za> <20091108162648.GA4099@mail.bytereef.org> Message-ID: <5c6f2a5d0911090501n23254cc7x8e1eaad34508f8fa@mail.gmail.com> On Sun, Nov 8, 2009 at 4:26 PM, Stefan Krah wrote: > Hi, > > in a (misguided) bugreport (http://bugs.python.org/issue7279) I was > questioning the reasons for allowing NaN comparisons with == and != > rather than raising InvalidOperation. Some quick recent history: For reference, the current behaviour dates from r60630. Before this, comparisons involving nans behaved even less coherently. See http://bugs.python.org/issue1979 for details. Apart from Python's use of __eq__, the other motivation for the current behaviour comes from the IEEE 854 standard; given the absence of helpful information in the Decimal standard, IEEE 854 is an obvious next place to look. There's an unofficial copy of the standard available at: http://754r.ucbtest.org/standards/854.pdf Section 5.7 describes twenty-six(!) distinct comparison operators. So it's not immediately clear which of those twenty-six comparison operators each of Python's six comparison operators should map to. However, in the standard, the first six operations in the table are somewhat distinguished: they're the ones that are marked as corresponding to the usual mathematical operations, and to Fortran's usual comparison operators (.EQ., etc.). Given this, and given that this behaviour seemed to fit well with Python's needs for __eq__, it seemed to make sense at the time to map Python's six operators to the first 6 operators in table 3. > I think two main issues emerge from the brief discussion: > > 1. Should the comparison operators follow the 'compare' function > ? from the standard? That's a possibility. But see below. > 2. What is the use of == and != outside the decimal scope? > > Mark mentions that Python uses == to test for set and dict memberships, > but that you cannot put decimal NaNs into sets: > > 'TypeError: Cannot hash a NaN value' > > > I want to add that Decimal('NaN') == Decimal('NaN') gives False, which > should somewhat limit the uses of == for NaNs outside the decimal realm > anyway. > > > Are there cases where == and != are actually needed to give a result > for NaNs? Well, when running in some form of 'non-stop' mode, where (quiet) NaN results are supposed to be propagated to the end of a computation, you certainly want equality comparisons with nan just to silently return false. E.g., in code like: if x == 0: else: nans should just end up in the second branch, without the programmer having had to think about it too hard. So I think comparisons with nans should always return either True or False when InvalidOperation is not trapped. The question is whether comparisons should always signal when InvalidOperation is trapped (which is what happens with the default context). I'm -0.5 on changing the current behaviour: it may not be exactly right, and if I were implementing Decimal from scratch I might well do things differently, but I don't think it's terribly wrong either. While not based on the Decimal standard itself, it's based on the next most closely- related standard. It works with Python's needs for __eq__ and __ne__. And it's already out there in Python 2.6; making minute adjustments to existing behaviour without a good reason seems like asking for trouble. Mark From dickinsm at gmail.com Mon Nov 9 14:08:41 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 9 Nov 2009 13:08:41 +0000 Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs In-Reply-To: <20091109104249.GA3377@mail.bytereef.org> References: <20091108162648.GA4099@mail.bytereef.org> <4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1> <20091109104249.GA3377@mail.bytereef.org> Message-ID: <5c6f2a5d0911090508s1a67dccei734d497b91d08fac@mail.gmail.com> On Mon, Nov 9, 2009 at 10:42 AM, Stefan Krah wrote: > I can also give a decimal use case where the current behavior is problematic > A variable initialized to a signaling NaN should always cause an exception. > > But this doesn't: > > salary = Decimal("sNaN") > minimum_wage = 1000 > if (salary == minimum_wage): > ? ?print "do stuff" > else: > ? ?print "do other stuff" Hmm. This does look suspicious. It's possible we should be raising for signalling nans here. For most of what I wrote above I was thinking of quiet nans. Mark From stefan-usenet at bytereef.org Mon Nov 9 14:21:24 2009 From: stefan-usenet at bytereef.org (Stefan Krah) Date: Mon, 9 Nov 2009 14:21:24 +0100 Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs In-Reply-To: References: <20091108162648.GA4099@mail.bytereef.org> <4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1> <20091109104249.GA3377@mail.bytereef.org> <20091109122159.GA4383@mail.bytereef.org> Message-ID: <20091109132124.GA5063@mail.bytereef.org> Antoine Pitrou wrote: > > I see the point, but Decimal("NaN") does not hash: > > Ok but witness again: > > >>> L = [1, 2, Decimal("NaN"), 3] > >>> 3 in L > True > >>> class H(object): > ... def __eq__(self, other): raise ValueError > ... > >>> L = [1, 2, H(), 3] > >>> 3 in L > Traceback (most recent call last): > File "", line 1, in > File "", line 2, in __eq__ > ValueError Yes, but the list is already broken in two ways: >>> L.sort() Traceback (most recent call last): File "", line 1, in File "/usr/lib/python2.7/decimal.py", line 877, in __lt__ ans = self._compare_check_nans(other, context) File "/usr/lib/python2.7/decimal.py", line 782, in _compare_check_nans self) File "/usr/lib/python2.7/decimal.py", line 3755, in _raise_error raise error(explanation) decimal.InvalidOperation: comparison involving NaN >>> Decimal("NaN") in L False > (NB: interestingly, float("nan") does hash) I wonder if it should: >>> d = {float('nan'): 10, 0: 20} >>> 0 in d True >>> float('nan') in d False >>> d[float('nan')] Traceback (most recent call last): File "", line 1, in KeyError: nan >>> I guess my point is that NaNs in lists and dicts are broken in so many ways that it might be good to discourage this use. (And get the added benefit of safer mathematical behavior for == and !=.) Stefan Krah From dickinsm at gmail.com Mon Nov 9 14:23:03 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 9 Nov 2009 13:23:03 +0000 Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs In-Reply-To: <5c6f2a5d0911090501n23254cc7x8e1eaad34508f8fa@mail.gmail.com> References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za> <1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za> <20091108162648.GA4099@mail.bytereef.org> <5c6f2a5d0911090501n23254cc7x8e1eaad34508f8fa@mail.gmail.com> Message-ID: <5c6f2a5d0911090523tedd4f34geb5530ec10a72ef3@mail.gmail.com> On Mon, Nov 9, 2009 at 1:01 PM, Mark Dickinson wrote: > current behaviour comes from the IEEE 854 standard; ?given the > absence of helpful information in the Decimal standard, IEEE 854 > is an obvious next place to look. ?There's an unofficial copy of the > standard available at: > > http://754r.ucbtest.org/standards/854.pdf > > Section 5.7 describes twenty-six(!) distinct comparison operators. It's interesting to note that out of the 32 possible different comparison behaviours (two choices of result for each of {equal, lessthan, greaterthan, unordered}, together with a choice of whether to signal or not for unordered in each case), the two interesting operators that are missing from that IEEE 854 table are precisely the ones that we're discussing: signalling __eq__ (i.e., return False for lessthan, greaterthan, unordered, True for equal, and signal on unordered), and signalling __ne__ (the reverse of the above, but still signalling on unordered). (The other four missing operators are the uninteresting ones that always return True or always return False.) Mark From dickinsm at gmail.com Mon Nov 9 14:25:42 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 9 Nov 2009 13:25:42 +0000 Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs In-Reply-To: <20091109132124.GA5063@mail.bytereef.org> References: <20091108162648.GA4099@mail.bytereef.org> <4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1> <20091109104249.GA3377@mail.bytereef.org> <20091109122159.GA4383@mail.bytereef.org> <20091109132124.GA5063@mail.bytereef.org> Message-ID: <5c6f2a5d0911090525oc84f864jb1e336d32d0c6741@mail.gmail.com> On Mon, Nov 9, 2009 at 1:21 PM, Stefan Krah wrote: > Antoine Pitrou wrote: >> (NB: interestingly, float("nan") does hash) > > > I wonder if it should: > >>>> d = {float('nan'): 10, 0: 20} >>>> 0 in d > True >>>> float('nan') in d > False >>>> d[float('nan')] > Traceback (most recent call last): > ?File "", line 1, in > KeyError: nan That's because you're creating two different float nans. Compare with: Python 3.2a0 (py3k:76132M, Nov 6 2009, 14:47:39) [GCC 4.2.1 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> nan = float('nan') >>> d = {nan: 10, 0: 20} >>> nan in d True >>> d[nan] 10 Mark From solipsis at pitrou.net Mon Nov 9 14:29:16 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 9 Nov 2009 13:29:16 +0000 (UTC) Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs References: <20091108162648.GA4099@mail.bytereef.org> <4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1> <20091109104249.GA3377@mail.bytereef.org> <20091109122159.GA4383@mail.bytereef.org> <20091109132124.GA5063@mail.bytereef.org> Message-ID: Stefan Krah bytereef.org> writes: > > I guess my point is that NaNs in lists and dicts are broken in so many > ways that it might be good to discourage this use. (And get the added > benefit of safer mathematical behavior for == and !=.) Giving users seemingly random and unexplainable exceptions would not be a good way to discourage it, though. From ncoghlan at gmail.com Mon Nov 9 14:32:15 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 09 Nov 2009 23:32:15 +1000 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it In-Reply-To: <4AF7D5E7.4080901@v.loewis.de> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <4AF341BD.2010203@v.loewis.de> <7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> <4AF7D5E7.4080901@v.loewis.de> Message-ID: <4AF819DF.6030107@gmail.com> Martin v. L?wis wrote: >> The problem arises because we're systematically unbalancing the hash >> table. The iterator returns the first valid entry in the hash table, >> which we remove. Repeat several times and pretty soon the iterator has >> to spend O(n) time scanning through empty entries to find the first >> remaining valid entry. > > Interesting. Something goes wrong, it seems: if items get removed over > and over again, I think the set should shrink (not sure whether it > actually does). Then, I think you should end up with an amortized O(1) > for selecting an element (assuming that the underlying hashes don't > collide). The behaviour is inherited (in spirit) from dicts. Courtesy of dictnotes.txt: """ * Shrinkage rate upon exceeding maximum sparseness. The current CPython code never even checks sparseness when deleting a key. When a new key is added, it resizes based on the number of active keys, so that the addition may trigger shrinkage rather than growth. """ """ Dictionary operations involving only a single key can be O(1) unless resizing is possible. By checking for a resize only when the dictionary can grow (and may *require* resizing), other operations remain O(1), and the odds of resize thrashing or memory fragmentation are reduced. In particular, an algorithm that empties a dictionary by repeatedly invoking .pop will see no resizing, which might not be necessary at all because the dictionary is eventually discarded entirely. """ So the rationale is to ensure that only add operations perform a resize and so that sequential pop() operations don't incur excessive resizing costs. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From amk at amk.ca Mon Nov 9 15:16:09 2009 From: amk at amk.ca (A.M. Kuchling) Date: Mon, 9 Nov 2009 09:16:09 -0500 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com> <9d153b7c0911051613o62be5dc9l52102f7805cd4a0e@mail.gmail.com> <52dc1c820911081256q7fc3f7fdk87b2f12932dce11d@mail.gmail.com> Message-ID: <20091109141609.GA8499@amk-desktop.matrixgroup.net> On Sun, Nov 08, 2009 at 10:27:46PM +0100, Georg Brandl wrote: > Good point, I'll make that change if AMK agrees. It's certainly fine with me. Do we want to only make that change to the 2.7 "What's New", or should we also do it for the 2.6 one? --amk From daniel at stutzbachenterprises.com Mon Nov 9 16:09:54 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 9 Nov 2009 09:09:54 -0600 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it In-Reply-To: <4AF7D5E7.4080901@v.loewis.de> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <4AF341BD.2010203@v.loewis.de> <7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> <4AF7D5E7.4080901@v.loewis.de> Message-ID: On Mon, Nov 9, 2009 at 2:42 AM, "Martin v. L?wis" wrote: > Interesting. Something goes wrong, it seems: if items get removed over > and over again, I think the set should shrink (not sure whether it > actually does). Then, I think you should end up with an amortized O(1) > for selecting an element (assuming that the underlying hashes don't > collide). > I'm not sure if Python's implementation shrinks or not, but even if it did shrink, it would still be amortized O(n). Imagine a completely full hash table that currently contains n elements in n slots (I know this cannot happen in Python's implementation but it's useful for illustrative purposes). Assume it will shrink when it gets down to n/2 elements. Here is my pathological algorithm again, for reference purposes: while s: for i in s: break # Imagine a complex algorithm here that depends on i still being in s s.remove(i) We repeatedly search through the slots sequentially and remove the first element we find. The first removal requires checking 1 slot, the second removal requires checking 2 slots, the third removal requires checking 3 slots, and so forth. The hash table will shrink after the n/2-th removal, when we have checked 1 + 2 + 3 + ... + n/2 = O(n**2) slots for n/2 removals (or amortized O(n) per removal). It's too late for shrinking to save us; we've already performed too much work. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.belopolsky at gmail.com Mon Nov 9 16:36:27 2009 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 9 Nov 2009 10:36:27 -0500 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it In-Reply-To: References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <4AF341BD.2010203@v.loewis.de> <7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> <4AF7D5E7.4080901@v.loewis.de> Message-ID: On Mon, Nov 9, 2009 at 10:09 AM, Daniel Stutzbach wrote: > On Mon, Nov 9, 2009 at 2:42 AM, "Martin v. L?wis" > wrote: >> >> Interesting. Something goes wrong, it seems: if items get removed over >> and over again, I think the set should shrink (not sure whether it >> actually does). Then, I think you should end up with an amortized O(1) >> for selecting an element (assuming that the underlying hashes don't >> collide). > > I'm not sure if Python's implementation shrinks or not, It does not: >>> s = set(range(100000)) >>> from sys import getsizeof >>> getsizeof(s) 4194536 >>> while s: x = s.pop() ... >>> getsizeof(s) 4194536 >>> s.clear() >>> getsizeof(s) 232 From bjourne at gmail.com Mon Nov 9 17:40:11 2009 From: bjourne at gmail.com (=?ISO-8859-1?Q?Bj=F6rn_Lindqvist?=) Date: Mon, 9 Nov 2009 17:40:11 +0100 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it In-Reply-To: <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <4AF341BD.2010203@v.loewis.de> <7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> Message-ID: <740c3aec0911090840w7d3947a2ocd36b2d4ae5d70ff@mail.gmail.com> 2009/11/6 Raymond Hettinger : > [me] >> >> Why not write a short, fast get_first() function for your utils directory >> and be done with it? >> That could work with sets, mappings, generators, and other containers and >> iterators. >> No need to fatten the set/frozenset API for something so trivial and so >> rarely needed. > > Forgot to post the code. ?It is short, fast, and easy. ?It is explicit about > handing the case with an empty input. ?And it is specific about which value > it returns (always the first iterated value; not an arbitrary one). ?There's > no guessing about what it does. ?It gets the job done. > > def first(iterable): > ? 'Return the first value from a container or iterable. ?If empty, raises > LookupError' > ? for value in iterable: > ? ? ? return value > ? raise LookupError('no first value; iterable is empty') > > If desired, it is not hard to change to the last time to return a default > value or some other exception. That function is very nice and genericly lisp-like. Couldn't that one be in the builtins? It would both solve the problem and avoid fattening the set API. -- mvh Bj?rn From brett at python.org Mon Nov 9 18:12:28 2009 From: brett at python.org (Brett Cannon) Date: Mon, 9 Nov 2009 09:12:28 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <200911081115.01803.steve@pearwood.info> <200911091006.26433.steve@pearwood.info> Message-ID: On Sun, Nov 8, 2009 at 19:50, geremy condra wrote: > On Sun, Nov 8, 2009 at 9:41 PM, Guido van Rossum wrote: >> On Sun, Nov 8, 2009 at 5:45 PM, geremy condra wrote: >>> I quote: >>> >>> "This PEP proposes a temporary moratorium (suspension) of all changes >>> to the Python language syntax, semantics, and built-ins for a period >>> of *at least two years* from the release of Python 3.1." >>> >>> Emphasis mine. >> >> I since added this: >> >> """In particular, >> the moratorium would include Python 3.2 (to be released 18-24 months >> after 3.1) but (unless explicitly extended) allow Python 3.3 to once >> again include language changes.""" >> >>> Like I say, a definite end point would be much preferred to n > 2. >> >> My time machine doesn't work very well in the future. So I can't tell >> what we'll find necessary 2 years from now. But I would be fine with >> defining the time limit to be max(time(3.1) + 2 years, time(3.2)). >> I.e. the moratorium (unless explicitly extended) ends as soon as 3.2 >> has been released *and* at least 2 years have passed since 3.1. > > Ok, thanks for the clarification. Could you spell out what you would > consider grounds for a future extension? > We feel it's necessary as a group or Guido does, simple as that. You can't plan it since it's over two years away. If the time comes and people feel the moratorium has been beneficial and should go longer, we will extend it. It will most likely be for the same reasons we started it. >>> If possible, I'd also like to hear some of Steven's other points addressed. >> >> They haven't changed my mind. > > Ok, but the fact that you (or Steven) hold a particular set of beliefs > is a singularly unconvincing argument. I disagree. Guido is the BDFL so his set of beliefs is enough unless faced with a huge number of people disagreeing. That has not occurred on this topic. >Could you explain why you > don't agree, if only for the record? Enough happens on python-dev based on gut feeling that there isn't a need. If we had to spell out objections to every email we received while discussing a PEP, threads would never end. Heck, I think this PEP discussion as gone on long enough and that Guido could pronounce at this point. -Brett From jasper at humppa.nl Mon Nov 9 18:26:23 2009 From: jasper at humppa.nl (Jasper Lievisse Adriaanse) Date: Mon, 9 Nov 2009 18:26:23 +0100 Subject: [Python-Dev] Python 2.6: OverflowError: signed integer is greater than maximum Message-ID: <20091109172623.GA15620@gurthang.humppa.nl> Hi, while trying to get Python 2.6 working on OpenBSD/sgi (64-bit port) I ran into the following during build: OverflowError: signed integer is greater than maximum I ran the command that triggered this by hand with -v added: (sgi Python-2.6.3 40)$ export PATH; PATH="`pwd`:$PATH"; export PYTHONPATH; PYTHONPATH="`pwd`/Lib"; export DYLD_FRAMEWORK_PATH; DYLD_FRAMEWORK_PATH="`pwd`"; export EXE; EXE=""; cd ./Lib/plat-openbsd4; ./regen python$EXE -v ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/netinet/in.h # installing zipimport hook import zipimport # builtin # installed zipimport hook # /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/site.pyc matches /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/site.py import site # precompiled from /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/site.pyc 'import site' failed; traceback: Traceback (most recent call last): File "/usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/site.py", line 61, in import sys OverflowError: signed integer is greater than maximum import encodings # directory /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/encodings # /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/encodings/__init__.pyc matches /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/encodings/__init__.py import encodings # precompiled from /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/encodings/__init__.pyc Python 2.6.3 (r263:75183, Nov 6 2009, 09:50:33) [GCC 3.3.5 (propolice)] on openbsd4 Type "help", "copyright", "credits" or "license" for more information. # /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/re.pyc matches /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/re.py import re # precompiled from /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/re.pyc Traceback (most recent call last): File "../../Tools/scripts/h2py.py", line 24, in import sys, re, getopt, os File "/usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/re.py", line 104, in import sys OverflowError: signed integer is greater than maximum # clear __builtin__._ # clear sys.path # clear sys.argv # clear sys.ps1 # clear sys.ps2 # clear sys.exitfunc # clear sys.exc_type # clear sys.exc_value # clear sys.exc_traceback # clear sys.last_type # clear sys.last_value # clear sys.last_traceback # clear sys.path_hooks # clear sys.path_importer_cache # clear sys.meta_path # clear sys.flags # clear sys.float_info # restore sys.stdin # restore sys.stdout # restore sys.stderr # cleanup __main__ # cleanup[1] zipimport # cleanup[1] signal # cleanup[1] exceptions # cleanup[1] _warnings # cleanup sys # cleanup __builtin__ # cleanup ints: 3 unfreed ints # cleanup floats (sgi plat-openbsd4 41)$ Does anyone know what's going on here, or how to fix it? I can provide a full build and configure log if needed. Cheers, Jasper -- "Intelligence should guide our actions, but in harmony with the texture of the situation at hand" -- Francisco Varela From guido at python.org Mon Nov 9 18:56:34 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 9 Nov 2009 09:56:34 -0800 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: References: <200911081115.01803.steve@pearwood.info> <200911091006.26433.steve@pearwood.info> Message-ID: Thanks Brett. I've moved the moratorium PEP to Status: Accepted. I've added the words about inclusion of 3.2 and exclusion of 3.3 (which were eaten by a svn conflict when I previously tried to add them) and added a section to th end stating that an extension will require another PEP. --Guido On Mon, Nov 9, 2009 at 9:12 AM, Brett Cannon wrote: > On Sun, Nov 8, 2009 at 19:50, geremy condra wrote: >> On Sun, Nov 8, 2009 at 9:41 PM, Guido van Rossum wrote: >>> On Sun, Nov 8, 2009 at 5:45 PM, geremy condra wrote: >>>> I quote: >>>> >>>> "This PEP proposes a temporary moratorium (suspension) of all changes >>>> to the Python language syntax, semantics, and built-ins for a period >>>> of *at least two years* from the release of Python 3.1." >>>> >>>> Emphasis mine. >>> >>> I since added this: >>> >>> """In particular, >>> the moratorium would include Python 3.2 (to be released 18-24 months >>> after 3.1) but (unless explicitly extended) allow Python 3.3 to once >>> again include language changes.""" >>> >>>> Like I say, a definite end point would be much preferred to n > 2. >>> >>> My time machine doesn't work very well in the future. So I can't tell >>> what we'll find necessary 2 years from now. But I would be fine with >>> defining the time limit to be max(time(3.1) + 2 years, time(3.2)). >>> I.e. the moratorium (unless explicitly extended) ends as soon as 3.2 >>> has been released *and* at least 2 years have passed since 3.1. >> >> Ok, thanks for the clarification. Could you spell out what you would >> consider grounds for a future extension? >> > > We feel it's necessary as a group or Guido does, simple as that. You > can't plan it since it's over two years away. If the time comes and > people feel the moratorium has been beneficial and should go longer, > we will extend it. It will most likely be for the same reasons we > started it. > >>>> If possible, I'd also like to hear some of Steven's other points addressed. >>> >>> They haven't changed my mind. >> >> Ok, but the fact that you (or Steven) hold a particular set of beliefs >> is a singularly unconvincing argument. > > I disagree. Guido is the BDFL so his set of beliefs is enough unless > faced with a huge number of people disagreeing. That has not occurred > on this topic. > >>Could you explain why you >> don't agree, if only for the record? > > Enough happens on python-dev based on gut feeling that there isn't a > need. If we had to spell out objections to every email we received > while discussing a PEP, threads would never end. Heck, I think this > PEP discussion as gone on long enough and that Guido could pronounce > at this point. > > -Brett > -- --Guido van Rossum (python.org/~guido) From dickinsm at gmail.com Mon Nov 9 20:15:20 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 9 Nov 2009 19:15:20 +0000 Subject: [Python-Dev] Python 2.6: OverflowError: signed integer is greater than maximum In-Reply-To: <20091109172623.GA15620@gurthang.humppa.nl> References: <20091109172623.GA15620@gurthang.humppa.nl> Message-ID: <5c6f2a5d0911091115k1ab306ffnc264c70db6dc237a@mail.gmail.com> On Mon, Nov 9, 2009 at 5:26 PM, Jasper Lievisse Adriaanse wrote: > Hi, > > while trying to get Python 2.6 working on OpenBSD/sgi (64-bit port) > I ran into the following during build: > > OverflowError: signed integer is greater than maximum > > I ran the command that triggered this by hand with -v added: [traceback snipped] > Does anyone know what's going on here, or how to fix it? > I can provide a full build and configure log if needed. No idea. Please could you file a report at http://bugs.python.org? The build and configure log would be useful. If you can diagnose the source of the failure yourself any further that would be more useful; I suspect that the core developers will have a hard time figuring the problem out without this. I may well be barking up the wrong tree here, but as a first guess it looks as though something in the _PySys_Init function in Python/sysmodule.c is (directly or indirectly) causing the OverflowError to be raised. I'd try adding a bunch of printf calls to that function that print the value of PyErr_Occurred(), to see where (if at all) the Python exception is being set. (I suggest building with the --with-pydebug configure option for this.) Mark From g.brandl at gmx.net Mon Nov 9 20:51:39 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 09 Nov 2009 19:51:39 +0000 Subject: [Python-Dev] PEP 3003 - Python Language Moratorium In-Reply-To: <20091109141609.GA8499@amk-desktop.matrixgroup.net> References: <4222a8490911051521r1b9c8165id6e0f12d62da0613@mail.gmail.com> <48c184130911051555r9a5b78cs3a13cb1345d3cbf7@mail.gmail.com> <9d153b7c0911051613o62be5dc9l52102f7805cd4a0e@mail.gmail.com> <52dc1c820911081256q7fc3f7fdk87b2f12932dce11d@mail.gmail.com> <20091109141609.GA8499@amk-desktop.matrixgroup.net> Message-ID: A.M. Kuchling schrieb: > On Sun, Nov 08, 2009 at 10:27:46PM +0100, Georg Brandl wrote: >> Good point, I'll make that change if AMK agrees. > > It's certainly fine with me. Do we want to only make that change to > the 2.7 "What's New", or should we also do it for the 2.6 one? Why not for 2.6 as well, it's an immediate benefit, especially since the list of deprecations in 2.6 is huge. Georg From steve at pearwood.info Mon Nov 9 21:16:56 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 10 Nov 2009 07:16:56 +1100 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it In-Reply-To: <740c3aec0911090840w7d3947a2ocd36b2d4ae5d70ff@mail.gmail.com> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> <740c3aec0911090840w7d3947a2ocd36b2d4ae5d70ff@mail.gmail.com> Message-ID: <200911100716.57045.steve@pearwood.info> On Tue, 10 Nov 2009 03:40:11 am Bj?rn Lindqvist wrote: > 2009/11/6 Raymond Hettinger : > > [me] > > > >> Why not write a short, fast get_first() function for your utils > >> directory and be done with it? > >> That could work with sets, mappings, generators, and other > >> containers and iterators. > >> No need to fatten the set/frozenset API for something so trivial > >> and so rarely needed. > > > > Forgot to post the code. ?It is short, fast, and easy. ?It is > > explicit about handing the case with an empty input. ?And it is > > specific about which value it returns (always the first iterated > > value; not an arbitrary one). ?There's no guessing about what it > > does. ?It gets the job done. > > > > def first(iterable): > > ? 'Return the first value from a container or iterable. ?If empty, > > raises LookupError' > > ? for value in iterable: > > ? ? ? return value > > ? raise LookupError('no first value; iterable is empty') > > > > If desired, it is not hard to change to the last time to return a > > default value or some other exception. > > That function is very nice and genericly lisp-like. Couldn't that one > be in the builtins? It would both solve the problem and avoid > fattening the set API. I'm not sure, but isn't that thread-unsafe? Because sets aren't directly iterable, `for value in iterable` builds a set_iterator operator. If another thread modifies the set after the set_iterator is built, but before the value is looked up, you will get a mysterious RuntimeError almost impossible to debug. >>> def first(): # simplified ... for value in iterator: ... return value ... >>> dis.dis(first) 2 0 SETUP_LOOP 15 (to 18) 3 LOAD_GLOBAL 0 (iterator) 6 GET_ITER 7 FOR_ITER 7 (to 17) 10 STORE_FAST 0 (value) 3 13 LOAD_FAST 0 (value) 16 RETURN_VALUE >> 17 POP_BLOCK >> 18 LOAD_CONST 0 (None) 21 RETURN_VALUE Is it possible for another thread to be called between the GET_ITER and STORE_FAST? -- Steven D'Aprano From greg.ewing at canterbury.ac.nz Mon Nov 9 22:22:51 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 10 Nov 2009 10:22:51 +1300 Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs In-Reply-To: References: <20091108162648.GA4099@mail.bytereef.org> <4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1> <20091109104249.GA3377@mail.bytereef.org> Message-ID: <4AF8882B.4040200@canterbury.ac.nz> Antoine Pitrou wrote: > The problem is when searching for /another/ object which hashes the same as > Decimal("NaN"). Maybe decimal NaNs should be unhashable, so that you can't put them in a dictionary in the first place. -- Greg From ncoghlan at gmail.com Mon Nov 9 22:44:10 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 10 Nov 2009 07:44:10 +1000 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it In-Reply-To: References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <4AF341BD.2010203@v.loewis.de> <7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> <4AF7D5E7.4080901@v.loewis.de> Message-ID: <4AF88D2A.1020306@gmail.com> Alexander Belopolsky wrote: > On Mon, Nov 9, 2009 at 10:09 AM, Daniel Stutzbach > wrote: >> On Mon, Nov 9, 2009 at 2:42 AM, "Martin v. L?wis" >> wrote: >>> Interesting. Something goes wrong, it seems: if items get removed over >>> and over again, I think the set should shrink (not sure whether it >>> actually does). Then, I think you should end up with an amortized O(1) >>> for selecting an element (assuming that the underlying hashes don't >>> collide). >> I'm not sure if Python's implementation shrinks or not, > > It does not: > >>>> s = set(range(100000)) >>>> from sys import getsizeof >>>> getsizeof(s) > 4194536 >>>> while s: x = s.pop() > ... >>>> getsizeof(s) > 4194536 >>>> s.clear() >>>> getsizeof(s) > 232 Interestingly, it looks like there the sparseness check isn't triggering on addition of items either (when dictnotes.txt says it should): >>> from sys import getsizeof >>> s = set(range(100000)) >>> getsizeof(s) 2097264 >>> while s: x = s.pop() ... >>> getsizeof(s) 2097264 >>> s.add(1) >>> getsizeof(s) 2097264 I did a similar test with dict.fromkeys and that also didn't resize until .clear() was invoked. I don't know the current criteria settings for the sparseness check, but it seems odd that going from 100k active keys to none wouldn't cause a resize... Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From martin at v.loewis.de Mon Nov 9 22:50:11 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 09 Nov 2009 22:50:11 +0100 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it In-Reply-To: <4AF819DF.6030107@gmail.com> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <4AF0D0D7.3090403@canterbury.ac.nz> <200911042234.36796.steve@pearwood.info> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <4AF341BD.2010203@v.loewis.de> <7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> <4AF7D5E7.4080901@v.loewis.de> <4AF819DF.6030107@gmail.com> Message-ID: <4AF88E93.4010905@v.loewis.de> > So the rationale is to ensure that only add operations perform a resize > and so that sequential pop() operations don't incur excessive resizing > costs. I agree that the use case of repeated .pop() operations is reasonable, and (IIUC) that case is also special-cased using a finger/index. I think for regular removal, the same logic should not apply: if a series of removals is performed, then further (non-pop) removals see increasing costs, as do regular lookups. So I think that a removal should trigger shrinking (with appropriate thresholds) unless it's a .pop. Regards, Martin From martin at v.loewis.de Mon Nov 9 23:10:56 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 09 Nov 2009 23:10:56 +0100 Subject: [Python-Dev] IDLE as default Python editor In-Reply-To: <4AF80E73.5090104@gmail.com> References: <87iqdk2n7b.fsf@benfinney.id.au> <4AF80E73.5090104@gmail.com> Message-ID: <4AF89370.2040206@v.loewis.de> Nick Coghlan wrote: > Ben Finney wrote: >> anatoly techtonik writes: >> >>> Quite an interesting question recently popped up in pygame community >>> that I'd like to ask to Python developers. >> This forum is specifically about development *of* Python. > > Anatoly's question is actually a fair one for python-dev - we're the > ones that *ship* Idle, so it is legitimate to ask our reasons for > continuing to do so. But he didn't - instead, he asked how many of us use it. Usage by committers is not (or shouldn't be) a primary criterion for including or not including something. Instead, usage in the community should be, and python-dev is indeed the wrong place to estimate that. OTOH, the second (or, rather, third) question (does anybody think it should be replaced) is indeed on-topic for python-dev. I didn't really answer that question before, so I do now: I have not personally plans to replace it, and I'm skeptical wrt. anybody else's plans unless there is specific code in existence that IDLE could be replaced *with*. Regards, Martin From martin at v.loewis.de Mon Nov 9 23:19:13 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 09 Nov 2009 23:19:13 +0100 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it In-Reply-To: <200911100716.57045.steve@pearwood.info> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> <740c3aec0911090840w7d3947a2ocd36b2d4ae5d70ff@mail.gmail.com> <200911100716.57045.steve@pearwood.info> Message-ID: <4AF89561.9010408@v.loewis.de> > I'm not sure, but isn't that thread-unsafe? You are right; it's thread-unsafe. I would fix it by catching the RuntimeError, and retrying. Given the current GIL strategy (including proposed changes to it), it won't happen two times in a row, so the number of retries would be bounded. Regards, Martin From martin at v.loewis.de Mon Nov 9 23:23:11 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 09 Nov 2009 23:23:11 +0100 Subject: [Python-Dev] Python 2.6: OverflowError: signed integer is greater than maximum In-Reply-To: <5c6f2a5d0911091115k1ab306ffnc264c70db6dc237a@mail.gmail.com> References: <20091109172623.GA15620@gurthang.humppa.nl> <5c6f2a5d0911091115k1ab306ffnc264c70db6dc237a@mail.gmail.com> Message-ID: <4AF8964F.7020104@v.loewis.de> > I may well be barking up the wrong tree here, but as a first guess > it looks as though something in the _PySys_Init function in > Python/sysmodule.c is (directly or indirectly) causing the > OverflowError to be raised. My theory would be different. There is a pending unchecked OverflowError before the import, and the err-occurred check after the import picks it up. Of course, your guess is as good as mine. While barking up trees: My guess is that it's a compiler bug (i.e. the compiler generating bad code). Regards, Martin From martin at v.loewis.de Mon Nov 9 23:39:04 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 09 Nov 2009 23:39:04 +0100 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it In-Reply-To: References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <99BB308202224D31AA5300223DB59912@RaymondLaptop1> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <4AF341BD.2010203@v.loewis.de> <7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> <4AF7D5E7.4080901@v.loewis.de> Message-ID: <4AF89A08.5070106@v.loewis.de> > We repeatedly search through the slots sequentially and remove the first > element we find. The first removal requires checking 1 slot, the second > removal requires checking 2 slots, the third removal requires checking 3 > slots, and so forth. The hash table will shrink after the n/2-th > removal, when we have checked 1 + 2 + 3 + ... + n/2 = O(n**2) slots for > n/2 removals (or amortized O(n) per removal). It's too late for > shrinking to save us; we've already performed too much work. Ah, right. Perhaps people writing the loop the way you proposed deserve to get bad performance, as they should use .pop in the first place. Regards, Martin From daniel at stutzbachenterprises.com Mon Nov 9 23:44:53 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 9 Nov 2009 16:44:53 -0600 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it In-Reply-To: <4AF88E93.4010905@v.loewis.de> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <7b0045ed0911051243q18f8324ak9abaa98ac26948e9@mail.gmail.com> <4AF341BD.2010203@v.loewis.de> <7b0045ed0911051419i76f4221dye8ecccc424179ec6@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> <4AF7D5E7.4080901@v.loewis.de> <4AF819DF.6030107@gmail.com> <4AF88E93.4010905@v.loewis.de> Message-ID: On Mon, Nov 9, 2009 at 3:50 PM, "Martin v. L?wis" wrote: > I think for regular removal, the same logic should not apply: if a > series of removals is performed, then further (non-pop) removals > see increasing costs, as do regular lookups. So I think that a removal > should trigger shrinking (with appropriate thresholds) unless it's a > .pop. > Minor technicality, but it's the next() operation of the iterator that has increasing cost as the set/dict becomes sparse. Removals are always O(1). The removal uses the hash to find the item quickly. The iterator has to scan through the table for non-empty entries. (the above assumes a good hash function with few collisions, of course) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Mon Nov 9 23:51:09 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 09 Nov 2009 17:51:09 -0500 Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs In-Reply-To: <5c6f2a5d0911090525oc84f864jb1e336d32d0c6741@mail.gmail.com> References: <20091108162648.GA4099@mail.bytereef.org> <4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1> <20091109104249.GA3377@mail.bytereef.org> <20091109122159.GA4383@mail.bytereef.org> <20091109132124.GA5063@mail.bytereef.org> <5c6f2a5d0911090525oc84f864jb1e336d32d0c6741@mail.gmail.com> Message-ID: Mark Dickinson wrote: > That's because you're creating two different float nans. > Compare with: > > Python 3.2a0 (py3k:76132M, Nov 6 2009, 14:47:39) > [GCC 4.2.1 (SUSE Linux)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> nan = float('nan') >>>> d = {nan: 10, 0: 20} >>>> nan in d > True >>>> d[nan] > 10 This also suggests to me that nan should be a singleton, or at least that the doc should recommend that programs should make it be such for the program. tjr From jasper at humppa.nl Tue Nov 10 00:05:27 2009 From: jasper at humppa.nl (Jasper Lievisse Adriaanse) Date: Tue, 10 Nov 2009 00:05:27 +0100 Subject: [Python-Dev] Python 2.6: OverflowError: signed integer is greater than maximum In-Reply-To: <5c6f2a5d0911091115k1ab306ffnc264c70db6dc237a@mail.gmail.com> References: <20091109172623.GA15620@gurthang.humppa.nl> <5c6f2a5d0911091115k1ab306ffnc264c70db6dc237a@mail.gmail.com> Message-ID: <20091109230527.GA7091@gurthang.humppa.nl> On Mon, Nov 09, 2009 at 07:15:20PM +0000, Mark Dickinson wrote: > On Mon, Nov 9, 2009 at 5:26 PM, Jasper Lievisse Adriaanse > wrote: > > Hi, > > > > while trying to get Python 2.6 working on OpenBSD/sgi (64-bit port) > > I ran into the following during build: > > > > OverflowError: signed integer is greater than maximum > > > > I ran the command that triggered this by hand with -v added: > > [traceback snipped] > > > Does anyone know what's going on here, or how to fix it? > > I can provide a full build and configure log if needed. > > No idea. Please could you file a report at http://bugs.python.org? > The build and configure log would be useful. If you can diagnose > the source of the failure yourself any further that would be more > useful; I suspect that the core developers will have a hard time > figuring the problem out without this. > > I may well be barking up the wrong tree here, but as a first guess > it looks as though something in the _PySys_Init function in > Python/sysmodule.c is (directly or indirectly) causing the > OverflowError to be raised. I'd try adding a bunch of > printf calls to that function that print the value of > PyErr_Occurred(), to see where (if at all) the Python exception > is being set. (I suggest building with the --with-pydebug configure > option for this.) > > Mark Hi, for the record, I filed the bug as: http://bugs.python.org/issue7296 In the meantime I'll try digging into this issue. Cheers, Jasper -- "Intelligence should guide our actions, but in harmony with the texture of the situation at hand" -- Francisco Varela From guido at python.org Tue Nov 10 00:20:29 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 9 Nov 2009 15:20:29 -0800 Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs In-Reply-To: References: <20091108162648.GA4099@mail.bytereef.org> <4FC3B5BCE9034A79AFB0530C748F019E@RaymondLaptop1> <20091109104249.GA3377@mail.bytereef.org> <20091109122159.GA4383@mail.bytereef.org> <20091109132124.GA5063@mail.bytereef.org> <5c6f2a5d0911090525oc84f864jb1e336d32d0c6741@mail.gmail.com> Message-ID: On Mon, Nov 9, 2009 at 2:51 PM, Terry Reedy wrote: > This also suggests to me that nan should be a singleton, or at least that > the doc should recommend that programs should make it be such for the > program. The IEEE std disagreed -- there's extra info hidden in the mantissa bits. And the Python float implementation makes it pretty impractical to do this at the application level since x+y will generate a new NaN-valued float object each time it is called (if the outcome is NaN). -- --Guido van Rossum (python.org/~guido) From sparks.m at gmail.com Tue Nov 10 02:12:13 2009 From: sparks.m at gmail.com (Michael Sparks) Date: Tue, 10 Nov 2009 01:12:13 +0000 Subject: [Python-Dev] People want CPAN :-) In-Reply-To: References: <87iqdm7nkp.fsf@benfinney.id.au> Message-ID: <20e5c9660911091712y3548145yf0b48a0fa075220f@mail.gmail.com> [ I'm posting this comment in reply to seeing this thread: * http://thread.gmane.org/gmane.comp.python.distutils.devel/11359 Which has been reposted around - and I've read that thread. I lurk on this list, in case anything comes up that I'd hope to be able to say something useful to. I don't know if this will be, but that's my reason for posting. If this is the wrong place, my apologies, I don't sub to distutils-sig :-/ ] On Sat, Nov 7, 2009 at 2:30 PM, ssteinerX at gmail.com wrote: > On Nov 7, 2009, at 3:20 AM, Ben Finney wrote: > >> Guido van Rossum writes: >> >>> On Fri, Nov 6, 2009 at 2:52 PM, David Lyon >>> wrote: >>>> [ lots of snippage ] ... > All in all, I think this could be a big leap forward for the Python > distribution ecosystem whether or not we eventually write the PyPan I wished > for as a new Perl refugee. Speaking as someone who left the perl world for the python world, many years ago now, primarily due to working on one project, the thing I really miss about Perl is CPAN. It's not the fact that you know you do perl Makefile.PL && make && make test && make install. Nor the fact that it's trivial to set up a skeleton package setup that makes that work for you. It's not the fact that there's an installer than can download & track dependencies. The thing that makes the difference IMHO is two points: * In a language which has a core ethos "There's more than one way to do it", packaging is the one place where there is one, and only one obvious way to do it. (Oddly with python, with packaging this is flipped - do I as a random project use distutils? pip? setuptools? distribute? virtualenv?) * It has a managed namespace or perhaps better - a co-ordinated namespace. CPAN may have lots of ills, and bad aspects about it (I've never really trusted the auto installer due to seeing one too many people having their perl installation as a whole upgraded due to a bug that was squashed 6-8 years ago), but these two points are pretty much killer. All the other aspects like auto download, upload, dependency tracking, auto doc extraction for the website etc, really follow from the managed namespace really. I realise that various efforts like easy_install & distribute & friends make that sort of step implicitly - since there can only be one http://pypi.python.org/pypi/flibble . But it's not quite the same - due to externally hosted packages. For more detail about this aspect: * http://www.cpan.org/modules/04pause.html#namespace I'm really mentioning this because I didn't see it listed, and I really think that it's very easy to underestimate this aspect of CPAN. IMHO, it is what matters the most about CPAN. The fact that they've nabbed the CTAN idea of having an archive network for storing, mirroring and grabbing stuff from is by comparison /almost/ irrelevant IMHO. It is the sort of thing that leads to the DBI::DBD type stuff that is being simple to use, because of the encouragement to talk and share a namespace. The biggest issue with this is retrofitting this to an existing world. Personal opinion, I hope it's useful, and going back into lurk mode (I hope :). If this annoys you, please just ignore it. Michael. From arfrever.fta at gmail.com Tue Nov 10 11:41:40 2009 From: arfrever.fta at gmail.com (Arfrever Frehtes Taifersar Arahesis) Date: Tue, 10 Nov 2009 11:41:40 +0100 Subject: [Python-Dev] 2.7/3.2 release schedule In-Reply-To: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> Message-ID: <71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com> 2009-11-02 21:00 Benjamin Peterson napisa?(a): > I've updated PEP 373 with my proposed release schedule: > > - 2.7/3.2 alpha 1 2009-12-05 > - 2.7/3.2 alpha 2 2010-01-09 > - 2.7/3.2 alpha 3 2010-02-06 > - 2.7/3.2 alpha 4 2010-03-06 > - 2.7/3.2 beta 1 2010-04-03 > - 2.7/3.2 beta 2 2010-05-01 > - 2.7/3.2 rc1 2010-05-29 > - 2.7/3.2 rc2 2010-06-12 > - 2.7/3.2 final 2010-06-26 PEP 3003 states that Python 3.2 will be released 18-24 months after Python 3.1. Python 3.1 was released on June 2009-06-27 [1], so theoretically Python 3.2 should be released not before 2010-12-19 [2]. [1] http://python.org/download/releases/3.1/ [2] datetime.date(2009, 6, 27) + datetime.timedelta(18 * 30) -- Arfrever Frehtes Taifersar Arahesis From ncoghlan at gmail.com Tue Nov 10 11:47:05 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 10 Nov 2009 20:47:05 +1000 Subject: [Python-Dev] IDLE as default Python editor In-Reply-To: <4AF89370.2040206@v.loewis.de> References: <87iqdk2n7b.fsf@benfinney.id.au> <4AF80E73.5090104@gmail.com> <4AF89370.2040206@v.loewis.de> Message-ID: <4AF944A9.2030108@gmail.com> Martin v. L?wis wrote: > Nick Coghlan wrote: >> Ben Finney wrote: >>> anatoly techtonik writes: >>> >>>> Quite an interesting question recently popped up in pygame community >>>> that I'd like to ask to Python developers. >>> This forum is specifically about development *of* Python. >> Anatoly's question is actually a fair one for python-dev - we're the >> ones that *ship* Idle, so it is legitimate to ask our reasons for >> continuing to do so. > > But he didn't - instead, he asked how many of us use it. Usage by > committers is not (or shouldn't be) a primary criterion for including > or not including something. Instead, usage in the community should be, > and python-dev is indeed the wrong place to estimate that. I took it as being a somewhat relevant leading question. If the response had been "No, none of us use it" then the obvious follow-up question would be "Why do you ship something that you don't consider worth using?". I agree I was responding to a question that wasn't actually written in the email though (and as I said in my other message, agree that it can't be replaced without someone putting up a serious alternative for consideration that ticks all the same boxes that IDLE currently ticks and then offers significant improvements over and above that). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Tue Nov 10 11:51:19 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 10 Nov 2009 20:51:19 +1000 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it In-Reply-To: <4AF89561.9010408@v.loewis.de> References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> <740c3aec0911090840w7d3947a2ocd36b2d4ae5d70ff@mail.gmail.com> <200911100716.57045.steve@pearwood.info> <4AF89561.9010408@v.loewis.de> Message-ID: <4AF945A7.5090904@gmail.com> Martin v. L?wis wrote: >> I'm not sure, but isn't that thread-unsafe? > > You are right; it's thread-unsafe. > > I would fix it by catching the RuntimeError, and retrying. Given the > current GIL strategy (including proposed changes to it), it won't happen > two times in a row, so the number of retries would be bounded. It's also one of the major reasons for not sharing mutable containers between threads if you can avoid it (and serialising access to them if you can't) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From wa2n39 at gmail.com Wed Nov 11 03:17:19 2009 From: wa2n39 at gmail.com (wa2n39 at gmail.com) Date: Wed, 11 Nov 2009 02:17:19 +0000 Subject: [Python-Dev] Python-Dev Digest, Vol 76, Issue 83 Message-ID: <258427219-1257851199-cardhu_decombobulator_blackberry.rim.net-1355761493-@bda056.bisx.prodap.on.blackberry> Unreg ------Pesan Asli------ Dari:python-dev-request at python.org Pengirim:python-dev-bounces+wa2n39=gmail.com at python.org Ke:python-dev at python.org Balas Ke:python-dev at python.org Perihal:Python-Dev Digest, Vol 76, Issue 83 Terkirim:10 Nov 2009 03:00 Send Python-Dev mailing list submissions to python-dev at python.org To subscribe or unsubscribe via the World Wide Web, visit http://mail.python.org/mailman/listinfo/python-dev or, via email, send a message with subject or body 'help' to python-dev-request at python.org You can reach the person managing the list at python-dev-owner at python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Python-Dev digest..." Today's Topics: 1. Re: Retrieve an arbitrary element from asetwithoutremoving it (Nick Coghlan) ---------------------------------------------------------------------- Message: 1 Date: Tue, 10 Nov 2009 20:51:19 +1000 From: Nick Coghlan To: "Martin v. L?wis" Cc: python-dev at python.org Subject: Re: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it Message-ID: <4AF945A7.5090904 at gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Martin v. L?wis wrote: >> I'm not sure, but isn't that thread-unsafe? > > You are right; it's thread-unsafe. > > I would fix it by catching the RuntimeError, and retrying. Given the > current GIL strategy (including proposed changes to it), it won't happen > two times in a row, so the number of retries would be bounded. It's also one of the major reasons for not sharing mutable containers between threads if you can avoid it (and serialising access to them if you can't) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- ------------------------------ _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev End of Python-Dev Digest, Vol 76, Issue 83 ****************************************** Sent from my BlackBerry? powered by Sinyal Kuat INDOSAT From ncoghlan at gmail.com Tue Nov 10 14:13:28 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 10 Nov 2009 23:13:28 +1000 Subject: [Python-Dev] 2.7/3.2 release schedule In-Reply-To: <71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com> References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> <71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com> Message-ID: <4AF966F8.5040008@gmail.com> Arfrever Frehtes Taifersar Arahesis wrote: > 2009-11-02 21:00 Benjamin Peterson napisa?(a): >> I've updated PEP 373 with my proposed release schedule: >> >> - 2.7/3.2 alpha 1 2009-12-05 >> - 2.7/3.2 alpha 2 2010-01-09 >> - 2.7/3.2 alpha 3 2010-02-06 >> - 2.7/3.2 alpha 4 2010-03-06 >> - 2.7/3.2 beta 1 2010-04-03 >> - 2.7/3.2 beta 2 2010-05-01 >> - 2.7/3.2 rc1 2010-05-29 >> - 2.7/3.2 rc2 2010-06-12 >> - 2.7/3.2 final 2010-06-26 > > PEP 3003 states that Python 3.2 will be released 18-24 months after > Python 3.1. Python 3.1 was released on June 2009-06-27 [1], so > theoretically Python 3.2 should be released not before 2010-12-19 [2]. The PEP 3003 text isn't allowing for the fact that 3.1 is "3.0 as it should have been", so the starting point for the 18-24 month rule of thumb is actually back when 3.0 was released in Dec 2008. This was discussed a fair bit back when the decision was made to do the short release cycle between 3.0 and 3.1 in order to address some of the more glaring shortcomings of the 3.0 release. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From solipsis at pitrou.net Tue Nov 10 14:55:48 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 10 Nov 2009 13:55:48 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Retrieve_an_arbitrary_element_from=09asetw?= =?utf-8?q?ithoutremoving_it?= References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> <740c3aec0911090840w7d3947a2ocd36b2d4ae5d70ff@mail.gmail.com> <200911100716.57045.steve@pearwood.info> <4AF89561.9010408@v.loewis.de> <4AF945A7.5090904@gmail.com> Message-ID: Nick Coghlan gmail.com> writes: > > It's also one of the major reasons for not sharing mutable containers > between threads if you can avoid it (and serialising access to them if > you can't) Not necessarily, for example it is common to rely on the fact that list.append() is atomic. Regards Antoine. From barry at python.org Mon Nov 9 15:12:02 2009 From: barry at python.org (Barry Warsaw) Date: Mon, 9 Nov 2009 08:12:02 -0600 Subject: [Python-Dev] OpenSSL vulnerability In-Reply-To: <4AF71457.3050705@v.loewis.de> References: <40373.1257545902@parc.com> <20091106223634.3229.2049231320.divmod.xquotient.232@localhost.localdomain> <4AF71457.3050705@v.loewis.de> Message-ID: <3BC6D0BD-9677-40E2-9CF8-2353E55278FD@python.org> On Nov 8, 2009, at 12:56 PM, Martin v. L?wis wrote: >>> Also, for Python 2.5 and earlier, any SSL-based code is vulnerable >>> to a MitM >>> anyway, so this can only be an issue for code using the new APIs >>> in Python >>> 2.6. >> >> That's not going to stop the >> wannabe-self-proclaimed-so-called-vulnerability-"experts" from >> whining >> about Python not releasing updated binary distributions though. :-( > > The Windows binaries currently build with 0.9.8g. Since changing that > would be a source code change (even though just a single line), I > think > a full source release would be necessary (most likely then for both > 2.6 > and 3.1). I don't think it's worth making a quick 2.6.5 release for this if it's primary intent is to produce new Windows binaries. I'm okay with making the changes to the tree, but we'll release 2.6.5 on a "normal" schedule. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 194 bytes Desc: This is a digitally signed message part URL: From ncoghlan at gmail.com Tue Nov 10 15:27:30 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 11 Nov 2009 00:27:30 +1000 Subject: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it In-Reply-To: References: <7b0045ed0910301843y7059d316s8a851142b78b7aeb@mail.gmail.com> <97069B15530D4C66AC1D4BCCB70E2FD5@RaymondLaptop1> <740c3aec0911090840w7d3947a2ocd36b2d4ae5d70ff@mail.gmail.com> <200911100716.57045.steve@pearwood.info> <4AF89561.9010408@v.loewis.de> <4AF945A7.5090904@gmail.com> Message-ID: <4AF97852.9020100@gmail.com> Antoine Pitrou wrote: > Nick Coghlan gmail.com> writes: >> It's also one of the major reasons for not sharing mutable containers >> between threads if you can avoid it (and serialising access to them if >> you can't) > > Not necessarily, for example it is common to rely on the fact that list.append() > is atomic. s/"mutable containers"/"mutable-containers-that-object-loudly-to-their-size-changing-during-iteration -like-sets-and-dicts" :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Tue Nov 10 15:28:50 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 11 Nov 2009 00:28:50 +1000 Subject: [Python-Dev] OpenSSL vulnerability In-Reply-To: <3BC6D0BD-9677-40E2-9CF8-2353E55278FD@python.org> References: <40373.1257545902@parc.com> <20091106223634.3229.2049231320.divmod.xquotient.232@localhost.localdomain> <4AF71457.3050705@v.loewis.de> <3BC6D0BD-9677-40E2-9CF8-2353E55278FD@python.org> Message-ID: <4AF978A2.2090304@gmail.com> Barry Warsaw wrote: > I don't think it's worth making a quick 2.6.5 release for this if it's > primary intent is to produce new Windows binaries. I'm okay with making > the changes to the tree, but we'll release 2.6.5 on a "normal" schedule. Perhaps publish a source patch relative to 2.6.4 for people that would like to rebuild their own Windows binaries with just that change? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From barry at python.org Tue Nov 10 15:34:40 2009 From: barry at python.org (Barry Warsaw) Date: Tue, 10 Nov 2009 08:34:40 -0600 Subject: [Python-Dev] OpenSSL vulnerability In-Reply-To: <4AF978A2.2090304@gmail.com> References: <40373.1257545902@parc.com> <20091106223634.3229.2049231320.divmod.xquotient.232@localhost.localdomain> <4AF71457.3050705@v.loewis.de> <3BC6D0BD-9677-40E2-9CF8-2353E55278FD@python.org> <4AF978A2.2090304@gmail.com> Message-ID: <4A6F4151-3083-4745-B76A-F16921966ED9@python.org> On Nov 10, 2009, at 8:28 AM, Nick Coghlan wrote: > Barry Warsaw wrote: >> I don't think it's worth making a quick 2.6.5 release for this if >> it's >> primary intent is to produce new Windows binaries. I'm okay with >> making >> the changes to the tree, but we'll release 2.6.5 on a "normal" >> schedule. > > Perhaps publish a source patch relative to 2.6.4 for people that would > like to rebuild their own Windows binaries with just that change? +1. A link to that could easily go on the 2.6.4 page. Perhaps the right way to do it is to put the blessed patch in roundup and add a link to that patch file on the website. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 194 bytes Desc: This is a digitally signed message part URL: From guido at python.org Tue Nov 10 16:55:11 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Nov 2009 07:55:11 -0800 Subject: [Python-Dev] 2.7/3.2 release schedule In-Reply-To: <4AF966F8.5040008@gmail.com> References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> <71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com> <4AF966F8.5040008@gmail.com> Message-ID: On Tue, Nov 10, 2009 at 5:13 AM, Nick Coghlan wrote: > Arfrever Frehtes Taifersar Arahesis wrote: >> 2009-11-02 21:00 Benjamin Peterson napisa?(a): >>> I've updated PEP 373 with my proposed release schedule: >>> >>> - 2.7/3.2 alpha 1 2009-12-05 >>> - 2.7/3.2 alpha 2 2010-01-09 >>> - 2.7/3.2 alpha 3 2010-02-06 >>> - 2.7/3.2 alpha 4 2010-03-06 >>> - 2.7/3.2 beta 1 2010-04-03 >>> - 2.7/3.2 beta 2 2010-05-01 >>> - 2.7/3.2 rc1 2010-05-29 >>> - 2.7/3.2 rc2 2010-06-12 >>> - 2.7/3.2 final 2010-06-26 >> >> PEP 3003 states that Python 3.2 will be released 18-24 months after >> Python 3.1. Python 3.1 was released on June 2009-06-27 [1], so >> theoretically Python 3.2 should be released not before 2010-12-19 [2]. > > The PEP 3003 text isn't allowing for the fact that 3.1 is "3.0 as it > should have been", so the starting point for the 18-24 month rule of > thumb is actually back when 3.0 was released in Dec 2008. This was > discussed a fair bit back when the decision was made to do the short > release cycle between 3.0 and 3.1 in order to address some of the more > glaring shortcomings of the 3.0 release. Was this discussed somewhere? When I agreed to an early 3.1 release (or did I propose it?) I'm quite sure that I expected 3.2 to come the usual time (i.e., 18-24 months) after 3.1. I think I said something to the extent of "we'll treat 3.1 the same way we treat any release" which IMO implies a lifetime of 18-24 months. -- --Guido van Rossum (python.org/~guido) From solipsis at pitrou.net Tue Nov 10 17:09:53 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 10 Nov 2009 16:09:53 +0000 (UTC) Subject: [Python-Dev] 2.7/3.2 release schedule References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> <71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com> <4AF966F8.5040008@gmail.com> Message-ID: Guido van Rossum python.org> writes: > > Was this discussed somewhere? I don't remember so, except for a short subthread on python-ideas where you indeed mentioned (to my disappointment :-)) that you were against a one-year release period. Regards Antoine. From techtonik at gmail.com Tue Nov 10 17:20:58 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Tue, 10 Nov 2009 18:20:58 +0200 Subject: [Python-Dev] Replacing IDLE Message-ID: On Tue, Nov 10, 2009 at 12:10 AM, "Martin v. L?wis" wrote: >> >> Anatoly's question is actually a fair one for python-dev - we're the >> ones that *ship* Idle, so it is legitimate to ask our reasons for >> continuing to do so. > > OTOH, the second (or, rather, third) question (does anybody think it > should be replaced) is indeed on-topic for python-dev. That is the exact question I have in my mind. Sorry for not being able to state it clearly. I'll be more straightforward next time. > I didn't really answer that question before, so I do now: I have not > personally plans to replace it, and I'm skeptical wrt. anybody else's > plans unless there is specific code in existence that IDLE could be > replaced *with*. There are a lot of Python editors. It is unlikely that they implement all IDLE features, but I believe that 20% of IDLE features that is used by 80% of users are guaranteed by any editor out of the box. Assuming that they are more extensible than IDLE anyway. (I would say "perfection is achieved when there is nothing more to add...", but I promised to be straightforward). If we filter list of http://wiki.python.org/moin/PythonEditors by language/license/framework, we will be able to see if there is any suitable open source Python code to replace IDLE's. Then there will be another issue - all editors are based upon some frameworks - I didn't see any popular cross-platform GUI toolkits in Python, so we will inevitably face the need to replace Tkinter with other default GUI toolkit. Should we place an editor first and combine the list a of unique features of IDLE that we (as developers) will surely miss OR should we think what GUI framework will be more beneficial for Python users if shipped by default (keeping in mind that we might have to add missing features to whatever editor we choose later)? -- anatoly t. From ronaldoussoren at mac.com Tue Nov 10 17:44:29 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Tue, 10 Nov 2009 17:44:29 +0100 Subject: [Python-Dev] Replacing IDLE In-Reply-To: References: Message-ID: <59691AA2-995E-452B-B21D-0C7F8861C7A6@mac.com> On 10 Nov, 2009, at 17:20, anatoly techtonik wrote: > On Tue, Nov 10, 2009 at 12:10 AM, "Martin v. L?wis" wrote: >>> >>> Anatoly's question is actually a fair one for python-dev - we're the >>> ones that *ship* Idle, so it is legitimate to ask our reasons for >>> continuing to do so. >> >> OTOH, the second (or, rather, third) question (does anybody think it >> should be replaced) is indeed on-topic for python-dev. > > That is the exact question I have in my mind. Sorry for not being able > to state it clearly. I'll be more straightforward next time. What's so bad about IDLE that you'd like to replace it? The obvious alternative to replacing IDLE is to fix whatever is wrong with it. I sometimes use IDLE and it's not too bad as a simple Python editor. A major reason for not using it more often is that the keystrokes for my prefered text-editor are in my muscle memory and IDLE is not that much of an improvement over my current habits that I'm willing the re-educate myself. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From guido at python.org Tue Nov 10 18:26:31 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Nov 2009 09:26:31 -0800 Subject: [Python-Dev] 2.7/3.2 release schedule In-Reply-To: References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> <71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com> <4AF966F8.5040008@gmail.com> Message-ID: On Tue, Nov 10, 2009 at 8:09 AM, Antoine Pitrou wrote: > Guido van Rossum python.org> writes: >> >> Was this discussed somewhere? > > I don't remember so, except for a short subthread on python-ideas where you > indeed mentioned (to my disappointment :-)) that you were against a one-year > release period. Trust me on this too, please. We used to have releases once a year and we got really big serious feedback from our biggest users that the release cycle was going too fast. We discussed it amply and agreed on a minimum time of 18 months between releases. This was quite a while ago (I recall it being somewhere between 2000 and 2004) but I don't see that the situation has really changed -- if anything, we need to slow down more. We should really have a PEP for this, like we do for bugfix releases (PEP 6), but at the time we weren't so anal about PEPs for process. I realize this can be frustrating for developers who want to see their code released. I had the same feeling at the time. But when it was explained to me what a version upgrade looks like for a typical large company who have 100,000 or more lines of Python, often with insufficient tests, written by people who no longer work for the company or expensive consultants, I realized that releasing once a year was a break-neck pace for such users. There was wide discussion and community agreement on the 18 months minimum, as a compromise between the needs of large users (who would have been happier with two years) and the desires of developers (who, like you, preferred shorter release cycles). I really don't think we should change this "contract with our users" now. If necessary, we'll have to write a PEP to describe and explain it. -- --Guido van Rossum (python.org/~guido) From regebro at gmail.com Tue Nov 10 18:27:00 2009 From: regebro at gmail.com (Lennart Regebro) Date: Tue, 10 Nov 2009 18:27:00 +0100 Subject: [Python-Dev] raw binary data and 2to3 In-Reply-To: <1afaf6160911081748u3f47f5dche74b1f0171c5bcff@mail.gmail.com> References: <1afaf6160911081748u3f47f5dche74b1f0171c5bcff@mail.gmail.com> Message-ID: <319e029f0911100927k78a05be4o1d3e82ba5fb12873@mail.gmail.com> 2009/11/9 Benjamin Peterson : > Not to my knowledge. I would prefer to not add a fixer for this > directly to 2to3 because it is not correct for most programs. However, > I think 2to3 should grow some sort of plugin system, so custom fixers > can easily be written and used. Well, 2to3 is only plugins, so. :) The difficulty in writing fixers is understanding the abstract parse tree or whatever it's called and how to use it. Documentation on that for people who don't have a doctorate in computer linguistics would probably be a good idea. ;) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From python at rcn.com Tue Nov 10 19:26:13 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 10 Nov 2009 10:26:13 -0800 Subject: [Python-Dev] 2.7/3.2 release schedule References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> <71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com> <4AF966F8.5040008@gmail.com> Message-ID: <5FFB0E2375804AD28576A8C3EE953A9B@RaymondLaptop1> [Guido van Rossum] >. We used to have releases once a year and > we got really big serious feedback from our biggest users that the > release cycle was going too fast. We discussed it amply and agreed on > a minimum time of 18 months between releases. If the language moratorium goes into effect, would shorter release cycles still have a negative impact? Do people possibly want slower changes to the language and faster updates to the library? I don't know the answer. Just asking how this matches up with the feedback you have gotten previously. Raymond From tseaver at palladion.com Tue Nov 10 19:42:58 2009 From: tseaver at palladion.com (Tres Seaver) Date: Tue, 10 Nov 2009 13:42:58 -0500 Subject: [Python-Dev] 2.7/3.2 release schedule In-Reply-To: <5FFB0E2375804AD28576A8C3EE953A9B@RaymondLaptop1> References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> <71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com> <4AF966F8.5040008@gmail.com> <5FFB0E2375804AD28576A8C3EE953A9B@RaymondLaptop1> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Raymond Hettinger wrote: > [Guido van Rossum] >> . We used to have releases once a year and >> we got really big serious feedback from our biggest users that the >> release cycle was going too fast. We discussed it amply and agreed on >> a minimum time of 18 months between releases. > > If the language moratorium goes into effect, would shorter release cycles > still have a negative impact? Do people possibly want slower changes > to the language and faster updates to the library? > > I don't know the answer. Just asking how this matches up with the > feedback you have gotten previously. Depends on the kind of changes you are talking about: backward- incompatible ones (like the hashlib / medusa ones in 2.6, for instance) are likely going to be unpopular with the "Python with a tie" crowd (I think that was the name of the IPC BoF from which the original feedback coalesced). Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkr5tC0ACgkQ+gerLs4ltQ4QzQCg1eUsdnAEyIficxjeFQevR9ul pv0An2qpDswkGhqAHkM2REtGE9Zpz8+V =t5kK -----END PGP SIGNATURE----- From fuzzyman at voidspace.org.uk Tue Nov 10 20:26:47 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 10 Nov 2009 20:26:47 +0100 Subject: [Python-Dev] Replacing IDLE In-Reply-To: References: Message-ID: <4AF9BE77.5050100@voidspace.org.uk> anatoly techtonik wrote: > On Tue, Nov 10, 2009 at 12:10 AM, "Martin v. L?wis" wrote: > >>> Anatoly's question is actually a fair one for python-dev - we're the >>> ones that *ship* Idle, so it is legitimate to ask our reasons for >>> continuing to do so. >>> >> OTOH, the second (or, rather, third) question (does anybody think it >> should be replaced) is indeed on-topic for python-dev. >> > > That is the exact question I have in my mind. Sorry for not being able > to state it clearly. I'll be more straightforward next time. > > >> I didn't really answer that question before, so I do now: I have not >> personally plans to replace it, and I'm skeptical wrt. anybody else's >> plans unless there is specific code in existence that IDLE could be >> replaced *with*. >> > > There are a lot of Python editors. It is unlikely that they implement > all IDLE features, but I believe that 20% of IDLE features that is > used by 80% of users are guaranteed by any editor out of the box. > Assuming that they are more extensible than IDLE anyway. (I would say > "perfection is achieved when there is nothing more to add...", but I > promised to be straightforward). > > If we filter list of http://wiki.python.org/moin/PythonEditors by > language/license/framework, we will be able to see if there is any > suitable open source Python code to replace IDLE's. > > Why replace IDLE? > Then there will be another issue - all editors are based upon some > frameworks - I didn't see any popular cross-platform GUI toolkits in > Python, so we will inevitably face the need to replace Tkinter with > other default GUI toolkit. > > This just *won't* happen. (At least not in the forseeable future.) Better filter your list to only those IDEs that use Tkinter or have a GUI not dependent on packages that aren't already in the standard library. Michael -- http://www.ironpythoninaction.com/ From martin at v.loewis.de Tue Nov 10 21:49:28 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Tue, 10 Nov 2009 21:49:28 +0100 Subject: [Python-Dev] Replacing IDLE In-Reply-To: References: Message-ID: <4AF9D1D8.3060406@v.loewis.de> > If we filter list of http://wiki.python.org/moin/PythonEditors by > language/license/framework, we will be able to see if there is any > suitable open source Python code to replace IDLE's. This is not how it works. We cannot incorporate something into Python without explicit consent and support from the author(s). So for any editor to be incorporated as a replacement (along with all libraries it depends on) we would need a commitment from the author(s) that they support the fork that we create (and eventually abandon their stand-alone release in favor of the fork). > Should we place an editor first and combine the list a of unique > features of IDLE that we (as developers) will surely miss OR should > we think what GUI framework will be more beneficial for Python users > if shipped by default (keeping in mind that we might have to add > missing features to whatever editor we choose later)? I think neither approach can possibly work. It's always the authors of contribution who need to act first. Regards, Martin From martin at v.loewis.de Tue Nov 10 22:05:41 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Tue, 10 Nov 2009 22:05:41 +0100 Subject: [Python-Dev] 2.7/3.2 release schedule In-Reply-To: <4AF966F8.5040008@gmail.com> References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> <71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com> <4AF966F8.5040008@gmail.com> Message-ID: <4AF9D5A5.5000707@v.loewis.de> >> PEP 3003 states that Python 3.2 will be released 18-24 months after >> Python 3.1. Python 3.1 was released on June 2009-06-27 [1], so >> theoretically Python 3.2 should be released not before 2010-12-19 [2]. > > The PEP 3003 text isn't allowing for the fact that 3.1 is "3.0 as it > should have been", so the starting point for the 18-24 month rule of > thumb is actually back when 3.0 was released in Dec 2008. This was > discussed a fair bit back when the decision was made to do the short > release cycle between 3.0 and 3.1 in order to address some of the more > glaring shortcomings of the 3.0 release. I agree with everybody else who said that a) there was *no* consensus that the release cycle for 3.2 should be shortened because of 3.1 being released early. I also remember the opposite. b) whatever past discussion may have been, it would be a mistake to release 3.2 earlier than 18 months after 3.1. Of course, 2.7 *could* be released by the proposed schedule; it just would have to be decoupled from 3.2 (just as 2.6 eventually got decoupled from 3.0). I personally think that decoupling the releases would be best, i.e. not start thinking about 3.2 for another 6 months. Regards, Martin From ncoghlan at gmail.com Tue Nov 10 22:16:50 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 11 Nov 2009 07:16:50 +1000 Subject: [Python-Dev] 2.7/3.2 release schedule In-Reply-To: <4AF9D5A5.5000707@v.loewis.de> References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> <71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com> <4AF966F8.5040008@gmail.com> <4AF9D5A5.5000707@v.loewis.de> Message-ID: <4AF9D842.5000808@gmail.com> Martin v. L?wis wrote: >>> PEP 3003 states that Python 3.2 will be released 18-24 months after >>> Python 3.1. Python 3.1 was released on June 2009-06-27 [1], so >>> theoretically Python 3.2 should be released not before 2010-12-19 [2]. >> The PEP 3003 text isn't allowing for the fact that 3.1 is "3.0 as it >> should have been", so the starting point for the 18-24 month rule of >> thumb is actually back when 3.0 was released in Dec 2008. This was >> discussed a fair bit back when the decision was made to do the short >> release cycle between 3.0 and 3.1 in order to address some of the more >> glaring shortcomings of the 3.0 release. > > I agree with everybody else who said that > > a) there was *no* consensus that the release cycle for 3.2 should be > shortened because of 3.1 being released early. I also remember the > opposite. > > b) whatever past discussion may have been, it would be a mistake to > release 3.2 earlier than 18 months after 3.1. Fair enough - I didn't remember that discussion all that well, but assumed it had reached that consensus due to the lack of comment when Benjamin originally posted his proposed schedule. Your response and Guido's clearly indicate otherwise :) If that wasn't the consensus, then all the dates should slide back 6 months (i.e. no alphas until June 2010). (I actually prefer that since it gives me a chance to find a cleaner approach to the runpy.run_path problem, but didn't want to rehash a discussion I thought we had already had) > Of course, 2.7 *could* be released by the proposed schedule; it just > would have to be decoupled from 3.2 (just as 2.6 eventually got > decoupled from 3.0). That leads to a 2.x version with features that aren't yet available in a 3.x version though. I thought we weren't planning on doing that anymore. > I personally think that decoupling the releases would be best, i.e. > not start thinking about 3.2 for another 6 months. Or just have the timing be 18 months for 3.2 and 24 months for 2.7 (i.e. push the first alpha of both back to June next year). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From solipsis at pitrou.net Tue Nov 10 22:26:24 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 10 Nov 2009 21:26:24 +0000 (UTC) Subject: [Python-Dev] Reworking the GIL References: <1256502140.5621.163.camel@localhost> <1257613317.3437.63.camel@localhost> Message-ID: Hello again, I've now removed priority requests, tried to improve the internal doc a bit, and merged the changes into py3k. Afterwards, the new Windows 7 buildbot has hung in test_multiprocessing, but I don't know whether it's related. Regards Antoine. Guido van Rossum python.org> writes: > > > I would remove them -- if and when we find there's a need for > something like them I suspect it won't look like what you currently > have, so it's better not to complexificate your current patch with > them. (I would remove all traces, including the conditions passed into > the end macros.) > > >> My only suggestion so far: the description could use more explicit > >> documentation on the various variables and macros and how they > >> combine. > > > > Is it before or after > > http://mail.python.org/pipermail/python-checkins/2009- November/087482.html ? > > After. While that is already really helpful, not all the code is > easily linked back to paragraphs in that comment block, and some > variables are not mentioned by name in the block. > From solipsis at pitrou.net Tue Nov 10 22:50:26 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 10 Nov 2009 21:50:26 +0000 (UTC) Subject: [Python-Dev] Status of the Buildbot fleet and related bugs References: Message-ID: Le Thu, 05 Nov 2009 22:53:27 -0500, R. David Murray a ?crit?: > The buildbot waterfall is much greener now. Thanks to all who have > contributed to making it so (and it hasn't just been Mark and Antoine > and I, though we've been the most directly active (and yes, Mark, you > did contribute several fixes!)). The buildbots still show occasional oddities. For example, right now in the page "http://www.python.org/dev/buildbot/3.x/", some results have disappeared (the columns for "AMD64 Ubuntu" builders have become empty). Moreover, some buildslaves have gone back in time (they are building r76188 after having built and tested r76195)... I swear the new GIL doesn't include a time machine. Regards Antoine. From martin at v.loewis.de Tue Nov 10 23:06:14 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Tue, 10 Nov 2009 23:06:14 +0100 Subject: [Python-Dev] Status of the Buildbot fleet and related bugs In-Reply-To: References: Message-ID: <4AF9E3D6.2000807@v.loewis.de> >> The buildbot waterfall is much greener now. Thanks to all who have >> contributed to making it so (and it hasn't just been Mark and Antoine >> and I, though we've been the most directly active (and yes, Mark, you >> did contribute several fixes!)). > > The buildbots still show occasional oddities. For example, right now in > the page "http://www.python.org/dev/buildbot/3.x/", some results have > disappeared (the columns for "AMD64 Ubuntu" builders have become empty). Yes, I noticed it too. It will go away after some page reloads. > Moreover, some buildslaves have gone back in time (they are building > r76188 after having built and tested r76195)... I swear the new GIL > doesn't include a time machine. That's because I resubmitted these changes after restarting the master. Regards, Martin From benjamin at python.org Tue Nov 10 23:06:46 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 10 Nov 2009 16:06:46 -0600 Subject: [Python-Dev] 2.7/3.2 release schedule In-Reply-To: <4AF9D5A5.5000707@v.loewis.de> References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> <71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com> <4AF966F8.5040008@gmail.com> <4AF9D5A5.5000707@v.loewis.de> Message-ID: <1afaf6160911101406t6da78accq8918eb4ceb1adca5@mail.gmail.com> 2009/11/10 "Martin v. L?wis" : > I personally think that decoupling the releases would be best, i.e. > not start thinking about 3.2 for another 6 months. The problem with that is that there is a period of time where 2.x has features which 3.x doesn't. My preference is to move back the whole schedule 6 months. -- Regards, Benjamin From martin at v.loewis.de Tue Nov 10 23:09:42 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Tue, 10 Nov 2009 23:09:42 +0100 Subject: [Python-Dev] 2.7/3.2 release schedule In-Reply-To: <1afaf6160911101406t6da78accq8918eb4ceb1adca5@mail.gmail.com> References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> <71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com> <4AF966F8.5040008@gmail.com> <4AF9D5A5.5000707@v.loewis.de> <1afaf6160911101406t6da78accq8918eb4ceb1adca5@mail.gmail.com> Message-ID: <4AF9E4A6.8070304@v.loewis.de> Benjamin Peterson wrote: > 2009/11/10 "Martin v. L?wis" : >> I personally think that decoupling the releases would be best, i.e. >> not start thinking about 3.2 for another 6 months. > > The problem with that is that there is a period of time where 2.x has > features which 3.x doesn't. My preference is to move back the whole > schedule 6 months. Ok, fine with me as well. FWIW, I don't see that (2.7 released before 3.2) as a problem (but I understand that others might, for what I would consider formal reasons). Regards, Martin From g.brandl at gmx.net Tue Nov 10 23:27:09 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 10 Nov 2009 22:27:09 +0000 Subject: [Python-Dev] IDLE as default Python editor In-Reply-To: <4AF944A9.2030108@gmail.com> References: <87iqdk2n7b.fsf@benfinney.id.au> <4AF80E73.5090104@gmail.com> <4AF89370.2040206@v.loewis.de> <4AF944A9.2030108@gmail.com> Message-ID: Nick Coghlan schrieb: > Martin v. L?wis wrote: >> Nick Coghlan wrote: >>> Ben Finney wrote: >>>> anatoly techtonik writes: >>>> >>>>> Quite an interesting question recently popped up in pygame community >>>>> that I'd like to ask to Python developers. >>>> This forum is specifically about development *of* Python. >>> Anatoly's question is actually a fair one for python-dev - we're the >>> ones that *ship* Idle, so it is legitimate to ask our reasons for >>> continuing to do so. >> >> But he didn't - instead, he asked how many of us use it. Usage by >> committers is not (or shouldn't be) a primary criterion for including >> or not including something. Instead, usage in the community should be, >> and python-dev is indeed the wrong place to estimate that. > > I took it as being a somewhat relevant leading question. If the response > had been "No, none of us use it" then the obvious follow-up question > would be "Why do you ship something that you don't consider worth using?". Quite simple: because we can't possibly ship Emacs. cheers, Georg From wa2n39 at gmail.com Wed Nov 11 14:36:29 2009 From: wa2n39 at gmail.com (wa2n39 at gmail.com) Date: Wed, 11 Nov 2009 13:36:29 +0000 Subject: [Python-Dev] Python-Dev Digest, Vol 76, Issue 87 In-Reply-To: References: Message-ID: <873739028-1257892142-cardhu_decombobulator_blackberry.rim.net-1348400255-@bda056.bisx.prodap.on.blackberry> Stop email Sent from my BlackBerry? powered by Sinyal Kuat INDOSAT -----Original Message----- From: python-dev-request at python.org Date: Tue, 10 Nov 2009 23:09:45 To: Subject: Python-Dev Digest, Vol 76, Issue 87 Send Python-Dev mailing list submissions to python-dev at python.org To subscribe or unsubscribe via the World Wide Web, visit http://mail.python.org/mailman/listinfo/python-dev or, via email, send a message with subject or body 'help' to python-dev-request at python.org You can reach the person managing the list at python-dev-owner at python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Python-Dev digest..." Today's Topics: 1. Re: 2.7/3.2 release schedule (Martin v. L?wis) 2. Re: 2.7/3.2 release schedule (Nick Coghlan) 3. Re: Reworking the GIL (Antoine Pitrou) 4. Re: Status of the Buildbot fleet and related bugs (Antoine Pitrou) 5. Re: Status of the Buildbot fleet and related bugs (Martin v. L?wis) 6. Re: 2.7/3.2 release schedule (Benjamin Peterson) 7. Re: 2.7/3.2 release schedule (Martin v. L?wis) ---------------------------------------------------------------------- Message: 1 Date: Tue, 10 Nov 2009 22:05:41 +0100 From: "Martin v. L?wis" To: Nick Coghlan Cc: Python Development Subject: Re: [Python-Dev] 2.7/3.2 release schedule Message-ID: <4AF9D5A5.5000707 at v.loewis.de> Content-Type: text/plain; charset=UTF-8 >> PEP 3003 states that Python 3.2 will be released 18-24 months after >> Python 3.1. Python 3.1 was released on June 2009-06-27 [1], so >> theoretically Python 3.2 should be released not before 2010-12-19 [2]. > > The PEP 3003 text isn't allowing for the fact that 3.1 is "3.0 as it > should have been", so the starting point for the 18-24 month rule of > thumb is actually back when 3.0 was released in Dec 2008. This was > discussed a fair bit back when the decision was made to do the short > release cycle between 3.0 and 3.1 in order to address some of the more > glaring shortcomings of the 3.0 release. I agree with everybody else who said that a) there was *no* consensus that the release cycle for 3.2 should be shortened because of 3.1 being released early. I also remember the opposite. b) whatever past discussion may have been, it would be a mistake to release 3.2 earlier than 18 months after 3.1. Of course, 2.7 *could* be released by the proposed schedule; it just would have to be decoupled from 3.2 (just as 2.6 eventually got decoupled from 3.0). I personally think that decoupling the releases would be best, i.e. not start thinking about 3.2 for another 6 months. Regards, Martin ------------------------------ Message: 2 Date: Wed, 11 Nov 2009 07:16:50 +1000 From: Nick Coghlan To: "Martin v. L?wis" Cc: Python Development Subject: Re: [Python-Dev] 2.7/3.2 release schedule Message-ID: <4AF9D842.5000808 at gmail.com> Content-Type: text/plain; charset=UTF-8 Martin v. L?wis wrote: >>> PEP 3003 states that Python 3.2 will be released 18-24 months after >>> Python 3.1. Python 3.1 was released on June 2009-06-27 [1], so >>> theoretically Python 3.2 should be released not before 2010-12-19 [2]. >> The PEP 3003 text isn't allowing for the fact that 3.1 is "3.0 as it >> should have been", so the starting point for the 18-24 month rule of >> thumb is actually back when 3.0 was released in Dec 2008. This was >> discussed a fair bit back when the decision was made to do the short >> release cycle between 3.0 and 3.1 in order to address some of the more >> glaring shortcomings of the 3.0 release. > > I agree with everybody else who said that > > a) there was *no* consensus that the release cycle for 3.2 should be > shortened because of 3.1 being released early. I also remember the > opposite. > > b) whatever past discussion may have been, it would be a mistake to > release 3.2 earlier than 18 months after 3.1. Fair enough - I didn't remember that discussion all that well, but assumed it had reached that consensus due to the lack of comment when Benjamin originally posted his proposed schedule. Your response and Guido's clearly indicate otherwise :) If that wasn't the consensus, then all the dates should slide back 6 months (i.e. no alphas until June 2010). (I actually prefer that since it gives me a chance to find a cleaner approach to the runpy.run_path problem, but didn't want to rehash a discussion I thought we had already had) > Of course, 2.7 *could* be released by the proposed schedule; it just > would have to be decoupled from 3.2 (just as 2.6 eventually got > decoupled from 3.0). That leads to a 2.x version with features that aren't yet available in a 3.x version though. I thought we weren't planning on doing that anymore. > I personally think that decoupling the releases would be best, i.e. > not start thinking about 3.2 for another 6 months. Or just have the timing be 18 months for 3.2 and 24 months for 2.7 (i.e. push the first alpha of both back to June next year). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- ------------------------------ Message: 3 Date: Tue, 10 Nov 2009 21:26:24 +0000 (UTC) From: Antoine Pitrou To: python-dev at python.org Subject: Re: [Python-Dev] Reworking the GIL Message-ID: Content-Type: text/plain; charset=UTF-8 Hello again, I've now removed priority requests, tried to improve the internal doc a bit, and merged the changes into py3k. Afterwards, the new Windows 7 buildbot has hung in test_multiprocessing, but I don't know whether it's related. Regards Antoine. Guido van Rossum python.org> writes: > > > I would remove them -- if and when we find there's a need for > something like them I suspect it won't look like what you currently > have, so it's better not to complexificate your current patch with > them. (I would remove all traces, including the conditions passed into > the end macros.) > > >> My only suggestion so far: the description could use more explicit > >> documentation on the various variables and macros and how they > >> combine. > > > > Is it before or after > > http://mail.python.org/pipermail/python-checkins/2009- November/087482.html ? > > After. While that is already really helpful, not all the code is > easily linked back to paragraphs in that comment block, and some > variables are not mentioned by name in the block. > ------------------------------ Message: 4 Date: Tue, 10 Nov 2009 21:50:26 +0000 (UTC) From: Antoine Pitrou To: python-dev at python.org Subject: Re: [Python-Dev] Status of the Buildbot fleet and related bugs Message-ID: Content-Type: text/plain; charset=UTF-8 Le Thu, 05 Nov 2009 22:53:27 -0500, R. David Murray a ?crit?: > The buildbot waterfall is much greener now. Thanks to all who have > contributed to making it so (and it hasn't just been Mark and Antoine > and I, though we've been the most directly active (and yes, Mark, you > did contribute several fixes!)). The buildbots still show occasional oddities. For example, right now in the page "http://www.python.org/dev/buildbot/3.x/", some results have disappeared (the columns for "AMD64 Ubuntu" builders have become empty). Moreover, some buildslaves have gone back in time (they are building r76188 after having built and tested r76195)... I swear the new GIL doesn't include a time machine. Regards Antoine. ------------------------------ Message: 5 Date: Tue, 10 Nov 2009 23:06:14 +0100 From: "Martin v. L?wis" To: Antoine Pitrou Cc: python-dev at python.org Subject: Re: [Python-Dev] Status of the Buildbot fleet and related bugs Message-ID: <4AF9E3D6.2000807 at v.loewis.de> Content-Type: text/plain; charset=UTF-8 >> The buildbot waterfall is much greener now. Thanks to all who have >> contributed to making it so (and it hasn't just been Mark and Antoine >> and I, though we've been the most directly active (and yes, Mark, you >> did contribute several fixes!)). > > The buildbots still show occasional oddities. For example, right now in > the page "http://www.python.org/dev/buildbot/3.x/", some results have > disappeared (the columns for "AMD64 Ubuntu" builders have become empty). Yes, I noticed it too. It will go away after some page reloads. > Moreover, some buildslaves have gone back in time (they are building > r76188 after having built and tested r76195)... I swear the new GIL > doesn't include a time machine. That's because I resubmitted these changes after restarting the master. Regards, Martin ------------------------------ Message: 6 Date: Tue, 10 Nov 2009 16:06:46 -0600 From: Benjamin Peterson To: Martin v. L?wis Cc: Nick Coghlan , Python Development Subject: Re: [Python-Dev] 2.7/3.2 release schedule Message-ID: <1afaf6160911101406t6da78accq8918eb4ceb1adca5 at mail.gmail.com> Content-Type: text/plain; charset=UTF-8 2009/11/10 "Martin v. L?wis" : > I personally think that decoupling the releases would be best, i.e. > not start thinking about 3.2 for another 6 months. The problem with that is that there is a period of time where 2.x has features which 3.x doesn't. My preference is to move back the whole schedule 6 months. -- Regards, Benjamin ------------------------------ Message: 7 Date: Tue, 10 Nov 2009 23:09:42 +0100 From: "Martin v. L?wis" To: Benjamin Peterson Cc: Nick Coghlan , Python Development Subject: Re: [Python-Dev] 2.7/3.2 release schedule Message-ID: <4AF9E4A6.8070304 at v.loewis.de> Content-Type: text/plain; charset=UTF-8 Benjamin Peterson wrote: > 2009/11/10 "Martin v. L?wis" : >> I personally think that decoupling the releases would be best, i.e. >> not start thinking about 3.2 for another 6 months. > > The problem with that is that there is a period of time where 2.x has > features which 3.x doesn't. My preference is to move back the whole > schedule 6 months. Ok, fine with me as well. FWIW, I don't see that (2.7 released before 3.2) as a problem (but I understand that others might, for what I would consider formal reasons). Regards, Martin ------------------------------ _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev End of Python-Dev Digest, Vol 76, Issue 87 ****************************************** From python at rcn.com Tue Nov 10 23:30:22 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 10 Nov 2009 14:30:22 -0800 Subject: [Python-Dev] 2.7/3.2 release schedule References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com><71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com><4AF966F8.5040008@gmail.com> <4AF9D5A5.5000707@v.loewis.de> <1afaf6160911101406t6da78accq8918eb4ceb1adca5@mail.gmail.com> Message-ID: [MvL] >> I personally think that decoupling the releases would be best, i.e. >> not start thinking about 3.2 for another 6 months. [Benjamin] > The problem with that is that there is a period of time where 2.x has > features which 3.x doesn't. My preference is to move back the whole > schedule 6 months. My preference is to decouple and let 2.7 go out 18 months after 2.6. In general, 2.x users should not have to pay a price for whatever we do with 3.x. Raymond From techtonik at gmail.com Wed Nov 11 02:48:19 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Wed, 11 Nov 2009 03:48:19 +0200 Subject: [Python-Dev] Replacing IDLE In-Reply-To: <59691AA2-995E-452B-B21D-0C7F8861C7A6@mac.com> References: <59691AA2-995E-452B-B21D-0C7F8861C7A6@mac.com> Message-ID: On Tue, Nov 10, 2009 at 6:44 PM, Ronald Oussoren wrote: > > What's so bad about IDLE that you'd like to replace it? That was exactly my previous question. You don't use IDLE either, so why not to replace it with something that you can actually use, with something that is at least extensible? So people will be interested to learn and contribute. > The obvious alternative to replacing IDLE is to fix whatever is wrong with it. It is also obvious that developer would better download a decent editor than bog their mind with once perspective, but now dead technology IDLE was build upon. -- anatoly t. From techtonik at gmail.com Wed Nov 11 03:33:20 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Wed, 11 Nov 2009 04:33:20 +0200 Subject: [Python-Dev] Replacing IDLE In-Reply-To: <4AF9BE77.5050100@voidspace.org.uk> References: <4AF9BE77.5050100@voidspace.org.uk> Message-ID: On Tue, Nov 10, 2009 at 9:26 PM, Michael Foord wrote: > >> Then there will be another issue - all editors are based upon some >> frameworks - I didn't see any popular cross-platform GUI toolkits in >> Python, so we will inevitably face the need to replace Tkinter with >> other default GUI toolkit. > > This just *won't* happen. (At least not in the forseeable future.) Better > filter your list to only those IDEs that use Tkinter or have a GUI not > dependent on packages that aren't already in the standard library. That is a real stumbling block. There aren't any GUI toolkits in pure Python to be included alongside the editor, and those Tkinter bindings that compiled for every platform are just useless. If there is a reason that we just *won't* let any GUI framework (except Tkinter) slip into Python distribution then of course, any talk about editor is just a waste of time. But I'd like to see is that reason is real to know when to expect the changes. I've done a small research about Tkinter editors to investigate the solution you proposed. http://wiki.python.org/moin/PythonEditors shows that these were made with Tk: Pye - not true, either switched or wrong project, but PyQT4 - http://code.google.com/p/pye-editor/ xRope - GPL and probably worse than IDLE - http://sourceforge.net/projects/xrope/ IDEStudio - enhanced IDLE, dead - http://wiki.python.org/moin/IDEStudio SourceNavigator - GPL, C and not really an editor - http://sourcenav.sourceforge.net/ There is only one perspective project called Leo (with MIT license) that was historically written in Tkinter with the goal (I believe) to construct pure Python editor. It is editor with outlines unique in the way that it is able to rewrite itself literally at runtime. I believe only using such advanced tool one can cope with Tkinter inconsistencies and limitations. Since the beginning Leo has also developed PyQT4 interface. -- anatoly t. From guido at python.org Wed Nov 11 03:46:50 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Nov 2009 18:46:50 -0800 Subject: [Python-Dev] 2.7/3.2 release schedule In-Reply-To: <5FFB0E2375804AD28576A8C3EE953A9B@RaymondLaptop1> References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> <71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com> <4AF966F8.5040008@gmail.com> <5FFB0E2375804AD28576A8C3EE953A9B@RaymondLaptop1> Message-ID: On Tue, Nov 10, 2009 at 10:26 AM, Raymond Hettinger wrote: > > [Guido van Rossum] >> >> . We used to have releases once a year and >> we got really big serious feedback from our biggest users that the >> release cycle was going too fast. We discussed it amply and agreed on >> a minimum time of 18 months between releases. > > If the language moratorium goes into effect, "If"? It's already Approved. :-) > would shorter release cycles > still have a negative impact? ?Do people possibly want slower changes > to the language and faster updates to the library? > > I don't know the answer. ?Just asking how this matches up with the > feedback you have gotten previously. I expect it wouldn't make a difference since backwards compatibility testing involves a lot more than language syntax. -- --Guido van Rossum (python.org/~guido) From techtonik at gmail.com Wed Nov 11 03:50:03 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Wed, 11 Nov 2009 04:50:03 +0200 Subject: [Python-Dev] Replacing IDLE In-Reply-To: <4AF9D1D8.3060406@v.loewis.de> References: <4AF9D1D8.3060406@v.loewis.de> Message-ID: On Tue, Nov 10, 2009 at 10:49 PM, "Martin v. L?wis" wrote: > > This is not how it works. We cannot incorporate something into Python > without explicit consent and support from the author(s). So for any > editor to be incorporated as a replacement (along with all libraries > it depends on) we would need a commitment from the author(s) that they > support the fork that we create (and eventually abandon their > stand-alone release in favor of the fork). > Does that mean even if authors of some imaginary editor agree to incorporate their code into Python, the framework that it is built upon will have to be incorporated into Python also (and eventually abandoned at original location)? In this situation I see no way we can provide any decent tools for GUI like for default editor. Thanks for support. -- anatoly t. From stephen at xemacs.org Wed Nov 11 05:06:36 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 11 Nov 2009 13:06:36 +0900 Subject: [Python-Dev] Replacing IDLE In-Reply-To: References: <4AF9D1D8.3060406@v.loewis.de> Message-ID: <87639h4sdv.fsf@uwakimon.sk.tsukuba.ac.jp> anatoly techtonik writes: > Does that mean even if authors of some imaginary editor agree to > incorporate their code into Python, the framework that it is built > upon will have to be incorporated into Python also (and eventually > abandoned at original location)? I would assume so. How else can Python guarantee its maintenance? > In this situation I see no way we can provide any decent tools for GUI > like for default editor. IDLE *is* a "decent tool". You don't have to like it, but it's good enough to get started. By the time you realize its limitations, you should have a handle on easy_install. The promise is "batteries included." Nobody promised you a nickel metal hydride battery that you can use as a replacement in your Prius. From greg.ewing at canterbury.ac.nz Wed Nov 11 07:07:56 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 11 Nov 2009 19:07:56 +1300 Subject: [Python-Dev] Replacing IDLE In-Reply-To: References: <59691AA2-995E-452B-B21D-0C7F8861C7A6@mac.com> Message-ID: <4AFA54BC.90401@canterbury.ac.nz> anatoly techtonik wrote: > why not to replace it with something that you can actually use, with > something that is at least extensible? So people will be interested to > learn and contribute. IDLE is written in Python, so it's about as extensible as you can get. Seems to me the only kind of IDE that it makes sense to ship with Python is one that is written in Python and maintained by the core developers. Anything else is best left as a third party package for download by those who want to use it. Tkinter is arguably not a very good basis for it from a technical point of view, but so far all the alternatives are too bloated and/or nonportable to consider bundling with the standard distribution. So the only realistic alternatives at the moment seem to be either IDLE+Tkinter or nothing. Quite likely most experienced developers use something else, and wouldn't care if IDLE were removed and not replaced with anything. But there are benefits to having a standard, lightweight IDE present in most Python installations. Books and tutorials can be designed around it, for example, with fairly high assurance that it will be available to beginners who aren't comfortable finding their way around text editors and command shells. So, I'd say that, like democracy, it's not very good, but it's better than any of the alternatives. :-) -- Greg From martin at v.loewis.de Wed Nov 11 07:17:20 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Wed, 11 Nov 2009 07:17:20 +0100 Subject: [Python-Dev] Replacing IDLE In-Reply-To: References: <4AF9D1D8.3060406@v.loewis.de> Message-ID: <4AFA56F0.1090704@v.loewis.de> > Does that mean even if authors of some imaginary editor agree to > incorporate their code into Python, the framework that it is built > upon will have to be incorporated into Python also (and eventually > abandoned at original location)? It depends. It should work the same way as IDLE: it's ok that Tcl/Tk itself is not included in Python, however, Tkinter is. If that imaginary editor has lots of build dependencies that make it really difficult to use it, I would be opposed to including it. If it requires one library that is typically already available on a Linux system, it would be fine with me. Regards, Martin From greg.ewing at canterbury.ac.nz Wed Nov 11 08:01:38 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 11 Nov 2009 20:01:38 +1300 Subject: [Python-Dev] Replacing IDLE In-Reply-To: <4AFA56F0.1090704@v.loewis.de> References: <4AF9D1D8.3060406@v.loewis.de> <4AFA56F0.1090704@v.loewis.de> Message-ID: <4AFA6152.50900@canterbury.ac.nz> Martin v. L?wis wrote: > If that imaginary editor has lots of build dependencies that make > it really difficult to use it, I would be opposed to including it. > If it requires one library that is typically already available on > a Linux system, it would be fine with me. If I manage to get PyGUI into a practically usable state and it ever gets included in the core distribution, then it might be a viable basis for a modernised version of IDLE. That's probably a fairly long way off, though. -- Greg From vmanis at telus.net Wed Nov 11 07:30:06 2009 From: vmanis at telus.net (Vincent Manis) Date: Tue, 10 Nov 2009 22:30:06 -0800 Subject: [Python-Dev] Replacing IDLE In-Reply-To: <4AFA54BC.90401@canterbury.ac.nz> References: <59691AA2-995E-452B-B21D-0C7F8861C7A6@mac.com> <4AFA54BC.90401@canterbury.ac.nz> Message-ID: On 2009-11-10, at 22:07, Greg Ewing wrote: > So, I'd say that, like democracy, [IDLE is] not very good, but > it's better than any of the alternatives. :-) Speaking purely as a Python user, I am very happy that IDLE is part of the Python distribution. Personally, I use and like emacs too much, and have no use for IDLE for my own programming at all. But I have taught a lot of Python classes, and being able to tell the IT staff `please install Python x.y on the machines in the lab', and then knowing that students have everything they need to do their lab work, is a major plus. (I found that many of the students stayed with IDLE even after their course was over, too.) As a side observation, I think the main reasons why Tk is so resistant to removal from Python (and several other dynamic languages) is that its developers took very seriously the idea of being lightweight and portable, while many other toolkits grew more and more complex. Not all GUI toolkits are like that; I have nice things to say about FLTK, for example. Perhaps one day somebody will do a revision of Tk (sans Tcl) for the 21st century, but until then, Greg's Winston Churchill quote applies not just to IDLE but also to Tk. [`This horse is dead, please stop flogging it.' `No it's not, it's just observing the moratorium'] -- v From vmanis at telus.net Wed Nov 11 07:32:25 2009 From: vmanis at telus.net (Vincent Manis) Date: Tue, 10 Nov 2009 22:32:25 -0800 Subject: [Python-Dev] Replacing IDLE In-Reply-To: <4AFA56F0.1090704@v.loewis.de> References: <4AF9D1D8.3060406@v.loewis.de> <4AFA56F0.1090704@v.loewis.de> Message-ID: On 2009-11-10, at 22:17, Martin v. L?wis wrote: > If that imaginary editor has lots of build dependencies that make > it really difficult to use it, I would be opposed to including it. > If it requires one library that is typically already available on > a Linux system, it would be fine with me. s/Linux/Linux, Mac (Aqua, not X11), and Windows/ -- v From ronaldoussoren at mac.com Wed Nov 11 08:27:06 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 11 Nov 2009 08:27:06 +0100 Subject: [Python-Dev] Replacing IDLE In-Reply-To: References: <59691AA2-995E-452B-B21D-0C7F8861C7A6@mac.com> Message-ID: On 11 Nov, 2009, at 2:48, anatoly techtonik wrote: > On Tue, Nov 10, 2009 at 6:44 PM, Ronald Oussoren wrote: >> >> What's so bad about IDLE that you'd like to replace it? > > That was exactly my previous question. You don't use IDLE either, so > why not to replace it with something that you can actually use, with > something that is at least extensible? So people will be interested to > learn and contribute. I don't use IDLE because I'm too used to my current editor, which btw. is the cmd-line version of vim. That offers everything I need from a Python editor. That said, I have used IDLE occasionaly and it works fine for a basic Python environment. IDLE is extensible, I've seen a number of IDLE extensions on the IDLE-dev mailinglist (). That list would be a better location to discus this. BTW. You still haven't mentioned why you think IDLE should be replaced. > >> The obvious alternative to replacing IDLE is to fix whatever is wrong with it. > > It is also obvious that developer would better download a decent > editor than bog their mind with once perspective, but now dead > technology IDLE was build upon. Tk is actively developed, it is not dead technology. Replacing IDLE by an editor which uses another toolkit would either mean that the default editor uses libraries not in the stdlib, or the inclusion of that toolkit in the stdlib. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3567 bytes Desc: not available URL: From stefan-usenet at bytereef.org Wed Nov 11 12:36:13 2009 From: stefan-usenet at bytereef.org (Stefan Krah) Date: Wed, 11 Nov 2009 12:36:13 +0100 Subject: [Python-Dev] [format] Restrict fill characters for numerical values Message-ID: <20091111113613.GA4802@mail.bytereef.org> Hi, I think http://www.python.org/dev/peps/pep-3101/ is too liberal with the choice of fill characters for numerical values. As far as I can see, this is quite legal: Python 2.7a0 (trunk:76132M, Nov 6 2009, 15:20:35) [GCC 4.1.3 20080623 (prerelease) (Ubuntu 4.1.2-23ubuntu3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from decimal import * >>> format(Decimal(4), "9<10") '4999999999' >>> format(Decimal(4), "9>10") '9999999994' >>> format(Decimal(4), "->2") '-4' >>> I propose to disallow digits and '+-' fill characters for numerical values, except for the combination '0, left-padding'. Actually, I'd prefer to allow only whitespace and '0, left-padding'. See also: http://bugs.python.org/issue6871 http://bugs.python.org/issue6902 Stefan Krah From eric at trueblade.com Wed Nov 11 13:39:26 2009 From: eric at trueblade.com (Eric Smith) Date: Wed, 11 Nov 2009 07:39:26 -0500 Subject: [Python-Dev] [format] Restrict fill characters for numerical values In-Reply-To: <20091111113613.GA4802@mail.bytereef.org> References: <20091111113613.GA4802@mail.bytereef.org> Message-ID: <4AFAB07E.2030707@trueblade.com> Stefan Krah wrote: > Hi, > > I think http://www.python.org/dev/peps/pep-3101/ is too liberal with the > choice of fill characters for numerical values. As far as I can see, this > is quite legal: > > > Python 2.7a0 (trunk:76132M, Nov 6 2009, 15:20:35) > [GCC 4.1.3 20080623 (prerelease) (Ubuntu 4.1.2-23ubuntu3)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> from decimal import * >>>> format(Decimal(4), "9<10") > '4999999999' >>>> format(Decimal(4), "9>10") > '9999999994' >>>> format(Decimal(4), "->2") > '-4' > > > I propose to disallow digits and '+-' fill characters for numerical values, > except for the combination '0, left-padding'. Actually, I'd prefer to allow > only whitespace and '0, left-padding'. Why? What problem are you trying to prevent that would justify the extra code to implement this and the extra documentation to explain it? Eric. From stefan-usenet at bytereef.org Wed Nov 11 14:06:34 2009 From: stefan-usenet at bytereef.org (Stefan Krah) Date: Wed, 11 Nov 2009 14:06:34 +0100 Subject: [Python-Dev] [format] Restrict fill characters for numerical values In-Reply-To: <4AFAB07E.2030707@trueblade.com> References: <20091111113613.GA4802@mail.bytereef.org> <4AFAB07E.2030707@trueblade.com> Message-ID: <20091111130634.GA618@yoda.bytereef.org> Eric Smith wrote: > Stefan Krah wrote: > >Hi, > > > >I think http://www.python.org/dev/peps/pep-3101/ is too liberal with the > >choice of fill characters for numerical values. As far as I can see, this > >is quite legal: > > > > > >Python 2.7a0 (trunk:76132M, Nov 6 2009, 15:20:35) > >[GCC 4.1.3 20080623 (prerelease) (Ubuntu 4.1.2-23ubuntu3)] on linux2 > >Type "help", "copyright", "credits" or "license" for more information. > >>>>from decimal import * > >>>>format(Decimal(4), "9<10") > >'4999999999' > >>>>format(Decimal(4), "9>10") > >'9999999994' > >>>>format(Decimal(4), "->2") > >'-4' > > > > > >I propose to disallow digits and '+-' fill characters for numerical values, > >except for the combination '0, left-padding'. Actually, I'd prefer to allow > >only whitespace and '0, left-padding'. > > Why? > > What problem are you trying to prevent that would justify the extra code > to implement this and the extra documentation to explain it? I simply think that apart from rounding, the output of format should not change the numerical value of its argument. The format functions in C do not allow this to happen. Are there other languages where this is possible? Stefan Krah From ziade.tarek at gmail.com Wed Nov 11 14:36:06 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 11 Nov 2009 14:36:06 +0100 Subject: [Python-Dev] PEP 382 status Message-ID: <94bdd2610911110536n60a882acj8b2cd63ced3af25b@mail.gmail.com> Hi, I was wondering what's the status of PEP 382. Is anyone (MvL?) is going to start to work on its implementation for Python 2.7/3.2 inclusion ? Regards Tarek From stefan-usenet at bytereef.org Wed Nov 11 14:34:33 2009 From: stefan-usenet at bytereef.org (Stefan Krah) Date: Wed, 11 Nov 2009 14:34:33 +0100 Subject: [Python-Dev] [format] Restrict fill characters for numerical values In-Reply-To: <20091111130634.GA618@yoda.bytereef.org> References: <20091111113613.GA4802@mail.bytereef.org> <4AFAB07E.2030707@trueblade.com> <20091111130634.GA618@yoda.bytereef.org> Message-ID: <20091111133433.GA2645@yoda.bytereef.org> Stefan Krah wrote: > I simply think that apart from rounding, the output of format should not > change the numerical value of its argument. The format functions in C do > not allow this to happen. Are there other languages where this is possible? Actually there are other cases: strfmon() allows this, too. Let's drop it. Stefan Krah From drobinow at gmail.com Wed Nov 11 15:00:41 2009 From: drobinow at gmail.com (David Robinow) Date: Wed, 11 Nov 2009 06:00:41 -0800 Subject: [Python-Dev] Replacing IDLE In-Reply-To: <4AFA54BC.90401@canterbury.ac.nz> References: <59691AA2-995E-452B-B21D-0C7F8861C7A6@mac.com> <4AFA54BC.90401@canterbury.ac.nz> Message-ID: <4eb0089f0911110600l6c4a895j5da0c7bdb367d708@mail.gmail.com> On Tue, Nov 10, 2009 at 10:07 PM, Greg Ewing wrote: > Seems to me the only kind of IDE that it makes sense to > ship with Python is one that is written in Python and > maintained by the core developers. Anything else is best > left as a third party package for download by those > who want to use it. > > Tkinter is arguably not a very good basis for it from a > technical point of view, but so far all the alternatives > are too bloated and/or nonportable to consider bundling > with the standard distribution. > > So the only realistic alternatives at the moment seem > to be either IDLE+Tkinter or nothing. > My immediate reaction? What about stdwin? Then I found this: http://mail.python.org/pipermail/python-dev/2000-October/010113.html From eric at trueblade.com Wed Nov 11 15:11:48 2009 From: eric at trueblade.com (Eric Smith) Date: Wed, 11 Nov 2009 09:11:48 -0500 Subject: [Python-Dev] [format] Restrict fill characters for numerical values In-Reply-To: <20091111130634.GA618@yoda.bytereef.org> References: <20091111113613.GA4802@mail.bytereef.org> <4AFAB07E.2030707@trueblade.com> <20091111130634.GA618@yoda.bytereef.org> Message-ID: <4AFAC624.8070007@trueblade.com> Stefan Krah wrote: > Eric Smith wrote: >> Stefan Krah wrote: >>> Hi, >>> >>> I think http://www.python.org/dev/peps/pep-3101/ is too liberal with the >>> choice of fill characters for numerical values. As far as I can see, this >>> is quite legal: >>> >>> >>> Python 2.7a0 (trunk:76132M, Nov 6 2009, 15:20:35) >>> [GCC 4.1.3 20080623 (prerelease) (Ubuntu 4.1.2-23ubuntu3)] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>>>> >from decimal import * >>>>>> format(Decimal(4), "9<10") >>> '4999999999' >>>>>> format(Decimal(4), "9>10") >>> '9999999994' >>>>>> format(Decimal(4), "->2") >>> '-4' >>> >>> >>> I propose to disallow digits and '+-' fill characters for numerical values, >>> except for the combination '0, left-padding'. Actually, I'd prefer to allow >>> only whitespace and '0, left-padding'. >> Why? >> >> What problem are you trying to prevent that would justify the extra code >> to implement this and the extra documentation to explain it? > > > I simply think that apart from rounding, the output of format should not > change the numerical value of its argument. The format functions in C do > not allow this to happen. Are there other languages where this is possible? I don't know if there are other languages that allow it. But I definitely have use cases for padding with '*', for example. I don't see the point in just disallowing numbers 1 through 9. If you don't want your numbers padded with 9's, don't put a 9 in your format string. Eric. From eric at trueblade.com Wed Nov 11 15:18:11 2009 From: eric at trueblade.com (Eric Smith) Date: Wed, 11 Nov 2009 09:18:11 -0500 Subject: [Python-Dev] PEP 382 status In-Reply-To: <94bdd2610911110536n60a882acj8b2cd63ced3af25b@mail.gmail.com> References: <94bdd2610911110536n60a882acj8b2cd63ced3af25b@mail.gmail.com> Message-ID: <4AFAC7A3.4000801@trueblade.com> Tarek Ziad? wrote: > I was wondering what's the status of PEP 382. Is anyone (MvL?) is > going to start to work on its implementation for Python 2.7/3.2 > inclusion ? If Martin isn't interested in doing it, I'll take a try at it. But I'll need some rough guidance on the implementation approach. Eric. From pluskid at gmail.com Wed Nov 11 17:16:43 2009 From: pluskid at gmail.com (Zhang Chiyuan) Date: Thu, 12 Nov 2009 00:16:43 +0800 Subject: [Python-Dev] Is this a bug of the HTMLParser? Message-ID: Hi all, I'm using BeautifulSoup to parsing an HTML page and find it refused to parse the page. By looking at the backtrace, I found it is a problem with the python built-in HTMLParser.py. In fact, the web page I'm parsing is with some Chinese characters. there is a tag like ?? , note this is legacy html page where the attributes are not quoted. However, the regexp defined in HTMLParser.py is : attrfind = re.compile( r'\s*([a-zA-Z_][-.:a-zA-Z_0-9]*)(\s*=\s*' r'(\'[^\']*\'|"[^"]*"|[-a-zA-Z0-9./,:;+*%?!&$\(\)_#=~@]*))?') Note that the Chinese character (also any other non-english characters), so it fire an error parsing this. I'm not sure whether the HTML standard allow un-quoted non-ASCII characters in the attributes. If it allows, this seems to be a bug. and the regexp to better be [^>\s] IMHO. BTW: It seems something like : can not be parsed. :-/ -- pluskid http://blog.pluskid.org From fuzzyman at voidspace.org.uk Wed Nov 11 17:24:02 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 11 Nov 2009 17:24:02 +0100 Subject: [Python-Dev] Is this a bug of the HTMLParser? In-Reply-To: References: Message-ID: <4AFAE522.2090403@voidspace.org.uk> Hello Zhang Chiyuan, Can you file a bug on the Python issue tracker please: http://bugs.python.org Thanks Michael Foord Zhang Chiyuan wrote: > Hi all, > > I'm using BeautifulSoup to parsing an HTML page and find it refused to > parse the page. By looking at the backtrace, I found it is a problem > with the python built-in HTMLParser.py. In fact, the web page I'm > parsing is with some Chinese characters. there is a tag like src=/foo/bar.png alt=??> , note this is legacy html page where the > attributes are not quoted. However, the regexp defined in > HTMLParser.py is : > > attrfind = re.compile( > r'\s*([a-zA-Z_][-.:a-zA-Z_0-9]*)(\s*=\s*' > r'(\'[^\']*\'|"[^"]*"|[-a-zA-Z0-9./,:;+*%?!&$\(\)_#=~@]*))?') > > Note that the Chinese character (also any other non-english > characters), so it fire an error parsing this. I'm not sure whether > the HTML standard allow un-quoted non-ASCII characters in the > attributes. If it allows, this seems to be a bug. and the regexp to > better be [^>\s] IMHO. > > BTW: It seems something like : > > > > can not be parsed. :-/ > > -- > pluskid > http://blog.pluskid.org > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ From oshecho at gmail.com Wed Nov 11 18:59:47 2009 From: oshecho at gmail.com (Echo) Date: Wed, 11 Nov 2009 12:59:47 -0500 Subject: [Python-Dev] IDLE as default Python editor In-Reply-To: References: <87iqdk2n7b.fsf@benfinney.id.au> <4AF80E73.5090104@gmail.com> <4AF89370.2040206@v.loewis.de> <4AF944A9.2030108@gmail.com> Message-ID: <910885da0911110959m6c86edbepf15934902982c96a@mail.gmail.com> On Tue, Nov 10, 2009 at 5:27 PM, Georg Brandl wrote: > > Quite simple: because we can't possibly ship Emacs. > > cheers, > Georg > We just need a PyEmacs. Written in python, extensible in elist and python. Nice and simple ;-D -- -Brandon Singer From fdrake at acm.org Wed Nov 11 19:50:05 2009 From: fdrake at acm.org (Fred Drake) Date: Wed, 11 Nov 2009 13:50:05 -0500 Subject: [Python-Dev] IDLE as default Python editor In-Reply-To: <910885da0911110959m6c86edbepf15934902982c96a@mail.gmail.com> References: <87iqdk2n7b.fsf@benfinney.id.au> <4AF80E73.5090104@gmail.com> <4AF89370.2040206@v.loewis.de> <4AF944A9.2030108@gmail.com> <910885da0911110959m6c86edbepf15934902982c96a@mail.gmail.com> Message-ID: <9cee7ab80911111050k5f2a077en36d41a134d2dbfb2@mail.gmail.com> On Wed, Nov 11, 2009 at 12:59 PM, Echo wrote: > We just need a PyEmacs. Written in python, extensible in elist and > python. Nice and simple ;-D I'd even give up the elisp support if I could have Python in my Emacs. -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller From martin at v.loewis.de Wed Nov 11 21:04:00 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 11 Nov 2009 21:04:00 +0100 Subject: [Python-Dev] PEP 382 status In-Reply-To: <94bdd2610911110536n60a882acj8b2cd63ced3af25b@mail.gmail.com> References: <94bdd2610911110536n60a882acj8b2cd63ced3af25b@mail.gmail.com> Message-ID: <4AFB18B0.3050800@v.loewis.de> > I was wondering what's the status of PEP 382. Is anyone (MvL?) is > going to start to work on its implementation for Python 2.7/3.2 > inclusion ? I'll be working on an implementation, but contributions are welcome. Unfortunately, I'm really short on free software time recently (and keep hoping that this will improve). One thing that I always wondered about is the status of importlib. Will that be used in 3.2? In addition, I still owe a few answers to comments from the previous discussion. Regards, Martin From drobinow at gmail.com Wed Nov 11 21:17:44 2009 From: drobinow at gmail.com (David Robinow) Date: Wed, 11 Nov 2009 15:17:44 -0500 Subject: [Python-Dev] IDLE as default Python editor In-Reply-To: <9cee7ab80911111050k5f2a077en36d41a134d2dbfb2@mail.gmail.com> References: <87iqdk2n7b.fsf@benfinney.id.au> <4AF80E73.5090104@gmail.com> <4AF89370.2040206@v.loewis.de> <4AF944A9.2030108@gmail.com> <910885da0911110959m6c86edbepf15934902982c96a@mail.gmail.com> <9cee7ab80911111050k5f2a077en36d41a134d2dbfb2@mail.gmail.com> Message-ID: <4eb0089f0911111217n688cc830x5ba768dec9df53a@mail.gmail.com> On Wed, Nov 11, 2009 at 1:50 PM, Fred Drake wrote: > On Wed, Nov 11, 2009 at 12:59 PM, Echo wrote: >> We just need a PyEmacs. Written in python, extensible in elist and >> python. Nice and simple ;-D > I'd even give up the elisp support if I could have Python in my Emacs. Have you tried Pymacs? From ncoghlan at gmail.com Wed Nov 11 22:18:23 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 12 Nov 2009 07:18:23 +1000 Subject: [Python-Dev] 2.7/3.2 release schedule In-Reply-To: References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com><71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com><4AF966F8.5040008@gmail.com> <4AF9D5A5.5000707@v.loewis.de> <1afaf6160911101406t6da78accq8918eb4ceb1adca5@mail.gmail.com> Message-ID: <4AFB2A1F.5060307@gmail.com> Raymond Hettinger wrote: > > [MvL] >>> I personally think that decoupling the releases would be best, i.e. >>> not start thinking about 3.2 for another 6 months. > > [Benjamin] >> The problem with that is that there is a period of time where 2.x has >> features which 3.x doesn't. My preference is to move back the whole >> schedule 6 months. > > My preference is to decouple and let 2.7 go out 18 months after 2.6. > In general, 2.x users should not have to pay a price for whatever > we do with 3.x. And I guess anyone doing forward ports is likely going to be maintaining 2.6 compatibility anyway, so waiting until 3.2 comes out before using new 2.7 features wouldn't be a major burden. I think I've decided I don't mind either way, so I'm fine with whichever approach is easier for Benjamin and the platform installer builders to manage. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From guido at python.org Wed Nov 11 22:23:47 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 11 Nov 2009 13:23:47 -0800 Subject: [Python-Dev] 2.7/3.2 release schedule In-Reply-To: <4AFB2A1F.5060307@gmail.com> References: <1afaf6160911021200m1bcf27cck4a78db471db038a7@mail.gmail.com> <71ac8a310911100241x2567c396pc2e5ef4653ced917@mail.gmail.com> <4AF966F8.5040008@gmail.com> <4AF9D5A5.5000707@v.loewis.de> <1afaf6160911101406t6da78accq8918eb4ceb1adca5@mail.gmail.com> <4AFB2A1F.5060307@gmail.com> Message-ID: On Wed, Nov 11, 2009 at 1:18 PM, Nick Coghlan wrote: > I think I've decided I don't mind either way, so I'm fine with whichever > approach is easier for Benjamin and the platform installer builders to > manage. +1 -- --Guido van Rossum (python.org/~guido) From naughtysriram at gmail.com Thu Nov 12 10:23:50 2009 From: naughtysriram at gmail.com (Sriram Srinivasan) Date: Thu, 12 Nov 2009 14:23:50 +0500 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' Message-ID: I guess why every programming language has some kind of a 'standard library' built in within it. In my view it must not be called as a 'library' at all. what it does is like a 'bunch of built-in programs ready-made to do stuff'. Lets see what a 'library' does: 1. offers books for customers 1.1 lets user select a book by genre, etc 1.2 lets user to use different books of same genre, etc 1.3 lets user to use books by same author, etc for different genre 2. keeps track of all the books + their genre 2.1 first knows what all books it has at present 2.2 when new book comes it is added to the particular shelf sorted by genre,author,edition, etc. 2.3 when books become old they are kept separately for future reference 2.4 very old books can be sent to a museum/discarded I guess no standard library does the minimum of this but wants to be called a library. As a python user I always wanted the standard library to have such features so the user/developer decides to use what set of libraries he want. consider the libraries for 2.5 ,2.6, 3K are all available to the user, the user selects what he wants with something like. use library 2.5 or use library 2.6 etc. The 2 main things that the library management interface has to do is intra library management and inter library management. intra library mgmt- consider books to be different libraries (standard, commercial, private, hobby, etc) inter library mgmt- consider books to be modules inside a library ( standard, commercial, private, hobby, etc) if somehow we could accomplish this kind of mother of a all plugin/ad- hoc system that is a real breakthrough. Advantages: 1. new modules can be added to the stream quickly 2. let the user select what he want to do 3. modules (that interdepend on each other) can be packed into small distribution and added to the stream quickly without waiting for new releases 4. solution to problems like py 2.x and 3.x 5. users can be up to date 6. documentation becomes easy + elaborate to users 7. bug managing is easy too 8. more feed back 9. testing also becomes easy 10. many more , i don't know.. you have to find. Python already has some thing like that __future__ stuff. but my question is how many people know that? and how many use that? most of them wait until old crust gets totally removed. that is bad for user and python. that is why problems like py2.x py3.x originate. If there is a newer book collection it must always be available at the library. i must not go to another library to get that book. I am not an expert, I am just another python learner. These are just my views on the state of the standard libraries and to make them state-of-the-art..! ;) -- Regards, Sriram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ludvig at lericson.se Thu Nov 12 10:44:32 2009 From: ludvig at lericson.se (Ludvig Ericson) Date: Thu, 12 Nov 2009 10:44:32 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? Message-ID: Why are there comments on PyPI? Moreso, why are there comments which I cannot control as a package author on my very own packages? That's just absurd. It's *my* package, and so should be *my* choice if I want user input or not. And ratings? I thought it was the Python Package Index, not the Python Anonymous Package Tribunal. As I see it, there are only two ways to fix these misguided steps of development: throw them out, or make them opt-in settings. The comments I simply do not understand. Why not instead provide a form for mailing the package author? The ratings are just not what PyPI should be doing, is about, or what I signed up for. -L From fuzzyman at voidspace.org.uk Thu Nov 12 11:15:11 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 12 Nov 2009 11:15:11 +0100 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: References: Message-ID: <4AFBE02F.6020302@voidspace.org.uk> Sriram Srinivasan wrote: > I guess why every programming language has some kind of a 'standard > library' built in within it. In my view it must not be called as a > 'library' at all. what it does > is like a 'bunch of built-in programs ready-made to do stuff'. > > Lets see what a 'library' does: > > 1. offers books for customers > 1.1 lets user select a book by genre, etc > 1.2 lets user to use different books of same genre, etc > 1.3 lets user to use books by same author, etc for different genre > > 2. keeps track of all the books + their genre > 2.1 first knows what all books it has at present > 2.2 when new book comes it is added to the particular shelf sorted by > genre,author,edition, etc. > 2.3 when books become old they are kept separately for future reference > 2.4 very old books can be sent to a museum/discarded > > I guess no standard library does the minimum of this but wants to be > called a library. > I don't really understand your requirements, but it sound like you want a package management system. The standard library just provides a standard set of tools (it is the books not the book management system - although part of what you want is in the standard library in the form of distutils which is currently receiving a radical overhaul). You should look at Distribute and virtualenv, which gets you pretty much what you are suggesting (as far as I can tell): http://pypi.python.org/pypi/virtualenv http://pypi.python.org/pypi/distribute All the best, Michael Foord > As a python user I always wanted the standard library to have such > features so the user/developer decides to use what set of libraries he > want. > > consider the libraries for 2.5 ,2.6, 3K are all available to the user, > the user selects what he wants with something like. > > use library 2.5 or use library 2.6 etc. > > The 2 main things that the library management interface has to do is > intra library management and inter library management. > > intra library mgmt- consider books to be different libraries > (standard, commercial, private, hobby, etc) > inter library mgmt- consider books to be modules inside a library > ( standard, commercial, private, hobby, etc) > > if somehow we could accomplish this kind of mother of a all plugin/ad- > hoc system that is a real breakthrough. > > Advantages: > > 1. new modules can be added to the stream quickly > 2. let the user select what he want to do > 3. modules (that interdepend on each other) can be packed into small > distribution and added to the stream quickly without waiting for new > releases > 4. solution to problems like py 2.x and 3.x > 5. users can be up to date > 6. documentation becomes easy + elaborate to users > 7. bug managing is easy too > 8. more feed back > 9. testing also becomes easy > 10. many more , i don't know.. you have to find. > > Python already has some thing like that __future__ stuff. but my > question is how many people know that? and how many use that? most of > them wait until old crust gets totally removed. that is bad for user > and python. that is why problems like py2.x py3.x originate. If there > is a newer book collection it must always be available at the library. > i must not go to another library to get that book. > > I am not an expert, I am just another python learner. These are just > my views on the state of the standard libraries and to > make them state-of-the-art..! ;) > > -- > Regards, > Sriram. > ------------------------------------------------------------------------ > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ From naughtysriram at gmail.com Thu Nov 12 11:54:58 2009 From: naughtysriram at gmail.com (Sriram Srinivasan) Date: Thu, 12 Nov 2009 15:54:58 +0500 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: <4AFBE02F.6020302@voidspace.org.uk> References: <4AFBE02F.6020302@voidspace.org.uk> Message-ID: > > I don't really understand your requirements, but it sound like you want a > package management system. The standard library just provides a standard set > of tools (it is the books not the book management system - although part of > what you want is in the standard library in the form of distutils which is > currently receiving a radical overhaul). > > You should look at Distribute and virtualenv, which gets you pretty much > what you are suggesting (as far as I can tell): > > http://pypi.python.org/pypi/virtualenv > http://pypi.python.org/pypi/distribute > > All the best, > > Michael Foord > I don't know if you have used Dev-C++. It has a 'package management' mechanism for the standard libraries. please see the http://devpaks.org/ webpage where all the packaged libraries are stored. In python we have the PyPI which is equivalent to the http://devpacks.orgbut in PyPI the packages are all user made applications. What I want is similar to PyPI but for the python standard libraries, so that they (libraries) are as add-on as possible. I guess you understand what I am thinking... and do pardon my english too.. -- Regards, Sriram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eckhardt at satorlaser.com Thu Nov 12 12:14:09 2009 From: eckhardt at satorlaser.com (Ulrich Eckhardt) Date: Thu, 12 Nov 2009 12:14:09 +0100 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: References: <4AFBE02F.6020302@voidspace.org.uk> Message-ID: <200911121214.09958.eckhardt@satorlaser.com> On Thursday 12 November 2009, Sriram Srinivasan wrote: > I don't know if you have used Dev-C++. It has a 'package management' > mechanism for the standard libraries. I disagree. It has a package management system for libraries, not for the standard libraries. The point is that the Python standard library is supplied as part of Python itself, as is e.g. the C++ standard library as part of a C++ compiler. > In python we have the PyPI which is equivalent to the > http://devpacks.org but in PyPI the packages are all user made > applications. ...as in devpacks.org. Please scroll down on that page and read the warning that mentions than any person can upload a package possibly containing malware or broken code or whatever. > What I want is similar to PyPI but for the python standard libraries, so > that they (libraries) are as add-on as possible. So, you want to make libs that are currently part of the standard library external libs available via PyPI? If that is what you want, it would be helpful if you provided a reason and also which libraries you're talking about. Uli -- Sator Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Sator Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at ************************************************************************************** Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder anderweitig benutzt werden. E-Mails k?nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ?nderungen enthalten. Sator Laser GmbH ist f?r diese Folgen nicht verantwortlich. ************************************************************************************** From naughtysriram at gmail.com Thu Nov 12 12:36:30 2009 From: naughtysriram at gmail.com (Sriram Srinivasan) Date: Thu, 12 Nov 2009 16:36:30 +0500 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: <200911121214.09958.eckhardt@satorlaser.com> References: <4AFBE02F.6020302@voidspace.org.uk> <200911121214.09958.eckhardt@satorlaser.com> Message-ID: > I disagree. It has a package management system for libraries, not for the > standard libraries. The point is that the Python standard library is > supplied > as part of Python itself, as is e.g. the C++ standard library as part of a > C++ compiler. > > standard libraries i meant the standard libraries used. and what i asked for is for(in python) both the standard-libraries and the standard libraries used. c the term (intra and inter library management) which includes the default standard libraries and other standard libraries > > ...as in devpacks.org. Please scroll down on that page and read the > warning > that mentions than any person can upload a package possibly containing > malware or broken code or whatever. > > if you talk about security then nothing is secure for sure! in PyPI ppl can upload their application as egg. what if there is another site where only eggs of python.org containing add-on standard libraries are distributed and noone is allowed to upload stuff except the python.org or PSF. > > So, you want to make libs that are currently part of the standard library > external libs available via PyPI? If that is what you want, it would be > helpful if you provided a reason and also which libraries you're talking > about. > > > Not exactly via PyPI but something similar to it. and don't compare C++ with python. both have their own + & - -- Regards, Sriram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew at matthewwilkes.co.uk Thu Nov 12 13:03:27 2009 From: matthew at matthewwilkes.co.uk (Matthew Wilkes) Date: Thu, 12 Nov 2009 12:03:27 +0000 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: References: <4AFBE02F.6020302@voidspace.org.uk> <200911121214.09958.eckhardt@satorlaser.com> Message-ID: On 2009-11-12, at 1136, Sriram Srinivasan wrote: > standard libraries i meant the standard libraries used. > and what i asked for is for(in python) both the standard-libraries > and the standard libraries used. > c the term (intra and inter library management) which includes the > default standard libraries and other standard libraries I don't think you're using the correct terminology. The standard library is what is shipped with the interpreter. It's the only thing that's common between all users of that interpreter, i.e. it's standard. > what if there is another site where only eggs of python.org > containing add-on standard libraries are distributed > and noone is allowed to upload stuff except the python.org or PSF. The PSF does not maintain add-on libraries, nor can it make guarantees about other add-ons libraries. >> So, you want to make libs that are currently part of the standard >> library >> external libs available via PyPI? > Not exactly via PyPI but something similar to it. Again, why? > and don't compare C++ with python. both have their own + & - Frankly, comparing and contrasting with other languages is very useful, I have little to no idea what you're talking about, and doubt it's got anything to do with the development of the Python language. Matthew From naughtysriram at gmail.com Thu Nov 12 13:08:22 2009 From: naughtysriram at gmail.com (Sriram Srinivasan) Date: Thu, 12 Nov 2009 17:08:22 +0500 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: References: <4AFBE02F.6020302@voidspace.org.uk> <200911121214.09958.eckhardt@satorlaser.com> Message-ID: ok let me make it more clear.. forget how you use python now.. i am talking about __futuristic__ python programming. these are just my ideas.. more over i can say imaginations. there is no more python2.x or python3.x or python y.x releases. there is only updates of python and standard library say 1.1.5 and 1.1.6. let the difference be an old xml library updated with a new regex support. i am coding my program now. i want my application to be compatible with 1.1.5 library use library 1.1.5 import blah form blahblah ... ... i cannot use regex feature of xml in this application i then update my library in the evening now both the libraries are present in my system. now i try using the new feature use library 1.1.6 #thats all now i get all features import blah from blahblah ... ... -- Regards, Sriram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew at matthewwilkes.co.uk Thu Nov 12 13:13:39 2009 From: matthew at matthewwilkes.co.uk (Matthew Wilkes) Date: Thu, 12 Nov 2009 12:13:39 +0000 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: References: <4AFBE02F.6020302@voidspace.org.uk> <200911121214.09958.eckhardt@satorlaser.com> Message-ID: Oh, I see. > use library 1.1.5 versus > use library 1.1.6 #thats all now i get all features That's part of pkg_resources. It looks like this: pkg_resources.require("mylibrary==1.1.6") import mylibrary There are plenty of other ways to manage this, most people use systems like virtualenv or buildout, but that's a discussion for the normal python mailing list, not the development one. Hope that helps, Matt From ncoghlan at gmail.com Thu Nov 12 13:26:56 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 12 Nov 2009 22:26:56 +1000 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: References: <4AFBE02F.6020302@voidspace.org.uk> <200911121214.09958.eckhardt@satorlaser.com> Message-ID: <4AFBFF10.7010800@gmail.com> Matthew Wilkes wrote: > There are plenty of other ways to manage this, most people use systems > like virtualenv or buildout, but that's a discussion for the normal > python mailing list, not the development one. Indeed. Please take this to python-list (it isn't even a coherent enough suggestion for python-ideas at this point - there are way too many mistaken assumptions and/or unconventional uses of terminology) Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From exarkun at twistedmatrix.com Thu Nov 12 14:06:57 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Thu, 12 Nov 2009 13:06:57 -0000 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: Message-ID: <20091112130657.3229.1629790699.divmod.xquotient.653@localhost.localdomain> On 09:44 am, ludvig at lericson.se wrote: >Why are there comments on PyPI? Moreso, why are there comments which I >cannot control as a package author on my very own packages? That's >just absurd. > >It's *my* package, and so should be *my* choice if I want user input >or not. > >And ratings? I thought it was the Python Package Index, not the Python >Anonymous Package Tribunal. > >As I see it, there are only two ways to fix these misguided steps of >development: throw them out, or make them opt-in settings. > >The comments I simply do not understand. Why not instead provide a >form for mailing the package author? >The ratings are just not what PyPI should be doing, is about, or what >I signed up for. See the various catalog-sig threads on this topic. Jean-Paul From steve at pearwood.info Thu Nov 12 14:38:03 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 13 Nov 2009 00:38:03 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: Message-ID: <200911130038.05516.steve@pearwood.info> On Thu, 12 Nov 2009 08:44:32 pm Ludvig Ericson wrote: > Why are there comments on PyPI? Moreso, why are there comments which > I cannot control as a package author on my very own packages? That's > just absurd. No, what's absurd is thinking that the act of publishing software somehow gives you the right to demand control over what others say about your software. I don't suppose that this rant of yours has something to do with the comment posted today? http://pypi.python.org/pypi/spypam/1.0 -- Steven D'Aprano From jnoller at gmail.com Thu Nov 12 15:06:31 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 12 Nov 2009 09:06:31 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <200911130038.05516.steve@pearwood.info> References: <200911130038.05516.steve@pearwood.info> Message-ID: <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> On Thu, Nov 12, 2009 at 8:38 AM, Steven D'Aprano wrote: > On Thu, 12 Nov 2009 08:44:32 pm Ludvig Ericson wrote: >> Why are there comments on PyPI? Moreso, why are there comments which >> I cannot control as a package author on my very own packages? That's >> just absurd. > > No, what's absurd is thinking that the act of publishing software > somehow gives you the right to demand control over what others say > about your software. > > I don't suppose that this rant of yours has something to do with the > comment posted today? Frankly, I agree with him. As implemented, I *and others* think this is broken. I've taken the stance of not publishing things to PyPi until A> I find the time to contribute to make it better or B> It changes. jesse From barry at python.org Thu Nov 12 15:25:56 2009 From: barry at python.org (Barry Warsaw) Date: Thu, 12 Nov 2009 08:25:56 -0600 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote: > Frankly, I agree with him. As implemented, I *and others* think this > is broken. I've taken the stance of not publishing things to PyPi > until A> I find the time to contribute to make it better or B> It > changes. That's distressing. For better or worse PyPI is the central repository of 3rd party packages. It should be easy, desirable, fun and socially encouraged to get your packages there. I personally think a ratings system can be useful, but you should be able to opt-out of it if you want. Or just write such awesome software that the bogus bad reviews will be buried by an avalanche of kudos. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 194 bytes Desc: This is a digitally signed message part URL: From regebro at gmail.com Thu Nov 12 15:28:38 2009 From: regebro at gmail.com (Lennart Regebro) Date: Thu, 12 Nov 2009 15:28:38 +0100 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: References: <4AFBE02F.6020302@voidspace.org.uk> Message-ID: <319e029f0911120628o1ec4aed1qba81b29e190bcac3@mail.gmail.com> 2009/11/12 Sriram Srinivasan : > In python we have the PyPI which is equivalent to the http://devpacks.org > but in PyPI the packages are all user made applications. > What I want is similar to PyPI but for the python standard libraries, so > that they (libraries) are as add-on as possible. They are. The mechanism for deciding if you want to use them is called "importing". Now of you don't want them installed, the easiest is to remove the ones you don't want. This is useful on embedded systems, etc, but not generally useful otehrwise. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From amnorvend at gmail.com Thu Nov 12 15:29:17 2009 From: amnorvend at gmail.com (Jason Baker) Date: Thu, 12 Nov 2009 08:29:17 -0600 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: <816c4abb0911120629i479d05d2i5d331854ff90b55d@mail.gmail.com> On Thu, Nov 12, 2009 at 8:06 AM, Jesse Noller wrote: > On Thu, Nov 12, 2009 at 8:38 AM, Steven D'Aprano wrote: >> On Thu, 12 Nov 2009 08:44:32 pm Ludvig Ericson wrote: >>> Why are there comments on PyPI? Moreso, why are there comments which >>> I cannot control as a package author on my very own packages? That's >>> just absurd. >> >> No, what's absurd is thinking that the act of publishing software >> somehow gives you the right to demand control over what others say >> about your software. >> >> I don't suppose that this rant of yours has something to do with the >> comment posted today? > > Frankly, I agree with him. As implemented, I *and others* think this > is broken. I've taken the stance of not publishing things to PyPi > until A> I find the time to contribute to make it better or B> It > changes. I'm not sure I see the utility of ratings, but I think comments can be useful as long as they don't carry over from release to release. For instance, suppose there's a bug in my package and someone leaves a comment about it. I don't want that comment still hanging around 3 years after I've already fixed the bug. From jnoller at gmail.com Thu Nov 12 15:32:11 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 12 Nov 2009 09:32:11 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: <4222a8490911120632j43035c3ndd08292de5ada60a@mail.gmail.com> On Thu, Nov 12, 2009 at 9:25 AM, Barry Warsaw wrote: > On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote: > >> Frankly, I agree with him. As implemented, I *and others* think this >> is broken. I've taken the stance of not publishing things to PyPi >> until A> I find the time to contribute to make it better or B> It >> changes. > > That's distressing. ?For better or worse PyPI is the central repository of > 3rd party packages. ?It should be easy, desirable, fun and socially > encouraged to get your packages there. > > I personally think a ratings system can be useful, but you should be able to > opt-out of it if you want. ?Or just write such awesome software that the > bogus bad reviews will be buried by an avalanche of kudos. > > -Barry I completely and totally agree with you. That's why it's a self-imposed thing for me, I want to help, but don't have the time. In the same breath, I don't want to support it as-is. PyPI isn't a place to file bugs, complain something didn't work for you if you didn't even have the common decency in some cases to email them. Being unable, as an author, to remove, moderate, or even respond to issues there bothers me quite a bit. Heck, I would even be for requiring packages have a mailing list and / or bug tracker to even upload, rather than comments. At least then the authors or maintainers have a fighting chance. From dalcinl at gmail.com Thu Nov 12 16:01:19 2009 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Thu, 12 Nov 2009 13:01:19 -0200 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4222a8490911120632j43035c3ndd08292de5ada60a@mail.gmail.com> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> <4222a8490911120632j43035c3ndd08292de5ada60a@mail.gmail.com> Message-ID: On Thu, Nov 12, 2009 at 12:32 PM, Jesse Noller wrote: > On Thu, Nov 12, 2009 at 9:25 AM, Barry Warsaw wrote: >> On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote: >> >>> Frankly, I agree with him. As implemented, I *and others* think this >>> is broken. I've taken the stance of not publishing things to PyPi >>> until A> I find the time to contribute to make it better or B> It >>> changes. >> >> That's distressing. ?For better or worse PyPI is the central repository of >> 3rd party packages. ?It should be easy, desirable, fun and socially >> encouraged to get your packages there. >> > > PyPI isn't a place to file bugs, complain something didn't work for > you if you didn't even have the common decency in some cases to email > them. Being unable, as an author, to remove, moderate, or even respond > to issues there bothers me quite a bit. > I also agree with you. I do not see the point to make PyPI yet another social network. -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From exarkun at twistedmatrix.com Thu Nov 12 16:23:01 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Thu, 12 Nov 2009 15:23:01 -0000 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> <4222a8490911120632j43035c3ndd08292de5ada60a@mail.gmail.com> Message-ID: <20091112152301.3229.485210589.divmod.xquotient.654@localhost.localdomain> On 03:01 pm, dalcinl at gmail.com wrote: >On Thu, Nov 12, 2009 at 12:32 PM, Jesse Noller >wrote: >>On Thu, Nov 12, 2009 at 9:25 AM, Barry Warsaw >>wrote: >>>On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote: >>>>Frankly, I agree with him. As implemented, I *and others* think this >>>>is broken. I've taken the stance of not publishing things to PyPi >>>>until A> I find the time to contribute to make it better or B> It >>>>changes. >>> >>>That's distressing. ?For better or worse PyPI is the central >>>repository of >>>3rd party packages. ?It should be easy, desirable, fun and socially >>>encouraged to get your packages there. >> >>PyPI isn't a place to file bugs, complain something didn't work for >>you if you didn't even have the common decency in some cases to email >>them. Being unable, as an author, to remove, moderate, or even respond >>to issues there bothers me quite a bit. > >I also agree with you. I do not see the point to make PyPI yet another >social network. +1 Jean-Paul From olemis at gmail.com Thu Nov 12 16:33:55 2009 From: olemis at gmail.com (Olemis Lang) Date: Thu, 12 Nov 2009 10:33:55 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <200911130038.05516.steve@pearwood.info> References: <200911130038.05516.steve@pearwood.info> Message-ID: <24ea26600911120733n58e22403hbed0afa289e3fd60@mail.gmail.com> On Thu, Nov 12, 2009 at 8:38 AM, Steven D'Aprano wrote: > On Thu, 12 Nov 2009 08:44:32 pm Ludvig Ericson wrote: >> Why are there comments on PyPI? Moreso, why are there comments which >> I cannot control as a package author on my very own packages? That's >> just absurd. > > No, what's absurd is thinking that the act of publishing software > somehow gives you the right to demand control over what others say > about your software. > Both of you are right , but I agree with Mr. Ludvig Ericson opinion (> 90%). The package author probably has no ?right to demand control over what others say about her/his software? , but he has the right to decide where such comments should be posted and also if he/she wants to focus on (opinions | comments | ... ) or more useful feedback like issues or support request . For example, in my case, I prefer to have either custom ticket types in the project's Trac environment or a plugin to receive this kind of feedback *in the project's site* . IMHO PyPI is not the right place . /me probably wrong IMO -0.1 for votes and comments in PyPI and therefore +1 for including settings to let coders decide (somehow ;o) whether to allow this or not -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: From olemis at gmail.com Thu Nov 12 16:50:23 2009 From: olemis at gmail.com (Olemis Lang) Date: Thu, 12 Nov 2009 10:50:23 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4222a8490911120632j43035c3ndd08292de5ada60a@mail.gmail.com> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> <4222a8490911120632j43035c3ndd08292de5ada60a@mail.gmail.com> Message-ID: <24ea26600911120750k427ffcbj4a10afd0ff288747@mail.gmail.com> On Thu, Nov 12, 2009 at 9:32 AM, Jesse Noller wrote: > On Thu, Nov 12, 2009 at 9:25 AM, Barry Warsaw wrote: >> On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote: >> >>> Frankly, I agree with him. As implemented, I *and others* think this >>> is broken. I've taken the stance of not publishing things to PyPi >>> until A> I find the time to contribute to make it better or B> It >>> changes. >> >> That's distressing. ?For better or worse PyPI is the central repository of >> 3rd party packages. ?It should be easy, desirable, fun and socially >> encouraged to get your packages there. >> >> I personally think a ratings system can be useful, but you should be able to >> opt-out of it if you want. ?Or just write such awesome software that the >> bogus bad reviews will be buried by an avalanche of kudos. >> >> -Barry > > I completely and totally agree with you. That's why it's a > self-imposed thing for me, I want to help, but don't have the time. In > the same breath, I don't want to support it as-is. > > PyPI isn't a place to file bugs, complain something didn't work for > you if you didn't even have the common decency in some cases to email > them. Being unable, as an author, to remove, moderate, or even respond > to issues there bothers me quite a bit. > +1 > Heck, I would even be for requiring packages have a mailing list and / > or bug tracker to even upload, rather than comments. At least then the > authors or maintainers have a fighting chance. > Intention = suggestion + crazy idea => for a better PyPI But there's probably a chance to display what people said in the project's site . If PyPI would be able to retrieve that information from the project's site (e.g. that'd be possible for Trac and other PMS via RPC ) and also some of the aforementioned (Jesse's) issues might be solved with added benefits (data being cached and discarded from time to time, better performance, less DB space, ...) or not . IMO, what's missing in my reasoning is whether there is a common std API for e.g. issues . But there's a popular API for wikis (i.e. WikiRPC) so probably there's something std (I repeat, that I don't know :-/ ) out there . At least Trac's TicketRPC is very simple (i.e. two simple methods) and extensible (e.g. custom ticket fields ;o) PS: I don't really know the exact details of the impl of votes and comments in PyPI. -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: From guido at python.org Thu Nov 12 17:02:46 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Nov 2009 08:02:46 -0800 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <200911130038.05516.steve@pearwood.info> References: <200911130038.05516.steve@pearwood.info> Message-ID: On Thu, Nov 12, 2009 at 5:38 AM, Steven D'Aprano wrote: > On Thu, 12 Nov 2009 08:44:32 pm Ludvig Ericson wrote: >> Why are there comments on PyPI? Moreso, why are there comments which >> I cannot control as a package author on my very own packages? That's >> just absurd. > > No, what's absurd is thinking that the act of publishing software > somehow gives you the right to demand control over what others say > about your software. > > I don't suppose that this rant of yours has something to do with the > comment posted today? > > http://pypi.python.org/pypi/spypam/1.0 If you were to ask me, the people arguing against ratings and user comments are fighting a losing battle. If they had an iPhone or Android phone (or some other device with an "app store" kind of place to find downloads) they'd know the value (for prospective downloaders) of ratings and comments. Now, I think PyPI can use some (perhaps a lot of) improvement in the details of how it works, e.g. there should be a way to flag inappropriate messages (and users who post many inappropriate messages) and the software author should be able to talk back, but the general idea is here and won't go away by wishing it away. -- --Guido van Rossum (python.org/~guido) From solipsis at pitrou.net Thu Nov 12 17:19:09 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 12 Nov 2009 16:19:09 +0000 (UTC) Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <200911130038.05516.steve@pearwood.info> Message-ID: Guido van Rossum python.org> writes: > > If you were to ask me, the people arguing against ratings and user > comments are fighting a losing battle. If they had an iPhone or > Android phone (or some other device with an "app store" kind of place > to find downloads) they'd know the value (for prospective downloaders) > of ratings and comments. Now, I think PyPI can use some (perhaps a lot > of) improvement in the details of how it works, e.g. there should be a > way to flag inappropriate messages (and users who post many > inappropriate messages) and the software author should be able to talk > back, but the general idea is here and won't go away by wishing it > away. Perhaps the suggestions and appreciations about the PyPI ratings and comment system can go to http://pypi.python.org/pypi/pypi/ :-) (more seriously, the problem with a comment system is that once it takes off, you need a whole array of functionalities to maintain a good S/N ratio. Just allowing people to "comment" without any sort of moderation, filtering or community building doesn't work) Regards Antoine. From pluskid at gmail.com Thu Nov 12 17:27:07 2009 From: pluskid at gmail.com (Zhang Chiyuan) Date: Fri, 13 Nov 2009 00:27:07 +0800 Subject: [Python-Dev] Is this a bug of the HTMLParser? In-Reply-To: <4AFAE522.2090403@voidspace.org.uk> References: <4AFAE522.2090403@voidspace.org.uk> Message-ID: filed: http://bugs.python.org/issue7311 On Thu, Nov 12, 2009 at 12:24 AM, Michael Foord wrote: > Hello Zhang Chiyuan, > > Can you file a bug on the Python issue tracker please: > > http://bugs.python.org > > Thanks > > Michael Foord > > Zhang Chiyuan wrote: > >> Hi all, >> >> I'm using BeautifulSoup to parsing an HTML page and find it refused to >> parse the page. By looking at the backtrace, I found it is a problem >> with the python built-in HTMLParser.py. In fact, the web page I'm >> parsing is with some Chinese characters. there is a tag like > src=/foo/bar.png alt=??> , note this is legacy html page where the >> attributes are not quoted. However, the regexp defined in >> HTMLParser.py is : >> >> attrfind = re.compile( >> r'\s*([a-zA-Z_][-.:a-zA-Z_0-9]*)(\s*=\s*' >> r'(\'[^\']*\'|"[^"]*"|[-a-zA-Z0-9./,:;+*%?!&$\(\)_#=~@]*))?') >> >> Note that the Chinese character (also any other non-english >> characters), so it fire an error parsing this. I'm not sure whether >> the HTML standard allow un-quoted non-ASCII characters in the >> attributes. If it allows, this seems to be a bug. and the regexp to >> better be [^>\s] IMHO. >> >> BTW: It seems something like : >> >> >> >> can not be parsed. :-/ >> >> -- >> pluskid >> http://blog.pluskid.org >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk >> >> > > > -- > http://www.ironpythoninaction.com/ > > -- pluskid http://blog.pluskid.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jnoller at gmail.com Thu Nov 12 17:31:29 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 12 Nov 2009 11:31:29 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> Message-ID: <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> On Thu, Nov 12, 2009 at 11:02 AM, Guido van Rossum wrote: > If you were to ask me, the people arguing against ratings and user > comments are fighting a losing battle. If they had an iPhone or > Android phone (or some other device with an "app store" kind of place > to find downloads) they'd know the value (for prospective downloaders) > of ratings and comments. Now, I think PyPI can use some (perhaps a lot > of) improvement in the details of how it works, e.g. there should be a > way to flag inappropriate messages (and users who post many > inappropriate messages) and the software author should be able to talk > back, but the general idea is here and won't go away by wishing it > away. > > -- > --Guido van Rossum (python.org/~guido) I don't want us to impersonate the mindless, sub-useful drivel found in App store/YouTube/etc comments 99% of the time or the broken "5 star ratings" systems, etc. It's too easy to game. I'm not arguing something like this *shouldn't* exist; but that the current implementation is a far cry from something actually *good and useful*. If we want forums, let's put in forums. If we want a real review system, then do that. But before we even did those; why not have mandatory links for entries to bug trackers, mailing lists, source repositories, etc? I'm saying saying this doesn't seem well thought out, and the current implementation is broken by design. Of course, as I said earlier; since I don't have time to patch it; I'll simply just not participate. jesse From amnorvend at gmail.com Thu Nov 12 17:59:27 2009 From: amnorvend at gmail.com (Jason Baker) Date: Thu, 12 Nov 2009 10:59:27 -0600 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> Message-ID: <816c4abb0911120859x482fdfe8r53715730e6f05ec5@mail.gmail.com> On Thu, Nov 12, 2009 at 10:19 AM, Antoine Pitrou wrote: > (more seriously, the problem with a comment system is that once it takes off, > you need a whole array of functionalities to maintain a good S/N ratio. Just > allowing people to "comment" without any sort of moderation, filtering or > community building doesn't work) Why not allow ratings on comments as well? From dstanek at dstanek.com Thu Nov 12 18:02:59 2009 From: dstanek at dstanek.com (David Stanek) Date: Thu, 12 Nov 2009 12:02:59 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: On Thu, Nov 12, 2009 at 9:06 AM, Jesse Noller wrote: > On Thu, Nov 12, 2009 at 8:38 AM, Steven D'Aprano > wrote: > > On Thu, 12 Nov 2009 08:44:32 pm Ludvig Ericson wrote: > >> Why are there comments on PyPI? Moreso, why are there comments which > >> I cannot control as a package author on my very own packages? That's > >> just absurd. > > > > No, what's absurd is thinking that the act of publishing software > > somehow gives you the right to demand control over what others say > > about your software. > > > > I don't suppose that this rant of yours has something to do with the > > comment posted today? > > Frankly, I agree with him. As implemented, I *and others* think this > is broken. I've taken the stance of not publishing things to PyPi > until A> I find the time to contribute to make it better or B> It > changes. > Where is the code for PyPi? I took a quick look and didn't turn up anything. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek -------------- next part -------------- An HTML attachment was scrubbed... URL: From jacob at jacobian.org Thu Nov 12 18:06:36 2009 From: jacob at jacobian.org (Jacob Kaplan-Moss) Date: Thu, 12 Nov 2009 11:06:36 -0600 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: On Thu, Nov 12, 2009 at 11:02 AM, David Stanek wrote: > Where is the code for PyPi? I took a quick look and didn't turn up anything. https://svn.python.org/packages/trunk/pypi/ I've already started on a patch to make comments an option that package maintainers could turn on or off, but I don't want to waste any more time fighting this code unless I have some assurance it'll be checked in. Jacob From masklinn at masklinn.net Thu Nov 12 18:08:27 2009 From: masklinn at masklinn.net (Masklinn) Date: Thu, 12 Nov 2009 18:08:27 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> Message-ID: <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> On 12 Nov 2009, at 17:31 , Jesse Noller wrote: > > But before we even did those; why not have mandatory links for entries > to bug trackers, mailing lists, source repositories, etc? I'm saying > saying this doesn't seem well thought out, and the current > implementation is broken by design. Of course, as I said earlier; > since I don't have time to patch it; I'll simply just not participate. I think having links to those is a very good idea and more important than a comment/notation system. They shouldn't be mandatory though, not every library has a mailing list, or even a (public anyway) bug tracker. Giving users and easy way to contact the author would be a must as well ("package index owner" and "package index maintainer" should link to the users's profiles, and users should be able to setup contact informations e.g. a mail address or a website). See how search.cpan does it, the information is worth a lot to users. Currently, finding how to send feedback to a package owner usually requires wading through the description text itself, which in some cases (e.g. distribute) amounts to pages of document. On 12 Nov 2009, at 17:31 , Jesse Noller wrote: > I don't want us to impersonate the mindless, sub-useful drivel found > in App store/YouTube/etc comments 99% of the time or the broken "5 > star ratings" systems, etc. It's too easy to game. I think I read a paper on that subject a short time ago (but can't seem to find it right now), basically saying 5-star systems weren't very useful, a simple +1/0/-1 was just as good (if not better) and 0/+1 worked as well. From dstanek at dstanek.com Thu Nov 12 18:14:11 2009 From: dstanek at dstanek.com (David Stanek) Date: Thu, 12 Nov 2009 12:14:11 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: On Thu, Nov 12, 2009 at 12:06 PM, Jacob Kaplan-Moss wrote: > On Thu, Nov 12, 2009 at 11:02 AM, David Stanek > wrote: > > Where is the code for PyPi? I took a quick look and didn't turn up > anything. > > https://svn.python.org/packages/trunk/pypi/ > > I've already started on a patch to make comments an option that > package maintainers could turn on or off, but I don't want to waste > any more time fighting this code unless I have some assurance it'll be > checked in. > Thanks. If I have some spare time I'm going take a look. Should I post patches to the regular Python bug tracker? -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek -------------- next part -------------- An HTML attachment was scrubbed... URL: From arcriley at gmail.com Thu Nov 12 18:27:19 2009 From: arcriley at gmail.com (Arc Riley) Date: Thu, 12 Nov 2009 12:27:19 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <200911130038.05516.steve@pearwood.info> References: <200911130038.05516.steve@pearwood.info> Message-ID: Nobody is claiming right to censor what people say about their software. This is the Internet. There are blogs. Google and other search engines find blogs quickly, and people who agree with the viewpoints expressed link to them thus making the blog postings more visible. There are countless other social networks and outlets for people to flame and slander (or praise and promote, in a much less common case) software. It would be more useful to provide a PyPI mechanism to publish a link to file bugs on the project's own website and leave project ratings the work of other sites such as Ohloh. On Thu, Nov 12, 2009 at 8:38 AM, Steven D'Aprano wrote: > > No, what's absurd is thinking that the act of publishing software > somehow gives you the right to demand control over what others say > about your software. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at rcn.com Thu Nov 12 18:32:15 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 12 Nov 2009 09:32:15 -0800 Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <200911130038.05516.steve@pearwood.info><4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: <6B1D2192FBED4006B98E6BBEE952097E@RaymondLaptop1> [Jacob Kaplan-Moss] > I've already started on a patch to make comments an option that > package maintainers could turn on or off, but I don't want to waste > any more time fighting this code unless I have some assurance it'll be > checked in. I support your efforts. Raymond From amk at amk.ca Thu Nov 12 18:40:10 2009 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 12 Nov 2009 12:40:10 -0500 Subject: [Python-Dev] FYI: LWN article on 2.x, 3.x, and the moratorium Message-ID: <20091112174010.GA11309@amk-desktop.matrixgroup.net> FYI: I've written an article for Linux Weekly News on the moratorium & related issues. The article is subscribers-only for a week, but here's a free link: http://lwn.net/SubscriberLink/361266/ef88bdbed5369800/ If you find this sort of thing useful/interesting, please consider subscribing to LWN. --amk From phd at phd.pp.ru Thu Nov 12 19:03:33 2009 From: phd at phd.pp.ru (Oleg Broytman) Date: Thu, 12 Nov 2009 21:03:33 +0300 Subject: [Python-Dev] FYI: LWN article on 2.x, 3.x, and the moratorium In-Reply-To: <20091112174010.GA11309@amk-desktop.matrixgroup.net> References: <20091112174010.GA11309@amk-desktop.matrixgroup.net> Message-ID: <20091112180333.GD5080@phd.pp.ru> On Thu, Nov 12, 2009 at 12:40:10PM -0500, A.M. Kuchling wrote: > FYI: I've written an article for Linux Weekly News on the moratorium & > related issues. > > The article is subscribers-only for a week, but here's a free link: > > http://lwn.net/SubscriberLink/361266/ef88bdbed5369800/ > > If you find this sort of thing useful/interesting, please consider > subscribing to LWN. +1. I've found LWN the best magazine (among both printed and electronic magazines) in its area, and despite the fact its articles are freed after a week (and many articles are published free from the very beginning) I subscribed to it to support the magazine ("Patronize" business model). They say their site is written in Python using Quixote as the web framework; they promised to publish the code but didn't publish it yet - the fact I consider the biggest drawback of the site. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From tjreedy at udel.edu Thu Nov 12 19:30:15 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 12 Nov 2009 13:30:15 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: Barry Warsaw wrote: > On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote: > >> Frankly, I agree with him. As implemented, I *and others* think this >> is broken. I've taken the stance of not publishing things to PyPi >> until A> I find the time to contribute to make it better or B> It >> changes. > > That's distressing. For better or worse PyPI is the central repository > of 3rd party packages. It should be easy, desirable, fun and socially > encouraged to get your packages there. I think his point is that a new book announcement servive is different from a book review and rating service. And that mixing the two is 'socially discouraging'. I do not know what the answer is > I personally think a ratings system can be useful, but you should be > able to opt-out of it if you want. Or just write such awesome software > that the bogus bad reviews will be buried by an avalanche of kudos. From guido at python.org Thu Nov 12 19:54:24 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Nov 2009 10:54:24 -0800 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: On Thu, Nov 12, 2009 at 10:30 AM, Terry Reedy wrote: > Barry Warsaw wrote: >> >> On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote: >> >>> Frankly, I agree with him. As implemented, I *and others* think this >>> is broken. I've taken the stance of not publishing things to PyPi >>> until A> I find the time to contribute to make it better or B> It >>> changes. >> >> That's distressing. ?For better or worse PyPI is the central repository of >> 3rd party packages. ?It should be easy, desirable, fun and socially >> encouraged to get your packages there. > > I think his point is that a new book announcement servive is different from > a book review and rating service. ?And that mixing the two is 'socially > discouraging'. I do not know what the answer is I would say that publishers disagree -- they seem to really like adding "social" stuff to their book announcement service. See e.g. Amazon (which combines all functions: announcement/promotion, ordering/download, review/comments/rate/popularity). I agree that creating a good social app is not easy, and if we can't improve the social app embedded in PyPI quickly enough, we should at least give authors the option to disable comments. Of course, as a user, I might not trust a module that has no reviews or ratings. -- --Guido van Rossum (python.org/~guido) From olemis at gmail.com Thu Nov 12 20:21:57 2009 From: olemis at gmail.com (Olemis Lang) Date: Thu, 12 Nov 2009 14:21:57 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: <24ea26600911121121w122c1ea3lfc2a5fc0e6fd059a@mail.gmail.com> Intention = precision => for a better PyPI On Thu, Nov 12, 2009 at 1:54 PM, Guido van Rossum wrote: > On Thu, Nov 12, 2009 at 10:30 AM, Terry Reedy wrote: >> Barry Warsaw wrote: >>> >>> On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote: >>> >>>> Frankly, I agree with him. As implemented, I *and others* think this >>>> is broken. I've taken the stance of not publishing things to PyPi >>>> until A> I find the time to contribute to make it better or B> It >>>> changes. >>> >>> That's distressing. ?For better or worse PyPI is the central repository of >>> 3rd party packages. ?It should be easy, desirable, fun and socially >>> encouraged to get your packages there. >> >> I think his point is that a new book announcement servive is different from >> a book review and rating service. ?And that mixing the two is 'socially >> discouraging'. I do not know what the answer is > > I would say that publishers disagree -- they seem to really like > adding "social" stuff to their book announcement service. See e.g. > Amazon (which combines all functions: announcement/promotion, > ordering/download, review/comments/rate/popularity). > ... but (most) book writers don't use an issue tracker to manage and get *useful* feedback from their readers (I know there are exceptions to the rule ;o) and fix the book chapters or anything else . Besides there are some differences between software and books and the way both of them are created, used and enhanced . What I don't like (today) about comments + votes is that I have to do the same thing in two different places (especially because one of the sources is *very* noisy). If there's a way to integrate both and ?reduce? the noise , that would be nice . ;o) > I agree that creating a good social app is not easy, and if we can't > improve the social app embedded in PyPI quickly enough, we should at > least give authors the option to disable comments. +1 > Of course, as a > user, I might not trust a module that has no reviews or ratings. > Not really sure. For example, if a user access the page for setuptools (just an example ;o) soon she/he will realize that other people use it very often and also has a high kwalitee score, therefore it is quite unlikely that such package be ?irrelevant? or ?untrusted? (this is IMHO) . -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: From jnoller at gmail.com Thu Nov 12 20:25:42 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 12 Nov 2009 14:25:42 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: <4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com> On Thu, Nov 12, 2009 at 1:54 PM, Guido van Rossum wrote: > > I would say that publishers disagree -- they seem to really like > adding "social" stuff to their book announcement service. See e.g. > Amazon (which combines all functions: announcement/promotion, > ordering/download, review/comments/rate/popularity). > > I agree that creating a good social app is not easy, and if we can't > improve the social app embedded in PyPI quickly enough, we should at > least give authors the option to disable comments. Of course, as a > user, I might not trust a module that has no reviews or ratings. > > -- > --Guido van Rossum (python.org/~guido) I'd not trust a package without a bug tracker, mailing list or link to the source a lot sooner than something without comments and ratings. Especially with ratings like milk and wolf shirts get: http://www.amazon.com/Tuscan-Whole-Milk-Gallon-128/dp/B00032G1S0/ref=sr_1_13?ie=UTF8&s=grocery&qid=1258053581&sr=1-13 http://www.amazon.com/Mountain-Mens-Short-Sleeve-Large/dp/B001VMZFPQ/ref=sr_1_1?ie=UTF8&s=apparel&qid=1258053663&sr=8-1 What about astroturfing? What's to stop me from writing a script to create a pile of accounts and then bumping packages I like with glowing ratings and reviews? Who is going to be the moderator, and how to decide between spam, incorrect comment, etc? From guido at python.org Thu Nov 12 20:30:27 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Nov 2009 11:30:27 -0800 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> <4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com> Message-ID: On Thu, Nov 12, 2009 at 11:25 AM, Jesse Noller wrote: > I'd not trust a package without a bug tracker, mailing list or link to > the source a lot sooner than something without comments and ratings. Yeah, but you're not exactly an average user. Most users don't know how to use a bug tracker. Also, there's a large variety of packages on PyPI. Not every developer has the same attitude, but they all live happily together on PyPI. (Or did you want someone to start a separate CPAN "for the rest of them" ? :-) [...] > What about astroturfing? What's to stop me from writing a script to > create a pile of accounts and then bumping packages I like with > glowing ratings and reviews? Who is going to be the moderator, and how > to decide between spam, incorrect comment, etc? Those are all hard problems, though all of them have at least partial solutions in the other worlds (Amazon, Wikipedia, Apple app store, etc.). Maybe there should be a standard "social app" that you can just customize for a specific purpose. Sounds like an interesting project, actually. -- --Guido van Rossum (python.org/~guido) From jnoller at gmail.com Thu Nov 12 20:38:21 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 12 Nov 2009 14:38:21 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> <4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com> Message-ID: <4222a8490911121138s2b7a26f5w6f68a03a9067c927@mail.gmail.com> On Thu, Nov 12, 2009 at 2:30 PM, Guido van Rossum wrote: > On Thu, Nov 12, 2009 at 11:25 AM, Jesse Noller wrote: >> I'd not trust a package without a bug tracker, mailing list or link to >> the source a lot sooner than something without comments and ratings. > > Yeah, but you're not exactly an average user. Most users don't know > how to use a bug tracker. Also, there's a large variety of packages on > PyPI. Not every developer has the same attitude, but they all live > happily together on PyPI. (Or did you want someone to start a separate > CPAN "for the rest of them" ? :-) True, but if you make entries for them mandatory (bug trackers, source, etc), and you encourage users to use them, you begin being to be the change you want to be, which is making PyPi *less* of an "app store" where the consumer doesn't collaborate with the authors. Or maybe rather than *putting* this stuff into Pypi; pypi allows plugins to allow authors to link in RSS feeds to their bug trackers, wiki streams, what have you. I think everyone can co exist, just not one at the cost of another ;) > [...] >> What about astroturfing? What's to stop me from writing a script to >> create a pile of accounts and then bumping packages I like with >> glowing ratings and reviews? Who is going to be the moderator, and how >> to decide between spam, incorrect comment, etc? > > Those are all hard problems, though all of them have at least partial > solutions in the other worlds (Amazon, Wikipedia, Apple app store, > etc.). Maybe there should be a standard "social app" that you can just > customize for a specific purpose. Sounds like an interesting project, > actually. Yup, reddit's source is out there, and I think there's lots of possibilities, I guess for me I'd rather have nothing than 50% of something that doesn't account for the various problems and the pretty valid opinions of authors and maintainers. jesse From martin at v.loewis.de Thu Nov 12 20:38:40 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 12 Nov 2009 20:38:40 +0100 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: References: Message-ID: <4AFC6440.1040008@v.loewis.de> > I am not an expert, I am just another python learner. These are just my > views on the state of the standard libraries and to > make them state-of-the-art..! ;) If I understand correctly, you want the (current) standard library to be separated from the Python implementation, and available separately. Interestingly enough, people are very much split over whether that would be a good thing or not. Some like it the way Python does, some dislike it (and some quite strongly so). In any case, many Python users consider it a good thing that it comes "with batteries included", ie. with no need to add extra stuff for many tasks. Some of the Python maintainers have recently started objecting to this setup, asking that the standard library should be split into separate packages that are released and distributed independent of Python. Others of us feel strongly that such a change should not be made. So don't expect any immediate change to happen. Regards, Martin From martin at v.loewis.de Thu Nov 12 20:44:41 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 12 Nov 2009 20:44:41 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> Message-ID: <4AFC65A9.8060304@v.loewis.de> > If you were to ask me, the people arguing against ratings and user > comments are fighting a losing battle. If they had an iPhone or > Android phone (or some other device with an "app store" kind of place > to find downloads) they'd know the value (for prospective downloaders) > of ratings and comments. Now, I think PyPI can use some (perhaps a lot > of) improvement in the details of how it works, e.g. there should be a > way to flag inappropriate messages (and users who post many > inappropriate messages) There is, although it's admittedly tedious: users can file a support request (follow the link "Get Help"). > and the software author should be able to talk > back That is actually the case: the author *can* talk back. > but the general idea is here and won't go away by wishing it > away. I'm going to take a poll RSN, and see what the majority of users think (rather than their vocal fraction). Then we can see what to do about it. Regards, Martin From jacob at jacobian.org Thu Nov 12 20:45:45 2009 From: jacob at jacobian.org (Jacob Kaplan-Moss) Date: Thu, 12 Nov 2009 13:45:45 -0600 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> <4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com> Message-ID: On Thu, Nov 12, 2009 at 1:30 PM, Guido van Rossum wrote: > Yeah, but you're not exactly an average user. Most users don't know > how to use a bug tracker. But they do know how to use mailing lists. Or IRC chats. Or support forums. Those places have (for many projects) tens, hundreds, or even thousands of peers who are able and willing to help new users get started. Only the package maintainers see comments on PyPI, meaning we've got to deal with requests for support there manually. This isn't academic; just this morning a user asked a question on Django's PyPI listing that would have been better asked on any of the support channels we provide. I have no way of directing him there besides lamely commenting after the fact, and then it just seems like I'm giving him the runaround. Look, nobody's asking to kill the feature. We're asking to *make it optional*, and to allow us to link to a more appropriate support forum instead. Can you please explain to me what's wrong with that? Jacob From martin at v.loewis.de Thu Nov 12 20:46:44 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 12 Nov 2009 20:46:44 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> Message-ID: <4AFC6624.2070802@v.loewis.de> > (more seriously, the problem with a comment system is that once it takes off, > you need a whole array of functionalities to maintain a good S/N ratio. Just > allowing people to "comment" without any sort of moderation, filtering or > community building doesn't work) The current rate is roughly 1 comment per day (with peaks of 5 comments), so it takes of rather slowly. Regards, Martin From g.brandl at gmx.net Thu Nov 12 20:50:43 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 12 Nov 2009 20:50:43 +0100 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: References: Message-ID: Sriram Srinivasan schrieb: > I guess why every programming language has some kind of a 'standard > library' built in within it. In my view it must not be called as a > 'library' at all. what it does > is like a 'bunch of built-in programs ready-made to do stuff'. > > Lets see what a 'library' does: > > 1. offers books for customers > 1.1 lets user select a book by genre, etc > 1.2 lets user to use different books of same genre, etc > 1.3 lets user to use books by same author, etc for different genre > > 2. keeps track of all the books + their genre > 2.1 first knows what all books it has at present > 2.2 when new book comes it is added to the particular shelf sorted by > genre,author,edition, etc.. > 2.3 when books become old they are kept separately for future reference > 2.4 very old books can be sent to a museum/discarded > > I guess no standard library does the minimum of this but wants to be > called a library. I guess you're simply stretching the "library" metaphor far beyond its usefulness :) Georg From jnoller at gmail.com Thu Nov 12 20:57:45 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 12 Nov 2009 14:57:45 -0500 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: <4AFC6440.1040008@v.loewis.de> References: <4AFC6440.1040008@v.loewis.de> Message-ID: <4222a8490911121157i622fbab8q5fe82d32fe9ea8f@mail.gmail.com> On Thu, Nov 12, 2009 at 2:38 PM, "Martin v. L?wis" wrote: >> I am not an expert, I am just another python learner. These are just my >> views on the state of the standard libraries and to >> make them state-of-the-art..! ;) > > If I understand correctly, you want the (current) standard library to be > separated from the Python implementation, and available separately. > > Interestingly enough, people are very much split over whether that would > be a good thing or not. Some like it the way Python does, some dislike > it (and some quite strongly so). > > In any case, many Python users consider it a good thing that it comes > "with batteries included", ie. with no need to add extra stuff for many > tasks. > > Some of the Python maintainers have recently started objecting to this > setup, asking that the standard library should be split into separate > packages that are released and distributed independent of Python. Others > of us feel strongly that such a change should not be made. > > So don't expect any immediate change to happen. > > Regards, > Martin Martin is correct; this came up on distutils-sig a month or so ago; I proposed offering multiple downloads "with batteries" and "without batteries (with the batteries on the side)". We decided as a group to hold off on that until further in the future. jesse From guido at python.org Thu Nov 12 21:01:08 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Nov 2009 12:01:08 -0800 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4AFC65A9.8060304@v.loewis.de> References: <200911130038.05516.steve@pearwood.info> <4AFC65A9.8060304@v.loewis.de> Message-ID: On Thu, Nov 12, 2009 at 11:44 AM, "Martin v. L?wis" wrote: > I'm going to take a poll RSN, and see what the majority of users > think (rather than their vocal fraction). Then we can see what to do > about it. Or (ironically) the vocal fraction can write scripts to stuff the ballot. :-) -- --Guido van Rossum (python.org/~guido) From guido at python.org Thu Nov 12 21:04:01 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Nov 2009 12:04:01 -0800 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> <4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com> Message-ID: On Thu, Nov 12, 2009 at 11:45 AM, Jacob Kaplan-Moss wrote: > On Thu, Nov 12, 2009 at 1:30 PM, Guido van Rossum wrote: >> Yeah, but you're not exactly an average user. Most users don't know >> how to use a bug tracker. > > But they do know how to use mailing lists. Or IRC chats. Or support forums. > > Those places have (for many projects) tens, hundreds, or even > thousands of peers who are able and willing to help new users get > started. Only the package maintainers see comments on PyPI, meaning > we've got to deal with requests for support there manually. > > This isn't academic; just this morning a user asked a question on > Django's PyPI listing that would have been better asked on any of the > support channels we provide. I have no way of directing him there > besides lamely commenting after the fact, and then it just seems like > I'm giving him the runaround. Maybe that's an example of a user who doesn't know how to use those support channels? I know I wouldn't bother with IRC even if it was the only way to get in touch with users, I hate it with a vengeance. (Though arguably I'm a special case -- whenever I show up everyone goes "ooooh, Guido is here." :-) And I might not want to sign up for a mailing list for a casual question. And what exactly is a "forum"? > Look, nobody's asking to kill the feature. We're asking to *make it > optional*, and to allow us to link to a more appropriate support forum > instead. Can you please explain to me what's wrong with that? I already said it was fine to make it opt-out. What more do you want? -- --Guido van Rossum (python.org/~guido) From dalcinl at gmail.com Thu Nov 12 21:13:48 2009 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Thu, 12 Nov 2009 18:13:48 -0200 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4AFC6624.2070802@v.loewis.de> References: <200911130038.05516.steve@pearwood.info> <4AFC6624.2070802@v.loewis.de> Message-ID: On Thu, Nov 12, 2009 at 5:46 PM, "Martin v. L?wis" wrote: >> (more seriously, the problem with a comment system is that once it takes off, >> you need a whole array of functionalities to maintain a good S/N ratio. Just >> allowing people to "comment" without any sort of moderation, filtering or >> community building doesn't work) > > The current rate is roughly 1 comment per day (with peaks of 5 > comments), so it takes of rather slowly. > Until spammers decide to attack... -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From martin at v.loewis.de Thu Nov 12 21:19:33 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 12 Nov 2009 21:19:33 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4AFC6624.2070802@v.loewis.de> Message-ID: <4AFC6DD5.7070702@v.loewis.de> >> The current rate is roughly 1 comment per day (with peaks of 5 >> comments), so it takes of rather slowly. >> > > Until spammers decide to attack... Sure. However, spambots have avoided PyPI so far, and manual spamming only had one incident (of somebody creating dozens of packages on a single day). My understanding is that spambots target systems where a comment form is present even in the unloggedin state. In PyPI, you need to create the account before you can comment. Regards, Martin From olemis at gmail.com Thu Nov 12 21:30:17 2009 From: olemis at gmail.com (Olemis Lang) Date: Thu, 12 Nov 2009 15:30:17 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4222a8490911121138s2b7a26f5w6f68a03a9067c927@mail.gmail.com> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> <4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com> <4222a8490911121138s2b7a26f5w6f68a03a9067c927@mail.gmail.com> Message-ID: <24ea26600911121230n536a50a5n56effad77fdba35e@mail.gmail.com> Intention = personal opinion => for a better PyPI On Thu, Nov 12, 2009 at 2:38 PM, Jesse Noller wrote: > On Thu, Nov 12, 2009 at 2:30 PM, Guido van Rossum wrote: >> On Thu, Nov 12, 2009 at 11:25 AM, Jesse Noller wrote: >>> I'd not trust a package without a bug tracker, mailing list or link to >>> the source a lot sooner than something without comments and ratings. >> >> Yeah, but you're not exactly an average user. Most users don't know >> how to use a bug tracker. Also, there's a large variety of packages on >> PyPI. Not every developer has the same attitude, but they all live >> happily together on PyPI. (Or did you want someone to start a separate >> CPAN "for the rest of them" ? :-) > > True, but if you make entries for them mandatory (bug trackers, > source, etc), and you encourage users to use them, you begin being to > be the change you want to be, which is making PyPi *less* of an "app > store" where the consumer doesn't collaborate with the authors. > > Or maybe rather than *putting* this stuff into Pypi; pypi allows > plugins to allow authors to link in RSS feeds to their bug trackers, > wiki streams, what have you. > IMO plugins could be a little bit complicated because PyPI would need to be extended, and there's also the problem of installing, upgrading and maintaining each plugin . OTOH if PyPI relies on a single API based on open standards (e.g. RPC or something RESTful ;o) then that would represent less overhead for PyPI maintainers . Instead of votes + comments I'd prefer a similar user interface but doing things as follows (feel free to filter things; besides I'll mention how it should work using Trac XmlRpcPlugin , but should be similar for other PMS ;o) : - Previous comments retrieved from third party site and shown (e.g. no more than 5 and only most recent shown like TH.org) as well as a link to navigate to third party site in order to look for further issues . (ticket.getAll + ticket.get) - Text input would be the description of a ticket (ticket.create) - A combobox to select either comment | defect | support (ticket.create) - Ratings could be interpreted as a priority or severity ... (labels = ticket.priority.getAll + ticket.priority.getAll , values for single item = ticket.get ) Implementing all this might require to add more information to the index (I am not sure) and also config options in the site for package maintainers, but since it'd be more useful (for me) probable (me and others) will prefer something like that : *and users won't even notice the changes* ;o) Nonetheless plugins approach is more general and flexible, and it is also possible to develop a plugin to support the RPC-based integration with external issue trackers . The main difference is maintenance effort once it's up and running . ;o) > I think everyone can co exist, just not one at the cost of another ;) > +1 ... keeping relevant data in single place -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: From amk at amk.ca Thu Nov 12 21:30:57 2009 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 12 Nov 2009 15:30:57 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> <4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com> Message-ID: <20091112203057.GA14396@amk-desktop.matrixgroup.net> On Thu, Nov 12, 2009 at 11:30:27AM -0800, Guido van Rossum wrote: > etc.). Maybe there should be a standard "social app" that you can just > customize for a specific purpose. Sounds like an interesting project, > actually. For comments, haloscan and disqus are third-party comment-hosting services; http://redalt.com/blog/comment-services has a longer list. I don't know if any of them support rating of the posts or objects being commented on (as opposed to rating other comments, which is supported by some of them). Or if any of them can delegate moderation to the module authors, as opposed to the PyPI admins. PyPI's REST-style URLs also work nicely as keys or RDF identifiers, so it would be straightforward to use them for identifying ratings or comment threads. --amk From robertc at robertcollins.net Thu Nov 12 21:34:20 2009 From: robertc at robertcollins.net (Robert Collins) Date: Fri, 13 Nov 2009 07:34:20 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: <1258058060.25002.4.camel@lifeless-64> On Thu, 2009-11-12 at 08:25 -0600, Barry Warsaw wrote: > > > That's distressing. For better or worse PyPI is the central > repository of 3rd party packages. It should be easy, desirable, fun > and socially encouraged to get your packages there. Its already socially encouraged: heck, if package foo is not on PyPI, it doesn't exist. As an author who hosts code elsewhere, I really want to get feedback on my packages. I don't really see the point of having a comment system in PyPI ((*for me*), but I would really like to be able to have a link to the appropriate web page for folk that want to ask questions. To be clear, I mean a specific link, not just the 'link to a website'. -Rob -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: From tjreedy at udel.edu Thu Nov 12 21:42:37 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 12 Nov 2009 15:42:37 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: Guido van Rossum wrote: > On Thu, Nov 12, 2009 at 10:30 AM, Terry Reedy wrote: >> Barry Warsaw wrote: >>> On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote: >>> >>>> Frankly, I agree with him. As implemented, I *and others* think this >>>> is broken. I've taken the stance of not publishing things to PyPi >>>> until A> I find the time to contribute to make it better or B> It >>>> changes. >>> That's distressing. For better or worse PyPI is the central repository of >>> 3rd party packages. It should be easy, desirable, fun and socially >>> encouraged to get your packages there. >> I think his point is that a new book announcement servive is different from >> a book review and rating service. And that mixing the two is 'socially >> discouraging'. I do not know what the answer is > > I would say that publishers disagree -- they seem to really like > adding "social" stuff to their book announcement service. See e.g. > Amazon (which combines all functions: announcement/promotion, > ordering/download, review/comments/rate/popularity). I use user reviews on both Amazon and Netflix. I notice that both let people rate the reviews (helpful or not), and I sometimes look at those also. Both list items without the say-so of creators, though most will tolerate possible bad reviews in exchange for sales. I very seldom see an item with only one review, so there is usually a mix. There are also ratings averaged across a lot more people. Part of the pypi problem is a startup problem of initially low numbers. If the only people who bother to log in to rate are the disgruntled, then the ratings/reviews will be biased. I wonder how many of the people promoting the new feature have themselves logged in to systematically rate and possibly comment on every package they have looked at, and thereby kickstart the system with fair responses. Authors can often respond to magazine/journal reviews in Letters to the Editor. Publishers tend to exercise some editorial control over reviews so as to not make the publication look bad with grossly bad reviews. Even on Amazon, an author could, I presume, add a response to a factually incorrect review. Terry Jan Reedy From ben+python at benfinney.id.au Thu Nov 12 21:51:06 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 13 Nov 2009 07:51:06 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: <87zl6rwjph.fsf@benfinney.id.au> Barry Warsaw writes: > On Nov 12, 2009, at 8:06 AM, Jesse Noller wrote: > > > Frankly, I agree with him. As implemented, I *and others* think this > > is broken. I've taken the stance of not publishing things to PyPi > > until A> I find the time to contribute to make it better or B> It > > changes. I'm updating my existing packages, but I have zero interest in publishing new packages to PyPI while it's trying to be a popularity rating and comment system. > That's distressing. For better or worse PyPI is the central repository > of 3rd party packages. That's exactly what we want it to be. The problem is the current implementation is diluting that focus and making it much less attractive to have one's packages there. -- \ ?Rightful liberty is unobstructed action, according to our | `\ will, within limits drawn around us by the equal rights of | _o__) others.? ?Thomas Jefferson | Ben Finney -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From ncoghlan at gmail.com Thu Nov 12 21:59:08 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 13 Nov 2009 06:59:08 +1000 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: <4AFC771C.50608@gmail.com> Barry Warsaw wrote: > I personally think a ratings system can be useful, but you should be > able to opt-out of it if you want. Or just write such awesome software > that the bogus bad reviews will be buried by an avalanche of kudos. One of the problems I have with online rating/comment systems for software are I see them as inherently biased. Happy users are more likely to be busy coding, unhappy users are more likely to be frustrated and looking for somewhere to vent. If the package author can't even *respond* to mistaken or misguided comments then the review system is fundamentally broken. Better not to have one at all - let people vent on their own blogs and other sites, and let potential users let Google do its thing (and if Google turns up nothing, then that in itself is a data point). Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From chris at simplistix.co.uk Thu Nov 12 22:01:16 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Thu, 12 Nov 2009 21:01:16 +0000 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4AFC65A9.8060304@v.loewis.de> References: <200911130038.05516.steve@pearwood.info> <4AFC65A9.8060304@v.loewis.de> Message-ID: <4AFC779C.6040608@simplistix.co.uk> Martin v. L?wis wrote: > I'm going to take a poll RSN, and see what the majority of users > think (rather than their vocal fraction). Then we can see what to do > about it. Yes please! I've been silently waiting for this and have (surprisingly for me!) managed to resist joining in the rant. I'm of the crowd who just wants to make comments (I don't care about ratings one way or another, and I can't, sadly, see anyone giving me access as to who-voted-less-than-5 so I can follow up with them and see what needs to be improved) optional. I'm quite okay with having a banner saying "This package has opted not to receive comments". I believe the software I release on PyPI is strong enough on its own merits that people will use it ;-) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Thu Nov 12 22:03:58 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Thu, 12 Nov 2009 21:03:58 +0000 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> Message-ID: <4AFC783E.5020006@simplistix.co.uk> Arc Riley wrote: > It would be more useful to provide a PyPI mechanism to publish a link to > file bugs on the project's own website and leave project ratings the > work of other sites such as Ohloh. Yes, I really wish I could include all the links in the sections on, say, http://www.simplistix.co.uk/software/python/errorhandler in my package metadata, and have that show on PyPI. Sadly, there are many PEP battles and much gnashing of teeth before that will ever happen :-( Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From ben+python at benfinney.id.au Thu Nov 12 22:11:03 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 13 Nov 2009 08:11:03 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> Message-ID: <87vdhfwis8.fsf@benfinney.id.au> Masklinn writes: > On 12 Nov 2009, at 17:31 , Jesse Noller wrote: > > But before we even did those; why not have mandatory links for > > entries to bug trackers, mailing lists, source repositories, etc? > > I'm saying saying this doesn't seem well thought out, and the > > current implementation is broken by design. Of course, as I said > > earlier; since I don't have time to patch it; I'll simply just not > > participate. > I think having links to those is a very good idea and more important > than a comment/notation system. They shouldn't be mandatory though, > not every library has a mailing list, or even a (public anyway) bug > tracker. I think Jesse's point (or, if he's not willing to claim it, my point) is that, compared to the mandatory comment system, it makes much *more* sense to have a mandatory field for ?URL to the BTS for this project?. At least it can be expected that in many cases project maintainers will *want* to use a conventional BTS, VCS, discussion forum, etc. So that route makes more sense than a mandatory comment system outside the project maintainer's control, while providing the user-participation that is the purported motivation. -- \ ?Spam will be a thing of the past in two years' time.? ?Bill | `\ Gates, 2004-01-24 | _o__) | Ben Finney From solipsis at pitrou.net Thu Nov 12 22:26:34 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 12 Nov 2009 21:26:34 +0000 (UTC) Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> <4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com> <20091112203057.GA14396@amk-desktop.matrixgroup.net> Message-ID: A.M. Kuchling amk.ca> writes: > > For comments, haloscan and disqus are third-party comment-hosting > services; http://redalt.com/blog/comment-services has a longer list. They are horrible for page loading times; and besides, I don't know how you can trust such third-party to provide an important function of your Web site, /and/ manage its data. From ncoghlan at gmail.com Thu Nov 12 22:31:17 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 13 Nov 2009 07:31:17 +1000 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4AFC779C.6040608@simplistix.co.uk> References: <200911130038.05516.steve@pearwood.info> <4AFC65A9.8060304@v.loewis.de> <4AFC779C.6040608@simplistix.co.uk> Message-ID: <4AFC7EA5.5080700@gmail.com> Chris Withers wrote: > I'm quite okay with having a banner > saying "This package has opted not to receive comments". Particularly if the developer is able to add a prominent link to the project's own support site or mailing list. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From rdmurray at bitdance.com Thu Nov 12 22:53:00 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 12 Nov 2009 16:53:00 -0500 (EST) Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: On Thu, 12 Nov 2009 at 15:42, Terry Reedy wrote: > Part of the pypi problem is a startup problem of initially low numbers. If > the only people who bother to log in to rate are the disgruntled, then the > ratings/reviews will be biased. I wonder how many of the people promoting the > new feature have themselves logged in to systematically rate and possibly > comment on every package they have looked at, and thereby kickstart the > system with fair responses. For what it's worth, I never look at PyPI. I get my packages either through Gentoo's portage or, if it isn't there (yet), by finding the package's home site through Google. So I'm a happy user of a number of packages, whose comments will never show up on PyPI. --David (RDM) From foom at fuhm.net Thu Nov 12 22:53:47 2009 From: foom at fuhm.net (James Y Knight) Date: Thu, 12 Nov 2009 16:53:47 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <87vdhfwis8.fsf@benfinney.id.au> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> <87vdhfwis8.fsf@benfinney.id.au> Message-ID: <3132DFCC-BA06-4AF6-863F-534C715A13E2@fuhm.net> On Nov 12, 2009, at 4:11 PM, Ben Finney wrote: > I think Jesse's point (or, if he's not willing to claim it, my > point) is > that, compared to the mandatory comment system, it makes much *more* > sense to have a mandatory field for ?URL to the BTS for this project?. One might look at the "competition" for inspiration. Looking at CPAN. There's no "comments" feature, but there is a "CPAN RT" bug-tracker which appears to be a way for users to submit comments/problems about packages in a way common to all packages in CPAN, but distinct from upstream's bug trackers/lists/etc. I'd assume that gets emailed to the listed maintainer of the package as well as being accessible to other users, although I don't really have any idea. e.g. http://search.cpan.org/~capttofu/DBD-mysql/lib/DBD/mysql.pm There might be something to be said for providing users a way to provide feedback that doesn't require making a accounts in a bazillion separate bugtrackers. *shrug* James From martin at v.loewis.de Thu Nov 12 22:58:02 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 12 Nov 2009 22:58:02 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4AFC7EA5.5080700@gmail.com> References: <200911130038.05516.steve@pearwood.info> <4AFC65A9.8060304@v.loewis.de> <4AFC779C.6040608@simplistix.co.uk> <4AFC7EA5.5080700@gmail.com> Message-ID: <4AFC84EA.9070003@v.loewis.de> Nick Coghlan wrote: > Chris Withers wrote: >> I'm quite okay with having a banner >> saying "This package has opted not to receive comments". > > Particularly if the developer is able to add a prominent link to the > project's own support site or mailing list. It's really puzzling that people always assume that people would use comments primarily to get help, or to report problems. It appears that nobody expects users to merely comment, voicing their opinion - yet the comment that triggered this particular thread was precisely that. Regards, Martin From martin at v.loewis.de Thu Nov 12 23:08:35 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 12 Nov 2009 23:08:35 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: <4AFC8763.9080300@v.loewis.de> > Part of the pypi problem is a startup problem of initially low numbers. > If the only people who bother to log in to rate are the disgruntled, > then the ratings/reviews will be biased. Fortunately, that isn't actually the case. The majority of comments is positive (from scanning the full list of comments, without actually counting). > Even on Amazon, an author could, I presume, add a response to a > factually incorrect review. And so they can on PyPI. Regards, Martin From martin at v.loewis.de Thu Nov 12 23:12:18 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 12 Nov 2009 23:12:18 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <87vdhfwis8.fsf@benfinney.id.au> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> <87vdhfwis8.fsf@benfinney.id.au> Message-ID: <4AFC8842.5050904@v.loewis.de> > At least it can be expected that in many cases project maintainers will > *want* to use a conventional BTS, VCS, discussion forum, etc. So that > route makes more sense than a mandatory comment system outside the > project maintainer's control, while providing the user-participation > that is the purported motivation. I think you are missing the point of the commenting system: these comments are *not* directed towards the package author. Instead, they are directed towards fellow users of the package. For this kind of message, a bugtracker is completely inappropriate, as is a mailing list, or any other kind of support forum. I agree that users asking for support should not use the PyPI commenting system. Regards, Martin From g.brandl at gmx.net Thu Nov 12 23:12:02 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 12 Nov 2009 23:12:02 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: R. David Murray schrieb: > On Thu, 12 Nov 2009 at 15:42, Terry Reedy wrote: >> Part of the pypi problem is a startup problem of initially low numbers. If >> the only people who bother to log in to rate are the disgruntled, then the >> ratings/reviews will be biased. I wonder how many of the people promoting the >> new feature have themselves logged in to systematically rate and possibly >> comment on every package they have looked at, and thereby kickstart the >> system with fair responses. > > For what it's worth, I never look at PyPI. I get my packages either > through Gentoo's portage or, if it isn't there (yet), by finding the > package's home site through Google. So I'm a happy user of a number of > packages, whose comments will never show up on PyPI. The other large group of easy_install users will also never get to the individual package pages on PyPI. Except when they have a problem, and then they are likely to only complain through the comments. Georg From masklinn at masklinn.net Thu Nov 12 23:23:09 2009 From: masklinn at masklinn.net (Masklinn) Date: Thu, 12 Nov 2009 23:23:09 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <3132DFCC-BA06-4AF6-863F-534C715A13E2@fuhm.net> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> <87vdhfwis8.fsf@benfinney.id.au> <3132DFCC-BA06-4AF6-863F-534C715A13E2@fuhm.net> Message-ID: <524B006B-E69A-4697-A75A-F22C98BFE441@masklinn.net> On 12 Nov 2009, at 22:53 , James Y Knight wrote: > On Nov 12, 2009, at 4:11 PM, Ben Finney wrote: >> I think Jesse's point (or, if he's not willing to claim it, my point) is >> that, compared to the mandatory comment system, it makes much *more* >> sense to have a mandatory field for ?URL to the BTS for this project?. > > One might look at the "competition" for inspiration. Looking at CPAN. There's no "comments" feature There is, on search.cpan.org. See http://search.cpan.org/~petdance/ack/ for instance, the link leads to http://cpanratings.perl.org/ (a pretty interesting example of the "distributed" nature of cpan in fact). From ben+python at benfinney.id.au Thu Nov 12 23:39:01 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 13 Nov 2009 09:39:01 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> <4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com> Message-ID: <87my2rwepm.fsf@benfinney.id.au> Guido van Rossum writes: > Maybe that's an example of a user who doesn't know how to use those > support channels? I know I wouldn't bother with IRC even if it was the > only way to get in touch with users, I hate it with a vengeance. > (Though arguably I'm a special case -- whenever I show up everyone > goes "ooooh, Guido is here." :-) And I might not want to sign up for a > mailing list for a casual question. And what exactly is a "forum"? By my understanding of English, a ?forum? is any assembly for open discussion of a topic. Mailing lists, IRC channels, Usenet groups, and gatherings in a pub all merit the term ?forum?; and no crappy web applications get to usurp its meaning exclusively. (Nothing to do with PyPI, just an appeal to keeping terms useful.) -- \ ?I have had a perfectly wonderful evening, but this wasn't it.? | `\ ?Groucho Marx | _o__) | Ben Finney From foom at fuhm.net Thu Nov 12 23:44:16 2009 From: foom at fuhm.net (James Y Knight) Date: Thu, 12 Nov 2009 17:44:16 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <524B006B-E69A-4697-A75A-F22C98BFE441@masklinn.net> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> <87vdhfwis8.fsf@benfinney.id.au> <3132DFCC-BA06-4AF6-863F-534C715A13E2@fuhm.net> <524B006B-E69A-4697-A75A-F22C98BFE441@masklinn.net> Message-ID: <111E097E-53C3-4F91-BDD6-2AAFC2E7AC4E@fuhm.net> On Nov 12, 2009, at 5:23 PM, Masklinn wrote: > On 12 Nov 2009, at 22:53 , James Y Knight wrote: >> On Nov 12, 2009, at 4:11 PM, Ben Finney wrote: >>> I think Jesse's point (or, if he's not willing to claim it, my >>> point) is >>> that, compared to the mandatory comment system, it makes much *more* >>> sense to have a mandatory field for ?URL to the BTS for this >>> project?. >> >> One might look at the "competition" for inspiration. Looking at >> CPAN. There's no "comments" feature > There is, on search.cpan.org. See http://search.cpan.org/~petdance/ack/ > for instance, the link leads to http://cpanratings.perl.org/ (a > pretty interesting example of the "distributed" nature of cpan in > fact). Ah, I see. I totally managed to miss that...I guess that's an interesting example of a bad web ui. :) James From martin at v.loewis.de Thu Nov 12 23:47:00 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 12 Nov 2009 23:47:00 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: <4AFC9064.20700@v.loewis.de> > Except when they have a problem, and then they are likely to only complain > through the comments. As this theory has been repeated often here, I decided to go through all comments and classify them, as: - good: (overall) positive evaluation (possibly including minor criticism/wishes) - bad: negative evaluation, complaint, bug report - neutral: statement of fact (typically in response to some help request) Here is what I got: - good: 20 - bad: 11 - neutral: 9 So far, the theory that only complainers will comment cannot be substantiated by facts. Of course, it could be that all the negative comments come from easy_install users, and all the positive comments from people who browse through PyPI... Notice however that such easy_install users often would have to create a PyPI account first. Regards, Martin From ben+python at benfinney.id.au Thu Nov 12 23:49:50 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 13 Nov 2009 09:49:50 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <200911130038.05516.steve@pearwood.info> <4AFC65A9.8060304@v.loewis.de> <4AFC779C.6040608@simplistix.co.uk> <4AFC7EA5.5080700@gmail.com> <4AFC84EA.9070003@v.loewis.de> Message-ID: <87iqdfwe7l.fsf@benfinney.id.au> "Martin v. L?wis" writes: > Nick Coghlan wrote: > > Particularly if the developer is able to add a prominent link to the > > project's own support site or mailing list. > > It's really puzzling that people always assume that people would use > comments primarily to get help, or to report problems. It appears that > nobody expects users to merely comment, voicing their opinion - yet > the comment that triggered this particular thread was precisely that. I'm reading ?the project's own mailing list? as a fine place to do exactly that: discuss the project. -- \ ?If nothing changes, everything will remain the same.? ?Barne's | `\ Law | _o__) | Ben Finney From masklinn at masklinn.net Thu Nov 12 23:50:25 2009 From: masklinn at masklinn.net (Masklinn) Date: Thu, 12 Nov 2009 23:50:25 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <111E097E-53C3-4F91-BDD6-2AAFC2E7AC4E@fuhm.net> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> <87vdhfwis8.fsf@benfinney.id.au> <3132DFCC-BA06-4AF6-863F-534C715A13E2@fuhm.net> <524B006B-E69A-4697-A75A-F22C98BFE441@masklinn.net> <111E097E-53C3-4F91-BDD6-2AAFC2E7AC4E@fuhm.net> Message-ID: <37F67F35-31EF-4DC9-A2D6-85FDA7A279E9@masklinn.net> On 12 Nov 2009, at 23:44 , James Y Knight wrote: > On Nov 12, 2009, at 5:23 PM, Masklinn wrote: >> On 12 Nov 2009, at 22:53 , James Y Knight wrote: >>> On Nov 12, 2009, at 4:11 PM, Ben Finney wrote: >>>> I think Jesse's point (or, if he's not willing to claim it, my point) is >>>> that, compared to the mandatory comment system, it makes much *more* >>>> sense to have a mandatory field for ?URL to the BTS for this project?. >>> >>> One might look at the "competition" for inspiration. Looking at CPAN. There's no "comments" feature >> There is, on search.cpan.org. See http://search.cpan.org/~petdance/ack/ for instance, the link leads to http://cpanratings.perl.org/ (a pretty interesting example of the "distributed" nature of cpan in fact). > > Ah, I see. I totally managed to miss that...I guess that's an interesting example of a bad web ui. :) I'm not sure it's so bad, it's just that it's at the root of the "cpan package" rather than in the POD (just click on "BDB-mysql" in the breadcrumb trail, landing at http://search.cpan.org/~capttofu/DBD-mysql/). Interestingly, the link to cpanratings from BDB-mysql is broken and yields a 404, even though its CPAN page lists 5 reviews and a score of ~3.5. From tseaver at palladion.com Thu Nov 12 23:52:08 2009 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 12 Nov 2009 17:52:08 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> <4222a8490911121125w67ffacfct9989db242d7c00f5@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Guido van Rossum wrote: > On Thu, Nov 12, 2009 at 11:25 AM, Jesse Noller wrote: >> I'd not trust a package without a bug tracker, mailing list or link to >> the source a lot sooner than something without comments and ratings. > > Yeah, but you're not exactly an average user. Most users don't know > how to use a bug tracker. Also, there's a large variety of packages on > PyPI. Not every developer has the same attitude, but they all live > happily together on PyPI. (Or did you want someone to start a separate > CPAN "for the rest of them" ? :-) > > [...] >> What about astroturfing? What's to stop me from writing a script to >> create a pile of accounts and then bumping packages I like with >> glowing ratings and reviews? Who is going to be the moderator, and how >> to decide between spam, incorrect comment, etc? > > Those are all hard problems, though all of them have at least partial > solutions in the other worlds (Amazon, Wikipedia, Apple app store, > etc.). Maybe there should be a standard "social app" that you can just > customize for a specific purpose. Sounds like an interesting project, > actually. The appstore analogy actually helps Jesse's case" "iFart in your general direction." (iFart is the top-rated app). Popularity and quality aren't related in any direct fashion. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkr8kZMACgkQ+gerLs4ltQ4ScQCeJtYU9KkAq2K1Dkk0jK9ffHvB IuwAoNBWpMPFR1YsdhQN31oS1L5m91UL =qmmQ -----END PGP SIGNATURE----- From martin at v.loewis.de Thu Nov 12 23:57:50 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 12 Nov 2009 23:57:50 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <87iqdfwe7l.fsf@benfinney.id.au> References: <200911130038.05516.steve@pearwood.info> <4AFC65A9.8060304@v.loewis.de> <4AFC779C.6040608@simplistix.co.uk> <4AFC7EA5.5080700@gmail.com> <4AFC84EA.9070003@v.loewis.de> <87iqdfwe7l.fsf@benfinney.id.au> Message-ID: <4AFC92EE.3080409@v.loewis.de> Ben Finney wrote: > "Martin v. L?wis" writes: > >> Nick Coghlan wrote: >>> Particularly if the developer is able to add a prominent link to the >>> project's own support site or mailing list. >> It's really puzzling that people always assume that people would use >> comments primarily to get help, or to report problems. It appears that >> nobody expects users to merely comment, voicing their opinion - yet >> the comment that triggered this particular thread was precisely that. > > I'm reading ?the project's own mailing list? as a fine place to do > exactly that: discuss the project. No no no: discuss with whom? The commenter may not want to discuss anything about the project, let alone with the package author. If it's a positive comment, you may likely only get "me too", anyway; if it's a negative comment, perhaps the commenter has already given up on the package, and just wants to warn other users. It's those other users that then will have problems finding reviews on the mailing list. Regards, Martin From solipsis at pitrou.net Fri Nov 13 00:00:52 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 12 Nov 2009 23:00:52 +0000 (UTC) Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> <87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de> Message-ID: Martin v. L?wis v.loewis.de> writes: > > I think you are missing the point of the commenting system: these > comments are *not* directed towards the package author. Instead, they > are directed towards fellow users of the package. For this kind of > message, a bugtracker is completely inappropriate, as is a mailing list, > or any other kind of support forum. Then why not simply add a sentence or two before the comment form warning that the comment system is not meant to ask for help, support or debugging about the package? Regards Antoine. From masklinn at masklinn.net Fri Nov 13 00:10:05 2009 From: masklinn at masklinn.net (Masklinn) Date: Fri, 13 Nov 2009 00:10:05 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> <87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de> Message-ID: <805B874E-6FD5-44DB-A908-922C93866458@masklinn.net> On 13 Nov 2009, at 00:00 , Antoine Pitrou wrote: > > Then why not simply add a sentence or two before the comment form warning that > the comment system is not meant to ask for help, support or debugging about the > package? Because users don't read warnings. The warning will therefore be promptly ignored, and then the aforementioned user will start ripping on the package because he didn't get help following his comment. From jnoller at gmail.com Fri Nov 13 00:11:23 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 12 Nov 2009 18:11:23 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4AFC9064.20700@v.loewis.de> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> <4AFC9064.20700@v.loewis.de> Message-ID: <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> On Thu, Nov 12, 2009 at 5:47 PM, "Martin v. L?wis" wrote: >> Except when they have a problem, and then they are likely to only complain >> through the comments. > > As this theory has been repeated often here, I decided to go through all > comments and classify them, as: > - good: (overall) positive evaluation (possibly including minor > ?criticism/wishes) > - bad: negative evaluation, complaint, bug report > - neutral: statement of fact (typically in response to some help > ?request) > > Here is what I got: > - good: ? 20 > - bad: ? ?11 > - neutral: 9 > > So far, the theory that only complainers will comment cannot be > substantiated by facts. Of course, it could be that all the negative > comments come from easy_install users, and all the positive comments > from people who browse through PyPI... Notice however that such > easy_install users often would have to create a PyPI account first. > > Regards, > Martin And how many of the "good" comments are astroturfers? What's so bad about package maintainers from having an opt-out? I'd rather run a pypi competitor at this point. From solipsis at pitrou.net Fri Nov 13 00:15:08 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 12 Nov 2009 23:15:08 +0000 (UTC) Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> <87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de> <805B874E-6FD5-44DB-A908-922C93866458@masklinn.net> Message-ID: Masklinn masklinn.net> writes: > > On 13 Nov 2009, at 00:00 , Antoine Pitrou wrote: > > > > Then why not simply add a sentence or two before the comment form warning that > > the comment system is not meant to ask for help, support or debugging about the > > package? > Because users don't read warnings. I don't like assuming users are idiots. > The warning will therefore be promptly ignored, and then the > aforementioned user will start ripping on the package because he didn't get > help following his comment. And then it's easy to point out that he was wrong if there was a warning in the first place. From martin at v.loewis.de Fri Nov 13 00:25:41 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 13 Nov 2009 00:25:41 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> Message-ID: <4AFC9975.7040204@v.loewis.de> > And how many of the "good" comments are astroturfers? If I understand that term correctly, it's about disguise: how would I be able to answer that question? > What's so bad about package maintainers from having an opt-out? PyPI is not just (and perhaps not even primarily) there for the package authors, but for the package users (and not surprisingly, it's primarily the package authors who ask for banning the user opinions). I'm just not willing to submit to one side; hence the poll. Regards, Martin From ben+python at benfinney.id.au Fri Nov 13 00:29:30 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 13 Nov 2009 10:29:30 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> <87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de> <805B874E-6FD5-44DB-A908-922C93866458@masklinn.net> Message-ID: <87eio3wcdh.fsf@benfinney.id.au> Antoine Pitrou writes: > Masklinn masklinn.net> writes: > > Because users don't read warnings. > > I don't like assuming users are idiots. You don't have to. You need only assume that users are busy, focussed on a task (?leave feedback?), and will therefore unconsciously filter out *anything* that is not the simplest path to complete that task. > > The warning will therefore be promptly ignored, and then the > > aforementioned user will start ripping on the package because he > > didn't get help following his comment. > > And then it's easy to point out that he was wrong if there was a > warning in the first place. I don't like having systems which make it easier to do the wrong thing than do the right thing, then blame those users for taking the obvious path to their goal. -- \ ?I'm a great lover, I'll bet.? ?Emo Philips | `\ | _o__) | Ben Finney From masklinn at masklinn.net Fri Nov 13 00:30:59 2009 From: masklinn at masklinn.net (Masklinn) Date: Fri, 13 Nov 2009 00:30:59 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> <87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de> <805B874E-6FD5-44DB-A908-922C93866458@masklinn.net> Message-ID: On 13 Nov 2009, at 00:15 , Antoine Pitrou wrote: > Masklinn masklinn.net> writes: >> >> On 13 Nov 2009, at 00:00 , Antoine Pitrou wrote: >>> >>> Then why not simply add a sentence or two before the comment form warning > that >>> the comment system is not meant to ask for help, support or debugging about > the >>> package? >> Because users don't read warnings. > I don't like assuming users are idiots. You don't have to. It's about expediency and care (or lack thereof) rather than idiocy. User(*) wants a solution, user finds place where he could ask, user asks. Users (which includes e.g. language users) tend to be lazy, rather than stupid. >> The warning will therefore be promptly ignored, and then the >> aforementioned user will start ripping on the package because he didn't get >> help following his comment. > And then it's easy to point out that he was wrong if there was a warning in the > first place. And then user will probably ask why you're not answering the question since you're here anyway, or might go as far as telling you that if you're not going to help you might as well not answer. *: not every user, but I believe a significant minority at least. From nyamatongwe at gmail.com Fri Nov 13 00:32:51 2009 From: nyamatongwe at gmail.com (Neil Hodgson) Date: Fri, 13 Nov 2009 10:32:51 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> <87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de> <805B874E-6FD5-44DB-A908-922C93866458@masklinn.net> Message-ID: <50862ebd0911121532l3fb20c60p5672ab1747dea491@mail.gmail.com> When SourceForge started having comments and ratings, I was a little upset at having poor negative comments there (like "not work!"). But after it has been running for a while it appears useful. I suppose it helps that Scintilla has 88% thumbs up from 134 respondents. Because there is voting on comments, the more useful comments have bubbled onto the front page. As the system is used more, you'll see a wider range of comments on projects and you'll be able to tell more from them. It should be seen as a completely separate thing to the existing fora and trackers that each project has. While you want people to become involved in your project, many are just having a quick look and don't want to sign up for mailing lists or to interact with project members. They may just want to quickly comment about whether it was suitable or not. Neil From martin at v.loewis.de Fri Nov 13 00:33:46 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 13 Nov 2009 00:33:46 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> <87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de> Message-ID: <4AFC9B5A.2070406@v.loewis.de> Antoine Pitrou wrote: > Martin v. L?wis v.loewis.de> writes: >> I think you are missing the point of the commenting system: these >> comments are *not* directed towards the package author. Instead, they >> are directed towards fellow users of the package. For this kind of >> message, a bugtracker is completely inappropriate, as is a mailing list, >> or any other kind of support forum. > > Then why not simply add a sentence or two before the comment form warning that > the comment system is not meant to ask for help, support or debugging about the > package? Done! Martin From jnoller at gmail.com Fri Nov 13 00:34:17 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 12 Nov 2009 18:34:17 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4AFC9975.7040204@v.loewis.de> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> Message-ID: <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> On Thu, Nov 12, 2009 at 6:25 PM, "Martin v. L?wis" wrote: >> And how many of the "good" comments are astroturfers? > > If I understand that term correctly, it's about disguise: how would > I be able to answer that question? It's unprovable. But I could see a group of people easily coordinating large amounts of negative, or positive feedback targeting particular packages, that looks legit. I know any "end user" rating and feedback system can be gamed. Just look at the reviews of milk on amazon. >> What's so bad about package maintainers from having an opt-out? > > PyPI is not just (and perhaps not even primarily) there for the package > authors, but for the package users (and not surprisingly, it's > primarily the package authors who ask for banning the user opinions). > > I'm just not willing to submit to one side; hence the poll. That's because as an author/maintainer - we have methods of giving feedback and communication. Why not rate ( or auto-rate) packages on objective criteria? E.g.: tests and test coverage, docs, installs on python version X, Y, Z, works on windows, etc? Quality can be measured. Me being a total failure and not reading the docs, and failing to install something is subjective. I don't disagree with the goal of giving *users* a voice, but is PyPI the right place for that? How many moderators do we have to watch comments? Can other users down vote comments by people which are simply *wrong*? Why can't we just disable it until we can come up with a better system that finds a balance between the rights of maintainers, and those of the user? From solipsis at pitrou.net Fri Nov 13 00:35:28 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 12 Nov 2009 23:35:28 +0000 (UTC) Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> <87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de> <805B874E-6FD5-44DB-A908-922C93866458@masklinn.net> Message-ID: Masklinn masklinn.net> writes: > > And then user will probably ask why you're not answering the question since > you're here anyway, or might go > as far as telling you that if you're not going to help you might as well not > answer. As I said, you are regarding the user as an idiot or as a troll. From martin at v.loewis.de Fri Nov 13 00:37:16 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 13 Nov 2009 00:37:16 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> <87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de> <805B874E-6FD5-44DB-A908-922C93866458@masklinn.net> Message-ID: <4AFC9C2C.9090408@v.loewis.de> > Users (which includes e.g. language users) tend to be lazy, rather than stupid. Then they likely won't comment on PyPI. To do so, they have to setup an account (which most don't have). They can't post comments without an account. Regards, Martin From jacob at jacobian.org Fri Nov 13 00:37:30 2009 From: jacob at jacobian.org (Jacob Kaplan-Moss) Date: Thu, 12 Nov 2009 17:37:30 -0600 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4AFC9975.7040204@v.loewis.de> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> Message-ID: On Thu, Nov 12, 2009 at 5:25 PM, "Martin v. L?wis" wrote: > I'm just not willing to submit to one side; hence the poll. Nobody's asking you to "submit" to anything! We're asking for the control to decide ourselves. Look, there's already a large faction of people who just want to write off PyPI and launch our own package server instead. I'm nearly on board, but we've had enough fragmentation in the packaging world lately, and I don't want to make the project worse. But you can bet your ass that if PyPI isn't made a good, neutral, central resource I'm going to leave for one that is. Do you really want a flood of package maintainers de-listing their packages just so that things work the way you think they should? I should clarify that I'm speaking personally and not in any official "Django capacity." I don't have personal control over whether or not Django would de-list from PyPI. Django's run by a community process, and I'd listen to the voice of the community before doing anything unilaterally. It's a good idea, this community process. We might want to apply it to PyPI one of these days. Jacob From martin at v.loewis.de Fri Nov 13 00:41:00 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 13 Nov 2009 00:41:00 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> Message-ID: <4AFC9D0C.5030500@v.loewis.de> > Why can't we just disable it until we can come up with a better system > that finds a balance between the rights of maintainers, and those of > the user? Because I want to wait for the outcome of the poll first. Regards, Martin From martin at v.loewis.de Fri Nov 13 00:42:32 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 13 Nov 2009 00:42:32 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> Message-ID: <4AFC9D68.9010904@v.loewis.de> > But you can bet your ass that if PyPI isn't made a good, neutral, > central resource I'm going to leave for one that is. Do you really > want a flood of package maintainers de-listing their packages just so > that things work the way you think they should? > > I should clarify that I'm speaking personally and not in any official > "Django capacity." I don't have personal control over whether or not > Django would de-list from PyPI. Django's run by a community process, > and I'd listen to the voice of the community before doing anything > unilaterally. It's a good idea, this community process. We might want > to apply it to PyPI one of these days. And indeed, I do: feel free to participate in the poll. Regards, Martin From catch-all at masklinn.net Fri Nov 13 00:44:30 2009 From: catch-all at masklinn.net (Xavier Morel) Date: Fri, 13 Nov 2009 00:44:30 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> Message-ID: <7C2A33E9-7E5A-4588-BA87-DBB0F8F8EF0F@masklinn.net> On 13 Nov 2009, at 00:34 , Jesse Noller wrote: > That's because as an author/maintainer - we have methods of giving > feedback and communication. Why not rate ( or auto-rate) packages on > objective criteria? > > E.g.: tests and test coverage, docs, installs on python version X, Y, > Z, works on windows, etc? Because there are lots of subjective criteria which are still very useful to users? The feeling of the API, the completeness of the library or its flexibility, etc?? If pypi one day has a CPAN-style buildbot farm allowing it to test the package on any platform under the sun, that can be included, the tests can be included as well but given the number of testing solutions (and coverage discovery associated) that would be quite an undertaking. And as far as docs go, what would be the criterion? "Has documentation"? How do you define "has documentation"? Has auto-extracted documentation from the docstrings? Has a README? Has a complete sphinx package? I don't think there's much that you can rate "objectively" about documentation. From catch-all at masklinn.net Fri Nov 13 00:45:40 2009 From: catch-all at masklinn.net (Xavier Morel) Date: Fri, 13 Nov 2009 00:45:40 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> <87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de> <805B874E-6FD5-44DB-A908-922C93866458@masklinn.net> Message-ID: On 13 Nov 2009, at 00:35 , Antoine Pitrou wrote: > Masklinn masklinn.net> writes: >> >> And then user will probably ask why you're not answering the question since >> you're here anyway, or might go >> as far as telling you that if you're not going to help you might as well not >> answer. > As I said, you are regarding the user as an idiot or as a troll. I don't think so, but we might disagree on either definition, or both. From masklinn at masklinn.net Fri Nov 13 00:46:08 2009 From: masklinn at masklinn.net (Masklinn) Date: Fri, 13 Nov 2009 00:46:08 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4AFC9C2C.9090408@v.loewis.de> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> <87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de> <805B874E-6FD5-44DB-A908-922C93866458@masklinn.net> <4AFC9C2C.9090408@v.loewis.de> Message-ID: On 13 Nov 2009, at 00:37 , Martin v. L?wis wrote: >> Users (which includes e.g. language users) tend to be lazy, rather than stupid. > Then they likely won't comment on PyPI. To do so, they have to setup an > account (which most don't have). They can't post comments without an > account. Fair point From ben+python at benfinney.id.au Fri Nov 13 00:54:50 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 13 Nov 2009 10:54:50 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> Message-ID: <87aayrwb79.fsf@benfinney.id.au> "Martin v. L?wis" writes: > PyPI is not just (and perhaps not even primarily) there for the > package authors, but for the package users (and not surprisingly, it's > primarily the package authors who ask for banning the user opinions). No-one here is asking for ?banning the user opinions?. As already pointed out, users are not mute; there are plenty of places for them to have a voice. I understand the mood here to be, not that user feedback is not wanted, but rather that PyPI in particular should be a place for *objective* data about a package. > I'm just not willing to submit to one side; hence the poll. I hope the poll question will not be as biased as the above mischaracterisation. If you ask ?do you want feedback from users?? or something similar, that's missing the point entirely, and a ?yes? answer doesn't speak to this discussion at all. -- \ ?Always code as if the guy who ends up maintaining your code | `\ will be a violent psychopath who knows where you live.? ?John | _o__) F. Woods | Ben Finney From ben+python at benfinney.id.au Fri Nov 13 01:00:00 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 13 Nov 2009 11:00:00 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> <87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de> <805B874E-6FD5-44DB-A908-922C93866458@masklinn.net> Message-ID: <87639fwayn.fsf@benfinney.id.au> Masklinn writes: > Users (which includes e.g. language users) tend to be lazy, rather > than stupid. I've found it useful to realise that, from the perspective of a program/website/feedback form, etc., the user has a tiny brain: but that's only because the user's big brain is *not* solely dedicated to the program/website/feedback form, etc. When designing a UI, one must realise that, though users are generally possessed of big brains, only a *tiny* portion of that brain can be assumed to be available for the UI; the rest is focussed on stuff the user actually cares about at the time. Assuming the user is stupid or lazy is bad, since it's false most of the time. Assuming that they're only willing to put forth as much attention or effort as absolutely necessary to complete the task, is *good*, since that turns out to be true most of the time. -- \ ?It's easy to play any musical instrument: all you have to do | `\ is touch the right key at the right time and the instrument | _o__) will play itself.? ?Johann Sebastian Bach | Ben Finney From david.lyon at preisshare.net Fri Nov 13 01:01:53 2009 From: david.lyon at preisshare.net (David Lyon) Date: Thu, 12 Nov 2009 19:01:53 -0500 Subject: [Python-Dev] =?utf-8?q?PyPI_comments_and_ratings=2C_*really*=3F?= In-Reply-To: <7C2A33E9-7E5A-4588-BA87-DBB0F8F8EF0F@masklinn.net> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <7C2A33E9-7E5A-4588-BA87-DBB0F8F8EF0F@masklinn.net> Message-ID: <1b4a243d79db54095040c7e068414474@preisshare.net> On Fri, 13 Nov 2009 00:44:30 +0100, Xavier Morel wrote: > If pypi one day has a CPAN-style buildbot farm allowing it to test the > package on any platform under the sun, that can be included, the tests can > be included as well but given the number of testing solutions (and coverage > discovery associated) that would be quite an undertaking. I'm working on such a thing in my spare time. Yep, it's a big time commitment. http://bitbucket.org/djlyon/pypi-package-testbot/ > And as far as docs go, what would be the criterion? "Has documentation"? Yes. > How do you define "has documentation"? Has auto-extracted documentation > from the docstrings? Yes. > Has a README? Yes Has a complete sphinx package? I don't think there's much that you can rate "objectively" about documentation. You can't do it objectively, but you can use a computer to count the number of lines and come up with a score. Daivd From martin at v.loewis.de Fri Nov 13 01:14:54 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 13 Nov 2009 01:14:54 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <1b4a243d79db54095040c7e068414474@preisshare.net> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <7C2A33E9-7E5A-4588-BA87-DBB0F8F8EF0F@masklinn.net> <1b4a243d79db54095040c7e068414474@preisshare.net> Message-ID: <4AFCA4FE.3090106@v.loewis.de> David Lyon wrote: > On Fri, 13 Nov 2009 00:44:30 +0100, Xavier Morel > wrote: >> If pypi one day has a CPAN-style buildbot farm allowing it to test the >> package on any platform under the sun, that can be included, the tests > can >> be included as well but given the number of testing solutions (and > coverage >> discovery associated) that would be quite an undertaking. > > I'm working on such a thing in my spare time. Yep, it's a big time > commitment. > > http://bitbucket.org/djlyon/pypi-package-testbot/ See also http://pycheesecake.org/ Regards, Martin From guido at python.org Fri Nov 13 01:17:50 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Nov 2009 16:17:50 -0800 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <87639fwayn.fsf@benfinney.id.au> References: <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> <7AF4743F-4602-4E4C-8B8F-2FBC34D3945F@masklinn.net> <87vdhfwis8.fsf@benfinney.id.au> <4AFC8842.5050904@v.loewis.de> <805B874E-6FD5-44DB-A908-922C93866458@masklinn.net> <87639fwayn.fsf@benfinney.id.au> Message-ID: On Thu, Nov 12, 2009 at 4:00 PM, Ben Finney wrote: > I've found it useful to realise that, from the perspective of a > program/website/feedback form, etc., the user has a tiny brain: [...] Actually it's the other way around. It's the program that has the tiny brain. :-) -- --Guido van Rossum (python.org/~guido) From david.lyon at preisshare.net Fri Nov 13 01:13:14 2009 From: david.lyon at preisshare.net (David Lyon) Date: Thu, 12 Nov 2009 19:13:14 -0500 Subject: [Python-Dev] =?utf-8?q?PyPI_comments_and_ratings=2C_*really*=3F?= In-Reply-To: <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> Message-ID: <3c0a8eeb1d731b5c7f05c841442cadc1@preisshare.net> Hi All, What do people think about this idea? I've actually started writing something to try to to do this and create sn automated scoring system for the packages on pypi. It was started last week based on Guido's comments on the distutils mailing list. > Why not rate ( or auto-rate) packages on > objective criteria? > > E.g.: tests and test coverage, docs, installs on python version X, Y, > Z, works on windows, etc? > > Quality can be measured. Me being a total failure and not reading the > docs, and failing to install something is subjective. I don't disagree > with the goal of giving *users* a voice, but is PyPI the right place > for that? How many moderators do we have to watch comments? Can other > users down vote comments by people which are simply *wrong*? > > Why can't we just disable it until we can come up with a better system > that finds a balance between the rights of maintainers, and those of > the user? From david.lyon at preisshare.net Fri Nov 13 01:15:57 2009 From: david.lyon at preisshare.net (David Lyon) Date: Thu, 12 Nov 2009 19:15:57 -0500 Subject: [Python-Dev] =?utf-8?q?PyPI_comments_and_ratings=2C_*really*=3F?= In-Reply-To: <4AFCA4FE.3090106@v.loewis.de> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <7C2A33E9-7E5A-4588-BA87-DBB0F8F8EF0F@masklinn.net> <1b4a243d79db54095040c7e068414474@preisshare.net> <4AFCA4FE.3090106@v.loewis.de> Message-ID: <5b8b56c7fd501914bec8f92f18cabebe@preisshare.net> On Fri, 13 Nov 2009 01:14:54 +0100, "Martin v. L?wis" wrote: > http://pycheesecake.org/ Ok, so what is the current status on it? David From martin at v.loewis.de Fri Nov 13 01:27:47 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 13 Nov 2009 01:27:47 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <5b8b56c7fd501914bec8f92f18cabebe@preisshare.net> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <7C2A33E9-7E5A-4588-BA87-DBB0F8F8EF0F@masklinn.net> <1b4a243d79db54095040c7e068414474@preisshare.net> <4AFCA4FE.3090106@v.loewis.de> <5b8b56c7fd501914bec8f92f18cabebe@preisshare.net> Message-ID: <4AFCA803.6040408@v.loewis.de> David Lyon wrote: > On Fri, 13 Nov 2009 01:14:54 +0100, "Martin v. L?wis" > wrote: > >> http://pycheesecake.org/ > > Ok, so what is the current status on it? Not sure; you would have to ask Grig. Apparently, there is a service running somewhere that computes cheesecake data for PyPI packages; it also sends them to PyPI. People have expressed to concerns that any kind of ranking based on kwalitee sounds fairly useless. Regards, Martin From ben+python at benfinney.id.au Fri Nov 13 01:44:42 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 13 Nov 2009 11:44:42 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> Message-ID: <87y6mbuubp.fsf@benfinney.id.au> "Martin v. L?wis" writes: > > Why can't we just disable it until we can come up with a better > > system that finds a balance between the rights of maintainers, and > > those of the user? > > Because I want to wait for the outcome of the poll first. There's a problem with the poll's placement: on the front page of the PyPI website. I only know about the poll because you said there was one, and I went hunting for it. The front page of PyPI is not one I ever visit, as a package maintainer; I'll visit the pages for the packages I maintain, or make a specific search of packages I'm looking for. So, the poll's audience is limited to those who visit the front page (which is hardly ever necessary for package maintainers), and those who already know it exists (e.g. through this discussion thread). You'll be missing the opinions of those maintainers who, like the OP of this thread, only discovered the behaviour much later. -- \ ?I do not believe in forgiveness as it is preached by the | `\ church. We do not need the forgiveness of God, but of each | _o__) other and of ourselves.? ?Robert G. Ingersoll | Ben Finney From sparks.m at gmail.com Fri Nov 13 01:52:44 2009 From: sparks.m at gmail.com (Michael Sparks) Date: Fri, 13 Nov 2009 00:52:44 +0000 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: <20e5c9660911121652s6c9c25caqf8856eb6c1685986@mail.gmail.com> On Thu, Nov 12, 2009 at 2:06 PM, Jesse Noller wrote: ... > Frankly, I agree with him. As implemented, I *and others* think this > is broken. I've taken the stance of not publishing things to PyPi > until A> I find the time to contribute to make it better or B> It > changes. Ditto, but maybe for different reasons. Personally, I'm interested in feedback - good and bad. That's the reason I choose odd names for projects, since it means I can create a google alert to find out random comments in bizarre locations (hence why when you wrote a blog entry, I responded). However, the reason I released anything onto PyPI (and relunctantly at that) was due to a random complaint that a user couldn't go "easy_install " and have it pick up code from PyPI. Going along with comments made elsewhere (by Guido I think) saying "but user's like reviews and rating when someone publishes a book", probably using Amazon, B&N & similar as examples, I agree they do. The closest equivalent here though IMO is somewhere like lulu.com - where people self-publish. Like PyPI that has a ratings system and comments, so you could say if it "works" there it should work for PyPI. The problem though is that software is much more mutable that a book. Taking the example listed - a comment added here: http://pypi.python.org/pypi/spypam/1.0 There's a note: "Inadequate docstrings give no clue about function arguments. Dumps core when I call it after reading the source to figure out the API. Cannot recommend." That's useful from a user perspective. Or is it? It's useful from a user perspective, until that issue is fixed. Then what? Is it still useful? Can the commenter remove it? Can they get notified it's changed? Can the maintainer say "this is fixed/changed?" I never look at the PyPI pages for stuff I create. Which means if someone is using it for support, they're wasting their time. (Why would I look at it? I know what the project is for and where to get it! :) (and also PyPI isn't the prime download for it either - so the download stats are irrelevant to me) I doubt I'm alone, so how many people's time are wasted asking questions there ? *shrug* I suppose, personally, I'm dubious about the idea of having unchanging comments and ratings associated with projects which are changing and improving - that feels like a mismatch. (Unlike a book, which generally is unchanging or has a separate edition and separate set of ratings and reviews) Incidentally, and perhaps probably more relevant to the discussion than my random opinion - some time back I created the twitter id http://twitter.com/pypi - using twitterfeed - since I wanted an easier way of following additions to pypi. There's currently 774 people following that. If there's interest, and if there's a survey to be done, I could forward a link to a survey through that twitterfeed - which I suspect is a mix of users of PyPI and uploaders to PyPI. (On a secondary note, if there's someone else who thinks they should own it, please let me know - it was a random convenience that people seem to find useful :-) Michael. From solipsis at pitrou.net Fri Nov 13 01:52:38 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 13 Nov 2009 00:52:38 +0000 (UTC) Subject: [Python-Dev] PyPI front page References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> Message-ID: Ben Finney benfinney.id.au> writes: > > There's a problem with the poll's placement: on the front page of the > PyPI website. Speaking of which, why is it that http://pypi.python.org/pypi and http://pypi.python.org/pypi/ (note the ending slash) return different contents (the latter being very voluminous)? I always mistake one for the other when entering the URL directly. cheers Antoine. From david.lyon at preisshare.net Fri Nov 13 01:51:08 2009 From: david.lyon at preisshare.net (David Lyon) Date: Thu, 12 Nov 2009 19:51:08 -0500 Subject: [Python-Dev] =?utf-8?q?PyPI_comments_and_ratings=2C_*really*=3F?= In-Reply-To: <4AFCA803.6040408@v.loewis.de> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <7C2A33E9-7E5A-4588-BA87-DBB0F8F8EF0F@masklinn.net> <1b4a243d79db54095040c7e068414474@preisshare.net> <4AFCA4FE.3090106@v.loewis.de> <5b8b56c7fd501914bec8f92f18cabebe@preisshare.net> <4AFCA803.6040408@v.loewis.de> Message-ID: <3df661fb01b68d13cc5bd01eaab5c935@preisshare.net> On Fri, 13 Nov 2009 01:27:47 +0100, "Martin v. L?wis" wrote: > Not sure; you would have to ask Grig. Apparently, there is a service > running somewhere that computes cheesecake data for PyPI packages; > it also sends them to PyPI. People have expressed to concerns that any > kind of ranking based on kwalitee sounds fairly useless. Of course. "Package Quality Metrics" would be a much better term. Still, the ideas they had were good. CPAN runs such a bot on all packages daily. Obviously they do it on seperate machines. Introducing any change is going to have people complain. Checking package authors packages is much like a dentist check. It mightn't be totally pleasant while its happening. But then if it isn't done, a user can then reflect and ask why nothing is being done to look at overall package quality. Which is currently the case. Processing so many packages for so many platforms is a monstrous task. Nobody should get the idea it can be done by the weekend. It will take a few months... well at the rate I am going anyway.. David From sparks.m at gmail.com Fri Nov 13 02:02:20 2009 From: sparks.m at gmail.com (Michael Sparks) Date: Fri, 13 Nov 2009 01:02:20 +0000 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <87y6mbuubp.fsf@benfinney.id.au> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> Message-ID: <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> On Fri, Nov 13, 2009 at 12:44 AM, Ben Finney wrote: > "Martin v. L?wis" writes: > >> > Why can't we just disable it until we can come up with a better >> > system that finds a balance between the rights of maintainers, and >> > those of the user? >> >> Because I want to wait for the outcome of the poll first. > > There's a problem with the poll's placement: on the front page of the > PyPI website. > > I only know about the poll because you said there was one, and I went > hunting for it. The front page of PyPI is not one I ever visit, as a > package maintainer; I'll visit the pages for the packages I maintain, or > make a specific search of packages I'm looking for. > > So, the poll's audience is limited to those who visit the front page > (which is hardly ever necessary for package maintainers), and those who > already know it exists (e.g. through this discussion thread). You'll be > missing the opinions of those maintainers who, like the OP of this > thread, only discovered the behaviour much later. This poll is only visible if you're logged into PyPI. This strikes me as a mistake. I went looking for a poll and didn't see it. I only found the poll by accident by wondering randomly what might change if I hit the login using open id button. So you can only vote in the poll if you a) get told about it b) realise you need to create an account to login and use in order to vote. I realise there's good reasons for that, but I think it's a mistake. (There's no guidance that you need to log in to see the poll for example) Michael. From amk at amk.ca Fri Nov 13 02:18:12 2009 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 12 Nov 2009 20:18:12 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <87y6mbuubp.fsf@benfinney.id.au> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> Message-ID: <20091113011812.GA75517@andrew-kuchlings-macbook.local> On Fri, Nov 13, 2009 at 11:44:42AM +1100, Ben Finney wrote: > There's a problem with the poll's placement: on the front page of the > PyPI website. I've posted a tweet to the ThePSF account about the poll. If the poll runs for a week or two, that would provide time for word of the poll to propagate through Twitter, blogs, etc. --amk From ianb at colorstudy.com Fri Nov 13 02:33:27 2009 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 12 Nov 2009 20:33:27 -0500 Subject: [Python-Dev] PyPI front page In-Reply-To: References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> Message-ID: On Thu, Nov 12, 2009 at 7:52 PM, Antoine Pitrou wrote: > Ben Finney benfinney.id.au> writes: > > > > There's a problem with the poll's placement: on the front page of the > > PyPI website. > > Speaking of which, why is it that http://pypi.python.org/pypi and > http://pypi.python.org/pypi/ (note the ending slash) return different > contents > (the latter being very voluminous)? I always mistake one for the other when > entering the URL directly. > easy_install replied on the behavior of /pypi/ (it uses the long list to do case-insensitive searches). Someone changed it, easy_install broke, and a compromise was to keep /pypi/ the way it was (but not /pypi). Probably this could be removed, as the /simple/ index is already case-insensitive, so easy_install shouldn't have to hit /pypi/ at all. -- Ian Bicking | http://blog.ianbicking.org | http://topplabs.org/civichacker -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Fri Nov 13 02:39:14 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 13 Nov 2009 12:39:14 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> Message-ID: <87tywzurst.fsf@benfinney.id.au> Michael Sparks writes: > On Fri, Nov 13, 2009 at 12:44 AM, Ben Finney wrote: > > So, the poll's audience is limited to those who visit the front page > > (which is hardly ever necessary for package maintainers), and those > > who already know it exists (e.g. through this discussion thread). > > You'll be missing the opinions of those maintainers who, like the OP > > of this thread, only discovered the behaviour much later. > > This poll is only visible if you're logged into PyPI. This strikes me > as a mistake. I went looking for a poll and didn't see it. The mistake, I think, is having a poll basically asking ?what should the PyPI maintainers do??, instead of weighing evidence and reasoned arguments. A poll may be good for gathering preferences and opinion, but it's a poor way to make a *decision*. -- \ ?I busted a mirror and got seven years bad luck, but my lawyer | `\ thinks he can get me five.? ?Steven Wright | _o__) | Ben Finney From brett at python.org Fri Nov 13 02:43:02 2009 From: brett at python.org (Brett Cannon) Date: Thu, 12 Nov 2009 17:43:02 -0800 Subject: [Python-Dev] PEP 382 status In-Reply-To: <4AFB18B0.3050800@v.loewis.de> References: <94bdd2610911110536n60a882acj8b2cd63ced3af25b@mail.gmail.com> <4AFB18B0.3050800@v.loewis.de> Message-ID: 2009/11/11 "Martin v. L?wis" : >> I was wondering what's the status of PEP 382. Is anyone (MvL?) is >> going to start to work on its implementation for Python 2.7/3.2 >> inclusion ? > > I'll be working on an implementation, but contributions are welcome. > Unfortunately, I'm really short on free software time recently (and > keep hoping that this will improve). > > One thing that I always wondered about is the status of importlib. > Will that be used in 3.2? > I doubt I will get to that as I suspect most people would rather I spent my time on getting the Hg conversion to happen or trying to help ramp up 3to2. So importlib will more than likely stay just a standard library package for 3.2 (unfortunately as that means making this work requires changes in two places, although prototyping in importlib should be the easiest thing to get working). -Brett > In addition, I still owe a few answers to comments from the previous > discussion. > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From robert.kern at gmail.com Fri Nov 13 03:29:42 2009 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 12 Nov 2009 20:29:42 -0600 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <20091113011812.GA75517@andrew-kuchlings-macbook.local> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20091113011812.GA75517@andrew-kuchlings-macbook.local> Message-ID: A.M. Kuchling wrote: > On Fri, Nov 13, 2009 at 11:44:42AM +1100, Ben Finney wrote: >> There's a problem with the poll's placement: on the front page of the >> PyPI website. > > I've posted a tweet to the ThePSF account about the poll. If the poll > runs for a week or two, that would provide time for word of the poll > to propagate through Twitter, blogs, etc. You should also make an announcement on python-announce. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From tseaver at palladion.com Fri Nov 13 05:08:04 2009 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 12 Nov 2009 23:08:04 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Michael Sparks wrote: > On Fri, Nov 13, 2009 at 12:44 AM, Ben Finney wrote: >> "Martin v. L?wis" writes: >> >>>> Why can't we just disable it until we can come up with a better >>>> system that finds a balance between the rights of maintainers, and >>>> those of the user? >>> Because I want to wait for the outcome of the poll first. >> There's a problem with the poll's placement: on the front page of the >> PyPI website. >> >> I only know about the poll because you said there was one, and I went >> hunting for it. The front page of PyPI is not one I ever visit, as a >> package maintainer; I'll visit the pages for the packages I maintain, or >> make a specific search of packages I'm looking for. >> >> So, the poll's audience is limited to those who visit the front page >> (which is hardly ever necessary for package maintainers), and those who >> already know it exists (e.g. through this discussion thread). You'll be >> missing the opinions of those maintainers who, like the OP of this >> thread, only discovered the behaviour much later. > > This poll is only visible if you're logged into PyPI. This strikes me > as a mistake. I went looking for a poll and didn't see it. > > I only found the poll by accident by wondering randomly what might > change if I hit the login using open id button. So you can only vote > in the poll if you a) get told about it b) realise you need to create > an account to login and use in order to vote. I realise there's good > reasons for that, but I think it's a mistake. (There's no guidance > that you need to log in to see the poll for example) I can see the information about the poll, and a link to view the results, without logging in. http://pypi.python.org/pypi (second paragraph there). That paragraph tells you that you need to log in to vote in the poll. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkr825UACgkQ+gerLs4ltQ74YwCgsMU6UvoF6DZQP0If5yDV8j6J GQIAnRVGlH0onr5nQwBFYUfGl/ni0SaR =Ba00 -----END PGP SIGNATURE----- From regebro at gmail.com Fri Nov 13 05:47:39 2009 From: regebro at gmail.com (Lennart Regebro) Date: Fri, 13 Nov 2009 05:47:39 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> Message-ID: <319e029f0911122047q4da642c7x53e217e8f6d4d682@mail.gmail.com> 2009/11/12 Guido van Rossum : > If you were to ask me, the people arguing against ratings and user > comments are fighting a losing battle. If they had an iPhone or > Android phone (or some other device with an "app store" kind of place > to find downloads) they'd know the value (for prospective downloaders) > of ratings and comments. Now, I think PyPI can use some (perhaps a lot > of) improvement in the details of how it works, e.g. there should be a > way to flag inappropriate messages (and users who post many > inappropriate messages) and the software author should be able to talk > back, but the general idea is here and won't go away by wishing it > away. Sure. But then we need moderation, spam filtering, flagging, etc. We need to implement a whole discussion forum basically. We also need to mark comments with the version they are relevant for and allow authors to mark that comments are not longer valid. In fact, we get a bug-tracker as well. Maybe we should just install a trac per package? :-) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From jacob at jacobian.org Fri Nov 13 05:55:08 2009 From: jacob at jacobian.org (Jacob Kaplan-Moss) Date: Thu, 12 Nov 2009 22:55:08 -0600 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4AFC9D0C.5030500@v.loewis.de> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> Message-ID: On Thu, Nov 12, 2009 at 5:41 PM, "Martin v. L?wis" wrote: > Because I want to wait for the outcome of the poll first. I'm curious: what criteria will you use to judge the outcome of the poll? That is, how will you translate the results of the poll into action? Right now, the results stand as Allow ratings and comments on all packages (status quo) 13 Allow package owners to disallow comments (ratings unmodified). 17 Allow comments, but only send them to package owners (ratings unmodified). 2 Disallow comments (ratings unmodified). 11 Disallow ratings and comments (status three months ago). 36 If the poll ended this moment, how would you judge? Would it just be mob rule (no comments)? Or some sort of spectrum -- there's 32 for comments in some capacity and 47 against, so does somehow translate to ratings but not comments? Or a weighted average? The average is 3.51 (1 being "allow comments" and 5 being "no comments")... what does *that* mean? On a deeper level, why are we voting at all? When else in the history of Python have we used popular vote to decide questions of this nature? Jacob [Martin, sorry for the repeat; I sent this privately first by mistake.] From henning.vonbargen at arcor.de Fri Nov 13 05:56:03 2009 From: henning.vonbargen at arcor.de (Henning von Bargen) Date: Fri, 13 Nov 2009 05:56:03 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: Message-ID: <4AFCE6E3.4070209@arcor.de> mode = "python-dev reader" Please excuse me if I'm wrong here, but I think python-dev just isn't the right place to discuss this topic, because it's about 3rd party packages and it's got nothing to do with the development *of the python language itself*, but generated a lot of traffic. mode = "package developer" I personally added a PyPI entry for my open source project just because it's there and it's a more or less standard, like CPAN in the Perl world, so I hoped others would perhaps find it there. On the other hand, if I am searching for software myself, I don't look at PyPI - I just use G**gle. I don't think PyPI needs any new features. For the "developer's social web" that support the feature, there are other good places, for example take a look at http://www.ohloh.net Henning From ben+python at benfinney.id.au Fri Nov 13 06:35:28 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 13 Nov 2009 16:35:28 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> Message-ID: <87ljibugv3.fsf@benfinney.id.au> Jacob Kaplan-Moss writes: > If the poll ended this moment, how would you judge? Would it just be > mob rule (no comments)? [?] Even though that's my preferred option, I *don't* want it chosen on the basis of a poll result, but on the basis of evidence and reasoned argument. > On a deeper level, why are we voting at all? When else in the history > of Python have we used popular vote to decide questions of this > nature? +1 -- \ ?I do not believe in immortality of the individual, and I | `\ consider ethics to be an exclusively human concern with no | _o__) superhuman authority behind it.? ?Albert Einstein, letter, 1953 | Ben Finney From steve at pearwood.info Fri Nov 13 06:38:18 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 13 Nov 2009 16:38:18 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: Message-ID: <200911131638.18786.steve@pearwood.info> On Fri, 13 Nov 2009 05:54:24 am Guido van Rossum wrote: > I agree that creating a good social app is not easy, and if we can't > improve the social app embedded in PyPI quickly enough, we should at > least give authors the option to disable comments. Of course, as a > user, I might not trust a module that has no reviews or ratings. As a user, I'd be more likely to trust a module with no reviews/ratings than one where the author disabled reviews/ratings. The first says "nobody hated it enough to complain", the second one says "the author is trying to hide something". -- Steven D'Aprano From ben+python at benfinney.id.au Fri Nov 13 06:54:28 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 13 Nov 2009 16:54:28 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <200911131638.18786.steve@pearwood.info> Message-ID: <87hbszufzf.fsf@benfinney.id.au> Steven D'Aprano writes: > On Fri, 13 Nov 2009 05:54:24 am Guido van Rossum wrote: > > > I agree that creating a good social app is not easy, and if we can't > > improve the social app embedded in PyPI quickly enough, we should at > > least give authors the option to disable comments. Of course, as a > > user, I might not trust a module that has no reviews or ratings. > > As a user, I'd be more likely to trust a module with no reviews/ratings > than one where the author disabled reviews/ratings. The first > says "nobody hated it enough to complain", the second one says "the > author is trying to hide something". Agreed, that's how I'd feel (and it's important to note that this would be an emotional, not necessarily entirely rational, reaction) as a user also. Package maintainers who also see that users would feel that way, and who agree with the purpose of PyPI as a common repository of all third-party packages, but who *don't* want to deal with PyPI's implementation of comments (whatever that may be at any time), have a clear option: to avoid hosting the package at PyPI at all. That's harmful, and I don't want it; but I don't see an alternative for such a maintainer. -- \ ?Dvorak users of the world flgkd!? ?Kirsten Chevalier, | `\ rec.humor.oracle.d | _o__) | Ben Finney From steve at pearwood.info Fri Nov 13 07:22:38 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 13 Nov 2009 17:22:38 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <7EAD464A-45AC-41EF-9E11-A9BAA3461DB9@lericson.se> References: <200911130038.05516.steve@pearwood.info> <7EAD464A-45AC-41EF-9E11-A9BAA3461DB9@lericson.se> Message-ID: <200911131722.38598.steve@pearwood.info> On Fri, 13 Nov 2009 04:27:48 am Ludvig Ericson wrote: > On 12 nov 2009, at 14:38, Steven D'Aprano wrote: > > On Thu, 12 Nov 2009 08:44:32 pm Ludvig Ericson wrote: > >> Why are there comments on PyPI? Moreso, why are there comments > >> which I cannot control as a package author on my very own > >> packages? That's just absurd. > > > > No, what's absurd is thinking that the act of publishing software > > somehow gives you the right to demand control over what others say > > about your software. > > ... on my own package's page. It's your package. It's the community's page about your package. I think of PyPI as a community-owned noticeboard. Its primary purpose is to allow the community to find good packages -- the benefit to the community is why it exists, not the benefit to the author of the package. In my opinion, the community is best served by a good comment/review system, one which avoids the worst trolling, and allows authors the right of reply, but does not allow authors to censor inconvenient but honest reviews. -- Steven D'Aprano From steve at pearwood.info Fri Nov 13 07:23:08 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 13 Nov 2009 17:23:08 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: Message-ID: <200911131723.08584.steve@pearwood.info> On Fri, 13 Nov 2009 07:42:37 am Terry Reedy wrote: > Part of the pypi problem is a startup problem of initially low > numbers. If the only people who bother to log in to rate are the > disgruntled, then the ratings/reviews will be biased. The package author who started this thread, Ludvig Ericson, had (as of last night) two comments, one positive, one negative. This admittedly tiny sample doesn't suggest to me that PyPI is suffering from the problem of biased ratings. -- Steven D'Aprano From ben+python at benfinney.id.au Fri Nov 13 08:00:08 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 13 Nov 2009 18:00:08 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <200911130038.05516.steve@pearwood.info> <7EAD464A-45AC-41EF-9E11-A9BAA3461DB9@lericson.se> <200911131722.38598.steve@pearwood.info> Message-ID: <87aayqvrif.fsf@benfinney.id.au> Steven D'Aprano writes: > In my opinion, the community is best served by a good comment/review > system, one which avoids the worst trolling, and allows authors the > right of reply, but does not allow authors to censor inconvenient but > honest reviews. I think you're right. I also think, though, that the community is best served by an objective repository of third-party Python packages, with information derived only directly from the package itself and objective data. That allows the least barrier to having a package maintainer want to register their package with such a service, which is in the interest of having it be as complete a registry of packages as can be. A community forum, on the other hand, has many characteristics that will be *disincentives* to a package manager for having their package appear there. It's never going to attract as many package maintainers as an impartial, objective registry; the many reasons already given here as to why some package maintainers *don't* want their packages in such a system are evidence of that. Those two purposes ? community forum, impartial registry ? are in conflict. I think PyPI has clearly already been serving the role of the registry, and that any community forum should be quite separate to encourage those who don't like it to still register their packages at PyPI. -- \ ?I'd take the awe of understanding over the awe of ignorance | `\ any day.? ?Douglas Adams | _o__) | Ben Finney From steve at pearwood.info Fri Nov 13 08:07:05 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 13 Nov 2009 18:07:05 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <87aayrwb79.fsf@benfinney.id.au> References: <4AFC9975.7040204@v.loewis.de> <87aayrwb79.fsf@benfinney.id.au> Message-ID: <200911131807.06132.steve@pearwood.info> On Fri, 13 Nov 2009 10:54:50 am Ben Finney wrote: > "Martin v. L?wis" writes: > > PyPI is not just (and perhaps not even primarily) there for the > > package authors, but for the package users (and not surprisingly, > > it's primarily the package authors who ask for banning the user > > opinions). > > No-one here is asking for ?banning the user opinions?. That's EXACTLY what people have asked for: the ability to ban user comments. Sure, some of them say they want to "opt-out", or "disable comments", but that's just a polite way of saying the same thing. Some of them -- 44% of the people who have responded to the poll as of a minute ago -- want to prohibit *all* comments, even for packages whose authors wants to receive comments. WTF? The original poster (Ludvig Ericson) started this thread insisting that there were only two acceptable options: prohibit all comments on PyPI, or make it opt-in. I quote: "As I see it, there are only two ways to fix these misguided steps of development: throw them out, or make them opt-in settings." > As already > pointed out, users are not mute; there are plenty of places for them > to have a voice. Ben, you've been talking recently about the dangers of fragmenting the Python community into multiple forums, but that's what you're arguing for on PyPI. I'm a user, I'm interested in a package, so I look for it on PyPI. I want to know what others think about it, before I commit to downloading it, installing it and seeing if it meets my needs. Instead of looking in the One Obvious place, the PyPI page, you want me to go off looking for blogs, mailing lists, or other forums, which might not even exist. Part of the value of a centralised place for comments on a package is that I can see whether other users agree with the comment or not. If I see a comment "This doesn't even work", and ten other comments saying "No, this is great, but you need to transmogrify the phlogiston first to get it to work", I've learned something useful. But if I stumble across a single blog that says "This doesn't work", I've actually been given negative knowledge: I've learned something that just isn't so. What *actual* problem are we trying to solve here? Is there a problem with spam on PyPI? No. Is there a problem with the comments being wildly biased, either for or against? Apparently not. Is there a problem with people using the comment system as a help forum? No. So what's the problem we're trying to solve? > I understand the mood here to be, not that user feedback is not > wanted, but rather that PyPI in particular should be a place for > *objective* data about a package. Who is responsible for gather this "objective" data? How do we decide what is objective and what isn't? People can't even agree on the validity of benchmarks! This is open source. The power of the bazaar, remember? I'm amazed at how many people are not just disinterested in, but actively hostile, to even *useful* comments from users. That's fine. If you, the package author, don't care about comments from users, don't read them. But they're not there for *your* benefit, they're there for the benefit of other users. -- Steven D'Aprano From martin at v.loewis.de Fri Nov 13 08:34:05 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 13 Nov 2009 08:34:05 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <87y6mbuubp.fsf@benfinney.id.au> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> Message-ID: <4AFD0BED.4040700@v.loewis.de> > There's a problem with the poll's placement: on the front page of the > PyPI website. You really should participate in the proper forum for the discussion of PyPI: catalog-sig. Then you would have noticed that I said I'll announce the poll later (i.e. today), which I'm doing right now. Feel free to announce it places where I didn't. Regards, Martin From kevin at bud.ca Fri Nov 13 08:35:09 2009 From: kevin at bud.ca (Kevin Teague) Date: Thu, 12 Nov 2009 23:35:09 -0800 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: <4222a8490911121157i622fbab8q5fe82d32fe9ea8f@mail.gmail.com> References: <4AFC6440.1040008@v.loewis.de> <4222a8490911121157i622fbab8q5fe82d32fe9ea8f@mail.gmail.com> Message-ID: <56195B13-39DF-44A3-9A79-5BDAFF244F40@bud.ca> On Nov 12, 2009, at 11:57 AM, Jesse Noller wrote: > On Thu, Nov 12, 2009 at 2:38 PM, "Martin v. L?wis" > wrote: >>> I am not an expert, I am just another python learner. These are >>> just my >>> views on the state of the standard libraries and to >>> make them state-of-the-art..! ;) >> >> If I understand correctly, you want the (current) standard library >> to be >> separated from the Python implementation, and available separately. >> >> Interestingly enough, people are very much split over whether that >> would >> be a good thing or not. Some like it the way Python does, some >> dislike >> it (and some quite strongly so). >> >> In any case, many Python users consider it a good thing that it comes >> "with batteries included", ie. with no need to add extra stuff for >> many >> tasks. >> >> Some of the Python maintainers have recently started objecting to >> this >> setup, asking that the standard library should be split into separate >> packages that are released and distributed independent of Python. >> Others >> of us feel strongly that such a change should not be made. >> >> So don't expect any immediate change to happen. >> >> Regards, >> Martin > > Martin is correct; this came up on distutils-sig a month or so ago; I > proposed offering multiple downloads "with batteries" and "without > batteries (with the batteries on the side)". We decided as a group to > hold off on that until further in the future. > > jesse It's also worth noting that there are three issues with respect to standard library packaging which are all orthologous: * Packaging for metadata: The standard library could be packaged so that there is consistent metadata about the distributions which compromise the standard library (version, author, maintainer, etc). This could be useful so that it would be easier to see at a glance which packages changed between any two Python releases. For example, I have a script which compares two working sets of packages and generates a web page with a colour-coded list of differences between those two working sets, based on the size of the version number bump, e.g. major.medium.minor (doesn't work with packages which use odd version numbering schemes). I've used this to compare working sets between different Grok releases or Plone releases to help see where the activity or major API changes might be happening inside any given Zope-based release. However, this script only works on 80% of the code, the other 20% is in the standard library, which is currently a black-box. Also, if PEP 376 is accepted, then the standard library will be inviolation of the stanard metadata for installed packages unless this happens. * Packaging for release: The standard library could be packaged so that releases happen on PyPI. This could be useful so that it's easier to consume newer versions of a package before those packages are included in the next Python release. It could also make Python upgrades easier, as you could hold-back packages which might otherwise break an application upon upgrade. I happened to find a bug in a standard library package today, and currently my option is to: Submit bug report to PyPI, wait for Python 2.7 release, then wait for Plone to update to Python 2.7. I'm looking at many years before this fix goes into my production application, so in the meantime I need to maintain a work-around for the bug as well account for the day when this bug is fixed. It makes a mess of the code. If it was possible to release this package on PyPI, this could reasonably happen in a few days or at worst a couple months. The setup.py for that package could be updated to require the newer version of the package, and the workaround could be removed from the code. * Packaging for resizing: The standard library could be packaged so that it could be made smaller or larger, or there could be different sized standard libraries made with different Python releases. So there are benefits to packaging the standard library which would be useful, for reasons which would have no impact on the "batteries included" aspect of Python. Plus, if the standard library was packaged and releases were made on PyPI (in addition to including them normally in the Python releases), we *might* be able to use PyPI's commenting infrastructure on this packages (~evil grin~). From martin at v.loewis.de Fri Nov 13 08:35:48 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 13 Nov 2009 08:35:48 +0100 Subject: [Python-Dev] PyPI front page In-Reply-To: References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> Message-ID: <4AFD0C54.1020709@v.loewis.de> > Ben Finney benfinney.id.au> writes: >> There's a problem with the poll's placement: on the front page of the >> PyPI website. > > Speaking of which, why is it that http://pypi.python.org/pypi and > http://pypi.python.org/pypi/ (note the ending slash) return different contents > (the latter being very voluminous)? I always mistake one for the other when > entering the URL directly. As Ian says: setuptools relies on it. It's part of the specification of the package index. Regards, Martin From martin at v.loewis.de Fri Nov 13 08:37:54 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 13 Nov 2009 08:37:54 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> Message-ID: <4AFD0CD2.6000304@v.loewis.de> > I only found the poll by accident by wondering randomly what might > change if I hit the login using open id button. So you can only vote > in the poll if you a) get told about it b) realise you need to create > an account to login and use in order to vote. I realise there's good > reasons for that, but I think it's a mistake. (There's no guidance > that you need to log in to see the poll for example) Why do you say that? "If you want to vote, you need to login to PyPI (on the right)." If that isn't clear to you, please propose a rephrasing (English is not my native language, nor the native language of MAL who proposed this specific wording) Regards, Martin From martin at v.loewis.de Fri Nov 13 08:39:59 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 13 Nov 2009 08:39:59 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20091113011812.GA75517@andrew-kuchlings-macbook.local> Message-ID: <4AFD0D4F.3090904@v.loewis.de> >> I've posted a tweet to the ThePSF account about the poll. If the poll >> runs for a week or two, that would provide time for word of the poll >> to propagate through Twitter, blogs, etc. > > You should also make an announcement on python-announce. On catalog-sig (the place where PyPI was discussed in recent years), I mentioned that I would announce it today, which I now have. Regards, Martin From martin at v.loewis.de Fri Nov 13 08:44:16 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 13 Nov 2009 08:44:16 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <20e5c9660911121652s6c9c25caqf8856eb6c1685986@mail.gmail.com> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> <20e5c9660911121652s6c9c25caqf8856eb6c1685986@mail.gmail.com> Message-ID: <4AFD0E50.5090309@v.loewis.de> > That's useful from a user perspective. Or is it? It's useful from a > user perspective, until that issue is fixed. Then what? Is it still > useful? Can the commenter remove it? Yes. > Can they get notified it's changed? Yes. > Can the maintainer say "this is fixed/changed?" Yes. > I never look at the PyPI pages for stuff I create. Which means if > someone is using it for support, they're wasting their time. (Why > would I look at it? I know what the project is for and where to get > it! :) (and also PyPI isn't the prime download for it either - so the > download stats are irrelevant to me) I doubt I'm alone, so how many > people's time are wasted asking questions there ? You'll get an email when someone comments, so you don't have to look at the page. > I suppose, personally, I'm dubious about the idea of having unchanging > comments and ratings associated with projects which are changing and > improving - that feels like a mismatch. That's why the comments are per-release. > If there's interest, and if there's a survey to be done, I could > forward a link to a survey through that twitterfeed - which I suspect > is a mix of users of PyPI and uploaders to PyPI. Feel free to annouce it on Twitter - I don't use that system myself. Regards, Martin From martin at v.loewis.de Fri Nov 13 08:45:39 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 13 Nov 2009 08:45:39 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4AFCE6E3.4070209@arcor.de> References: <4AFCE6E3.4070209@arcor.de> Message-ID: <4AFD0EA3.5050007@v.loewis.de> > mode = "python-dev reader" > > Please excuse me if I'm wrong here, > but I think python-dev just isn't the right place to discuss this topic, > because it's about 3rd party packages and it's got nothing to do with > the development *of the python language itself*, but generated a lot of > traffic. +1. The place to discuss pypi is catalog-sig. Regards, Martin From fuzzyman at voidspace.org.uk Fri Nov 13 09:47:30 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 13 Nov 2009 09:47:30 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <200911131722.38598.steve@pearwood.info> References: <200911130038.05516.steve@pearwood.info> <7EAD464A-45AC-41EF-9E11-A9BAA3461DB9@lericson.se> <200911131722.38598.steve@pearwood.info> Message-ID: <4AFD1D22.3050708@voidspace.org.uk> Steven D'Aprano wrote: > On Fri, 13 Nov 2009 04:27:48 am Ludvig Ericson wrote: > >> On 12 nov 2009, at 14:38, Steven D'Aprano wrote: >> >>> On Thu, 12 Nov 2009 08:44:32 pm Ludvig Ericson wrote: >>> >>>> Why are there comments on PyPI? Moreso, why are there comments >>>> which I cannot control as a package author on my very own >>>> packages? That's just absurd. >>>> >>> No, what's absurd is thinking that the act of publishing software >>> somehow gives you the right to demand control over what others say >>> about your software. >>> >> ... on my own package's page. >> > > It's your package. It's the community's page about your package. > > I think of PyPI as a community-owned noticeboard. Its primary purpose is > to allow the community to find good packages -- the benefit to the > community is why it exists, not the benefit to the author of the > package. > > In my opinion, the community is best served by a good comment/review > system, one which avoids the worst trolling, and allows authors the > right of reply, but does not allow authors to censor inconvenient but > honest reviews. > > +1 :-) Michael -- http://www.ironpythoninaction.com/ From naughtysriram at gmail.com Fri Nov 13 09:59:46 2009 From: naughtysriram at gmail.com (Sriram Srinivasan) Date: Fri, 13 Nov 2009 13:59:46 +0500 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: <56195B13-39DF-44A3-9A79-5BDAFF244F40@bud.ca> References: <4AFC6440.1040008@v.loewis.de> <4222a8490911121157i622fbab8q5fe82d32fe9ea8f@mail.gmail.com> <56195B13-39DF-44A3-9A79-5BDAFF244F40@bud.ca> Message-ID: you were thinking wrong. If suppose this feature is introduced it doesn't mean python will become batteries removed! you can ship the python release with the 'standard library packages' already installed. so what we get here is batteries included and ability to be changed after it is discharged! ;) users did not get this feature before. what we experience now is like 'use and throw' not the batteries but the whole python distribution (like torchlight/flashlight) itself. also users might get the ability to include other batteries too. also say for example some X company wants its own python distribution (including the standard libraries). they may have the standard libraries removed or another custom library on top of it (forget performance here). Now if the standard libraries were available as packages a new version may be installed and tested at no time. but if they get it by every release, there comes the problem. this not only makes things easier on both sides (releasing side and user side) it also makes the whole system transparent. even patches for standard libraries may be available as soon as a bug has been found, but not with the current system as Kevin mentioned. say for example we take an opensouce project, the linux kernel as example.( dont compare kernel with python each work differently and both scales are not equal) just think it was another opensource software. i can say mostly where there linux there is python (not talking about windows,embedded etc, both software go hand in hand, so i took it for comparison). if there is a bug/security issue found on the linux the patches are available in days/mostly by the end of next week. but for python it must take more time. may be some backward incompatible issue also comes along with it(eg. py3) then they have to wait for the dependencies being updated. i am only comparing the packaging and distributing system here not the softwares. althought the rate at which the kernel development is at a very high scale, the system is stable. and that is how open source works. every body can have look into everything. that is what makes open source software best. if it were (libraries) released as packages, by the time i install the patch package, i may also get the patch package of the dependency too. -- Regards, Sriram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ubershmekel at gmail.com Fri Nov 13 10:30:19 2009 From: ubershmekel at gmail.com (Yuval Greenfield) Date: Fri, 13 Nov 2009 11:30:19 +0200 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: <56195B13-39DF-44A3-9A79-5BDAFF244F40@bud.ca> References: <4AFC6440.1040008@v.loewis.de> <4222a8490911121157i622fbab8q5fe82d32fe9ea8f@mail.gmail.com> <56195B13-39DF-44A3-9A79-5BDAFF244F40@bud.ca> Message-ID: <9d153b7c0911130130v35ab44b7n5c98727505009409@mail.gmail.com> On Fri, Nov 13, 2009 at 9:35 AM, Kevin Teague wrote: > > It's also worth noting that there are three issues with respect to standard > library packaging which are all orthologous: > > ?* Packaging for metadata: The standard library could be packaged so that > there is consistent metadata about the distributions which compromise the > standard library (version, author, maintainer, etc). This could be useful so > that it would be easier to see at a glance which packages changed between > any two Python releases. For example, I have a script which compares two > working sets of packages and generates a web page with a colour-coded list > of differences between those two working sets, based on the size of the > version number bump, e.g. major.medium.minor (doesn't work with packages > which use odd version numbering schemes). I've used this to compare working > sets between different Grok releases or Plone releases to help see where the > activity or major API changes might be happening inside any given Zope-based > release. However, this script only works on 80% of the code, the other 20% > is in the standard library, which is currently a black-box. > > ? ?Also, if PEP 376 is accepted, then the standard library will be > inviolation of the stanard metadata for installed packages unless this > happens. > > ?* Packaging for release: ?The standard library could be packaged so that > releases happen on PyPI. This could be useful so that it's easier to consume > newer versions of a package before those packages are included in the next > Python release. It could also make Python upgrades easier, as you could > hold-back packages which might otherwise break an application upon upgrade. > I happened to find a bug in a standard library package today, and currently > my option is to: Submit bug report to PyPI, wait for Python 2.7 release, > then wait for Plone to update to Python 2.7. I'm looking at many years > before this fix goes into my production application, so in the meantime I > need to maintain a work-around for the bug as well account for the day when > this bug is fixed. It makes a mess of the code. If it was possible to > release this package on PyPI, this could reasonably happen in a few days or > at worst a couple months. The setup.py for that package could be updated to > require the newer version of the package, and the workaround could be > removed from the code. > > ?* Packaging for resizing: The standard library could be packaged so that it > could be made smaller or larger, or there could be different sized standard > libraries made with different Python releases. > > So there are benefits to packaging the standard library which would be > useful, for reasons which would have no impact on the "batteries included" > aspect of Python. > > Plus, if the standard library was packaged and releases were made on PyPI > (in addition to including them normally in the Python releases), we *might* > be able to use PyPI's commenting infrastructure on this packages (~evil > grin~). > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ubershmekel%40gmail.com > I kinda like the idea that the only packages included in the python trunk would be distribute and pip (maybe just svn:external). Upon packaging python for distribution (by installer or by source) the relevant batteries would be included. Moving packages in and out of the standard library just becomes so standard. But of course this really should be decided by the people who work most with the trunk and as I understand http://mail.python.org/pipermail/stdlib-sig/2009-September/000609.html this discussion's already run its course. --yuv From abpillai at gmail.com Fri Nov 13 11:36:10 2009 From: abpillai at gmail.com (Anand Balachandran Pillai) Date: Fri, 13 Nov 2009 16:06:10 +0530 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: <4AFC6440.1040008@v.loewis.de> References: <4AFC6440.1040008@v.loewis.de> Message-ID: <8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com> On Fri, Nov 13, 2009 at 1:08 AM, "Martin v. L?wis" wrote: > > I am not an expert, I am just another python learner. These are just my > > views on the state of the standard libraries and to > > make them state-of-the-art..! ;) > > If I understand correctly, you want the (current) standard library to be > separated from the Python implementation, and available separately. > > Interestingly enough, people are very much split over whether that would > be a good thing or not. Some like it the way Python does, some dislike > it (and some quite strongly so). > > In any case, many Python users consider it a good thing that it comes > "with batteries included", ie. with no need to add extra stuff for many > tasks. > > Some of the Python maintainers have recently started objecting to this > setup, asking that the standard library should be split into separate > packages that are released and distributed independent of Python. Others > of us feel strongly that such a change should not be made. > > So don't expect any immediate change to happen. > I think the recent postings on "CPAN for Python" in this list also stems from similar thoughts. As the size of the Python std library increases, and when it reaches a kind of implosion point (perhaps it is there already), it does make sense to split it into some thing like a "core" group of modules and add-ons which could be updated dynamically as and when required by a tool. It could be something as simple as a "require" keyword which could pull in the depdencies if not found. Perhaps at the top of your module, require (stuff, '1.27') import stuff Anyone who takes a quick look at the Python std library now is sure to feel that there is an overkill of stuff there, which could be classified and packaged better than dumping into the language build. > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/abpillai%40gmail.com > -- --Anand -------------- next part -------------- An HTML attachment was scrubbed... URL: From masklinn at masklinn.net Fri Nov 13 11:50:49 2009 From: masklinn at masklinn.net (Masklinn) Date: Fri, 13 Nov 2009 11:50:49 +0100 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: <8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com> References: <4AFC6440.1040008@v.loewis.de> <8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com> Message-ID: <45186ACD-29BA-4D5E-9ADB-25632021A539@masklinn.net> On 13 Nov 2009, at 11:36 , Anand Balachandran Pillai wrote: > I think the recent postings on "CPAN for Python" in this list also stems > from similar thoughts. No. The CPAN for Python messages (in and out of the list) are about third-party packages and a better way to manage (and a better experience of installing) those. Think Ruby, it has a significant stdlib (though not as extensive as Python's), but it also has gems (and the `gem` CLI util). Or Haskell, which also has a significant stdlib (GHC does anyway) but also has cabal and cabal-install. From wentland at cl.uni-heidelberg.de Fri Nov 13 11:19:06 2009 From: wentland at cl.uni-heidelberg.de (Wolodja Wentland) Date: Fri, 13 Nov 2009 11:19:06 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <87aayqvrif.fsf@benfinney.id.au> References: <200911130038.05516.steve@pearwood.info> <7EAD464A-45AC-41EF-9E11-A9BAA3461DB9@lericson.se> <200911131722.38598.steve@pearwood.info> <87aayqvrif.fsf@benfinney.id.au> Message-ID: <20091113101906.GA3246@kinakuta.local> On Fri, Nov 13, 2009 at 18:00 +1100, Ben Finney wrote: > Steven D'Aprano writes: > > > In my opinion, the community is best served by a good comment/review > > system, one which avoids the worst trolling, and allows authors the > > right of reply, but does not allow authors to censor inconvenient but > > honest reviews. > I also think, though, that the community is best served by an objective > repository of third-party Python packages, with information derived only > directly from the package itself and objective data. That allows the > least barrier to having a package maintainer want to register their > package with such a service, which is in the interest of having it be as > complete a registry of packages as can be. V> > A community forum, on the other hand, has many characteristics that will > be *disincentives* to a package manager for having their package appear > there. It's never going to attract as many package maintainers as an > impartial, objective registry; the many reasons already given here as to > why some package maintainers *don't* want their packages in such a > system are evidence of that. +1 I think that there are a lot of things that could be done to make PyPi a place where users can find *high wuality* packages and one that makes it desirerably for developers to have their packages listed there. Unfortunately comments *on the package main page* is not among them. I think the PyPi/Python developers should consider the following points: * PyPi is the de facto *official* repository of third party packages that form the "enhanced" stdlib * I as a developer chose PyPi to be the *official* website of my projects and I want to have control over this website. I could name a plethora of things I would do to enhance PyPi and here are some ideas: Better UI --------- Right now the package consists of a single page and all information is cramped in there. I would suggest having a "tab-style" bar containing things like: * Package description/Home (default) * Documentation Right now I have to place a link to the documentation I uploaded to PyPi in the package description myself? Why? Include that automatically on the package site as soon as it is uploaded and tell the developers about that feature! * Issue tracker (either hosted on PyPi or somewhere else) * Downloads (for different releases) * Comments Place comments on a different page, give developers the possibilty to enable this feature and react to comments. Maybe include a way to rate comments and those with a higher rating will bubble up reddit style. * Changelog * ... Automatic Package testing ------------------------- What I really like on PyPi is that my packages are tested automatically with Cheesecake and the order of packages when searching is determined by this rating. This gives a huge incentive to work on the package quality, which is a great thing IMHO. Ratings by users could influence this ranking as well... I would take this idea even further. All packages should be tested automatically and *REJECTED* if they don't meet certain quality criteria. PyPi should become a place where high quality packages can be found, not a dump for every single "Hello World" ever developed. Debian will run lintian (a Debian package checker) automatically against newly uploaded packages and reject those that don't meet a basic set of tests. Tests could reject packages that: * FTBFS * do not install/uninstall cleanly * miss important files (README, CHANGELOG, .. ) This could leverage platform specific information as in: If it is a *nix package and includes scripts, it should have man pages for them, .... * rate poorly on automatic tests (pep8, cheesecake, pylint, ...) * are not GPG signed * have not specified a license * ... I admit that this makes PyPi a much stricter place and suggest that sections are introduced that define different rules for inclusion (like Debian's stable, testing, unstable - main, contrib, non-free). There could be sections like this: * main - manually checked packages of high quality This would comprise packages of a high enough quality to be deemed acceptable in the stdlib * tested - packages not in main that rank high on automatic tests * ... * junk (better name required) Users could then decide which parts they want pip to retrieve packages from. We could even encourage github, bitbucket, sourceforge, ... to provide ways to automatically query their repositories from pip so they could be included in pip's "sources.list" and used to install dev-versions, packages of por quality. *IF* Python plans to develop a fully fledged platform agnostic package manager for Python programs/libraries it would be wise to learn from the solutions people came up with until now (aptitude, yum, pkg_add, emerge, ...) kind regards Wolodja Wentland -- .''`. Wolodja Wentland : :' : `. `'` 4096R/CAF14EFC `- 081C B7CD FF04 2BA9 94EA 36B2 8B7F 7D30 CAF1 4EFC -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From wa2n39 at gmail.com Sat Nov 14 03:44:14 2009 From: wa2n39 at gmail.com (wa2n39 at gmail.com) Date: Sat, 14 Nov 2009 02:44:14 +0000 Subject: [Python-Dev] Python-Dev Digest, Vol 76, Issue 114 Message-ID: <1038341212-1258112004-cardhu_decombobulator_blackberry.rim.net-1218332849-@bda056.bisx.prodap.on.blackberry> Please don't send me again this email ------Pesan Asli------ Dari:python-dev-request at python.org Pengirim:python-dev-bounces+wa2n39=gmail.com at python.org Ke:python-dev at python.org Balas Ke:python-dev at python.org Perihal:Python-Dev Digest, Vol 76, Issue 114 Terkirim:13 Nov 2009 03:00 Send Python-Dev mailing list submissions to python-dev at python.org To subscribe or unsubscribe via the World Wide Web, visit http://mail.python.org/mailman/listinfo/python-dev or, via email, send a message with subject or body 'help' to python-dev-request at python.org You can reach the person managing the list at python-dev-owner at python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Python-Dev digest..." Today's Topics: 1. Re: standard libraries don't behave like standard 'libraries' (Masklinn) ---------------------------------------------------------------------- Message: 1 Date: Fri, 13 Nov 2009 11:50:49 +0100 From: Masklinn To: Python-Dev Subject: Re: [Python-Dev] standard libraries don't behave like standard 'libraries' Message-ID: <45186ACD-29BA-4D5E-9ADB-25632021A539 at masklinn.net> Content-Type: text/plain; charset=us-ascii On 13 Nov 2009, at 11:36 , Anand Balachandran Pillai wrote: > I think the recent postings on "CPAN for Python" in this list also stems > from similar thoughts. No. The CPAN for Python messages (in and out of the list) are about third-party packages and a better way to manage (and a better experience of installing) those. Think Ruby, it has a significant stdlib (though not as extensive as Python's), but it also has gems (and the `gem` CLI util). Or Haskell, which also has a significant stdlib (GHC does anyway) but also has cabal and cabal-install. ------------------------------ _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev End of Python-Dev Digest, Vol 76, Issue 114 ******************************************* Sent from my BlackBerry? powered by Sinyal Kuat INDOSAT From naughtysriram at gmail.com Fri Nov 13 13:15:17 2009 From: naughtysriram at gmail.com (Sriram Srinivasan) Date: Fri, 13 Nov 2009 17:15:17 +0500 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: <8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com> References: <4AFC6440.1040008@v.loewis.de> <8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com> Message-ID: as people like to compare languages take ruby for example (i am confident that there will be no flame war here ;) ) we have PyPI they have RAA we have ? they have rubyforge i am seeing the rubyforge site now on my other tab, i find Communications (365 projects) Database (245 projects) Desktop Environment (80 projects) Education (81 projects) Games/Entertainment (225 projects) Internet (1339 projects) Multimedia (296 projects) Office/Business (226 projects) Other/Nonlisted Topic (82 projects) Printing (23 projects) Religion (7 projects) Scientific/Engineering (278 projects) Security (86 projects) Sociology (16 projects) Software Development (1164 projects) System (548 projects) Terminals (37 projects) Text Editors (136 projects) these are the ongoing library development. although they are small in number but it is put very clear. in python the only way i have found about the libraries we have is by using the doc (docs is great at this). but i find less/no details about standalone/libraries that are not part of python stdlib. (google is great at this) lets take RAA now, By Category Application 114 subcategories 553 projects Documentation 14 subcategories 31 projects Library 150 subcategories 1179 projects Ports 8 subcategories 10 projects * By Project(ALL): 286 subcategories, 1773 projects * By Owner: 905 owners first of all they can never ever beat python's record of packages. its funny i found a ruby project in pypi. on further inspection [rython 0.0.1] a library module for coding in ruby inside python code transparently... how awesome..! its happy to see that libraries already has taken the add-on feature. sad to see libraries mixed up with software packages (no one will find about the library amidst those 1000's of software! packaging libraries has some other advantages too, like accumulation of ideas happen. what if i design a *library* and no one uses it? whats the point in creating? but if i could upload that in pypi or something like that, also one python core active developer has had a look in to that, and might one day be integrated with the standard modules..! the best thing for now i propose is that segregating libraries and applications in pypi. if thats a first stage bring about an awesome library for distribution of packages (already in progress). third is chunk the standard library and start providing the packages. seems simple but already the python community is waiting for a long time. -- Regards, Sriram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From naughtysriram at gmail.com Fri Nov 13 13:33:38 2009 From: naughtysriram at gmail.com (Sriram Srinivasan) Date: Fri, 13 Nov 2009 17:33:38 +0500 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: References: <4AFC6440.1040008@v.loewis.de> <8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com> Message-ID: http://pypi.python.org/pypi?%3Aaction=search&term=library&submit=search this lists all the packages with the term 'library' in it. -- Regards, Sriram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Nov 13 14:11:59 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 13 Nov 2009 23:11:59 +1000 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: References: <4AFC6440.1040008@v.loewis.de> <8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com> Message-ID: <4AFD5B1F.1060900@gmail.com> Sriram Srinivasan wrote: > http://pypi.python.org/pypi?%3Aaction=search&term=library&submit=search > > > this lists all the packages with the term 'library' in it. Sriram, Please take this discussion to catalog-sig - python-dev isn't the place (the fact that many of us didn't immediately know the *right* place for the discussion indicates where PyPI sits on our personal active level of interest). You should find more interested (and knowledgable) folks and catalog-sig. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From steve at pearwood.info Fri Nov 13 14:14:43 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 14 Nov 2009 00:14:43 +1100 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: <8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com> References: <4AFC6440.1040008@v.loewis.de> <8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com> Message-ID: <200911140014.43627.steve@pearwood.info> On Fri, 13 Nov 2009 09:36:10 pm Anand Balachandran Pillai wrote: > It could be something as simple as a "require" keyword which could > ?pull in the depdencies if not found. Perhaps at the top of your > module, > > ?require (stuff, '1.27') > ?import stuff So Python, the *language*, has to become a package management system as well as a programming language? No thank you. At most, perhaps there could be a package management tool in the standard library: try: import stuff except ImportError: import packman found = packman.requires(stuff, '1.27') if found: import stuff else: fail() > Anyone who takes a quick look at the Python std library now is > ?sure to feel that there is an overkill of stuff there, which could > be classified and packaged better than dumping into the language > build. I don't. I think the std library could possibly be organised better, but just because something isn't useful to me right now, doesn't mean it isn't useful to someone, and may be useful to me tomorrow. -- Steven D'Aprano From abpillai at gmail.com Fri Nov 13 14:27:17 2009 From: abpillai at gmail.com (Anand Balachandran Pillai) Date: Fri, 13 Nov 2009 18:57:17 +0530 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: <200911140014.43627.steve@pearwood.info> References: <4AFC6440.1040008@v.loewis.de> <8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com> <200911140014.43627.steve@pearwood.info> Message-ID: <8548c5f30911130527g5494b825y357acd95abac3090@mail.gmail.com> On Fri, Nov 13, 2009 at 6:44 PM, Steven D'Aprano wrote: > On Fri, 13 Nov 2009 09:36:10 pm Anand Balachandran Pillai wrote: > > > It could be something as simple as a "require" keyword which could > > pull in the depdencies if not found. Perhaps at the top of your > > module, > > > > require (stuff, '1.27') > > import stuff > > So Python, the *language*, has to become a package management system as > well as a programming language? > > No thank you. > > At most, perhaps there could be a package management tool in the > standard library: > > try: > import stuff > except ImportError: > import packman > found = packman.requires(stuff, '1.27') > if found: > import stuff > else: > fail() > > > That was just a suggestion, don't take it literally. My intention was something similar to the stuff you have written above. It would be however useful to standardize the pkg manager "packman" to something in the standard library however. > > > Anyone who takes a quick look at the Python std library now is > > sure to feel that there is an overkill of stuff there, which could > > be classified and packaged better than dumping into the language > > build. > > I don't. > > I think the std library could possibly be organised better, but just > because something isn't useful to me right now, doesn't mean it isn't > useful to someone, and may be useful to me tomorrow. > > > Ah yes, the only problem is that for any updates/fixes to your module, you need to wait for the language release which is not very much an ideal situation IMHO. Especially now when we are in discussions about end of life of 2.x etc... W.r.t the original analogy, batteries v 1.1 are included with the torch 1.1, but if you want battery upgrade to 1.2, you need to buy torch 1.2.. > > -- > Steven D'Aprano > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/abpillai%40gmail.com > -- --Anand -------------- next part -------------- An HTML attachment was scrubbed... URL: From naughtysriram at gmail.com Fri Nov 13 15:17:04 2009 From: naughtysriram at gmail.com (Sriram Srinivasan) Date: Fri, 13 Nov 2009 19:17:04 +0500 Subject: [Python-Dev] Fwd: standard libraries don't behave like standard 'libraries' In-Reply-To: References: <4AFC6440.1040008@v.loewis.de> <8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com> <4AFD5B1F.1060900@gmail.com> Message-ID: Sriram, > > Please take this discussion to catalog-sig - python-dev isn't the place > (the fact that many of us didn't immediately know the *right* place for > the discussion indicates where PyPI sits on our personal active level of > interest). > > You should find more interested (and knowledgable) folks and catalog-sig. > > Regards, > Nick. > sorry for posting the link in a lone post, i must have posted it with my previous post, and the theme is how to make the standard libraries state-of-the-art and i many feel it must be in py-dev as it is directly connected with the development of python. As it required some comparison, i compared pypi with the ruby's *rubyforge*. in fact what i found astonishing is that there are a lot of libraries (3rd party) available in packaged forms. which is a very good sign, the problem is there is no sorted out list for libraries and software in pypi. *rubyforge* is like a pypi for *libraries* (standard and 3rd party) i guess some misplaced topics such as [Python-Dev] PyPI comments and ratings, *really*? , [Python-Dev] PyPI front page are also getting alarming responses in the py-dev list. including yours, ;) As Anand says requiring a new update of battery v1.5 requires update of python/flashlight which only occurs every release is the manin senario. I would also like to add that what if a company doesn't need the batteries that you give at the first place, they have developed their own batteries and want to use them(or use some from the default with theit own), also wants to use the same flashlight/python system? - it would be *easy* only if the batteries included had a feature of being removed. -- Regards, Sriram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From naughtysriram at gmail.com Fri Nov 13 15:18:54 2009 From: naughtysriram at gmail.com (Sriram Srinivasan) Date: Fri, 13 Nov 2009 19:18:54 +0500 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: References: <4AFC6440.1040008@v.loewis.de> <8548c5f30911130236j72d6b1a4m21d5bc714f451c43@mail.gmail.com> <4AFD5B1F.1060900@gmail.com> Message-ID: > Sriram, > > Please take this discussion to catalog-sig - python-dev isn't the place > (the fact that many of us didn't immediately know the *right* place for > the discussion indicates where PyPI sits on our personal active level of > interest). > > You should find more interested (and knowledgable) folks and catalog-sig. > > Regards, > Nick. > sorry for posting the link in a lone post, i must have posted it with my previous post, and the theme is how to make the standard libraries state-of-the-art and i many feel it must be in py-dev as it is directly connected with the development of python. As it required some comparison, i compared pypi with the ruby's *rubyforge*. in fact what i found astonishing is that there are a lot of libraries (3rd party) available in packaged forms. which is a very good sign, the problem is there is no sorted out list for libraries and software in pypi. *rubyforge* is like a pypi for *libraries* (standard and 3rd party) i guess some misplaced topics such as [Python-Dev] PyPI comments and ratings, *really*? , [Python-Dev] PyPI front page are also getting alarming responses in the py-dev list. including yours, ;) As Anand says requiring a new update of battery v1.5 requires update of python/flashlight which only occurs every release is the manin senario. I would also like to add that what if a company doesn't need the batteries that you give at the first place, they have developed their own batteries and want to use them(or use some from the default with theit own), also wants to use the same flashlight/python system? - it would be *easy* only if the batteries included had a feature of being removed. -- Regards, Sriram -------------- next part -------------- An HTML attachment was scrubbed... URL: From techtonik at gmail.com Fri Nov 13 17:14:37 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Fri, 13 Nov 2009 18:14:37 +0200 Subject: [Python-Dev] PyPI front page In-Reply-To: <4AFD0C54.1020709@v.loewis.de> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <4AFD0C54.1020709@v.loewis.de> Message-ID: On Fri, Nov 13, 2009 at 9:35 AM, "Martin v. L?wis" wrote: >> Ben Finney benfinney.id.au> writes: >>> There's a problem with the poll's placement: on the front page of the >>> PyPI website. >> >> Speaking of which, why is it that http://pypi.python.org/pypi and >> http://pypi.python.org/pypi/ (note the ending slash) return different contents >> (the latter being very voluminous)? I always mistake one for the other when >> entering the URL directly. > > As Ian says: setuptools relies on it. It's part of the specification of > the package index. Where is that specification? Does it require only http://pypi.python.org/pypi/ to be voluminous or also http://pypi.python.org/pypi to provide some search interface for automatic processing? There should be no problem in moving search page to the root http://pypi.python.org/ if spec is silent about the latter. -- anatoly t. From status at bugs.python.org Fri Nov 13 18:07:41 2009 From: status at bugs.python.org (Python tracker) Date: Fri, 13 Nov 2009 18:07:41 +0100 (CET) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20091113170741.A325B78364@psf.upfronthosting.co.za> ACTIVITY SUMMARY (11/06/09 - 11/13/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2491 open (+27) / 16645 closed (+14) / 19136 total (+41) Open issues with patches: 993 Average duration of open issues: 684 days. Median duration of open issues: 442 days. Open Issues Breakdown open 2455 (+27) pending 35 ( +0) Issues Created Or Reopened (42) _______________________________ test_debuglevel from test_telnetlib.py fails 11/06/09 CLOSED http://bugs.python.org/issue6748 reopened r.david.murray patch, buildbot json module apparently fails regression found at http://json.org 11/06/09 CLOSED http://bugs.python.org/issue7274 created DougShawhan CoverageResult fails to merge input file with non-empty callers 11/06/09 http://bugs.python.org/issue7275 created glinsvad patch UnboundLocalError scoping problem with nested functions 11/06/09 CLOSED http://bugs.python.org/issue7276 created olau online docs out-of-sync with sources and distributed .chm files 11/07/09 CLOSED http://bugs.python.org/issue7277 created gagenellina decimal.py: New instance vs. new reference 11/07/09 CLOSED http://bugs.python.org/issue7278 created skrah decimal.py: == and != comparisons involving NaNs 11/07/09 CLOSED http://bugs.python.org/issue7279 created skrah PCBuild instruction says to use nasmw.exe but it no longer exist 11/07/09 http://bugs.python.org/issue7280 created blep patch copysign() with NaN arguments on OpenSolaris 11/07/09 CLOSED http://bugs.python.org/issue7281 created skrah RLocks leak references when used in raw threads 11/08/09 CLOSED http://bugs.python.org/issue7282 created pitrou patch test_site failure when .local/lib/pythonX.Y/site-packages hasn't 11/08/09 http://bugs.python.org/issue7283 created pitrou optparse - display version in usage by default 11/08/09 http://bugs.python.org/issue7284 created techtonik multiprocessing module, example code error 11/08/09 http://bugs.python.org/issue7285 created hsmtkk patch odd exec() behavior or documentation 11/08/09 http://bugs.python.org/issue7286 created pmawhorter import hook demo does not work 11/08/09 http://bugs.python.org/issue7287 created blep Detect improper leading whitespace in C files for Vim 11/08/09 http://bugs.python.org/issue7288 created brett.cannon easy bad import in redemo.py script in v3.1 11/08/09 CLOSED http://bugs.python.org/issue7289 created alan.gauld at btinternet.com Update 'how do I set a global variable' faq entry 11/09/09 CLOSED http://bugs.python.org/issue7290 created r.david.murray patch urllib2 cannot handle https with proxy requiring auth 11/09/09 http://bugs.python.org/issue7291 created tsujikawa patch Multiprocessing Joinable race condition? 11/09/09 http://bugs.python.org/issue7292 created jonathan test_msvc9compiler test_reg_class failure on new Windows box 11/09/09 http://bugs.python.org/issue7293 created db3l MSDN URL in subprocess.py is incorrect 11/09/09 CLOSED http://bugs.python.org/issue7294 created eric.smith easy test_tarfile uses a hardcoded file name 11/09/09 CLOSED http://bugs.python.org/issue7295 created pitrou patch OverflowError: signed integer is greater than maximum on mips64 11/09/09 http://bugs.python.org/issue7296 created jasper --spam-- 11/10/09 CLOSED http://bugs.python.org/issue7297 created brandon.dixon reversed(range(x, -1, -1)) is empty when x > 1 11/10/09 http://bugs.python.org/issue7298 created ledave123 patch setup.py install doesn't honor PYTHONUSERBASE 11/10/09 http://bugs.python.org/issue7299 created doko patch, patch Unicode arguments in str.format() 11/10/09 http://bugs.python.org/issue7300 created doerwalter Add environment variable $PYTHONWARNINGS 11/10/09 http://bugs.python.org/issue7301 created barry Wrong link to Python Language Mapping Specification 11/10/09 http://bugs.python.org/issue7302 created efrerich pkgutil lacks documentation for useful functions 11/10/09 http://bugs.python.org/issue7303 created englabenny email.message.Message.set_payload and as_string given charset 'u 11/10/09 http://bugs.python.org/issue7304 created r.david.murray urllib2.urlopen() segfault using SSL on Solaris 11/11/09 http://bugs.python.org/issue7305 created rtarnell Patch - skip winsound tests if no default sound is configured 11/11/09 http://bugs.python.org/issue7306 created brian.curtin patch test_distutils failure under Windows 7 11/11/09 CLOSED http://bugs.python.org/issue7307 created pitrou Named group regex error 11/11/09 CLOSED http://bugs.python.org/issue7308 created pythonmeister crasher in str(Exception()) 11/12/09 http://bugs.python.org/issue7309 created arigo Unhelpful __repr__() in os.environ 11/12/09 http://bugs.python.org/issue7310 created kjohnson Bug on regexp of HTMLParser 11/12/09 http://bugs.python.org/issue7311 created pluskid Run some tests in a loop until failure 11/12/09 http://bugs.python.org/issue7312 created pitrou patch 2to3 fails when source file starts with BOM 11/13/09 http://bugs.python.org/issue7313 created gagenellina test_multiprocessing hangs on Windows 7 64 bit 11/13/09 http://bugs.python.org/issue7314 created krisvale 64bit Issues Now Closed (31) ______________________ CGIHTTPRequestHandler.run_cgi() does not run on Windows if sys.e 769 days http://bugs.python.org/issue1235 orsenthil patch Management of KeyboardInterrupt in cmd.py 710 days http://bugs.python.org/issue1294 gvanrossum patch RLock's are SLOW 530 days http://bugs.python.org/issue3001 pitrou patch Real segmentation fault handler 406 days http://bugs.python.org/issue3999 Rhamphoryncus patch No universal newline support for compile() when using bytes 337 days http://bugs.python.org/issue4628 benjamin.peterson Auto-detect indentation in C source in vimrc 223 days http://bugs.python.org/issue5611 brett.cannon patch Subclassing property doesn't preserve the auto __doc__ behavior 195 days http://bugs.python.org/issue5890 moriyoshi patch, needs review test_codecs fails when ran after test_zipimport and test_mailbox 114 days http://bugs.python.org/issue6551 pitrou patch test_debuglevel from test_telnetlib.py fails 0 days http://bugs.python.org/issue6748 jackdied patch, buildbot round(x) gives wrong result for large odd integers 34 days http://bugs.python.org/issue7070 mark.dickinson easy Add looping capability to regrtest 29 days http://bugs.python.org/issue7134 pitrou importlib/_bootstrap.py write_bytecode raises an IOError if it c 16 days http://bugs.python.org/issue7187 brett.cannon patch test_multiprocessing crashes under Windows when run in verbose m 18 days http://bugs.python.org/issue7197 pitrou patch 2to3 execfile conversion changes behavior 1 days http://bugs.python.org/issue7268 gagenellina patch Tests for synchronization primitives 0 days http://bugs.python.org/issue7270 pitrou patch Typo in turtle module documentation. 1 days http://bugs.python.org/issue7271 georg.brandl json module apparently fails regression found at http://json.org 2 days http://bugs.python.org/issue7274 DougShawhan UnboundLocalError scoping problem with nested functions 3 days http://bugs.python.org/issue7276 r.david.murray online docs out-of-sync with sources and distributed .chm files 0 days http://bugs.python.org/issue7277 r.david.murray decimal.py: New instance vs. new reference 1 days http://bugs.python.org/issue7278 rhettinger decimal.py: == and != comparisons involving NaNs 1 days http://bugs.python.org/issue7279 mark.dickinson copysign() with NaN arguments on OpenSolaris 2 days http://bugs.python.org/issue7281 mark.dickinson RLocks leak references when used in raw threads 2 days http://bugs.python.org/issue7282 pitrou patch bad import in redemo.py script in v3.1 0 days http://bugs.python.org/issue7289 benjamin.peterson Update 'how do I set a global variable' faq entry 2 days http://bugs.python.org/issue7290 r.david.murray patch MSDN URL in subprocess.py is incorrect 0 days http://bugs.python.org/issue7294 eric.smith easy test_tarfile uses a hardcoded file name 2 days http://bugs.python.org/issue7295 pitrou patch --spam-- 0 days http://bugs.python.org/issue7297 rhettinger test_distutils failure under Windows 7 1 days http://bugs.python.org/issue7307 amaury.forgeotdarc Named group regex error 0 days http://bugs.python.org/issue7308 mark.dickinson 'compile' built-in function failures when missing EOL 1293 days http://bugs.python.org/issue1479099 benjamin.peterson Top Issues Most Discussed (10) ______________________________ 19 Enable short float repr() on Solaris/x86 208 days open http://bugs.python.org/issue5792 10 RLock's are SLOW 530 days closed http://bugs.python.org/issue3001 7 reversed(range(x, -1, -1)) is empty when x > 1 3 days open http://bugs.python.org/issue7298 7 test_multiprocessing dictionary changed size errors and hang 40 days open http://bugs.python.org/issue7060 6 OverflowError: signed integer is greater than maximum on mips64 4 days open http://bugs.python.org/issue7296 6 CGIHTTPRequestHandler.run_cgi() does not run on Windows if sys. 769 days closed http://bugs.python.org/issue1235 5 Management of KeyboardInterrupt in cmd.py 710 days closed http://bugs.python.org/issue1294 4 Run some tests in a loop until failure 1 days open http://bugs.python.org/issue7312 4 copysign() with NaN arguments on OpenSolaris 2 days closed http://bugs.python.org/issue7281 4 test_debuglevel from test_telnetlib.py fails 0 days closed http://bugs.python.org/issue6748 From rhamph at gmail.com Fri Nov 13 19:18:45 2009 From: rhamph at gmail.com (Adam Olsen) Date: Fri, 13 Nov 2009 11:18:45 -0700 Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs In-Reply-To: <5c6f2a5d0911090501n23254cc7x8e1eaad34508f8fa@mail.gmail.com> References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za> <1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za> <20091108162648.GA4099@mail.bytereef.org> <5c6f2a5d0911090501n23254cc7x8e1eaad34508f8fa@mail.gmail.com> Message-ID: On Mon, Nov 9, 2009 at 06:01, Mark Dickinson wrote: > Well, when running in some form of 'non-stop' mode, where (quiet) NaN > results are supposed to be propagated to the end of a computation, you > certainly want equality comparisons with nan just to silently return false. > E.g., in code like: > > if x == 0: > ? ? > else: > ? ? > > nans should just end up in the second branch, without the programmer > having had to think about it too hard. if x != 0: else: nans should just end up in the first branch, without the programmer having had to think about it too hard. There is a more consistent alternative: have all comparisons involving NaN also return NaN, signifying they're unordered. Let bool coercion raise the exception. Thus, both examples would raise an exception, but a programmer who wants to handle NaN could do so explicitly: temp = x == 0 if temp.isnan() or temp: else: IEEE 754 is intended for a very different context. I don't think it makes sense to attempt literal conformance to it. -- Adam Olsen, aka Rhamphoryncus From p.f.moore at gmail.com Fri Nov 13 20:33:18 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 13 Nov 2009 19:33:18 +0000 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> Message-ID: <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> 2009/11/13 Tres Seaver : > I can see the information about the poll, and a link to view the > results, without logging in. > > ?http://pypi.python.org/pypi > > (second paragraph there). ?That paragraph tells you that you need to log > in to vote in the poll. I don't want to create a PyPI account (more account details I'll rarely use to remember) just to vote. I can see why anonymous votes are bad, but the sample's going to be self-selecting - people like me who don't want to create an account will be excluded. By the way, there's no way for me to vote that I don't care what option ends up being chosen, but I strongly oppose choosing something that would tend to make developers avoid using PyPI. The distutils/setuptools/distribute/... saga has fragmented Python's package ecosystem enough already. Let's not have PyPI go the same way :-( Paul. From g.brandl at gmx.net Fri Nov 13 23:04:25 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 13 Nov 2009 23:04:25 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <87tywzurst.fsf@benfinney.id.au> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <87tywzurst.fsf@benfinney.id.au> Message-ID: Ben Finney schrieb: > Michael Sparks writes: > >> On Fri, Nov 13, 2009 at 12:44 AM, Ben Finney wrote: >> > So, the poll's audience is limited to those who visit the front page >> > (which is hardly ever necessary for package maintainers), and those >> > who already know it exists (e.g. through this discussion thread). >> > You'll be missing the opinions of those maintainers who, like the OP >> > of this thread, only discovered the behaviour much later. >> >> This poll is only visible if you're logged into PyPI. This strikes me >> as a mistake. I went looking for a poll and didn't see it. > > The mistake, I think, is having a poll basically asking ?what should the > PyPI maintainers do??, instead of weighing evidence and reasoned > arguments. A poll may be good for gathering preferences and opinion, but > it's a poor way to make a *decision*. I haven't seen Martin commit to doing whatever the poll's preferred outcome was. I think it's just to see if there is really so much silent resistance or support as is claimed by both arguing sides. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From matthew at woodcraft.me.uk Fri Nov 13 22:34:32 2009 From: matthew at woodcraft.me.uk (Matthew Woodcraft) Date: Fri, 13 Nov 2009 21:34:32 +0000 (UTC) Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> Message-ID: Martin v. L?wis wrote: > Because I want to wait for the outcome of the poll first. The pypi front page says: | The poll will be closed on December 1, 2009. The option that receives | most votes will be implemented. As I write, the option with the most votes is /Allow both ratings and comments/. But between them, /Disallow comments/ and /Disallow both ratings and comments/ have more votes. It seems clear to me that, given those figures, allowing comments would not be following the result of the vote. So I think that simply implementing the option that receives most votes is not a wise plan. (There are two other options; I don't think their presence makes the plan any wiser.) -M- From solipsis at pitrou.net Fri Nov 13 22:38:31 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 13 Nov 2009 21:38:31 +0000 (UTC) Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> Message-ID: Matthew Woodcraft woodcraft.me.uk> writes: > > It seems clear to me that, given those figures, allowing comments would > not be following the result of the vote. > > So I think that simply implementing the option that receives most votes > is not a wise plan. Can we defer any further discussion about this until the poll is closed? Thank you Antoine. From ben+python at benfinney.id.au Fri Nov 13 22:43:25 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 14 Nov 2009 08:43:25 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> Message-ID: <87k4xut81u.fsf@benfinney.id.au> Paul Moore writes: > By the way, there's no way for me to vote that I don't care what > option ends up being chosen, but I strongly oppose choosing something > that would tend to make developers avoid using PyPI. There's also no option to vote that decisions on how to manage Python infrastructure (like PyPI) shouldn't be made by popular vote. -- \ ?I like to reminisce with people I don't know. Granted, it | `\ takes longer.? ?Steven Wright | _o__) | Ben Finney From dickinsm at gmail.com Fri Nov 13 22:50:57 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 13 Nov 2009 21:50:57 +0000 Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs In-Reply-To: References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za> <1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za> <20091108162648.GA4099@mail.bytereef.org> <5c6f2a5d0911090501n23254cc7x8e1eaad34508f8fa@mail.gmail.com> Message-ID: <5c6f2a5d0911131350r1e07af33pca49ebb1d794bdd3@mail.gmail.com> On Fri, Nov 13, 2009 at 6:18 PM, Adam Olsen wrote: > On Mon, Nov 9, 2009 at 06:01, Mark Dickinson wrote: >> Well, when running in some form of 'non-stop' mode, where (quiet) NaN >> results are supposed to be propagated to the end of a computation, you >> certainly want equality comparisons with nan just to silently return false. >> E.g., in code like: >> >> if x == 0: >> ? ? >> else: >> ? ? >> >> nans should just end up in the second branch, without the programmer >> having had to think about it too hard. > > if x != 0: > ? ? > else: > ? ? > > nans should just end up in the first branch, without the programmer > having had to think about it too hard. And they do: nan != 0 returns False. Maybe I'm missing your point here? > IEEE 754 is intended for a very different context. ?I don't think it > makes sense to attempt literal conformance to it. I disagree. The decimal specification is tied closely to IEEE 754 (the 2008 version), to the extent that even minor changes made by the 754r working group were mirrored in the decimal specification as it evolved; it only stopped evolving after IEEE 754-2008 was complete. IEEE 754-2008 also makes a point of targeting languages, rather than just floating-point hardware; to me it seems very much applicable to decimal.py. Mark From dickinsm at gmail.com Fri Nov 13 22:52:10 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 13 Nov 2009 21:52:10 +0000 Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs In-Reply-To: <5c6f2a5d0911131350r1e07af33pca49ebb1d794bdd3@mail.gmail.com> References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za> <1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za> <20091108162648.GA4099@mail.bytereef.org> <5c6f2a5d0911090501n23254cc7x8e1eaad34508f8fa@mail.gmail.com> <5c6f2a5d0911131350r1e07af33pca49ebb1d794bdd3@mail.gmail.com> Message-ID: <5c6f2a5d0911131352w47d46fffo3b54b8fa8995a332@mail.gmail.com> On Fri, Nov 13, 2009 at 9:50 PM, Mark Dickinson wrote: > And they do: ?nan != 0 returns False. ?Maybe I'm missing your point > here? Aargh! True! I meant to say True! Mark From rhamph at gmail.com Fri Nov 13 23:08:59 2009 From: rhamph at gmail.com (Adam Olsen) Date: Fri, 13 Nov 2009 15:08:59 -0700 Subject: [Python-Dev] decimal.py: == and != comparisons involving NaNs In-Reply-To: <5c6f2a5d0911131352w47d46fffo3b54b8fa8995a332@mail.gmail.com> References: <1257590632.12.0.893741146745.issue7279@psf.upfronthosting.co.za> <1257672530.83.0.836585441467.issue7279@psf.upfronthosting.co.za> <20091108162648.GA4099@mail.bytereef.org> <5c6f2a5d0911090501n23254cc7x8e1eaad34508f8fa@mail.gmail.com> <5c6f2a5d0911131350r1e07af33pca49ebb1d794bdd3@mail.gmail.com> <5c6f2a5d0911131352w47d46fffo3b54b8fa8995a332@mail.gmail.com> Message-ID: On Fri, Nov 13, 2009 at 14:52, Mark Dickinson wrote: > On Fri, Nov 13, 2009 at 9:50 PM, Mark Dickinson wrote: >> And they do: ?nan != 0 returns False. ?Maybe I'm missing your point >> here? > > Aargh! ?True! ?I meant to say True! Huh. Somewhere along the line I lost track of how python handled NaN. I thought "comparisons always evaluate to false" was the rule. -- Adam Olsen, aka Rhamphoryncus From amk at amk.ca Fri Nov 13 23:12:34 2009 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 13 Nov 2009 17:12:34 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <87k4xut81u.fsf@benfinney.id.au> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <87k4xut81u.fsf@benfinney.id.au> Message-ID: <20091113221234.GA15732@amk-desktop.matrixgroup.net> On Sat, Nov 14, 2009 at 08:43:25AM +1100, Ben Finney wrote: > There's also no option to vote that decisions on how to manage Python > infrastructure (like PyPI) shouldn't be made by popular vote. If the open source approach of 'the maintainer decides' is followed, well, both the maintainer of the code and the admin of the site is Martin. Comments stay, then. If 'BDFL decides' is followed, GvR thinks the idea is reasonable (http://mail.python.org/pipermail/python-dev/2009-November/094058.html), though the implementation might be improved. Again, comments stay. If popular vote is ruled out, I don't see who else could possibly make the decision to disable comments and/or ratings. --amk From jacob at jacobian.org Fri Nov 13 23:17:40 2009 From: jacob at jacobian.org (Jacob Kaplan-Moss) Date: Fri, 13 Nov 2009 16:17:40 -0600 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> Message-ID: On Fri, Nov 13, 2009 at 1:33 PM, Paul Moore wrote: > By the way, there's no way for me to vote that I don't care what > option ends up being chosen, but I strongly oppose choosing something > that would tend to make developers avoid using PyPI. I think the damage has already been done. There are enough people -- I'm one -- who feel this has revealed huge flaws in PyPI's stewardship. I'm pretty convinced, results of the poll or not, that the way PyPI is being run is fatally flawed. This "poll" business is just smoke and mirrors, anyway -- notice the way the "no comments" votors are split among three choices, while the "pro comments" voters have just two choices. It's also worded in a way that obscures the real debate. Regardless of the outcome, the poll's not going to change anyone's mind, and it certainly won't change the fact that PyPI's being run as a one-man show, not as a community resource. I'm just not comfortable with that. Unless PyPI changes its governance radically, I'm not going to invest any more effort in it. Jacob From theller at ctypes.org Fri Nov 13 23:55:24 2009 From: theller at ctypes.org (Thomas Heller) Date: Fri, 13 Nov 2009 23:55:24 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <20091113221234.GA15732@amk-desktop.matrixgroup.net> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <87k4xut81u.fsf@benfinney.id.au> <20091113221234.GA15732@amk-desktop.matrixgroup.net> Message-ID: A.M. Kuchling schrieb: > If the open source approach of 'the maintainer decides' is followed, > well, both the maintainer of the code and the admin of the site is > Martin. Comments stay, then. > > If 'BDFL decides' is followed, GvR thinks the idea is reasonable > (http://mail.python.org/pipermail/python-dev/2009-November/094058.html), > though the implementation might be improved. Again, comments stay. > > If popular vote is ruled out, I don't see who else could possibly make > the decision to disable comments and/or ratings. +1 -- Thanks, Thomas From ben+python at benfinney.id.au Fri Nov 13 23:57:18 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 14 Nov 2009 09:57:18 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <87k4xut81u.fsf@benfinney.id.au> <20091113221234.GA15732@amk-desktop.matrixgroup.net> Message-ID: <87bpj6t4mp.fsf@benfinney.id.au> "A.M. Kuchling" writes: > If popular vote is ruled out, I don't see who else could possibly make > the decision to disable comments and/or ratings. Reasoned argument with the person who decides. A bad idea with many people's support is still a bad idea; a good idea with few people's support is still a good idea. -- \ ?I have a microwave fireplace in my house. The other night I | `\ laid down in front of the fire for the evening in two minutes.? | _o__) ?Steven Wright | Ben Finney From steve at pearwood.info Sat Nov 14 00:18:12 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 14 Nov 2009 10:18:12 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <87bpj6t4mp.fsf@benfinney.id.au> References: <4AFC9064.20700@v.loewis.de> <20091113221234.GA15732@amk-desktop.matrixgroup.net> <87bpj6t4mp.fsf@benfinney.id.au> Message-ID: <200911141018.13536.steve@pearwood.info> On Sat, 14 Nov 2009 09:57:18 am Ben Finney wrote: > "A.M. Kuchling" writes: > > If popular vote is ruled out, I don't see who else could possibly > > make the decision to disable comments and/or ratings. > > Reasoned argument with the person who decides. A bad idea with many > people's support is still a bad idea; a good idea with few people's > support is still a good idea. Okay, let's talk reasoned debate. I understand the reason for making comments compulsory: they're for the benefit of the users, not the package owner. It helps prevent information about the package from being fragmented: there is One Obvious place to find out about a package on PyPI, which is the PyPI page, instead of having to search for blogs where people may or may not have made comments about the package. If individual package owners don't like this, too bad, because PyPI is run for the benefit of the community, not individual package owners. I understand the reason for making comments optional: personal choice of the package owner is valuable in and of itself, even if it is against the best interests of the community. But for the life of me, I can't understand the 1/3 of the votes that have been cast in favour of prohibiting comments for everybody, even those who want comments. -- Steven D'Aprano From steve at pearwood.info Sat Nov 14 00:19:17 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 14 Nov 2009 10:19:17 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> Message-ID: <200911141019.17917.steve@pearwood.info> On Sat, 14 Nov 2009 09:17:40 am Jacob Kaplan-Moss wrote: > This "poll" business is just smoke and mirrors, anyway -- notice the > way the "no comments" votors are split among three choices, while the > "pro comments" voters have just two choices. What? Allow ratings and comments on all packages (status quo) Allow package owners to disallow comments (ratings unmodified) Allow comments, but only send them to package owners (ratings unmodified) Disallow comments (ratings unmodified) Disallow ratings and comments (status three months ago) That's three choices for allowing comments, two for disallowing. -- Steven D'Aprano From steve at pearwood.info Sat Nov 14 00:20:43 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 14 Nov 2009 10:20:43 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <4AFC9D0C.5030500@v.loewis.de> Message-ID: <200911141020.43496.steve@pearwood.info> On Sat, 14 Nov 2009 08:34:32 am Matthew Woodcraft wrote: > As I write, the option with the most votes is /Allow both ratings and > comments/. > > But between them, /Disallow comments/ and /Disallow both ratings and > comments/ have more votes. If that's an accurate characterisation, then things must have changed *radically* in just a few minutes, because the voting at 8:38am (Melbourne time) was running 2:1 in favour of allowing comments versus disallowing them. At 10:17 it's still running at the same ratio, although the individual votes have changed slightly. > It seems clear to me that, given those figures, allowing comments > would not be following the result of the vote. Why do you think it is okay to combine the Disallow vote, without also combining the Allow vote? Less than a third of the total votes are in favour of disallowing comments, with two-thirds in favour of allowing them. -- Steven D'Aprano From steve at pearwood.info Sat Nov 14 00:30:14 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 14 Nov 2009 10:30:14 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <200911141019.17917.steve@pearwood.info> References: <200911141019.17917.steve@pearwood.info> Message-ID: <200911141030.15071.steve@pearwood.info> On Sat, 14 Nov 2009 10:19:17 am Steven D'Aprano wrote: > On Sat, 14 Nov 2009 09:17:40 am Jacob Kaplan-Moss wrote: > > This "poll" business is just smoke and mirrors, anyway -- notice > > the way the "no comments" votors are split among three choices, > > while the "pro comments" voters have just two choices. > > What? > > Allow ratings and comments on all packages (status quo) > Allow package owners to disallow comments (ratings unmodified) > Allow comments, but only send them to package owners (ratings > unmodified) > Disallow comments (ratings unmodified) > Disallow ratings and comments (status three months ago) > > That's three choices for allowing comments, two for disallowing. Sorry, I should have been more explicit: The first choice makes comments compulsory. In other words, comments are allowed. The second choice makes comments optional. Whether it is opt-in or opt-out, PyPI still allows comments as an option. The third choice makes comments available, but private. That's still allowing comments. Only the fourth and fifth options disallow comments. If you don't want there to be comments on PyPI, you only have two choices: 4 or 5. -- Steven D'Aprano From matthew at woodcraft.me.uk Sat Nov 14 01:00:52 2009 From: matthew at woodcraft.me.uk (Matthew Woodcraft) Date: Sat, 14 Nov 2009 00:00:52 +0000 (UTC) Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <4AFC9D0C.5030500@v.loewis.de> <200911141020.43496.steve@pearwood.info> Message-ID: Steven D'Aprano wrote: > Why do you think it is okay to combine the Disallow vote, without also > combining the Allow vote? Less than a third of the total votes are in > favour of disallowing comments, with two-thirds in favour of allowing > them. I don't. I was giving one example of the problem with the plan of not combining options at all. I wasn't writing to take sides. I was trying to say that using 'first past the post' voting with five options which are combinations of different issues (comments, ratings, whether to allow package owners to opt out) can't be expected to give a good decision. I think there is no safe way to pick a winner from this vote (unless more than half the voters choose a single option). (As a general point, in any discussion it tends to be unhelpful to ask "Why do you think X" unless the person you're asking has clearly indicated that they do in fact think X.) -M- From solipsis at pitrou.net Sat Nov 14 01:09:19 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 14 Nov 2009 00:09:19 +0000 (UTC) Subject: [Python-Dev] Status of the Buildbot fleet and related bugs References: <4AF9E3D6.2000807@v.loewis.de> Message-ID: Martin v. L?wis v.loewis.de> writes: > > > The buildbots still show occasional oddities. For example, right now in > > the page "http://www.python.org/dev/buildbot/3.x/", some results have > > disappeared (the columns for "AMD64 Ubuntu" builders have become empty). > > Yes, I noticed it too. It will go away after some page reloads. It is still happening more or less randomly unfortunately. http://www.python.org/dev/buildbot/3.x/ From robert.kern at gmail.com Sat Nov 14 01:10:16 2009 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 13 Nov 2009 18:10:16 -0600 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <200911141018.13536.steve@pearwood.info> References: <4AFC9064.20700@v.loewis.de> <20091113221234.GA15732@amk-desktop.matrixgroup.net> <87bpj6t4mp.fsf@benfinney.id.au> <200911141018.13536.steve@pearwood.info> Message-ID: [We really should be discussing this on catalog-sig, but moving things mid-thread never works, so here we go. I apologize to python-dev. I also apologize for the length.] On 2009-11-13 17:18 PM, Steven D'Aprano wrote: > On Sat, 14 Nov 2009 09:57:18 am Ben Finney wrote: >> "A.M. Kuchling" writes: >>> If popular vote is ruled out, I don't see who else could possibly >>> make the decision to disable comments and/or ratings. >> >> Reasoned argument with the person who decides. A bad idea with many >> people's support is still a bad idea; a good idea with few people's >> support is still a good idea. > > Okay, let's talk reasoned debate. > > I understand the reason for making comments compulsory: they're for the > benefit of the users, not the package owner. It helps prevent > information about the package from being fragmented: there is One > Obvious place to find out about a package on PyPI, which is the PyPI > page, instead of having to search for blogs where people may or may not > have made comments about the package. If individual package owners > don't like this, too bad, because PyPI is run for the benefit of the > community, not individual package owners. > > I understand the reason for making comments optional: personal choice of > the package owner is valuable in and of itself, even if it is against > the best interests of the community. > > But for the life of me, I can't understand the 1/3 of the votes that > have been cast in favour of prohibiting comments for everybody, even > those who want comments. While I do have a couple of packages on PyPI, I use PyPI as a consumer of packages much more frequently, every day in fact. I voted against comments and ratings because I think it is a detriment to my experience of PyPI as a user (I also think that they will be a detriment to the community, but that's a prediction, not a fact). Short form comments are simply not useful to me. Ratings are worse. They do not carry reliable information; they carry short statements of opinion from a wide variety of experiences, most of which are entirely unrelated to my own needs. To make an example, I have a lot of experience making ornery stuff build. A lot of other people don't. Their personal experience of not managing to install a package correctly turns into a weird, objective-seeming statement that the "package is difficult to build". People have different thresholds, different experiences, and different standards. When such opinions get numerically aggregated by a monolithic rating system, that's even worse. Even with short-form comments, they have the ability, though often not the propensity, to give a little bit of information ("I had trouble building it on Windows.") that can help me tease out whether their experiences will be relevant to mine, but one star is just one star. I *do* like to read long-form reviews where people relate what their needs were, what they tried to use the package for, and exactly where the package succeeded and where it failed. I can compare that context with my own needs and extract the relevant parts of their experience. Blogs are great for that. Now I do appreciate ratings and comments on shopping sites if they don't provide the capability for long-form reviews. But that's because the product is locked behind the barrier of payment and often shipping. There is no such barrier on PyPI. If I can get to a web view of their repository, thirty seconds with it will give a much better idea of whether the package is worth trying than any amount of comments I could read in that time. I can easily see how much documentation they have, if they have funny build requirements, if they are just prototyping, etc. without needing to trust that everyone else has needs and standards like mine. That's the key difference between comments and ratings on shopping sites and why I don't think that their success in that field translates to here. If you want one idea that would make my use of PyPI much more pleasant and informative, it would be to add a "Repository-URL" entry to the recommended PyPI metadata so that I could always start looking at the code in one click. Or integrate the source code browsing into PyPI itself; it could open up the sdists and eggs and show a WebVCS-like browser interface. Now, these are all reasons why commenting and rating are not beneficial to me. Where I think they are harmful is that when one is exposed to them, one cannot just ignore them. They have an emotional, unreasonable impact whether I want them to or not. And that's why I do not want to see them. Give me access to information, not opinions. If authors do want comments and ratings on their packages, let them use the services that already exist for *just* that purpose like Ohloh. They have the time and energy to implement these mechanisms with the care and attention that they need. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From chris at simplistix.co.uk Sat Nov 14 01:24:00 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Sat, 14 Nov 2009 00:24:00 +0000 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <200911141030.15071.steve@pearwood.info> References: <200911141019.17917.steve@pearwood.info> <200911141030.15071.steve@pearwood.info> Message-ID: <4AFDF8A0.1050100@simplistix.co.uk> Steven D'Aprano wrote: >> That's three choices for allowing comments, two for disallowing. > > Sorry, I should have been more explicit: > > The first choice makes comments compulsory. In other words, comments are > allowed. > > The second choice makes comments optional. Whether it is opt-in or > opt-out, PyPI still allows comments as an option. > > The third choice makes comments available, but private. That's still > allowing comments. > > Only the fourth and fifth options disallow comments. If you don't want > there to be comments on PyPI, you only have two choices: 4 or 5. The wording isn't ideal, unless you're in agreement with Martin: http://mail.python.org/pipermail/catalog-sig/2009-November/002288.html Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From skip at pobox.com Sat Nov 14 01:33:00 2009 From: skip at pobox.com (skip at pobox.com) Date: Fri, 13 Nov 2009 18:33:00 -0600 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: <19197.64188.94002.797250@montanaro.dyndns.org> >> Frankly, I agree with him. As implemented, I *and others* think this >> is broken. I've taken the stance of not publishing things to PyPi >> until A> I find the time to contribute to make it better or B> It >> changes. Barry> That's distressing. For better or worse PyPI is the central Barry> repository of 3rd party packages. It should be easy, desirable, Barry> fun and socially encouraged to get your packages there. I only realized people could comment on my packages when I got the poll email from Martin. It then took me awhile to figure out how to actually see comments about my packages. https://sourceforge.net/tracker/?func=detail&aid=2897527&group_id=66150&atid=513503 At the very least I think the feature needs to be easier for package authors to use. Skip From python at mrabarnett.plus.com Sat Nov 14 01:38:18 2009 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 14 Nov 2009 00:38:18 +0000 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <4AFC9D0C.5030500@v.loewis.de> <200911141020.43496.steve@pearwood.info> Message-ID: <4AFDFBFA.9030809@mrabarnett.plus.com> Matthew Woodcraft wrote: > Steven D'Aprano wrote: >> Why do you think it is okay to combine the Disallow vote, without also >> combining the Allow vote? Less than a third of the total votes are in >> favour of disallowing comments, with two-thirds in favour of allowing >> them. > > I don't. I was giving one example of the problem with the plan of not > combining options at all. > > I wasn't writing to take sides. I was trying to say that using 'first > past the post' voting with five options which are combinations of > different issues (comments, ratings, whether to allow package owners to > opt out) can't be expected to give a good decision. > > I think there is no safe way to pick a winner from this vote (unless > more than half the voters choose a single option). > > > (As a general point, in any discussion it tends to be unhelpful to ask > "Why do you think X" unless the person you're asking has clearly > indicated that they do in fact think X.) > Perhaps Approval voting should be used: http://en.wikipedia.org/wiki/Approval_voting From chris at simplistix.co.uk Sat Nov 14 01:44:02 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Sat, 14 Nov 2009 00:44:02 +0000 Subject: [Python-Dev] PyPI governance In-Reply-To: References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> Message-ID: <4AFDFD52.4070401@simplistix.co.uk> Jacob Kaplan-Moss wrote: > that obscures the real debate. Regardless of the outcome, the poll's > not going to change anyone's mind, and it certainly won't change the > fact that PyPI's being run as a one-man show, not as a community > resource. While I may not agree on his choices regarding comments and ratings, I don't feel this is fair to Martin. No one else, as far as I know, has offered to step up and help with the maintenance of PyPI. I don't know if Richard is still active, but the lack of response from him would suggest not. A test on your hypothesis would be to offer to help. I don't know who has access to the server the software runs on, but ultimately any core python committer can change the code in the subversion repository. I did just actually have a look over the code and was happy to find it was a lot simpler than I had feared but a bit dismayed at the style of coding and total lack of unit tests ;-) I might well be up for helping with the maintenance of PyPI if I'm welcome, especially as I'm already familiar with a lot of the components used, although I don't know how much time I can commit :-( Martin, are you interested in help? Chris PS: While I'm sure a lot of python-dev people are interested in this topic, I'm pretty sure this whole huge sprawling thread belongs on catalog-sig... -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From skip at pobox.com Sat Nov 14 01:45:17 2009 From: skip at pobox.com (skip at pobox.com) Date: Fri, 13 Nov 2009 18:45:17 -0600 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> References: <200911130038.05516.steve@pearwood.info> <4222a8490911120831y5233e547pbfd22de60bcf436e@mail.gmail.com> Message-ID: <19197.64925.119992.882456@montanaro.dyndns.org> Jesse> I don't want us to impersonate the mindless, sub-useful drivel Jesse> found in App store/YouTube/etc comments 99% of the time or the Jesse> broken "5 star ratings" systems, etc. It's too easy to game. I implemented a "5-star" system for Musi-Cal back in the day. Now, admittedly, rating musicians is a bit different than rating software, but you would probably be surprised at how much offense one musician can take when they discover that some other musician got a 4.7 rating and they only got a 4.2. :rolleyes: It did take some effort to reduce the chance that the system would be subverted. Similar to a free software site such as PyPI (and unlike a for-profit system such as the iTunes App Store), I think almost all people realized that the rating system was there to help the community and that by polluting the database they only hurt themselves. Skip From solipsis at pitrou.net Sat Nov 14 01:47:29 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 14 Nov 2009 00:47:29 +0000 (UTC) Subject: [Python-Dev] PyPI governance References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFDFD52.4070401@simplistix.co.uk> Message-ID: Chris Withers simplistix.co.uk> writes: > > PS: While I'm sure a lot of python-dev people are interested in this > topic, I'm pretty sure this whole huge sprawling thread belongs on > catalog-sig... Yes, please :) Antoine. From db3l.net at gmail.com Sat Nov 14 02:51:07 2009 From: db3l.net at gmail.com (David Bolen) Date: Fri, 13 Nov 2009 20:51:07 -0500 Subject: [Python-Dev] External module tests skipping on Windows build slaves? Message-ID: While working with the Windows (and now Windows 7) build slaves I've been noticing that a number of "unexpected" test skips are occurring, mostly for external modules like ssl, bz2, tk, etc... However, I know for a fact that those libraries are being fetched and built, so I was a little perplexed that they were being skipped for testing. I don't think it's a regression due to the recent slave work, but something that may have been happening for a while, if not always. Some manual testing I did would seem to indicate that there's a problem related to loading the right MS CRT when loading these modules. But they seem to reference both debug and non-debug versions, so even if I copy the CRT DLLs (from the SxS tre) into pcbuild manually, then I get a CRT assertion about a bad initialization. I suspect this wouldn't be a problem when everything is built in release mode. I would have sworn they used to get run, but now I'm not so sure. Perhaps I'm remembering older Python releases with VS.NET 2003, since the MSVC9 versions of the CRT and the SXS stuff was new with VS 2008 I think. Does anyone happen to know if these tests have ever run recently on the Windows build slaves? -- David From skip at pobox.com Sat Nov 14 02:54:31 2009 From: skip at pobox.com (skip at pobox.com) Date: Fri, 13 Nov 2009 19:54:31 -0600 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <200911130038.05516.steve@pearwood.info> <4222a8490911120606t3e138957kcdb21104261a1ead@mail.gmail.com> Message-ID: <19198.3543.344446.709885@montanaro.dyndns.org> Guido> Of course, as a user, I might not trust a module that has no Guido> reviews or ratings. I suspect the vast majority of projects will never acquire ratings or reviews. Skip From amnorvend at gmail.com Sat Nov 14 02:54:29 2009 From: amnorvend at gmail.com (Jason Baker) Date: Fri, 13 Nov 2009 19:54:29 -0600 Subject: [Python-Dev] PyPI governance In-Reply-To: <4AFDFD52.4070401@simplistix.co.uk> References: <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFDFD52.4070401@simplistix.co.uk> Message-ID: <816c4abb0911131754q38711732rbb9c3bfe43b3d4eb@mail.gmail.com> On Fri, Nov 13, 2009 at 6:44 PM, Chris Withers wrote: > PS: While I'm sure a lot of python-dev people are interested in this topic, > I'm pretty sure this whole huge sprawling thread belongs on catalog-sig... +100 From greg.ewing at canterbury.ac.nz Sat Nov 14 03:23:14 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 14 Nov 2009 15:23:14 +1300 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: <4AFC6440.1040008@v.loewis.de> References: <4AFC6440.1040008@v.loewis.de> Message-ID: <4AFE1492.7050401@canterbury.ac.nz> Martin v. L?wis wrote: > Some of the Python maintainers have recently started objecting to this > setup, asking that the standard library should be split into separate > packages that are released and distributed independent of Python. Others > of us feel strongly that such a change should not be made. I'd be worried, because I would no longer be able to release an app or package and say "requires Python x.y". I'd have to list the version numbers of all the micro packages making up the standard library that I use. Worse, I'd have to be aware of which ones I actually *do* use so I could mantain that list, something I don't have to think about at the moment. It mightn't be so bad if the whole stdlib were a *single* package, but I don't see much advantage in that. -- Greg From ben+python at benfinney.id.au Sat Nov 14 05:00:47 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 14 Nov 2009 15:00:47 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <4AFC9064.20700@v.loewis.de> <20091113221234.GA15732@amk-desktop.matrixgroup.net> <87bpj6t4mp.fsf@benfinney.id.au> <200911141018.13536.steve@pearwood.info> Message-ID: <873a4hu55c.fsf@benfinney.id.au> Steven D'Aprano writes: > But for the life of me, I can't understand the 1/3 of the votes that > have been cast in favour of prohibiting comments for everybody, even > those who want comments. You gave reason (and I agree with you) for why, on a service that allows comments and/or ratings, that users might think less of a project that chooses to be on the service but then chooses not to have comments and/or ratings. Thus, there's a good reason on such a system to *not* participate in the service at all. A barrier to entry, on something that should be *the* place to look for third-party Python packages. To encourage as many third-party packages to register on PyPI, it seems barriers like that should be removed. -- \ ?Please to bathe inside the tub.? ?hotel room, Japan | `\ | _o__) | Ben Finney From rdmurray at bitdance.com Sat Nov 14 05:19:31 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 13 Nov 2009 23:19:31 -0500 (EST) Subject: [Python-Dev] Status of the Buildbot fleet and related bugs In-Reply-To: References: <4AF9E3D6.2000807@v.loewis.de> Message-ID: On Sat, 14 Nov 2009 at 00:09, Antoine Pitrou wrote: > Martin v. L??wis v.loewis.de> writes: >> >>> The buildbots still show occasional oddities. For example, right now in >>> the page "http://www.python.org/dev/buildbot/3.x/", some results have >>> disappeared (the columns for "AMD64 Ubuntu" builders have become empty). >> >> Yes, I noticed it too. It will go away after some page reloads. > > It is still happening more or less randomly unfortunately. > http://www.python.org/dev/buildbot/3.x/ The buildbot pages appear to be pretty messed up now. I get many 404s (ex: the above url, the all stable builders page), although some seem to work (ex: the all builders page), and if I stick an 'all' into the URL for my buildbot page I can get to it, though that's is not the version of the URL linked from the 'all builders' table header. --David (RDM) From db3l.net at gmail.com Sat Nov 14 05:34:29 2009 From: db3l.net at gmail.com (David Bolen) Date: Fri, 13 Nov 2009 23:34:29 -0500 Subject: [Python-Dev] External module tests skipping on Windows build slaves? References: Message-ID: David Bolen writes: > (...) > I would have sworn they used to get run, but now I'm not so sure. > Perhaps I'm remembering older Python releases with VS.NET 2003, since > the MSVC9 versions of the CRT and the SXS stuff was new with VS 2008 I > think. > > Does anyone happen to know if these tests have ever run recently on > the Windows build slaves? Just to follow up - after finding an old 2.6a0 tree I had manually built that worked fine (so clearly it wasn't just VS 2008), I worked my way through some 2.6 tags and this broke as of 2.6.1, so I think the tests have been getting skipped since then. The root problem appears to be the change (r67121 for 2.6 merged from r67120 in trunk) that removed embedded manifests. I'll follow up on the issue tracker. -- David From db3l.net at gmail.com Sat Nov 14 05:40:12 2009 From: db3l.net at gmail.com (David Bolen) Date: Fri, 13 Nov 2009 23:40:12 -0500 Subject: [Python-Dev] Status of the Buildbot fleet and related bugs References: <4AF9E3D6.2000807@v.loewis.de> Message-ID: "R. David Murray" writes: > The buildbot pages appear to be pretty messed up now. I get many 404s > (ex: the above url, the all stable builders page), although some seem to > work (ex: the all builders page), and if I stick an 'all' into the URL > for my buildbot page I can get to it, though that's is not the version > of the URL linked from the 'all builders' table header. Yes, I think this just started happening. I'm guessing that the main site proxies the buildbot URL requests to the buildbot master process, and when it's down you get the 404s from the main server. I figured someone might be working on the master, though perhaps it just burped on its own :-) -- David From martin at v.loewis.de Sat Nov 14 09:43:50 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 14 Nov 2009 09:43:50 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <20091113101906.GA3246@kinakuta.local> References: <200911130038.05516.steve@pearwood.info> <7EAD464A-45AC-41EF-9E11-A9BAA3461DB9@lericson.se> <200911131722.38598.steve@pearwood.info> <87aayqvrif.fsf@benfinney.id.au> <20091113101906.GA3246@kinakuta.local> Message-ID: <4AFE6DC6.60308@v.loewis.de> > What I really like on PyPi is that my packages are tested automatically > with Cheesecake and the order of packages when searching is determined > by this rating. That is an (apparently widespread) myth. The Cheesecake rating is not considered in ranking search results; it's just the relevance of the search term that is. Regards, Martin From martin at v.loewis.de Sat Nov 14 09:45:04 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 14 Nov 2009 09:45:04 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> Message-ID: <4AFE6E10.3050307@v.loewis.de> Paul Moore wrote: > 2009/11/13 Tres Seaver : >> I can see the information about the poll, and a link to view the >> results, without logging in. >> >> http://pypi.python.org/pypi >> >> (second paragraph there). That paragraph tells you that you need to log >> in to vote in the poll. > > I don't want to create a PyPI account (more account details I'll > rarely use to remember) just to vote. I can see why anonymous votes > are bad, but the sample's going to be self-selecting - people like me > who don't want to create an account will be excluded. This is indeed intentional: people like you won't upload packages to PyPI, nor will they take part in the rating system, as both require a PyPI account. Regards, Martin From martin at v.loewis.de Sat Nov 14 09:52:58 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 14 Nov 2009 09:52:58 +0100 Subject: [Python-Dev] PyPI governance In-Reply-To: <4AFDFD52.4070401@simplistix.co.uk> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFDFD52.4070401@simplistix.co.uk> Message-ID: <4AFE6FEA.5070704@v.loewis.de> > Martin, are you interested in help? Certainly, yes. For the specific feature in question, I'd like to wait for the outcome of the poll. Otherwise, contributions are welcome. Regards, Martin From martin at v.loewis.de Sat Nov 14 10:09:44 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 14 Nov 2009 10:09:44 +0100 Subject: [Python-Dev] Status of the Buildbot fleet and related bugs In-Reply-To: References: <4AF9E3D6.2000807@v.loewis.de> Message-ID: <4AFE73D8.9020509@v.loewis.de> > Yes, I think this just started happening. I'm guessing that the main > site proxies the buildbot URL requests to the buildbot master process, > and when it's down you get the 404s from the main server. > > I figured someone might be working on the master, though perhaps it > just burped on its own :-) It was actually an Apache misconfiguration (the wrong virtual host would pick up requests, missing the reverse proxy configuration). I have fixed that now. Regards, Martin From wentland at cl.uni-heidelberg.de Sat Nov 14 14:18:13 2009 From: wentland at cl.uni-heidelberg.de (Wolodja Wentland) Date: Sat, 14 Nov 2009 14:18:13 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4AFE6DC6.60308@v.loewis.de> References: <200911130038.05516.steve@pearwood.info> <7EAD464A-45AC-41EF-9E11-A9BAA3461DB9@lericson.se> <200911131722.38598.steve@pearwood.info> <87aayqvrif.fsf@benfinney.id.au> <20091113101906.GA3246@kinakuta.local> <4AFE6DC6.60308@v.loewis.de> Message-ID: <20091114131813.GA3798@kinakuta.local> On Sat, Nov 14, 2009 at 09:43 +0100, "Martin v. L?wis" wrote: > > What I really like on PyPi is that my packages are tested automatically > > with Cheesecake and the order of packages when searching is determined > > by this rating. > That is an (apparently widespread) myth. The Cheesecake rating is not > considered in ranking search results; it's just the relevance of the > search term that is. Thanks for clarifying this! What about the other points? Crossposted to catalog-sig to move the discussion there. -- .''`. Wolodja Wentland : :' : `. `'` 4096R/CAF14EFC `- 081C B7CD FF04 2BA9 94EA 36B2 8B7F 7D30 CAF1 4EFC -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From martin at v.loewis.de Sat Nov 14 14:22:22 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 14 Nov 2009 14:22:22 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <20091114131813.GA3798@kinakuta.local> References: <200911130038.05516.steve@pearwood.info> <7EAD464A-45AC-41EF-9E11-A9BAA3461DB9@lericson.se> <200911131722.38598.steve@pearwood.info> <87aayqvrif.fsf@benfinney.id.au> <20091113101906.GA3246@kinakuta.local> <4AFE6DC6.60308@v.loewis.de> <20091114131813.GA3798@kinakuta.local> Message-ID: <4AFEAF0E.1040808@v.loewis.de> > >> That is an (apparently widespread) myth. The Cheesecake rating is not >> considered in ranking search results; it's just the relevance of the >> search term that is. > > Thanks for clarifying this! > > What about the other points? They are really off-topic for python-dev; PyPI discussion should take place on catalog-sig. I personally had problems following your message, as it seemed to mix too many aspects into one message. Regards, Martin From pje at telecommunity.com Sat Nov 14 15:39:24 2009 From: pje at telecommunity.com (P.J. Eby) Date: Sat, 14 Nov 2009 09:39:24 -0500 Subject: [Python-Dev] PyPI front page In-Reply-To: References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> Message-ID: <20091114143931.507E33A411A@sparrow.telecommunity.com> At 08:33 PM 11/12/2009 -0500, Ian Bicking wrote: >On Thu, Nov 12, 2009 at 7:52 PM, Antoine Pitrou ><solipsis at pitrou.net> wrote: >Ben Finney benfinney.id.au> writes: > > > > There's a problem with the poll's placement: on the front page of the > > PyPI website. > >Speaking of which, why is it that >http://pypi.python.org/pypi and >http://pypi.python.org/pypi/ (note the >ending slash) return different contents >(the latter being very voluminous)? I always mistake one for the other when >entering the URL directly. > > >easy_install replied on the behavior of /pypi/ (it uses the long >list to do case-insensitive searches).? Someone changed it, >easy_install broke, and a compromise was to keep /pypi/ the way it >was (but not /pypi). > >Probably this could be removed, as the /simple/ index is already >case-insensitive, so easy_install shouldn't have to hit /pypi/ at all. This was changed over a year ago; easy_install *does* use /simple by default. I would guess enough people are upgraded by now that PyPI need no longer continue to support it. From pje at telecommunity.com Sat Nov 14 15:52:21 2009 From: pje at telecommunity.com (P.J. Eby) Date: Sat, 14 Nov 2009 09:52:21 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <4AFE6E10.3050307@v.loewis.de> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> Message-ID: <20091114145222.084583A411A@sparrow.telecommunity.com> At 09:45 AM 11/14/2009 +0100, Martin v. L?wis wrote: >Paul Moore wrote: > > 2009/11/13 Tres Seaver : > >> I can see the information about the poll, and a link to view the > >> results, without logging in. > >> > >> http://pypi.python.org/pypi > >> > >> (second paragraph there). That paragraph tells you that you need to log > >> in to vote in the poll. > > > > I don't want to create a PyPI account (more account details I'll > > rarely use to remember) just to vote. I can see why anonymous votes > > are bad, but the sample's going to be self-selecting - people like me > > who don't want to create an account will be excluded. > >This is indeed intentional: people like you won't upload packages to >PyPI, nor will they take part in the rating system, as both require >a PyPI account. Which is bizarre, since Paul belongs to the group of people you say you care most about - i.e., those people browsing the index and looking for packages. The *consumers* of the comments, in other words. From martin at v.loewis.de Sat Nov 14 16:19:22 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 14 Nov 2009 16:19:22 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <20091114145222.084583A411A@sparrow.telecommunity.com> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> Message-ID: <4AFECA7A.8070400@v.loewis.de> >> > I don't want to create a PyPI account (more account details I'll >> > rarely use to remember) just to vote. I can see why anonymous votes >> > are bad, but the sample's going to be self-selecting - people like me >> > who don't want to create an account will be excluded. >> >> This is indeed intentional: people like you won't upload packages to >> PyPI, nor will they take part in the rating system, as both require >> a PyPI account. > > Which is bizarre, since Paul belongs to the group of people you say you > care most about - i.e., those people browsing the index and looking for > packages. The *consumers* of the comments, in other words. Ok: "intentional" is not the right attribute. It's more that I don't consider it too harmful. Regards, Martin From steve at pearwood.info Sat Nov 14 16:25:54 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 15 Nov 2009 02:25:54 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <20091114145222.084583A411A@sparrow.telecommunity.com> References: <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> Message-ID: <200911150225.55412.steve@pearwood.info> On Sun, 15 Nov 2009 01:52:21 am P.J. Eby wrote: > > > I don't want to create a PyPI account (more account details I'll > > > rarely use to remember) just to vote. I can see why anonymous > > > votes are bad, but the sample's going to be self-selecting - > > > people like me who don't want to create an account will be > > > excluded. > > > >This is indeed intentional: people like you won't upload packages to > >PyPI, nor will they take part in the rating system, as both require > >a PyPI account. > > Which is bizarre, since Paul belongs to the group of people you say > you care most about - i.e., those people browsing the index and > looking for packages. The *consumers* of the comments, in other > words. Not bizarre at all, practical. Without authenticated votes, gaming the system goes from technically challenging to simple enough anyone can do it. Martin may have the best interests of "consumers" in mind, but he can't force them to act in their best interest. If they choose to not vote, that's their choice to make. Think of it as voting registration. Even in countries like Australia with compulsory voting, you have to register first, to ensure that people make a single vote. Since Paul doesn't have an account, he can't make comments, or rate packages, or cast a vote. This is by design, not out of a desire to defranchise people like Paul, but because the alternatives -- easy comment spam, people voting multiple times -- are worse than some proportion of users being unable to vote. -- Steven D'Aprano From floris.bruynooghe at gmail.com Sat Nov 14 16:29:21 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Sat, 14 Nov 2009 15:29:21 +0000 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <4AFC9064.20700@v.loewis.de> <20091113221234.GA15732@amk-desktop.matrixgroup.net> <87bpj6t4mp.fsf@benfinney.id.au> <200911141018.13536.steve@pearwood.info> Message-ID: <20091114152921.GA28003@laurie.devork> On Fri, Nov 13, 2009 at 06:10:16PM -0600, Robert Kern wrote: > If you want one idea that would make my use of PyPI much more > pleasant and informative, it would be to add a "Repository-URL" > entry to the recommended PyPI metadata so that I could always start > looking at the code in one click. +1 Having a "Repository-URL", "Repository-Browse-URL" and a "Bug-Tracker-URL" field in PyPI would be a lot more usefule then comments and ratings. Regards Floris From arcriley at gmail.com Sat Nov 14 16:34:30 2009 From: arcriley at gmail.com (Arc Riley) Date: Sat, 14 Nov 2009 10:34:30 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <20091114152921.GA28003@laurie.devork> References: <4AFC9064.20700@v.loewis.de> <20091113221234.GA15732@amk-desktop.matrixgroup.net> <87bpj6t4mp.fsf@benfinney.id.au> <200911141018.13536.steve@pearwood.info> <20091114152921.GA28003@laurie.devork> Message-ID: > +1 > > Having a "Repository-URL", "Repository-Browse-URL" and a > "Bug-Tracker-URL" field in PyPI would be a lot more usefule then > comments and ratings. > > +1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From baptiste13z at free.fr Sat Nov 14 17:51:59 2009 From: baptiste13z at free.fr (Baptiste Carvello) Date: Sat, 14 Nov 2009 17:51:59 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <4AFC9064.20700@v.loewis.de> <20091113221234.GA15732@amk-desktop.matrixgroup.net> <87bpj6t4mp.fsf@benfinney.id.au> <200911141018.13536.steve@pearwood.info> Message-ID: Robert Kern a ?crit : > While I do have a couple of packages on PyPI, I use PyPI as a consumer > of packages much more frequently, every day in fact. Another "consumer" opinion: when investigating a new package, I usually look for the following things, in that order: 1) the "big picture" description: is there a coherent design? does it fit my needs? On PyPI, the description field should provide that. 2) the changelog: is the project still alive? are bugs fixed, or just features added? is the code rewritten from scratch for each release? Well, every project on PyPI should have a public changelog, so a link to that fits my need. 3) documentation: I don't necessarily care for the number of lines, but more whether it is understandable and goes into sufficient detail to not leave me guessing. Again, a link to the docs fits my needs. A single number (number of documentation lines,...) does not. 4) a mailing list archive (or newsgroup, or web forum), where I'm looking for signs of a healthy community. I usually go for the -devel list, but a -user list will do as well if the committers keep an eye on it. If a healthy community exists around a project, I will completely disregard comments, if present: time invested in the community speaks louder than the opinion of random bystanders. Only for small projects with no real community will I look at the comments (+ answers from the author) in order to make an opinion on the developpement process. I always disregard ratings. So as a conclusion about comments, they can have their use for projects without a publicly archived mailing list, but can otherwise be *replaced* by a direct link to a list archive. This could be a reasonable default for PyPI: disable comments when a link to a list archive is provided. While my experience may not be that of a typical user, I believe users will ultimately make use of all information they are provided. So it is important to provide the most relevant information, and not just what naive users ask for. Cheers, B. From tjreedy at udel.edu Sat Nov 14 21:31:00 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 14 Nov 2009 15:31:00 -0500 Subject: [Python-Dev] Too many Python accounts (was Re: PyPI comments and ratings, *really*?) In-Reply-To: <4AFECA7A.8070400@v.loewis.de> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> Message-ID: Martin v. L?wis wrote: >>> This is indeed intentional: people like you won't upload packages to >>> PyPI, nor will they take part in the rating system, as both require >>> a PyPI account. Why? I already have python tracker account and a python wiki account which is already 2 too many. The latter was a pain because the site lost my registration and as of a year ago, the registration process was broken. I would much prefer just one python site registration. Terry Jan Reedy From ben+python at benfinney.id.au Sat Nov 14 23:23:55 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 15 Nov 2009 09:23:55 +1100 Subject: [Python-Dev] Too many Python accounts References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> Message-ID: <87fx8gsq2s.fsf@benfinney.id.au> Terry Reedy writes: > Martin v. L?wis wrote: > > >>> This is indeed intentional: people like you won't upload packages > >>> to PyPI, nor will they take part in the rating system, as both > >>> require a PyPI account. > > Why? I already have python tracker account and a python wiki account > which is already 2 too many. The latter was a pain because the site > lost my registration and as of a year ago, the registration process > was broken. I would much prefer just one python site registration. And that registration should be using any OpenID, so that I don't need any new identities to participate on the Python sites: I can re-use existing identities. I would make a bug report for that, but the Python bug tracker also requires yet-another-identity. It's lunacy. -- \ ?Value your freedom or you will lose it, teaches history. | `\ ?Don't bother us with politics,? respond those who don't want | _o__) to learn.? ?Richard Stallman, 2002 | Ben Finney From stephen at xemacs.org Sun Nov 15 00:52:34 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 15 Nov 2009 08:52:34 +0900 Subject: [Python-Dev] Too many Python accounts In-Reply-To: <87fx8gsq2s.fsf@benfinney.id.au> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> <87fx8gsq2s.fsf@benfinney.id.au> Message-ID: <874oowbr5p.fsf@uwakimon.sk.tsukuba.ac.jp> Ben Finney writes: > I would make a bug report for that, but the Python bug tracker also > requires yet-another-identity. It's lunacy. No, it's legacy. The software predates the availability of OpenID. If you'd like to integrate Open ID authentication into Roundup, I will personally be happy to do the followup required to get the patch on the radar of the Python site and Roundup software maintainers. From ziade.tarek at gmail.com Sun Nov 15 02:21:41 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 15 Nov 2009 02:21:41 +0100 Subject: [Python-Dev] buildtime vs runtime in Distutils Message-ID: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> Hello, http://bugs.python.org/issue4359 reminds me that Distutils reads build files like Makefile or pyconfig.h to get some environment variables through the sysconfig module at *runtime*. This cannot work on all platforms, when our Makefile is not shipped with python but python-devel. (like Fedora) Since I am already working on the refactoring of this distutils/sysconfig module, so it lands in a stdlib module together with some elements from site.py (see a previous mail on the topic, and my tarek_sysconfig branch - work in progress), I was wondering if we couldn't make this module a template, and inject at its beginning new values extracted from Makefile and pyconfig.h. This could happen when "configure" is called, Regards, Tarek -- Tarek Ziad? | http://ziade.org |????????????! |???????????? From martin at v.loewis.de Sun Nov 15 02:36:26 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 15 Nov 2009 02:36:26 +0100 Subject: [Python-Dev] Too many Python accounts (was Re: PyPI comments and ratings, *really*?) In-Reply-To: References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> Message-ID: <4AFF5B1A.8090105@v.loewis.de> > Why? I already have python tracker account and a python wiki account > which is already 2 too many. The latter was a pain because the site lost > my registration and as of a year ago, the registration process was > broken. I would much prefer just one python site registration. Then I recommend that you get a google account for your email address, and register to PyPI using OpenID. Regards, Martin From martin at v.loewis.de Sun Nov 15 02:36:54 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 15 Nov 2009 02:36:54 +0100 Subject: [Python-Dev] Too many Python accounts In-Reply-To: <87fx8gsq2s.fsf@benfinney.id.au> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> <87fx8gsq2s.fsf@benfinney.id.au> Message-ID: <4AFF5B36.9030502@v.loewis.de> > And that registration should be using any OpenID, so that I don't need > any new identities to participate on the Python sites: I can re-use > existing identities. PyPI actually does support OpenID. Regards, Martin From martin at v.loewis.de Sun Nov 15 02:38:30 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 15 Nov 2009 02:38:30 +0100 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> Message-ID: <4AFF5B96.5030703@v.loewis.de> > http://bugs.python.org/issue4359 reminds me that Distutils reads build > files like Makefile or pyconfig.h to get some environment > variables through the sysconfig module at *runtime*. > > This cannot work on all platforms, when our Makefile is not shipped > with python but python-devel. (like Fedora) I don't see a problem with that: you'll need the python-devel package *anyway* when running distutils, for many packages. Regards, Martin From arfrever.fta at gmail.com Sun Nov 15 02:41:18 2009 From: arfrever.fta at gmail.com (Arfrever Frehtes Taifersar Arahesis) Date: Sun, 15 Nov 2009 02:41:18 +0100 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> Message-ID: <200911150241.26083.Arfrever.FTA@gmail.com> 2009-11-15 02:21:41 Tarek Ziad? napisa?(a): > Hello, > > http://bugs.python.org/issue4359 reminds me that Distutils reads build > files like Makefile or pyconfig.h to get some environment > variables through the sysconfig module at *runtime*. > > This cannot work on all platforms, when our Makefile is not shipped > with python but python-devel. (like Fedora) > > Since I am already working on the refactoring of this > distutils/sysconfig module, so it lands in a stdlib module together > with some > elements from site.py (see a previous mail on the topic, and my > tarek_sysconfig branch - work in progress), > > I was wondering if we couldn't make this module a template, and inject > at its beginning new values extracted from Makefile and pyconfig.h. > > This could happen when "configure" is called, It seems to be a good idea. You should create a .py.in file and use AC_SUBST and AC_CONFIG_FILES macros in configure.in. -- Arfrever Frehtes Taifersar Arahesis -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part. URL: From ben+python at benfinney.id.au Sun Nov 15 02:44:11 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 15 Nov 2009 12:44:11 +1100 Subject: [Python-Dev] Too many Python accounts References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> <87fx8gsq2s.fsf@benfinney.id.au> <874oowbr5p.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87y6m8r28k.fsf@benfinney.id.au> "Stephen J. Turnbull" writes: > Ben Finney writes: > > > I would make a bug report for that, but the Python bug tracker also > > requires yet-another-identity. It's lunacy. > > No, it's legacy. The software predates the availability of OpenID. I don't agree that's a ?no?; it's a ?yes, and? :-) > If you'd like to integrate Open ID authentication into Roundup, I will > personally be happy to do the followup required to get the patch on > the radar of the Python site and Roundup software maintainers. I'll give the usual caveat of ?this is but one of the many services I would like to be able to authenticate in with OpenID?, but it's a caveat rather than a rejection. I'll keep your kind offer in mind in case I get the bandwidth to take that on. -- \ ?Our products just aren't engineered for security.? ?Brian | `\ Valentine, senior vice-president of Microsoft Windows | _o__) development, 2002 | Ben Finney From ziade.tarek at gmail.com Sun Nov 15 02:54:43 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 15 Nov 2009 02:54:43 +0100 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: <4AFF5B96.5030703@v.loewis.de> References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <4AFF5B96.5030703@v.loewis.de> Message-ID: <94bdd2610911141754o37f45987ofa989b5267d282ba@mail.gmail.com> 2009/11/15 "Martin v. L?wis" : >> http://bugs.python.org/issue4359 reminds me that Distutils reads build >> files like Makefile or pyconfig.h to get some environment >> variables through the sysconfig module at *runtime*. >> >> This cannot work on all platforms, when our Makefile is not shipped >> with python but python-devel. (like Fedora) > > I don't see a problem with that: you'll need the python-devel package > *anyway* when running distutils, for many packages. The problem is that the main python distribution ("python") is not working as advertised since it contains distutils, which requires "python-devel" to work. This implies that "python" has a dependency on "python-devel", which does not make sense anymore for linux distros to have two distinct packages for Python. Having some of the makefile vars stored in stdlib solve this problem. Tarek From ziade.tarek at gmail.com Sun Nov 15 02:56:04 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 15 Nov 2009 02:56:04 +0100 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: <200911150241.26083.Arfrever.FTA@gmail.com> References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <200911150241.26083.Arfrever.FTA@gmail.com> Message-ID: <94bdd2610911141756o230e7dd9gd6abd385522454f0@mail.gmail.com> On Sun, Nov 15, 2009 at 2:41 AM, Arfrever Frehtes Taifersar Arahesis wrote: [..] >> >> This could happen when "configure" is called, > > It seems to be a good idea. You should create a .py.in file and use > AC_SUBST and AC_CONFIG_FILES macros in configure.in. I need to investigate on these, thx Tarek From martin at v.loewis.de Sun Nov 15 02:57:12 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 15 Nov 2009 02:57:12 +0100 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: <94bdd2610911141754o37f45987ofa989b5267d282ba@mail.gmail.com> References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <4AFF5B96.5030703@v.loewis.de> <94bdd2610911141754o37f45987ofa989b5267d282ba@mail.gmail.com> Message-ID: <4AFF5FF8.7040708@v.loewis.de> > The problem is that the main python distribution ("python") is not > working as advertised since > it contains distutils, which requires "python-devel" to work. > > This implies that "python" has a dependency on "python-devel", which > does not make sense > anymore for linux distros to have two distinct packages for Python. > > Having some of the makefile vars stored in stdlib solve this problem. I don't see how this can solve the problem. Distutils contains the build_ext command, which definitely will require the devel package. Regards, Martin From ben+python at benfinney.id.au Sun Nov 15 03:11:26 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 15 Nov 2009 13:11:26 +1100 Subject: [Python-Dev] Too many Python accounts References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> Message-ID: <87tywwr0z5.fsf@benfinney.id.au> "Martin v. L?wis" writes: > > And that registration should be using any OpenID, so that I don't > > need any new identities to participate on the Python sites: I can > > re-use existing identities. > > PyPI actually does support OpenID. I commend the PyPI administrators for this step, but the implementation is currently insufficient: it conflates a user's OpenID (their identity, as a URL) with their OpenID provider (the service which the person has chosen to do the authentication check and serve the data). That's what I meant by ?should be using any OpenID?. One of the best features of the OpenID system is identity delegation: that one's identity can be decoupled from the service behind the scenes which provides that identity. This is important, because it means I am not tied to a particular provider to maintain my identity; if they no longer provide my identity in a way I like, I can switch to a different provider while keeping the same identity. Fred can use his own OpenID ?fred.example.org?, initially set up behind the scenes to delegate to ?bigcorp.example.com? as the provider. Any time he likes, Fred can *change* which provider is actually used for authentication, without changing his OpenID. PyPI gets to find out which provider Fred is using for the identity ?fred.example.org? each time it performs discovery on that identity, not before. So, PyPI should not be asking the user ?what is your provider?? since that's (a) a detail irrelevant to the user if they just know their OpenID, (b) liable to change independent of the OpenID, and (c) discoverable from the OpenID auth process anyway. PyPI should instead ask the user for their OpenID (?fred.example.org?), without discussing providers. Then, attempt to authenticate that user, at which point PyPI automatically gets to find out which provider is in use (?bigcorp.example.com?). If you *then* want to be picky and complain that PyPI refuses identities provided by ?bigcorp.example.com?, that's the time to do it. I hope that makes more sense. -- \ ?Geeks like to think that they can ignore politics. You can | `\ leave politics alone, but politics won't leave you alone.? | _o__) ?Richard Stallman, 2002-07-26 | Ben Finney From ziade.tarek at gmail.com Sun Nov 15 03:13:53 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 15 Nov 2009 03:13:53 +0100 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: <4AFF5FF8.7040708@v.loewis.de> References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <4AFF5B96.5030703@v.loewis.de> <94bdd2610911141754o37f45987ofa989b5267d282ba@mail.gmail.com> <4AFF5FF8.7040708@v.loewis.de> Message-ID: <94bdd2610911141813qfc1fa43tb33b19e256819050@mail.gmail.com> 2009/11/15 "Martin v. L?wis" : >> The problem is that the main python distribution ("python") is not >> working as advertised since >> it contains distutils, which requires "python-devel" to work. >> >> This implies that "python" has a dependency on "python-devel", which >> does not make sense >> anymore for linux distros to have two distinct packages for Python. >> >> Having some of the makefile vars stored in stdlib solve this problem. > > I don't see how this can solve the problem. Distutils contains the > build_ext command, which definitely will require the devel package. Sure that's the edge case where files like Python.h is required for compiling C extensions. And compiler errors can make it pretty clear at this level, that the devel package is required. Not because of the need to read a Makefile, but to get the header files to compile. And it should be possible to compile extensions that don't require python-devel at all, when these extensions are not CPython extensions. Last, commands like "install" also requires using sysconfig, (thus reading Makefile) to get some values. I can change this for "install" it's not hard, but we would still provide some public APIs that are reading Makefile from within the stlib. right now, importing "disutils.sysconfig" triggers this behavior. That makes several use case of requiring python-devel just to read a few values that could be injected in the stdlib at build time. Regards Tarek From martin at v.loewis.de Sun Nov 15 03:31:04 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 15 Nov 2009 03:31:04 +0100 Subject: [Python-Dev] Too many Python accounts In-Reply-To: <87tywwr0z5.fsf@benfinney.id.au> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> Message-ID: <4AFF67E8.5060209@v.loewis.de> > Fred can use his own OpenID ?fred.example.org?, initially set up behind > the scenes to delegate to ?bigcorp.example.com? as the provider. Any > time he likes, Fred can *change* which provider is actually used for > authentication, without changing his OpenID. PyPI gets to find out which > provider Fred is using for the identity ?fred.example.org? each time it > performs discovery on that identity, not before. Does that actually work? What actual OpenID provider allows me to claim 'fred.example.org' as my OpenID? Sure, one can use authentication delegation, by means of the openid.delegate link. However, that still doesn't make the claimed identifier fred.example.com, but bigcorp.example.com/fred. So the only thing users gain with delegation is that they don't need to remember the tedious URL that their provider assigns them. When they switch providers, their claimed ID will still change, and they'll have to reregister in all services they use. Please correct me if I'm wrong. Regards, Martin From lists at cheimes.de Sun Nov 15 03:35:00 2009 From: lists at cheimes.de (Christian Heimes) Date: Sun, 15 Nov 2009 03:35:00 +0100 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> Message-ID: <4AFF68D4.9070103@cheimes.de> Tarek Ziad? wrote: > Hello, > > http://bugs.python.org/issue4359 reminds me that Distutils reads build > files like Makefile or pyconfig.h to get some environment > variables through the sysconfig module at *runtime*. > > This cannot work on all platforms, when our Makefile is not shipped > with python but python-devel. (like Fedora) Do we really want to change distutils to solve a problem of a third party packaging system when the problem was created by the very same third party in the first place? In other words: Should you spend your precious development time with fixing a problem that isn't our fault? The decision to split the header files into a separate package, that isn't installed by default, has already created tons of bad user experience in the past. Do you want to endorse the split even further? -0 from me Christian From tjreedy at udel.edu Sun Nov 15 04:02:32 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 14 Nov 2009 22:02:32 -0500 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: <4AFF68D4.9070103@cheimes.de> References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <4AFF68D4.9070103@cheimes.de> Message-ID: Christian Heimes wrote: > Tarek Ziad? wrote: >> Hello, >> >> http://bugs.python.org/issue4359 reminds me that Distutils reads build >> files like Makefile or pyconfig.h to get some environment >> variables through the sysconfig module at *runtime*. >> >> This cannot work on all platforms, when our Makefile is not shipped >> with python but python-devel. (like Fedora) > > Do we really want to change distutils to solve a problem of a third > party packaging system when the problem was created by the very same > third party in the first place? In other words: Should you spend your > precious development time with fixing a problem that isn't our fault? > The decision to split the header files into a separate package, that > isn't installed by default, has already created tons of bad user > experience in the past. Do you want to endorse the split even further? > > -0 from me If the third party disables distutils by removing some of the things it (sometimes) needs, then it seems to me that they should also remove disutils into the separate package. If *that* creates a problem, that should be *their* problem. From ncoghlan at gmail.com Sun Nov 15 04:06:48 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 15 Nov 2009 13:06:48 +1000 Subject: [Python-Dev] Using itertools in modules that are part of the build chain (Re: [Python-checkins] r76264 - python/branches/py3k/Lib/tokenize.py) In-Reply-To: <4afeda77.0702d00a.4fa1.488cSMTPIN_ADDED@mx.google.com> References: <4afeda77.0702d00a.4fa1.488cSMTPIN_ADDED@mx.google.com> Message-ID: <4AFF7048.7070701@gmail.com> benjamin.peterson wrote: > Modified: python/branches/py3k/Lib/tokenize.py > ============================================================================== > --- python/branches/py3k/Lib/tokenize.py (original) > +++ python/branches/py3k/Lib/tokenize.py Sat Nov 14 17:27:26 2009 > @@ -377,17 +377,12 @@ > The first token sequence will always be an ENCODING token > which tells you which encoding was used to decode the bytes stream. > """ > + # This import is here to avoid problems when the itertools module is not > + # built yet and tokenize is imported. > + from itertools import chain This is probably a bad idea - calling tokenize.tokenize() from a thread started as a side effect of importing a module will now deadlock on the import lock if the module import waits for that thread to finish. We tell people not to do that (starting and then waiting on threads as part of module import) for exactly this reason, but it is also the reason we avoid embedding import statements inside functions in the standard library (not to mention encouraging third party developers to also avoid embedding import statements inside functions). This does constrain where we can use itertools - if we want carte blanche to use it anywhere in the standard library, even those parts that are imported as part of the build chain, we'll need to bite the bullet and make it a builtin module rather than a separately built extension module. Cheers, Nick. P.S. The problem is easy to demonstrate on the current Py3k branch: 1. Put this in a module file in your py3k directory (e.g. "deadlock.py"): ----------- import threading import tokenize f = open(__file__, 'rU') def _deadlocks(): tokenize.tokenize(f.readline) t = threading.Thread(target=_deadlocks) t.start() t.join() ----------- 2. Then run: ./python -c "import deadlock" It will, as advertised, deadlock and you'll need to use Ctrl-Brk or kill -9 to get rid of it. (Note that preventing this kind of thing is one of the major reasons why direct execution and even the -m switch *don't* hang onto the import lock while running the __main__ module) -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From tjreedy at udel.edu Sun Nov 15 04:09:54 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 14 Nov 2009 22:09:54 -0500 Subject: [Python-Dev] Too many Python accounts (was Re: PyPI comments and ratings, *really*?) In-Reply-To: <4AFF5B1A.8090105@v.loewis.de> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> <4AFF5B1A.8090105@v.loewis.de> Message-ID: Martin v. L?wis wrote: >> Why? I already have python tracker account and a python wiki account >> which is already 2 too many. The latter was a pain because the site lost >> my registration and as of a year ago, the registration process was >> broken. I would much prefer just one python site registration. > > Then I recommend that you get a google account for your email address, > and register to PyPI using OpenID. OK, that worked after I hit the button a second time. Good enough. From ben+python at benfinney.id.au Sun Nov 15 04:53:57 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 15 Nov 2009 14:53:57 +1100 Subject: [Python-Dev] Too many Python accounts References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de> Message-ID: <87my2oqw8a.fsf@benfinney.id.au> "Martin v. L?wis" writes: > > Fred can use his own OpenID ?fred.example.org?, initially set up > > behind the scenes to delegate to ?bigcorp.example.com? as the > > provider. Any time he likes, Fred can *change* which provider is > > actually used for authentication, without changing his OpenID. PyPI > > gets to find out which provider Fred is using for the identity > > ?fred.example.org? each time it performs discovery on that identity, > > not before. > > Does that actually work? What actual OpenID provider allows me to > claim 'fred.example.org' as my OpenID? It's inherent in the protocol, and any standards-conformant provider supports it. If Fred controls ?fred.example.org? (i.e. can cause that URL to serve a document of his choosing), then Fred gets to delegate to any OpenID provider, and can change it at any time while keeping the same identity URL. If Albert, on the other hand, doesn't want to set up a URL under his control but instead just use an identity managed by someone else, that's a perfectly valid option: he can use ?bigcorp.example.com/albert? as his OpenID, and BigCorp will take care of being the endpoint as well. Albert is not getting any benefit from identity delegation (he's going to have a transition difficulty at some later date when he decides BigCorp isn't serving his identity needs any more), but neither is he required to know about delegation or other protocol details. > Sure, one can use authentication delegation, by means of the > openid.delegate link. However, that still doesn't make the claimed > identifier fred.example.com, but bigcorp.example.com/fred. That's not right (this is what I mean by unnecessarily conflating ?who provides the identity this time? with ?what is the identity?). The ?bigcorp.example.com/fred? URL is *not* Fred's identity in our example, it is only the ?endpoint URL?: where Fred has configured his delegation to go. Having control over ?fred.example.org?, Fred configures delegation, and then can happily forget the endpoint URL in his daily activities. When asked for his OpenID, Fred uses ?fred.example.org?. Albert, on the other hand, has punted on the whole issue of delegation; he uses ?bigcorp.example.com/albert? as his OpenID, because BigCorp causes that URL to be both an OpenID and an endpoint. That's still fine as far as the authentication process is concerned. But the relying party (the party requesting an OpenID from the user; in this case the relying party is PyPI) should not ask the user what their provider is. The user is asked only for their OpenID: Fred will provide ?fred.example.org?, Albert will provide ?bigcorp.example.com/albert?. PyPI discovers automatically, each time they use those identities, where the provider is this time by using the OpenID discovery step on the identity they provide. Fred is very happy for this, since he can keep using ?fred.example.org? even if he later changes which provider he's delegating to. Albert is very happy for this, because he doesn't have to know anything about ?provider? or ?delegation? or ?endpoint?, he just needs to remember his OpenID. PyPI's user interface becomes simpler, because it only needs to ask the user ?what is your OpenID?? and the rest is taken care of by the OpenID protocol implementation. > Please correct me if I'm wrong. I hope that helps. The documents at may be useful if you need more specifics. -- \ ?Computer perspective on Moore's Law: Human effort becomes | `\ twice as expensive roughly every two years.? ?anonymous | _o__) | Ben Finney From benjamin at python.org Sun Nov 15 05:01:07 2009 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 14 Nov 2009 22:01:07 -0600 Subject: [Python-Dev] [Python-checkins] Using itertools in modules that are part of the build chain (Re: r76264 - python/branches/py3k/Lib/tokenize.py) In-Reply-To: <4AFF7048.7070701@gmail.com> References: <4afeda77.0702d00a.4fa1.488cSMTPIN_ADDED@mx.google.com> <4AFF7048.7070701@gmail.com> Message-ID: <1afaf6160911142001m39d0f0ceyd3646e922d3bf703@mail.gmail.com> 2009/11/14 Nick Coghlan : > This does constrain where we can use itertools - if we want carte > blanche to use it anywhere in the standard library, even those parts > that are imported as part of the build chain, we'll need to bite the > bullet and make it a builtin module rather than a separately built > extension module. I have another unpleasant but slightly less hacky solution. We put detect_encoding in linecache where it is actually used. -- Regards, Benjamin From ncoghlan at gmail.com Sun Nov 15 07:04:36 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 15 Nov 2009 16:04:36 +1000 Subject: [Python-Dev] Modules that run other scripts Message-ID: <4AFF99F4.9000405@gmail.com> I'm in the process of adding a "run_path" function to the runpy module that allows Python code to execute scripts using the same rules as the CPython command line (i.e. accepting both source and compiled Python files in addition to zipfiles, directories and other valid sys.path entries containing a __main__.py file). After that I was considering looking at the standard library to see which modules can be used to execute other scripts (e.g. pdb, profile) and determine what would be involved in updating them to support the same flexibility as the main command line (possibly including adding their own "-m" option to run modules by name rather than file path location). For that second part: 1. Is it even worth doing at this stage? I'm not sure to what degree the new command line flexibility has even been adopted by third party application packagers, so I have no idea how large the pool of potential users might be. Should I instead wait until we start seeing complaints that these tools can't be used with script references that the main command line will handle quite happily? 2. Aside from runpy itself, pdb and profile are the only examples that spring to mind when I try to think of standard library modules that execute other Python scripts. Are there any others that I'm missing? (Even if we decide not to do anything at this stage, collating such a list may still be handy for future reference) Regards, Nick. P.S. pdb in particular may be messy to update, since the interaction between the way it provides exception post-mortem debugging support and the way the runpy module tries to avoid mutating the application's own __main__ module will likely require careful handling. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From robert.kern at gmail.com Sun Nov 15 07:37:00 2009 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 15 Nov 2009 00:37:00 -0600 Subject: [Python-Dev] Modules that run other scripts In-Reply-To: <4AFF99F4.9000405@gmail.com> References: <4AFF99F4.9000405@gmail.com> Message-ID: Nick Coghlan wrote: > For that second part: > 1. Is it even worth doing at this stage? I'm not sure to what degree the > new command line flexibility has even been adopted by third party > application packagers, so I have no idea how large the pool of potential > users might be. Should I instead wait until we start seeing complaints > that these tools can't be used with script references that the main > command line will handle quite happily? There is a small, but important class of "scripts that run scripts", which are mostly all development tools (e.g. coverage, my line_profiler, etc.). Doing this correctly in all of the corner cases is reasonably tricky, so I think this is a perfect case for having the functionality implemented once in the standard library. For what its worth, I think Ned Batchelder did the most thorough job of implementing this in the latest version of coverage: http://bitbucket.org/ned/coveragepy/src/tip/coverage/execfile.py -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ncoghlan at gmail.com Sun Nov 15 08:02:47 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 15 Nov 2009 17:02:47 +1000 Subject: [Python-Dev] Modules that run other scripts In-Reply-To: References: <4AFF99F4.9000405@gmail.com> Message-ID: <4AFFA797.4040907@gmail.com> Robert Kern wrote: > Nick Coghlan wrote: > >> For that second part: >> 1. Is it even worth doing at this stage? I'm not sure to what degree the >> new command line flexibility has even been adopted by third party >> application packagers, so I have no idea how large the pool of potential >> users might be. Should I instead wait until we start seeing complaints >> that these tools can't be used with script references that the main >> command line will handle quite happily? > > There is a small, but important class of "scripts that run scripts", > which are mostly all development tools (e.g. coverage, my line_profiler, > etc.). Doing this correctly in all of the corner cases is reasonably > tricky, so I think this is a perfect case for having the functionality > implemented once in the standard library. Yep, that part I'm convinced of the need for - hence runpy.run_path. It's whether the immediate demand is there to justify tinkering with the existing tools in the standard library that I'm not sure about (particularly when there is a risk of altering behaviour in some corner cases). > For what its worth, I think Ned Batchelder did the most thorough job of > implementing this in the latest version of coverage: > > http://bitbucket.org/ned/coveragepy/src/tip/coverage/execfile.py That's actually fairly similar to what run_path will do in the simple script case (zipfile and directory execution is a fair bit messier in practice, but based on a similar concept in principle). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From fuzzyman at voidspace.org.uk Sun Nov 15 12:41:23 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 15 Nov 2009 11:41:23 +0000 Subject: [Python-Dev] Too many Python accounts In-Reply-To: <4AFF67E8.5060209@v.loewis.de> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de> Message-ID: <4AFFE8E3.5020508@voidspace.org.uk> Martin v. L?wis wrote: >> Fred can use his own OpenID ?fred.example.org?, initially set up behind >> the scenes to delegate to ?bigcorp.example.com? as the provider. Any >> time he likes, Fred can *change* which provider is actually used for >> authentication, without changing his OpenID. PyPI gets to find out which >> provider Fred is using for the identity ?fred.example.org? each time it >> performs discovery on that identity, not before. >> > > Does that actually work? What actual OpenID provider allows me to claim > 'fred.example.org' as my OpenID? Sure, one can use authentication > delegation, by means of the openid.delegate link. However, that still > doesn't make the claimed identifier fred.example.com, but > bigcorp.example.com/fred. > > So the only thing users gain with delegation is that they don't need > to remember the tedious URL that their provider assigns them. When they > switch providers, their claimed ID will still change, and they'll have > to reregister in all services they use. > > No, the whole point of delegation is that I can use voidspace.org.uk as my openid - backed with any provider I want. I can then use voidspace.org.uk as my openid and not be tied to any provider. I'm afraid the PyPI implementation of openid is useless to me too - I want to use voidspace.org.uk as my openid but it doesn't let me. All the best, Michael > Please correct me if I'm wrong. > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From solipsis at pitrou.net Sun Nov 15 13:42:53 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 15 Nov 2009 12:42:53 +0000 (UTC) Subject: [Python-Dev] buildtime vs runtime in Distutils References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> Message-ID: Tarek Ziad? gmail.com> writes: > > This cannot work on all platforms, when our Makefile is not shipped > with python but python-devel. (like Fedora) This practice is stupid anyway, because it means you have to install python-devel even to install pure Python packages with setuptools/distribute. Just ask Fedora, Mandriva and friends to change their packaging practice (Mandriva already has a bug open for that by the way). In Debian/Ubuntu, the Makefile is correctly part of the main Python package: $ dpkg -S /usr/lib/python2.5/config/Makefile python2.5: /usr/lib/python2.5/config/Makefile Regards Antoine. From ziade.tarek at gmail.com Sun Nov 15 14:30:34 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 15 Nov 2009 14:30:34 +0100 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: <4AFF68D4.9070103@cheimes.de> References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <4AFF68D4.9070103@cheimes.de> Message-ID: <94bdd2610911150530y3191a192mb00d910e30e01c2e@mail.gmail.com> On Sun, Nov 15, 2009 at 3:35 AM, Christian Heimes wrote: [..] > Do we really want to change distutils to solve a problem of a third > party packaging system when the problem was created by the very same > third party in the first place? In other words: Should you spend your > precious development time with fixing a problem that isn't our fault? > The decision to split the header files into a separate package, that > isn't installed by default, has already created tons of bad user > experience in the past. Do you want to endorse the split even further? I didn't know the split story went like this. I took it like the "natural" split everyone agreed on, and I saw this distutils <-> Makefile link like something to fix. So, it sounds like a bad idea now :) Tarek. From g.brandl at gmx.net Sun Nov 15 14:31:45 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 15 Nov 2009 14:31:45 +0100 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> Message-ID: Antoine Pitrou schrieb: > Tarek Ziad? gmail.com> writes: >> >> This cannot work on all platforms, when our Makefile is not shipped >> with python but python-devel. (like Fedora) > > This practice is stupid anyway, because it means you have to install > python-devel even to install pure Python packages with setuptools/distribute. > Just ask Fedora, Mandriva and friends to change their packaging practice > (Mandriva already has a bug open for that by the way). +1. They are the ones splitting what "make install" installs into several packages, so they are the ones who have to fix the resulting dependency problems. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ziade.tarek at gmail.com Sun Nov 15 14:32:19 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 15 Nov 2009 14:32:19 +0100 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <4AFF68D4.9070103@cheimes.de> Message-ID: <94bdd2610911150532w2a91d0c3u61106231025bd6cd@mail.gmail.com> On Sun, Nov 15, 2009 at 4:02 AM, Terry Reedy wrote: > Christian Heimes wrote: >> >> Tarek Ziad? wrote: >>> >>> Hello, >>> >>> http://bugs.python.org/issue4359 reminds me that Distutils reads build >>> files like Makefile or pyconfig.h to get some environment >>> variables through the sysconfig module at *runtime*. >>> >>> This cannot work on all platforms, when our Makefile is not shipped >>> with python but python-devel. (like Fedora) >> >> Do we really want to change distutils to solve a problem of a third >> party packaging system when the problem was created by the very same >> third party in the first place? In other words: Should you spend your >> precious development time with fixing a problem that isn't our fault? >> The decision to split the header files into a separate package, that >> isn't installed by default, has already created tons of bad user >> experience in the past. Do you want to endorse the split even further? >> >> -0 from me > > If the third party disables distutils by removing some of the things it > (sometimes) needs, then it seems to me that they should also remove disutils > into the separate package. If *that* creates a problem, that should be > *their* problem. Ok. Fair enough, I'll work with them this way. Regards Tarek From g.brandl at gmx.net Sun Nov 15 14:33:34 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 15 Nov 2009 14:33:34 +0100 Subject: [Python-Dev] Status of the Buildbot fleet and related bugs In-Reply-To: <4AFE73D8.9020509@v.loewis.de> References: <4AF9E3D6.2000807@v.loewis.de> <4AFE73D8.9020509@v.loewis.de> Message-ID: Martin v. L?wis schrieb: >> Yes, I think this just started happening. I'm guessing that the main >> site proxies the buildbot URL requests to the buildbot master process, >> and when it's down you get the 404s from the main server. >> >> I figured someone might be working on the master, though perhaps it >> just burped on its own :-) > > It was actually an Apache misconfiguration (the wrong virtual host would > pick up requests, missing the reverse proxy configuration). I have fixed > that now. BTW, I noticed that the "cancel build" and "cancel all builds" buttons result in unhandled exceptions. (One build is cancelled nevertheless, so for "cancel build" it's just an inconvenience.) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From cournape at gmail.com Sun Nov 15 14:38:37 2009 From: cournape at gmail.com (David Cournapeau) Date: Sun, 15 Nov 2009 22:38:37 +0900 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: <94bdd2610911150532w2a91d0c3u61106231025bd6cd@mail.gmail.com> References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <4AFF68D4.9070103@cheimes.de> <94bdd2610911150532w2a91d0c3u61106231025bd6cd@mail.gmail.com> Message-ID: <5b8d13220911150538r380c24d1ic107c69b1e68526c@mail.gmail.com> On Sun, Nov 15, 2009 at 10:32 PM, Tarek Ziad? wrote: > > Ok. Fair enough, I'll work with them this way. Although packagers should certainly fix the problems they introduce in the first place, the second suggestion in the bug report would be useful, independently on how linux distributions package things. Especially if the data can be obtained for every build (autoconf and VS-based), this would help packages which use something else than distutils for their build. David From martin at v.loewis.de Sun Nov 15 19:57:04 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 15 Nov 2009 19:57:04 +0100 Subject: [Python-Dev] Too many Python accounts In-Reply-To: <4AFFE8E3.5020508@voidspace.org.uk> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk> Message-ID: <4B004F00.7080405@v.loewis.de> >> So the only thing users gain with delegation is that they don't need >> to remember the tedious URL that their provider assigns them. When they >> switch providers, their claimed ID will still change, and they'll have >> to reregister in all services they use. >> >> > No, the whole point of delegation is that I can use voidspace.org.uk as > my openid - backed with any provider I want. I can then use > voidspace.org.uk as my openid and not be tied to any provider. I'm fairly skeptical that you actually do use voidspace.org.uk. I can believe that this is what you type into the openid field - but the ID that is validated to the relying party is http://fuzzyman.myopenid.com/. So when you change the provider (and hence the openid.delegate value), you will have to create new accounts for all services that you use, right? > I'm afraid the PyPI implementation of openid is useless to me too - I > want to use voidspace.org.uk as my openid but it doesn't let me. See above - the services may let you *enter* that string, but it isn't your openid. Regards, Martin From fuzzyman at voidspace.org.uk Sun Nov 15 20:18:43 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 15 Nov 2009 19:18:43 +0000 Subject: [Python-Dev] Too many Python accounts In-Reply-To: <4B004F00.7080405@v.loewis.de> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk> <4B004F00.7080405@v.loewis.de> Message-ID: <4B005413.3010702@voidspace.org.uk> Martin v. L?wis wrote: >>> So the only thing users gain with delegation is that they don't need >>> to remember the tedious URL that their provider assigns them. When they >>> switch providers, their claimed ID will still change, and they'll have >>> to reregister in all services they use. >>> >>> >>> >> No, the whole point of delegation is that I can use voidspace.org.uk as >> my openid - backed with any provider I want. I can then use >> voidspace.org.uk as my openid and not be tied to any provider. >> > > I'm fairly skeptical that you actually do use voidspace.org.uk. > I can believe that this is what you type into the openid field - > but the ID that is validated to the relying party is > http://fuzzyman.myopenid.com/. > > So when you change the provider (and hence the openid.delegate > value), you will have to create new accounts for all services > that you use, right? > Well, when I login my registered ID is www.voidspace.org.uk and *not* fuzzyman.myopenid.com - so I believe you are incorrect (and in fact this very point was touted as one of the advantages of openid - that your account is independent of your provider and that you *can* change provider whilst retaining the same id). myopenid *does* the validation, but my registered openid is www.voidspace.org.uk and I *should* be able to change provider without creating a new account. Michael > >> I'm afraid the PyPI implementation of openid is useless to me too - I >> want to use voidspace.org.uk as my openid but it doesn't let me. >> > > See above - the services may let you *enter* that string, but it > isn't your openid. > > Regards, > Martin > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From martin at v.loewis.de Sun Nov 15 20:31:51 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 15 Nov 2009 20:31:51 +0100 Subject: [Python-Dev] Too many Python accounts In-Reply-To: <4B005413.3010702@voidspace.org.uk> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk> <4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk> Message-ID: <4B005727.4050004@v.loewis.de> > Well, when I login my registered ID is www.voidspace.org.uk and *not* > fuzzyman.myopenid.com - so I believe you are incorrect (and in fact this > very point was touted as one of the advantages of openid - that your > account is independent of your provider and that you *can* change > provider whilst retaining the same id). On the wire (between relying party and provider), voidspace.org.co.uk does never appear. From the OpenID 1.1 specification: # Now, when a Consumer sees that, it'll talk to # http://www.livejournal.com/openid/server.bml and ask if the End User # is exampleuser.livejournal.com, never mentioning www.example.com # anywhere on the wire. So all I (as a relying party) get verifyied is fuzzyman.myopenid.com. Why should I trust that voidspace.org.uk is actually a valid ID? Can't you then produce hundreds of IDs, all delegating to the same identity? IOW, why should I (as a relying party) pay any attention to the ID that you entered, rather than to what I get actually validated? Regards, Martin From ezio.melotti at gmail.com Sun Nov 15 20:55:00 2009 From: ezio.melotti at gmail.com (Ezio Melotti) Date: Sun, 15 Nov 2009 21:55:00 +0200 Subject: [Python-Dev] global statements outside functions/methods should raise SyntaxError Message-ID: <4B005C94.8060707@gmail.com> Python currently accepts global statements at the top level: >>> global foo >>> Beside being a meaningless operation, this might lead unexperienced user to make mistakes like: >>> foo = 5 >>> global foo # make foo global >>> def func(): ... print foo # access the global foo ... >>> func() 5 >>> # it works! "global foo" should raise a SyntaxError, similarly to what already happens with "return": >>> return foo File "", line 1 SyntaxError: 'return' outside function I opened an issue on the tracker (http://bugs.python.org/issue7329) and Benjamin suggested to discuss this here. The test he mentioned is in test_global.py: def test4(self): prog_text_4 = """\ global x x = 2 """ # this should work compile(prog_text_4, "", "exec") It just says that "it should work" but it doesn't say /why/. Any thoughts? From fuzzyman at voidspace.org.uk Sun Nov 15 20:59:43 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 15 Nov 2009 19:59:43 +0000 Subject: [Python-Dev] global statements outside functions/methods should raise SyntaxError In-Reply-To: <4B005C94.8060707@gmail.com> References: <4B005C94.8060707@gmail.com> Message-ID: <4B005DAF.8050904@voidspace.org.uk> Ezio Melotti wrote: > Python currently accepts global statements at the top level: >>>> global foo >>>> > > Beside being a meaningless operation, this might lead unexperienced > user to make mistakes like: >>>> foo = 5 >>>> global foo # make foo global >>>> def func(): > ... print foo # access the global foo > ... >>>> func() > 5 >>>> # it works! > > "global foo" should raise a SyntaxError, similarly to what already > happens with "return": >>>> return foo > File "", line 1 > SyntaxError: 'return' outside function > > > I opened an issue on the tracker (http://bugs.python.org/issue7329) > and Benjamin suggested to discuss this here. > The test he mentioned is in test_global.py: > > def test4(self): > prog_text_4 = """\ > global x > x = 2 > """ > # this should work > compile(prog_text_4, "", "exec") > > It just says that "it should work" but it doesn't say /why/. Well, personally I think it would be a good thing if this raised an exception during bytecode compilation - but it would fall under the moratorium and have to wait a few years. On the other hand it should be easy to get PyLint to include a checker for this and they are very responsive to feature requests. All the best, Michael > > Any thoughts? > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From benjamin at python.org Sun Nov 15 21:08:49 2009 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 15 Nov 2009 14:08:49 -0600 Subject: [Python-Dev] global statements outside functions/methods should raise SyntaxError In-Reply-To: <4B005DAF.8050904@voidspace.org.uk> References: <4B005C94.8060707@gmail.com> <4B005DAF.8050904@voidspace.org.uk> Message-ID: <1afaf6160911151208h6ff65337vca988b0a1e2f0707@mail.gmail.com> 2009/11/15 Michael Foord : > Well, personally I think it would be a good thing if this raised an > exception during bytecode compilation - but it would fall under the > moratorium and have to wait a few years. It could probably be considered a bug, though, since the global statement in that case silently has absolutely no effect. -- Regards, Benjamin From carey.tilden at gmail.com Sun Nov 15 21:20:05 2009 From: carey.tilden at gmail.com (Carey Tilden) Date: Sun, 15 Nov 2009 12:20:05 -0800 Subject: [Python-Dev] Too many Python accounts In-Reply-To: <4B005727.4050004@v.loewis.de> References: <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk> <4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk> <4B005727.4050004@v.loewis.de> Message-ID: On Sun, Nov 15, 2009 at 11:31 AM, "Martin v. L?wis" wrote: > > > Well, when I login my registered ID is www.voidspace.org.uk and *not* > > fuzzyman.myopenid.com - so I believe you are incorrect (and in fact this > > very point was touted as one of the advantages of openid - that your > > account is independent of your provider and that you *can* change > > provider whilst retaining the same id). > > On the wire (between relying party and provider), voidspace.org.co.uk > does never appear. From the OpenID 1.1 specification: > > # Now, when a Consumer sees that, it'll talk to > # http://www.livejournal.com/openid/server.bml and ask if the End User > # is exampleuser.livejournal.com, never mentioning www.example.com > # anywhere on the wire. > > So all I (as a relying party) get verifyied is fuzzyman.myopenid.com. > Why should I trust that voidspace.org.uk is actually a valid ID? Since the user entered voidspace.org.uk, they presumably believe it's an address they control. You have to assume they delegated to another provider on purpose. > Can't you then produce hundreds of IDs, all delegating to the same > identity? Yes. > IOW, why should I (as a relying party) pay any attention to the ID > that you entered, rather than to what I get actually validated? Because the user entered the value they wanted as their identity. This is the reason delegation even exists in the spec. In fact, the very next line after the section you quoted is: # The main advantage of this is that an End User can keep their Identifier # over many years, even as services come and go; they'll just keep # changing who they delegate to. If the provider dictates the identity, as you keep insisting, that sentence makes no sense whatsoever. The value entered as the identifier is the identifier you should use. Otherwise, what's the point of delegation at all? > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/carey.tilden%40gmail.com From solipsis at pitrou.net Sun Nov 15 21:23:43 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 15 Nov 2009 20:23:43 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?global_statements_outside_functions/method?= =?utf-8?q?s_should=09raise_SyntaxError?= References: <4B005C94.8060707@gmail.com> <4B005DAF.8050904@voidspace.org.uk> <1afaf6160911151208h6ff65337vca988b0a1e2f0707@mail.gmail.com> Message-ID: Benjamin Peterson python.org> writes: > > 2009/11/15 Michael Foord voidspace.org.uk>: > > Well, personally I think it would be a good thing if this raised an > > exception during bytecode compilation - but it would fall under the > > moratorium and have to wait a few years. > > It could probably be considered a bug, though, since the global > statement in that case silently has absolutely no effect. Indeed, and it's not like other implementations would be at a disadvantage if they didn't implement this error. Warning that the construct is meaningless can be helpful, especially for refugees from other languages. Regards Antoine. From skip at pobox.com Sun Nov 15 21:27:28 2009 From: skip at pobox.com (skip at pobox.com) Date: Sun, 15 Nov 2009 14:27:28 -0600 Subject: [Python-Dev] Too many Python accounts (was Re: PyPI comments and ratings, *really*?) In-Reply-To: <4AFF5B1A.8090105@v.loewis.de> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> <4AFF5B1A.8090105@v.loewis.de> Message-ID: <19200.25648.998776.238480@montanaro.dyndns.org> Martin> Then I recommend that you get a google account for your email Martin> address, and register to PyPI using OpenID. I've never found OpenID at all intuitive to use. Are there instructions on pypi.python.org which detail the steps necessary to use OpenID to login? I saw the "Claim OpenID" link on my PyPI profile page. So now I have an OpenID URL. What am I supposed to do with that? If I visit that URL it downloads a small bit of XML. I've tried using my Yahoo! and Luanchpad OpenIDs for other sites in the past. I've never successfully logged into any website with them, at least not as far as I can recall. I realize that maybe this is something that just doesn't click with me (maybe I'm an OpenID Luddite), but it seems to me that OpenID needs to be a bit easier (or obvious?) to use if it's to become some sort of de facto standard login mechanism. Skip From python at mrabarnett.plus.com Sun Nov 15 21:31:39 2009 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 15 Nov 2009 20:31:39 +0000 Subject: [Python-Dev] global statements outside functions/methods should raise SyntaxError In-Reply-To: <1afaf6160911151208h6ff65337vca988b0a1e2f0707@mail.gmail.com> References: <4B005C94.8060707@gmail.com> <4B005DAF.8050904@voidspace.org.uk> <1afaf6160911151208h6ff65337vca988b0a1e2f0707@mail.gmail.com> Message-ID: <4B00652B.6010008@mrabarnett.plus.com> Benjamin Peterson wrote: > 2009/11/15 Michael Foord : >> Well, personally I think it would be a good thing if this raised an >> exception during bytecode compilation - but it would fall under the >> moratorium and have to wait a few years. > > It could probably be considered a bug, though, since the global > statement in that case silently has absolutely no effect. > It wouldn't be adding a new feature, but it would be changing the behaviour of programs that currently have 'global' outside a function. Perhaps, due to the moratorium, there should be a warning before it's actually corrected. From tjreedy at udel.edu Sun Nov 15 21:38:26 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 15 Nov 2009 15:38:26 -0500 Subject: [Python-Dev] global statements outside functions/methods should raise SyntaxError In-Reply-To: <4B005C94.8060707@gmail.com> References: <4B005C94.8060707@gmail.com> Message-ID: Ezio Melotti wrote: > Python currently accepts global statements at the top level: > > I opened an issue on the tracker (http://bugs.python.org/issue7329) > and Benjamin suggested to discuss this here. > The test he mentioned is in test_global.py: > > def test4(self): > prog_text_4 = """\ > global x > x = 2 > """ > # this should work > compile(prog_text_4, "", "exec") > > It just says that "it should work" but it doesn't say /why/. > > Any thoughts? I make the same suggestion a couple of years ago, either this list or Py3k list, after newby reported 'problem' on python-list expecting module-level global to do something. Guido rejected it on the basis that he wanted to minimized differences between module code and function code. From brett at python.org Sun Nov 15 21:40:32 2009 From: brett at python.org (Brett Cannon) Date: Sun, 15 Nov 2009 12:40:32 -0800 Subject: [Python-Dev] [Python-checkins] Using itertools in modules that are part of the build chain (Re: r76264 - python/branches/py3k/Lib/tokenize.py) In-Reply-To: <1afaf6160911142001m39d0f0ceyd3646e922d3bf703@mail.gmail.com> References: <4afeda77.0702d00a.4fa1.488cSMTPIN_ADDED@mx.google.com> <4AFF7048.7070701@gmail.com> <1afaf6160911142001m39d0f0ceyd3646e922d3bf703@mail.gmail.com> Message-ID: On Sat, Nov 14, 2009 at 20:01, Benjamin Peterson wrote: > 2009/11/14 Nick Coghlan : >> This does constrain where we can use itertools - if we want carte >> blanche to use it anywhere in the standard library, even those parts >> that are imported as part of the build chain, we'll need to bite the >> bullet and make it a builtin module rather than a separately built >> extension module. > > I have another unpleasant but slightly less hacky solution. We put > detect_encoding in linecache where it is actually used. Well, it happens to be used by the standard library in linecache, but not all external uses of it necessarily tie into linecache (e.g. importlib uses detect_encoding() in some non-critical code). Might just have to live with sub-optimal code. -Brett From benjamin at python.org Sun Nov 15 21:43:10 2009 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 15 Nov 2009 14:43:10 -0600 Subject: [Python-Dev] [Python-checkins] Using itertools in modules that are part of the build chain (Re: r76264 - python/branches/py3k/Lib/tokenize.py) In-Reply-To: References: <4afeda77.0702d00a.4fa1.488cSMTPIN_ADDED@mx.google.com> <4AFF7048.7070701@gmail.com> <1afaf6160911142001m39d0f0ceyd3646e922d3bf703@mail.gmail.com> Message-ID: <1afaf6160911151243s23c56722lfa00010f14a0d6b3@mail.gmail.com> 2009/11/15 Brett Cannon : > On Sat, Nov 14, 2009 at 20:01, Benjamin Peterson wrote: >> 2009/11/14 Nick Coghlan : >>> This does constrain where we can use itertools - if we want carte >>> blanche to use it anywhere in the standard library, even those parts >>> that are imported as part of the build chain, we'll need to bite the >>> bullet and make it a builtin module rather than a separately built >>> extension module. >> >> I have another unpleasant but slightly less hacky solution. We put >> detect_encoding in linecache where it is actually used. > > Well, it happens to be used by the standard library in linecache, but > not all external uses of it necessarily tie into linecache (e.g. > importlib uses detect_encoding() in some non-critical code). Might > just have to live with sub-optimal code. Well, what I mean is that we'd do: def _detect_encoding(): in linecache and then "from linecache import _detect_encoding as detect_encoding" in tokenize.py. -- Regards, Benjamin From martin at v.loewis.de Sun Nov 15 21:44:43 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 15 Nov 2009 21:44:43 +0100 Subject: [Python-Dev] Too many Python accounts In-Reply-To: References: <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk> <4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk> <4B005727.4050004@v.loewis.de> Message-ID: <4B00683B.6090902@v.loewis.de> >> Can't you then produce hundreds of IDs, all delegating to the same >> identity? > > Yes. But then, users can easily create as many fake accounts as they want to. This is not something I want to happen (it's still possible to setup fake accounts, but it should be more difficult for the average script kiddy). > If the provider dictates the identity, as you keep insisting, that sentence > makes no sense whatsoever. The value entered as the identifier is the > identifier you should use. Otherwise, what's the point of delegation at all? It may help users to remember their openid more easily, and always fill in the same text into the login box. Regards, Martin From martin at v.loewis.de Sun Nov 15 21:46:29 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 15 Nov 2009 21:46:29 +0100 Subject: [Python-Dev] Too many Python accounts (was Re: PyPI comments and ratings, *really*?) In-Reply-To: <19200.25648.998776.238480@montanaro.dyndns.org> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> <4AFF5B1A.8090105@v.loewis.de> <19200.25648.998776.238480@montanaro.dyndns.org> Message-ID: <4B0068A5.2040605@v.loewis.de> > I've never found OpenID at all intuitive to use. Are there instructions on > pypi.python.org which detail the steps necessary to use OpenID to login? I > saw the "Claim OpenID" link on my PyPI profile page. So now I have an > OpenID URL. What am I supposed to do with that? If I visit that URL it > downloads a small bit of XML. > > I've tried using my Yahoo! and Luanchpad OpenIDs for other sites in the > past. I've never successfully logged into any website with them, at least > not as far as I can recall. I realize that maybe this is something that > just doesn't click with me (maybe I'm an OpenID Luddite), but it seems to me > that OpenID needs to be a bit easier (or obvious?) to use if it's to become > some sort of de facto standard login mechanism. That's indeed what PyPI attempts to do. At the "claim openid" place, follow the Launchpad link. It should guide you through the procedure. Then, when you want to login, again follow the Launchpad link on the front page. Regards, Martin From guido at python.org Sun Nov 15 21:46:26 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 15 Nov 2009 12:46:26 -0800 Subject: [Python-Dev] global statements outside functions/methods should raise SyntaxError In-Reply-To: References: <4B005C94.8060707@gmail.com> Message-ID: On Sun, Nov 15, 2009 at 12:38 PM, Terry Reedy wrote: > Ezio Melotti wrote: >> >> Python currently accepts global statements at the top level: >> >> I opened an issue on the tracker (http://bugs.python.org/issue7329) >> and Benjamin suggested to discuss this here. >> The test he mentioned is in test_global.py: >> >> ? def test4(self): >> ? ? ? prog_text_4 = """\ >> global x >> x = 2 >> """ >> ? ? ? # this should work >> ? ? ? compile(prog_text_4, "", "exec") >> >> It just says that "it should work" but it doesn't say /why/. >> >> Any thoughts? > > I make the same suggestion a couple of years ago, either this list or Py3k > list, after newby reported 'problem' on python-list expecting module-level > global to do something. Guido rejected it on the basis that he wanted to > minimized differences between module code and function code. That example should work because you could pass it to exec()/eval() with separate dicts for locals and globals: $ python3.0 Python 3.0 (py3k:67506, Dec 3 2008, 10:12:04) [GCC 4.0.1 (Apple Inc. build 5484)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> glo = {} >>> lo = {} >>> so = 'global x; x = 2' >>> co = compile(so, '', 'exec') >>> exec(co, glo, lo) >>> glo['x'] ['x'] >>> exec('x = 3', glo, lo) >>> glo['x'] 2 >>> lo['x'] 3 >>> -- --Guido van Rossum (python.org/~guido) From brett at python.org Sun Nov 15 21:51:10 2009 From: brett at python.org (Brett Cannon) Date: Sun, 15 Nov 2009 12:51:10 -0800 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> Message-ID: On Sun, Nov 15, 2009 at 05:31, Georg Brandl wrote: > Antoine Pitrou schrieb: >> Tarek Ziad? gmail.com> writes: >>> >>> This cannot work on all platforms, when our Makefile is not shipped >>> with python but python-devel. (like Fedora) >> >> This practice is stupid anyway, because it means you have to install >> python-devel even to install pure Python packages with setuptools/distribute. >> Just ask Fedora, Mandriva and friends to change their packaging practice >> (Mandriva already has a bug open for that by the way). > > +1. ?They are the ones splitting what "make install" installs into several > packages, so they are the ones who have to fix the resulting dependency > problems. +1 from me as well. Python is designed to run as a whole. If they choose to ignore our design decisions they do so at their own peril. Now if you want to put the time in, Tarek, to make sure distutils can be removed from the stdlib and have everything still work so the linux distros can add distutils to python-devel that's fine, but once again, they are the ones mucking around with things in a way we did not design for, so only do it if you really want to. -Brett From daniel at stutzbachenterprises.com Sun Nov 15 22:25:36 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sun, 15 Nov 2009 15:25:36 -0600 Subject: [Python-Dev] Too many Python accounts In-Reply-To: <4B00683B.6090902@v.loewis.de> References: <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk> <4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk> <4B005727.4050004@v.loewis.de> <4B00683B.6090902@v.loewis.de> Message-ID: On Sun, Nov 15, 2009 at 2:44 PM, "Martin v. L?wis" wrote: > But then, users can easily create as many fake accounts as they want to. > Why not do something more robust, then? For example, when a user enters an OpenID that hasn't been seen by PyPi before, make them enter a CAPTCHA. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Sun Nov 15 22:27:36 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 15 Nov 2009 21:27:36 +0000 Subject: [Python-Dev] Too many Python accounts In-Reply-To: <4B00683B.6090902@v.loewis.de> References: <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk> <4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk> <4B005727.4050004@v.loewis.de> <4B00683B.6090902@v.loewis.de> Message-ID: <4B007248.2020201@voidspace.org.uk> Martin v. L?wis wrote: >>> Can't you then produce hundreds of IDs, all delegating to the same >>> identity? >>> >> Yes. >> > > But then, users can easily create as many fake accounts as they want to. > This is not something I want to happen (it's still possible to setup > fake accounts, but it should be more difficult for the average script > kiddy). > > This doesn't seem to be a problem for all the other sites I use my openid with. Why not allow users to login with their own openid, but only allow one account to refer back to the same delegated account? Michael >> If the provider dictates the identity, as you keep insisting, that sentence >> makes no sense whatsoever. The value entered as the identifier is the >> identifier you should use. Otherwise, what's the point of delegation at all? >> > > It may help users to remember their openid more easily, and always fill > in the same text into the login box. > > Regards, > Martin > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From skip at pobox.com Sun Nov 15 22:34:40 2009 From: skip at pobox.com (skip at pobox.com) Date: Sun, 15 Nov 2009 15:34:40 -0600 Subject: [Python-Dev] Too many Python accounts (was Re: PyPI comments and ratings, *really*?) In-Reply-To: <4B0068A5.2040605@v.loewis.de> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> <4AFF5B1A.8090105@v.loewis.de> <19200.25648.998776.238480@montanaro.dyndns.org> <4B0068A5.2040605@v.loewis.de> Message-ID: <19200.29680.613310.668341@montanaro.dyndns.org> Martin> That's indeed what PyPI attempts to do. At the "claim openid" Martin> place, follow the Launchpad link. It should guide you through Martin> the procedure. Well, since I use Google a lot more I'd prefer to use that. If I click the Google OpenID link I now get OpenID is already claimed Martin> Then, when you want to login, again follow the Launchpad link on Martin> the front page. That seems to work, but I'm not sure how. Doesn't seem to use cookies. The Google OpenID link leads to http://pypi.python.org/pypi?:action=login&provider=Google which contains nothing about me. I saw a pypi.python.org cookie which expires "On Quit", so I restarted Camino and verified there were no pypi.python.org cookies, then clicked the Google OpenID link. It still works. I must be missing something obvious... S From martin at v.loewis.de Sun Nov 15 22:58:23 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 15 Nov 2009 22:58:23 +0100 Subject: [Python-Dev] Too many Python accounts In-Reply-To: <4B007248.2020201@voidspace.org.uk> References: <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk> <4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk> <4B005727.4050004@v.loewis.de> <4B00683B.6090902@v.loewis.de> <4B007248.2020201@voidspace.org.uk> Message-ID: <4B00797F.2080003@v.loewis.de> > This doesn't seem to be a problem for all the other sites I use my > openid with. Perhaps they don't care about fake accounts at all? That would, in particular, be the case if they accept arbitrary OpenID providers (which means that essentially no real authentication happens). > Why not allow users to login with their own openid, but > only allow one account to refer back to the same delegated account? That sounds good. I'm not sure how to implement a provider change in that scenario, though. Regards, Martin From ben+python at benfinney.id.au Sun Nov 15 22:58:50 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 16 Nov 2009 08:58:50 +1100 Subject: [Python-Dev] =?utf-8?q?OpenID_on_PyPI=3A_please_discuss_on_?= =?utf-8?q?=E2=80=98catalog-sig=E2=80=99_=28was=3A_Too_many_Python_account?= =?utf-8?q?s=29?= References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk> <4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk> Message-ID: <87vdhbpi05.fsf_-_@benfinney.id.au> Michael Foord writes: > myopenid *does* the validation, but my registered openid is > www.voidspace.org.uk and I *should* be able to change provider without > creating a new account. I'm very glad this issue is being discussed, but it's unfortunately taking place both here (where it's off-topic) and in private email (where it's not as visible as it should be), which surely doesn't make much sense. I have started a new sub-thread over on ?catalog-sig? for this topic, starting with Message-ID: <87zl6npiky.fsf_-_ at benfinney.id.au> . Please continue the discussion there; I'll be gathering together under that thread some of the salient messages I've seen. -- \ ?Anyone can do any amount of work provided it isn't the work he | `\ is supposed to be doing at the moment.? ?Robert Benchley | _o__) | Ben Finney From rdmurray at bitdance.com Sun Nov 15 23:09:19 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Sun, 15 Nov 2009 17:09:19 -0500 (EST) Subject: [Python-Dev] Status of the Buildbot fleet and related bugs In-Reply-To: References: <5c6f2a5d0911060027x6152a27ao77df7938a079f622@mail.gmail.com> Message-ID: There are non-stable buildbots that are failing consistently, but this message is about something else. Now that the biggest stability issues have been addressed some less-noisy stability issues are visible. The two that I have noticed most often are test_httpsservers, which hangs occasionally, and test_multiprocessing, which fails in various assertions occasionally. Since the buildbots are often slow and/or heavily loaded, I tried increasing DELTA in test_multiprocessing, but while it did seem to help some as evidenced by how how many times it ran in my buildbot using -F before and after, it did not prevent failures. --David (RDM) From martin at v.loewis.de Sun Nov 15 23:11:36 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 15 Nov 2009 23:11:36 +0100 Subject: [Python-Dev] Too many Python accounts (was Re: PyPI comments and ratings, *really*?) In-Reply-To: <19200.29680.613310.668341@montanaro.dyndns.org> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> <4AFF5B1A.8090105@v.loewis.de> <19200.25648.998776.238480@montanaro.dyndns.org> <4B0068A5.2040605@v.loewis.de> <19200.29680.613310.668341@montanaro.dyndns.org> Message-ID: <4B007C98.8010901@v.loewis.de> skip at pobox.com wrote: > Martin> That's indeed what PyPI attempts to do. At the "claim openid" > Martin> place, follow the Launchpad link. It should guide you through > Martin> the procedure. > > Well, since I use Google a lot more I'd prefer to use that. If I click the > Google OpenID link I now get > > OpenID is already claimed That means that you already did it. You can associate each OpenID only with one account (and it doesn't special-case that you are trying to do it again). > Martin> Then, when you want to login, again follow the Launchpad link on > Martin> the front page. > > That seems to work, but I'm not sure how. If it logs you in, it works. > Doesn't seem to use cookies. The > Google OpenID link leads to > > http://pypi.python.org/pypi?:action=login&provider=Google > > which contains nothing about me. Don't worry about that. That's how OpenID is supposed to work. > I saw a pypi.python.org cookie which > expires "On Quit", so I restarted Camino and verified there were no > pypi.python.org cookies, then clicked the Google OpenID link. It still > works. I must be missing something obvious... It's far from obvious. It's called "provider-driven identifier selection". PyPI redirects your browser to Google. Google looks at the Google cookie, and finds your identity; they also see that you have opted to automatically log into PyPI. So without further questions, they redirect you back to PyPI. PyPI finds your account, and displays a logged-in page. Look Ma, no ugly login box with tons of characters to type. Regards, Martin From fuzzyman at voidspace.org.uk Sun Nov 15 23:20:25 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 15 Nov 2009 22:20:25 +0000 Subject: [Python-Dev] Too many Python accounts In-Reply-To: <4B00797F.2080003@v.loewis.de> References: <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk> <4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk> <4B005727.4050004@v.loewis.de> <4B00683B.6090902@v.loewis.de> <4B007248.2020201@voidspace.org.uk> <4B00797F.2080003@v.loewis.de> Message-ID: <4B007EA9.1020702@voidspace.org.uk> Martin v. L?wis wrote: > [snip...] >> Why not allow users to login with their own openid, but >> only allow one account to refer back to the same delegated account? >> > > That sounds good. I'm not sure how to implement a provider change > in that scenario, though. > > Even not having provider changes supported would still allow me to use my openid with PyPI which would be great. The only problem with changing provider that I can see is when the provider a user changes to would then be a duplicate - in which case I think just refusing to allow the login would be fine (rather than changing the provider for that account). All the best, Michael > Regards, > Martin > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From martin at v.loewis.de Sun Nov 15 23:26:19 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 15 Nov 2009 23:26:19 +0100 Subject: [Python-Dev] Too many Python accounts In-Reply-To: <4B007EA9.1020702@voidspace.org.uk> References: <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk> <4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk> <4B005727.4050004@v.loewis.de> <4B00683B.6090902@v.loewis.de> <4B007248.2020201@voidspace.org.uk> <4B00797F.2080003@v.loewis.de> <4B007EA9.1020702@voidspace.org.uk> Message-ID: <4B00800B.7050302@v.loewis.de> > Even not having provider changes supported would still allow me to use > my openid with PyPI which would be great. The only problem with changing > provider that I can see is when the provider a user changes to would > then be a duplicate - in which case I think just refusing to allow the > login would be fine (rather than changing the provider for that account). In that case, it would be much easier to record your true openid (i.e. the one that your provider is able to validate). You can then enter the alias that you are used to, and PyPI would map that right away to the verifiable ID, and log you in with that. For this to work, you would have to upgrade your web page to OpenID 2, as this is the only protocol that PyPI supports. Regards, Martin From skip at pobox.com Mon Nov 16 03:20:19 2009 From: skip at pobox.com (skip at pobox.com) Date: Sun, 15 Nov 2009 20:20:19 -0600 Subject: [Python-Dev] Too many Python accounts (was Re: PyPI comments and ratings, *really*?) In-Reply-To: <4B007C98.8010901@v.loewis.de> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> <4AFECA7A.8070400@v.loewis.de> <4AFF5B1A.8090105@v.loewis.de> <19200.25648.998776.238480@montanaro.dyndns.org> <4B0068A5.2040605@v.loewis.de> <19200.29680.613310.668341@montanaro.dyndns.org> <4B007C98.8010901@v.loewis.de> Message-ID: <19200.46819.688129.940631@montanaro.dyndns.org> Martin> It's far from obvious. It's called "provider-driven identifier Martin> selection". PyPI redirects your browser to Google. Google looks Martin> at the Google cookie, and finds your identity; they also see Martin> that you have opted to automatically log into PyPI. So without Martin> further questions, they redirect you back to PyPI. PyPI finds Martin> your account, and displays a logged-in page. Thanks. Makes sense now... Skip From tseaver at palladion.com Mon Nov 16 05:14:30 2009 From: tseaver at palladion.com (Tres Seaver) Date: Sun, 15 Nov 2009 23:14:30 -0500 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <20091114145222.084583A411A@sparrow.telecommunity.com> References: <4AFC9064.20700@v.loewis.de> <4222a8490911121511s40eba937n2c7b1fde6f695c6c@mail.gmail.com> <4AFC9975.7040204@v.loewis.de> <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 P.J. Eby wrote: > At 09:45 AM 11/14/2009 +0100, Martin v. L?wis wrote: >> Paul Moore wrote: >>> 2009/11/13 Tres Seaver : >>>> I can see the information about the poll, and a link to view the >>>> results, without logging in. >>>> >>>> http://pypi.python.org/pypi >>>> >>>> (second paragraph there). That paragraph tells you that you need to log >>>> in to vote in the poll. >>> I don't want to create a PyPI account (more account details I'll >>> rarely use to remember) just to vote. I can see why anonymous votes >>> are bad, but the sample's going to be self-selecting - people like me >>> who don't want to create an account will be excluded. >> This is indeed intentional: people like you won't upload packages to >> PyPI, nor will they take part in the rating system, as both require >> a PyPI account. > > Which is bizarre, since Paul belongs to the group of people you say > you care most about - i.e., those people browsing the index and > looking for packages. The *consumers* of the comments, in other words. I agree with Martin that anonymous votes, like anonymous comments, are process-harmful here. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAksA0aYACgkQ+gerLs4ltQ4WEACfUJmt5LEFujv5xZNCl1tbDGzR CrIAoKUtK10tQVIiEbDljaHhyTssr4r5 =+UqM -----END PGP SIGNATURE----- From kevin at bud.ca Mon Nov 16 05:19:24 2009 From: kevin at bud.ca (Kevin Teague) Date: Sun, 15 Nov 2009 20:19:24 -0800 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: <4AFE1492.7050401@canterbury.ac.nz> References: <4AFC6440.1040008@v.loewis.de> <4AFE1492.7050401@canterbury.ac.nz> Message-ID: On Nov 13, 2009, at 6:23 PM, Greg Ewing wrote: > Martin v. L?wis wrote: > > > Some of the Python maintainers have recently started objecting to > this > > setup, asking that the standard library should be split into > separate > > packages that are released and distributed independent of Python. > Others > > of us feel strongly that such a change should not be made. > > I'd be worried, because I would no longer be able to > release an app or package and say "requires Python x.y". > I'd have to list the version numbers of all the micro > packages making up the standard library that I use. > > Worse, I'd have to be aware of which ones I actually > *do* use so I could mantain that list, something I don't > have to think about at the moment. "requires Python x.y" would imply a dependency on the working set of micro-packages which were shipped with that version of Python (or more specifically, any working set that was released within a particular Python release series). You would only need to list packages from the standard library as dependencies in special-case circumstances where you required a version higher or lower than what shipped with a particular release series of Python. It would perhaps increase the potential for people to get into situations where they update a Python with newer packages which makes it incompatibe with other Python applications. This problem would be mitgated by the fact that the standard library tends to be very API stable, so usually newer releases only contain minor bug fixes or performance enhancements and are unlikely to cause breakage. Package installation tools would also still continue to install into site- packages, or ideally in a virtualenv or script-generation environement like Buildout does. So installing updates to the standard library could be done only to support those applications which require them, but leave the default working set untouched for any other Python applications. Conversely, it may also help the standard library compatability in some situations, as I've seen people copy newer files into the standard library locations as a method of applying bug fixes, and given a better set of metadata it would then be more natural to use tools which allowed updates to happen in an orderly fashion. That's all given that splitting the standard library into individual packages also continues to release a single standard library. I don't really think releases small/medium/large sized standard libraries as was also discussed is a good idea. Maybe usage of tools such as virtualenv and buildout aren't yet widespread enough to alleviate people mucking up their installations in such a way that causes them pain. And this would also make it easier for people to develop applications which would be harder to pakcage into linux distributions or other package managers which don't allow for non-global updates. However, these are only theoretical concerns. It's concrete issue such as this one: http://stackoverflow.com/questions/1734373/including-package-data-with-distribute/ Where a developer uses something in the standard library, and a python- dev commiter provides a fix very quickly (yay Tarek!). But then that developer has to be told to wait a year until the next Python release, then wait until you've got the time to migrate the rest of your application to the new Python release, then you can finally use that fix, and in the meantime even though the issue has been solved you still need to workaround the problem! It's issues like this where it's hard not to want to avoid using standard library packages (beyond "core" modules which stable and only change very rarely lke os, sys, re, etc.) because there are unneccessary roadblocks between developer and user. From tseaver at palladion.com Mon Nov 16 05:27:29 2009 From: tseaver at palladion.com (Tres Seaver) Date: Sun, 15 Nov 2009 23:27:29 -0500 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: <94bdd2610911150530y3191a192mb00d910e30e01c2e@mail.gmail.com> References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <4AFF68D4.9070103@cheimes.de> <94bdd2610911150530y3191a192mb00d910e30e01c2e@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tarek Ziad? wrote: > On Sun, Nov 15, 2009 at 3:35 AM, Christian Heimes wrote: > [..] >> Do we really want to change distutils to solve a problem of a third >> party packaging system when the problem was created by the very same >> third party in the first place? In other words: Should you spend your >> precious development time with fixing a problem that isn't our fault? >> The decision to split the header files into a separate package, that >> isn't installed by default, has already created tons of bad user >> experience in the past. Do you want to endorse the split even further? > > I didn't know the split story went like this. I took it like the > "natural" split everyone > agreed on, and I saw this distutils <-> Makefile link like something to fix. > > So, it sounds like a bad idea now :) Parsing the Makefile at runtime seems like an insane choice anyway to me: +1 for your new module having constants generated at ./configure time. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAksA1LEACgkQ+gerLs4ltQ5gCQCgtYpBewlnocHJf5hp33TfkLjG 72IAnRW1d9n2CO8S2V+9ewcMb81oWNL+ =GG45 -----END PGP SIGNATURE----- From ncoghlan at gmail.com Mon Nov 16 05:43:05 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 16 Nov 2009 14:43:05 +1000 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <4AFF68D4.9070103@cheimes.de> <94bdd2610911150530y3191a192mb00d910e30e01c2e@mail.gmail.com> Message-ID: <4B00D859.7090202@gmail.com> Tres Seaver wrote: > Tarek Ziad? wrote: >> I didn't know the split story went like this. I took it like the >> "natural" split everyone >> agreed on, and I saw this distutils <-> Makefile link like something to fix. > >> So, it sounds like a bad idea now :) > > Parsing the Makefile at runtime seems like an insane choice anyway to > me: +1 for your new module having constants generated at ./configure time. I'm with Tres here - having distutils (aside from build_ext) depend on non-Python parts of the source tree seem a little strange. However, given that the recommended packaging includes the needed files, putting an RFE on the tracker to reduce the runtime dependency on the source code would be an acceptable response if there is something else you'd rather be working on. Cheers, Nick. _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ziade.tarek at gmail.com Mon Nov 16 10:38:11 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 16 Nov 2009 10:38:11 +0100 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <4AFC9064.20700@v.loewis.de> <20091113221234.GA15732@amk-desktop.matrixgroup.net> <87bpj6t4mp.fsf@benfinney.id.au> <200911141018.13536.steve@pearwood.info> <20091114152921.GA28003@laurie.devork> Message-ID: <94bdd2610911160138k3f973a15i78d49cd36534df23@mail.gmail.com> On Sat, Nov 14, 2009 at 4:34 PM, Arc Riley wrote: > >> +1 >> >> Having a "Repository-URL", "Repository-Browse-URL" and a >> "Bug-Tracker-URL" field in PyPI would be a lot more usefule then >> comments and ratings. >> > > +1 You mean in the Metadata ? We are currenty working on adding fields in them, for an upcoming 1.2 version (see PEP 345) I am crossposting to distutils-sig so we can discuss this point over there, because I think that would be a great triple of fields to add in 1.2 Tarek From fuzzyman at voidspace.org.uk Mon Nov 16 14:41:38 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 16 Nov 2009 13:41:38 +0000 Subject: [Python-Dev] Too many Python accounts In-Reply-To: <4B00800B.7050302@v.loewis.de> References: <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk> <4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk> <4B005727.4050004@v.loewis.de> <4B00683B.6090902@v.loewis.de> <4B007248.2020201@voidspace.org.uk> <4B00797F.2080003@v.loewis.de> <4B007EA9.1020702@voidspace.org.uk> <4B00800B.7050302@v.loewis.de> Message-ID: <4B015692.5020501@voidspace.org.uk> Martin v. L?wis wrote: >> Even not having provider changes supported would still allow me to use >> my openid with PyPI which would be great. The only problem with changing >> provider that I can see is when the provider a user changes to would >> then be a duplicate - in which case I think just refusing to allow the >> login would be fine (rather than changing the provider for that account). >> > > In that case, it would be much easier to record your true openid (i.e. > the one that your provider is able to validate). You can then enter the > alias that you are used to, and PyPI would map that right away to the > verifiable ID, and log you in with that. > > Would it be possible to detect a change of provider and then offer the option to migrate the account to the new provider (so long as it does not conflict with another account)? > For this to work, you would have to upgrade your web page to OpenID 2, > as this is the only protocol that PyPI supports. > This I can do. Thanks Michael > Regards, > Martin > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From martin at v.loewis.de Mon Nov 16 19:46:14 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 16 Nov 2009 19:46:14 +0100 Subject: [Python-Dev] Too many Python accounts In-Reply-To: <4B015692.5020501@voidspace.org.uk> References: <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk> <4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk> <4B005727.4050004@v.loewis.de> <4B00683B.6090902@v.loewis.de> <4B007248.2020201@voidspace.org.uk> <4B00797F.2080003@v.loewis.de> <4B007EA9.1020702@voidspace.org.uk> <4B00800B.7050302@v.loewis.de> <4B015692.5020501@voidspace.org.uk> Message-ID: <4B019DF6.9090903@v.loewis.de> > Would it be possible to detect a change of provider and then offer the > option to migrate the account to the new provider (so long as it does > not conflict with another account)? That would be possible - but again complicate the UI. Regards, Martin From fuzzyman at voidspace.org.uk Mon Nov 16 20:01:13 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 16 Nov 2009 19:01:13 +0000 Subject: [Python-Dev] Too many Python accounts In-Reply-To: <4B019DF6.9090903@v.loewis.de> References: <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk> <4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk> <4B005727.4050004@v.loewis.de> <4B00683B.6090902@v.loewis.de> <4B007248.2020201@voidspace.org.uk> <4B00797F.2080003@v.loewis.de> <4B007EA9.1020702@voidspace.org.uk> <4B00800B.7050302@v.loewis.de> <4B015692.5020501@voidspace.org.uk> <4B019DF6.9090903@v.loewis.de> Message-ID: <4B01A179.1040704@voidspace.org.uk> Martin v. L?wis wrote: >> Would it be possible to detect a change of provider and then offer the >> option to migrate the account to the new provider (so long as it does >> not conflict with another account)? >> > > That would be possible - but again complicate the UI. > Why? You should only need to present this option *after* the user has authenticated with the a different provider to the one associated with their account (or perhaps on detecting that the provider is different - but either way no UI changes until / unless the user actually has changed provider). That complicates the UI code perhaps, but shouldn't change the standard UI. I don't think it should be given as an option to the user before they have changed provider, given that one of the major use cases is to allow the user to switch provider if their current one becomes unavailable. A UI that only allows them to switch provider if they can first login via their current provider would lock them out if their provider goes down. Michael > Regards, > Martin > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From martin at v.loewis.de Mon Nov 16 20:13:10 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 16 Nov 2009 20:13:10 +0100 Subject: [Python-Dev] Too many Python accounts In-Reply-To: <4B01A179.1040704@voidspace.org.uk> References: <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk> <4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk> <4B005727.4050004@v.loewis.de> <4B00683B.6090902@v.loewis.de> <4B007248.2020201@voidspace.org.uk> <4B00797F.2080003@v.loewis.de> <4B007EA9.1020702@voidspace.org.uk> <4B00800B.7050302@v.loewis.de> <4B015692.5020501@voidspace.org.uk> <4B019DF6.9090903@v.loewis.de> <4B01A179.1040704@voidspace.org.uk> Message-ID: <4B01A446.4020708@v.loewis.de> Michael Foord wrote: > Martin v. L?wis wrote: >>> Would it be possible to detect a change of provider and then offer the >>> option to migrate the account to the new provider (so long as it does >>> not conflict with another account)? >>> >> >> That would be possible - but again complicate the UI. >> > > Why? You should only need to present this option *after* the user has > authenticated That's what I mean - it's another HTML form to present, in certain cases. It would be easier to change the underlying ID silently, and refuse login if that then would conflict with an existing account. Regards, Martin From a.badger at gmail.com Mon Nov 16 20:15:04 2009 From: a.badger at gmail.com (Toshio Kuratomi) Date: Mon, 16 Nov 2009 11:15:04 -0800 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> Message-ID: <20091116191504.GS25960@clingman.lan> On Sun, Nov 15, 2009 at 02:31:45PM +0100, Georg Brandl wrote: > Antoine Pitrou schrieb: > > Tarek Ziad? gmail.com> writes: > >> > >> This cannot work on all platforms, when our Makefile is not shipped > >> with python but python-devel. (like Fedora) > > > > This practice is stupid anyway, because it means you have to install > > python-devel even to install pure Python packages with setuptools/distribute. > > Just ask Fedora, Mandriva and friends to change their packaging practice > > (Mandriva already has a bug open for that by the way). > > +1. They are the ones splitting what "make install" installs into several > packages, so they are the ones who have to fix the resulting dependency > problems. > I agree with this, however, my point on the bug was more akin to this: Tres Seaver wrote: > Parsing the Makefile at runtime seems like an insane choice anyway to > me: +1 for your new module having constants generated at ./configure > time. Makefiles and C header files are not intended as general purpose data formats. Using them as such has a variety of disadvantages: * If someone else wants to get at the data, they have to go through the API in distutils. Any data that's not exposed by the API is unavailable. * Since disturils doesn't implement a full parser for the make and C syntax it is possible to break distutils when making legitimate changes to those build files. These are the reasons I opened the bug to get that information into a real data file rather than parsing the Makefile and header files. I'll also mention two further things: The reason that python-devel was split off was to make it more useful for livecds, olpc, embedded systems, and other places where disk space is at a premimum. Being able to combine an operating system that is used by people beyond your immediate community is great for finding and fixing bugs before your users run into them. Being able to program in a high level language on these platforms has benefits that I'm sure everyone here can appreciate. I've brought the issue of Makefile and pyconfig.h being needed for distutils to the attention of every new Fedora python maintainer since the package split was made. The current maintainer, David Malcolm, agrees that distutils.sysconfig needs to be able to use this data and he has moved the Makefile and header files into the main python package. This doesn't change the problems with using a Makefile and C header files as a data format for python. -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From fuzzyman at voidspace.org.uk Mon Nov 16 20:18:08 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 16 Nov 2009 19:18:08 +0000 Subject: [Python-Dev] Too many Python accounts In-Reply-To: <4B01A446.4020708@v.loewis.de> References: <87fx8gsq2s.fsf@benfinney.id.au> <4AFF5B36.9030502@v.loewis.de> <87tywwr0z5.fsf@benfinney.id.au> <4AFF67E8.5060209@v.loewis.de> <4AFFE8E3.5020508@voidspace.org.uk> <4B004F00.7080405@v.loewis.de> <4B005413.3010702@voidspace.org.uk> <4B005727.4050004@v.loewis.de> <4B00683B.6090902@v.loewis.de> <4B007248.2020201@voidspace.org.uk> <4B00797F.2080003@v.loewis.de> <4B007EA9.1020702@voidspace.org.uk> <4B00800B.7050302@v.loewis.de> <4B015692.5020501@voidspace.org.uk> <4B019DF6.9090903@v.loewis.de> <4B01A179.1040704@voidspace.org.uk> <4B01A446.4020708@v.loewis.de> Message-ID: <4B01A570.8000103@voidspace.org.uk> Martin v. L?wis wrote: > Michael Foord wrote: > >> Martin v. L?wis wrote: >> >>>> Would it be possible to detect a change of provider and then offer the >>>> option to migrate the account to the new provider (so long as it does >>>> not conflict with another account)? >>>> >>>> >>> That would be possible - but again complicate the UI. >>> >>> >> Why? You should only need to present this option *after* the user has >> authenticated >> > > That's what I mean - it's another HTML form to present, in certain > cases. > > It would be easier to change the underlying ID silently, and > refuse login if that then would conflict with an existing account. > > Sounds good to me. Michael > Regards, > Martin > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From floris.bruynooghe at gmail.com Mon Nov 16 20:35:05 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Mon, 16 Nov 2009 19:35:05 +0000 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <4AFF68D4.9070103@cheimes.de> <94bdd2610911150530y3191a192mb00d910e30e01c2e@mail.gmail.com> Message-ID: <20091116193505.GA29976@laurie.devork> On Sun, Nov 15, 2009 at 11:27:29PM -0500, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Tarek Ziad? wrote: > > On Sun, Nov 15, 2009 at 3:35 AM, Christian Heimes wrote: > > [..] > >> Do we really want to change distutils to solve a problem of a third > >> party packaging system when the problem was created by the very same > >> third party in the first place? In other words: Should you spend your > >> precious development time with fixing a problem that isn't our fault? > >> The decision to split the header files into a separate package, that > >> isn't installed by default, has already created tons of bad user > >> experience in the past. Do you want to endorse the split even further? > > > > I didn't know the split story went like this. I took it like the > > "natural" split everyone > > agreed on, and I saw this distutils <-> Makefile link like something to fix. > > > > So, it sounds like a bad idea now :) > > Parsing the Makefile at runtime seems like an insane choice anyway to > me: +1 for your new module having constants generated at ./configure time. +1 There have been bugs in the past about distutils.sysconfig.parse_makefile() not knowing the full make syntax and there will no doubt still be bugs like this. Substituting these variables at ./configure time into a .py.in file would be much safer. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From p.f.moore at gmail.com Mon Nov 16 21:39:09 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 16 Nov 2009 20:39:09 +0000 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: References: <4222a8490911121534h30572eabv73ae58649c77155c@mail.gmail.com> <4AFC9D0C.5030500@v.loewis.de> <87y6mbuubp.fsf@benfinney.id.au> <20e5c9660911121702r162caf28jb6d656383147d909@mail.gmail.com> <79990c6b0911131133i2987a5d4s5c4ef780a527732@mail.gmail.com> <4AFE6E10.3050307@v.loewis.de> <20091114145222.084583A411A@sparrow.telecommunity.com> Message-ID: <79990c6b0911161239y75760654yb70bcde05761da25@mail.gmail.com> 2009/11/16 Tres Seaver : >> Which is bizarre, since Paul belongs to the group of people you say >> you care most about - i.e., those people browsing the index and >> looking for packages. ?The *consumers* of the comments, in other words. > > I agree with Martin that anonymous votes, like anonymous comments, are > process-harmful here. FWIW, having started this, I've no problem with this. As long as people are aware of the self-selecting nature of the sample (and Martin clearly is) that's fine. Paul. From amcnabb at mcnabbs.org Mon Nov 16 21:10:09 2009 From: amcnabb at mcnabbs.org (Andrew McNabb) Date: Mon, 16 Nov 2009 13:10:09 -0700 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: <4AFF68D4.9070103@cheimes.de> References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <4AFF68D4.9070103@cheimes.de> Message-ID: <20091116201009.GB9127@mcnabbs.org> On Sun, Nov 15, 2009 at 03:35:00AM +0100, Christian Heimes wrote: > > Do we really want to change distutils to solve a problem of a third > party packaging system when the problem was created by the very same > third party in the first place? In other words: Should you spend your > precious development time with fixing a problem that isn't our fault? > The decision to split the header files into a separate package, that > isn't installed by default, has already created tons of bad user > experience in the past. Do you want to endorse the split even further? The problem being addressed is that distutils is using an artefact of the build system (a Makefile) as a data file. I think the issue of whether or not to "endorse the split" is a red herring. -- Andrew McNabb http://www.mcnabbs.org/andrew/ PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868 From dmalcolm at redhat.com Mon Nov 16 21:27:53 2009 From: dmalcolm at redhat.com (David Malcolm) Date: Mon, 16 Nov 2009 15:27:53 -0500 Subject: [Python-Dev] Static analysis of CPython using coccinelle/spatch Message-ID: <1258403275.31627.98.camel@brick> Has anyone else looked at using Coccinelle/spatch[1] on CPython source code? It's a GPL-licensed tool for matching semantic patterns in C source code. It's been used on the Linux kernel for detecting and fixing problems, and for autogenerating patches when refactoring (http://coccinelle.lip6.fr/impact_linux.php). Although it's implemented in OCaml, it is scriptable using Python. I've been experimenting with using it on CPython code, both on the core implementation, and on C extension modules. As a test, I've written a validator for the mini-language used by PyArg_ParseTuple and its variants. My code examines the types of the variables passed as varargs, and attempts to check that they are correct, according to the rules here http://docs.python.org/c-api/arg.html (and in Python/getargs.c) It can detect this old error (fixed in svn r34931): buggy.c:12:socket_htons:Mismatching type of argument 1 in ""i:htons"": expected "int *" but got "unsigned long *" Similarly, it finds the deliberate error in xxmodule.c: xxmodule.c:207:xx_roj:unknown format char in "O#:roj": '#' (Unfortunately, when run on the full source tree, I see numerous messages, and as far as I can tell, the others are false positives) You can see the code here: http://fedorapeople.org/gitweb?p=dmalcolm/public_git/check-cpython.git;a=tree and download using anonymous git in this manner: git clone git://fedorapeople.org/home/fedora/dmalcolm/public_git/check-cpython.git The .cocci file detects invocations of PyArg_ParseTuple and determines the types of the arguments. At each matching call site it invokes python code, passing the type information to validate.py's validate_types. (I suspect it's possible to use spatch to detect reference counting antipatterns; I've also attempted 2to3 refactoring of c code using semantic patches, but so far macros tend to get in the way). Alternatively, are there any other non-proprietary static analysis tools for CPython? Thoughts? Dave [1] http://coccinelle.lip6.fr/ From dmalcolm at redhat.com Mon Nov 16 22:09:33 2009 From: dmalcolm at redhat.com (David Malcolm) Date: Mon, 16 Nov 2009 16:09:33 -0500 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> Message-ID: <1258405773.3763.3.camel@brick> On Sun, 2009-11-15 at 12:42 +0000, Antoine Pitrou wrote: > Tarek Ziad? gmail.com> writes: > > > > This cannot work on all platforms, when our Makefile is not shipped > > with python but python-devel. (like Fedora) > > This practice is stupid anyway, because it means you have to install > python-devel even to install pure Python packages with setuptools/distribute. > Just ask Fedora, Mandriva and friends to change their packaging practice > (Mandriva already has a bug open for that by the way). > > In Debian/Ubuntu, the Makefile is correctly part of the main Python package: > > $ dpkg -S /usr/lib/python2.5/config/Makefile > python2.5: /usr/lib/python2.5/config/Makefile For what it's worth, this is fixed in Fedora as of python-2.6.4-2.fc13 ; the bug tracking it was: https://bugzilla.redhat.com/show_bug.cgi?id=531901 From benjamin at python.org Tue Nov 17 00:11:25 2009 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 16 Nov 2009 17:11:25 -0600 Subject: [Python-Dev] 2.7 and 3.2 release schedules Message-ID: <1afaf6160911161511v5c79f861xe7e4c84728d920af@mail.gmail.com> After more thought, I think that separating the 2.7 and 3.2 releases is not as big of an issue as I once thought. Therefore, I'd like to adopt the schedule I posted a few weeks back for 2.7 only. This only means some other lucky victi... I mean volunteer can do 3.2. :) -- Regards, Benjamin From greg at hewgill.com Tue Nov 17 11:31:07 2009 From: greg at hewgill.com (Greg Hewgill) Date: Tue, 17 Nov 2009 10:31:07 +0000 Subject: [Python-Dev] new unbounded memory leak in exception handling? Message-ID: <20091117103107.GO49760@occam.hewgill.net> I've constructed an example program that does not leak memory in Python 2.x, but causes unbounded memory allocation in Python 3.1. Here is the code: import gc import sys class E(Exception): def __init__(self, fn): self.fn = fn def call(self): self.fn() def f(): raise E(f) a = E(f) while True: print(len(gc.get_objects())) try: a.call() except E: # get exception value in a python2/3 portable way a = sys.exc_info()[1] Every time through the loop, the length of gc.get_objects() increases by 7 under Python 3.1. I believe this is a regression error, since Python 2.x does not exhibit the same behaviour. Can anybody confirm this? Greg Hewgill http://hewgill.com From solipsis at pitrou.net Tue Nov 17 12:46:18 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 17 Nov 2009 11:46:18 +0000 (UTC) Subject: [Python-Dev] Add an optional timeout to lock operations Message-ID: Hello, I've submitted a patch (*) to add an optional timeout to locking operations (Lock.acquire() etc.). Since it's a pretty basic functionality, I would like to know if there was any good reason for not doing it. (*) http://bugs.python.org/issue7316 Thank you Antoine. From rdmurray at bitdance.com Tue Nov 17 14:40:37 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 17 Nov 2009 08:40:37 -0500 (EST) Subject: [Python-Dev] new unbounded memory leak in exception handling? In-Reply-To: <20091117103107.GO49760@occam.hewgill.net> References: <20091117103107.GO49760@occam.hewgill.net> Message-ID: On Tue, 17 Nov 2009 at 10:31, Greg Hewgill wrote: > I've constructed an example program that does not leak memory in Python > 2.x, but causes unbounded memory allocation in Python 3.1. Here is the > code: > > import gc > import sys > > class E(Exception): > def __init__(self, fn): > self.fn = fn > def call(self): > self.fn() > > def f(): > raise E(f) > > a = E(f) > while True: > print(len(gc.get_objects())) > try: > a.call() > except E: > # get exception value in a python2/3 portable way > a = sys.exc_info()[1] > > Every time through the loop, the length of gc.get_objects() increases > by 7 under Python 3.1. I believe this is a regression error, since > Python 2.x does not exhibit the same behaviour. > > Can anybody confirm this? I think you want to take a look at PEP 3134. And then please file a doc bug to have someone update the documentation of sys.exc_info, since the advice in the warning box is no longer valid in Python 3. --David (RDM) From guido at python.org Tue Nov 17 17:01:49 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Nov 2009 08:01:49 -0800 Subject: [Python-Dev] Add an optional timeout to lock operations In-Reply-To: References: Message-ID: I think I can answer the "why" question: thread.c is *very* old code, in fact it predates the posix threads standard. When we (actually Sjoerd Mullender) wrote it, we had a number of OS-specific locking APIs to work with and the API was designed to fit all of them. I don't even recall the initial set, but I think it included SGI Irix and and old version of SunOS. So then over time new platforms were added, but adding a new method or parameter to the API was nearly impossible because someone would have to find out how to implement the new feature on all supported platforms. I think the number of platforms has dwindled to two or three (Posix, Windows, and maybe one minority OS?), so now's the time to do it. (IOW I think the idea of the patch is fine.) Will locks be interruptible with ^C? That is an oft-requested feature which also wasn't supported at that time; what's the situation nowadays? --Guido On Tue, Nov 17, 2009 at 3:46 AM, Antoine Pitrou wrote: > > Hello, > > I've submitted a patch (*) to add an optional timeout to locking operations > (Lock.acquire() etc.). Since it's a pretty basic functionality, I would like to > know if there was any good reason for not doing it. > > (*) http://bugs.python.org/issue7316 > > Thank you > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) From solipsis at pitrou.net Tue Nov 17 17:21:41 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 17 Nov 2009 16:21:41 +0000 (UTC) Subject: [Python-Dev] Add an optional timeout to lock operations References: Message-ID: Guido van Rossum python.org> writes: > > I think the number of platforms > has dwindled to two or three (Posix, Windows, and maybe one minority > OS?), so now's the time to do it. (IOW I think the idea of the patch > is fine.) Thanks. (the minority OS would be OS/2, I think) > Will locks be interruptible with ^C? That is an oft-requested feature > which also wasn't supported at that time; what's the situation > nowadays? They still aren't interruptible. From what I can read it may be possible to make them interruptible in the POSIX semaphore-based implementation, not in the POSIX condition variable-based implementation (which is used as a fallback when POSIX semaphores are not available, but I don't know whether this fallback is still useful). As for Windows I have absolutely no idea. Regards Antoine. From greg at hewgill.com Tue Nov 17 19:14:02 2009 From: greg at hewgill.com (Greg Hewgill) Date: Tue, 17 Nov 2009 18:14:02 +0000 Subject: [Python-Dev] new unbounded memory leak in exception handling? In-Reply-To: References: <20091117103107.GO49760@occam.hewgill.net> Message-ID: <20091117181402.GQ49760@occam.hewgill.net> On Tue, Nov 17, 2009 at 08:40:37AM -0500, R. David Murray wrote: > I think you want to take a look at PEP 3134. And then please file a doc > bug to have someone update the documentation of sys.exc_info, since the > advice in the warning box is no longer valid in Python 3. Thanks for the pointer, I understand the cause now. I had read the warning in the description of sys.exc_info, but I hadn't realised that Python was (now) implicitly saving the traceback for me. I was able to solve this memory leak by explicitly clearing __traceback__: except E: # get exception value in a python2/3 portable way a = sys.exc_info()[1] a.__traceback__ = None I have added a doc bug issue for this: http://bugs.python.org/issue7340 Greg Hewgill http://hewgill.com From brett at python.org Tue Nov 17 22:03:23 2009 From: brett at python.org (Brett Cannon) Date: Tue, 17 Nov 2009 13:03:23 -0800 Subject: [Python-Dev] Static analysis of CPython using coccinelle/spatch In-Reply-To: <1258403275.31627.98.camel@brick> References: <1258403275.31627.98.camel@brick> Message-ID: On Mon, Nov 16, 2009 at 12:27, David Malcolm wrote: > Has anyone else looked at using Coccinelle/spatch[1] on CPython source > code? Not that has been mentioned on the list before. > > It's a GPL-licensed tool for matching semantic patterns in C source > code. It's been used on the Linux kernel for detecting and fixing > problems, and for autogenerating patches when refactoring > (http://coccinelle.lip6.fr/impact_linux.php). ?Although it's implemented > in OCaml, it is scriptable using Python. > > I've been experimenting with using it on CPython code, both on the core > implementation, and on C extension modules. > > As a test, I've written a validator for the mini-language used by > PyArg_ParseTuple and its variants. ?My code examines the types of the > variables passed as varargs, and attempts to check that they are > correct, according to the rules here > http://docs.python.org/c-api/arg.html (and in Python/getargs.c) > > It can detect this old error (fixed in svn r34931): > buggy.c:12:socket_htons:Mismatching type of argument 1 in ""i:htons"": > expected "int *" but got "unsigned long *" > > Similarly, it finds the deliberate error in xxmodule.c: > xxmodule.c:207:xx_roj:unknown format char in "O#:roj": '#' > > (Unfortunately, when run on the full source tree, I see numerous > messages, and as far as I can tell, the others are false positives) > > You can see the code here: > http://fedorapeople.org/gitweb?p=dmalcolm/public_git/check-cpython.git;a=tree > and download using anonymous git in this manner: > git clone git://fedorapeople.org/home/fedora/dmalcolm/public_git/check-cpython.git > > The .cocci file detects invocations of PyArg_ParseTuple and determines > the types of the arguments. ?At each matching call site it invokes > python code, passing the type information to validate.py's > validate_types. > > (I suspect it's possible to use spatch to detect reference counting > antipatterns; I've also attempted 2to3 refactoring of c code using > semantic patches, but so far macros tend to get in the way). > > Alternatively, are there any other non-proprietary static analysis tools > for CPython? Specific to CPython? No. But I had a chance to run practically every major commercial static analysis tool over the code base back on 2006. We also occasionally run valgrind over the code. But thanks to have we have structured the code and taken performance shortcuts static analysis tools easily get tripped up by CPython (as you have discovered). > > Thoughts? Running the tool over the code base and reporting the found bugs would be appreciated. -Brett > Dave > > [1] http://coccinelle.lip6.fr/ > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From amk at amk.ca Tue Nov 17 22:41:27 2009 From: amk at amk.ca (A.M. Kuchling) Date: Tue, 17 Nov 2009 16:41:27 -0500 Subject: [Python-Dev] Static analysis of CPython using coccinelle/spatch In-Reply-To: <1258403275.31627.98.camel@brick> References: <1258403275.31627.98.camel@brick> Message-ID: <20091117214127.GA15423@amk-desktop.matrixgroup.net> On Mon, Nov 16, 2009 at 03:27:53PM -0500, David Malcolm wrote: > Has anyone else looked at using Coccinelle/spatch[1] on CPython source > code? For an excellent explanation of Coccinelle, see . --amk From g.brandl at gmx.net Tue Nov 17 23:55:51 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 17 Nov 2009 23:55:51 +0100 Subject: [Python-Dev] 2.7 and 3.2 release schedules In-Reply-To: <1afaf6160911161511v5c79f861xe7e4c84728d920af@mail.gmail.com> References: <1afaf6160911161511v5c79f861xe7e4c84728d920af@mail.gmail.com> Message-ID: Benjamin Peterson schrieb: > After more thought, I think that separating the 2.7 and 3.2 releases > is not as big of an issue as I once thought. Therefore, I'd like to > adopt the schedule I posted a few weeks back for 2.7 only. > > This only means some other lucky victi... I mean volunteer can do 3.2. :) If no one else wants to try and ruin Python 3, I'll do it . In the regular 18-month schedule, it is due around Jan 2011, when I will have finished my diploma and have enough time while looking for PhD opportunities. cheers, Georg From barry at python.org Wed Nov 18 00:12:44 2009 From: barry at python.org (Barry Warsaw) Date: Tue, 17 Nov 2009 17:12:44 -0600 Subject: [Python-Dev] 2.7 and 3.2 release schedules In-Reply-To: References: <1afaf6160911161511v5c79f861xe7e4c84728d920af@mail.gmail.com> Message-ID: On Nov 17, 2009, at 4:55 PM, Georg Brandl wrote: > Benjamin Peterson schrieb: >> After more thought, I think that separating the 2.7 and 3.2 releases >> is not as big of an issue as I once thought. Therefore, I'd like to >> adopt the schedule I posted a few weeks back for 2.7 only. >> >> This only means some other lucky victi... I mean volunteer can do >> 3.2. :) > > If no one else wants to try and ruin Python 3, I'll do it . Ha ha ha^H^H^H^H^H^H^H^H^HGreat! Benjamin's the expert now, but I'm here to help if needed. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 194 bytes Desc: This is a digitally signed message part URL: From solipsis at pitrou.net Wed Nov 18 01:09:01 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 18 Nov 2009 00:09:01 +0000 (UTC) Subject: [Python-Dev] 2.7 and 3.2 release schedules References: <1afaf6160911161511v5c79f861xe7e4c84728d920af@mail.gmail.com> Message-ID: Barry Warsaw python.org> writes: > > > If no one else wants to try and ruin Python 3, I'll do it . > > Ha ha ha^H^H^H^H^H^H^H^H^HGreat! Benjamin's the expert now, but I'm > here to help if needed. Well Georg isn't a novice when it comes to ruining things, especially documentation and commit statistics. Although I'm sure guidance from a veteran can be useful if we want to reach that ultimate degree of perfection which helps attract an endless stream of anonymous function syntax proposals. Regards Antoine, who is improving as well. From tjreedy at udel.edu Wed Nov 18 01:45:20 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 17 Nov 2009 19:45:20 -0500 Subject: [Python-Dev] Static analysis of CPython using coccinelle/spatch In-Reply-To: <20091117214127.GA15423@amk-desktop.matrixgroup.net> References: <1258403275.31627.98.camel@brick> <20091117214127.GA15423@amk-desktop.matrixgroup.net> Message-ID: A.M. Kuchling wrote: > On Mon, Nov 16, 2009 at 03:27:53PM -0500, David Malcolm wrote: >> Has anyone else looked at using Coccinelle/spatch[1] on CPython source >> code? > > For an excellent explanation of Coccinelle, see > . For those who have not looked, Coccinelle means ladybug (a bug-eating bug ;-) in French. Its principle use to to take C code and a SmPl file of high-level patch descriptions (fixers, in 2to3 talk) and produce a standard diff file. I wonder if this could be used to help people migrate C extensions to 3.1, by developing a SmPl file with the needed changes dictated by API changes. This is similar to its motivating application to Linux. From http://coccinelle.lip6.fr/ "Coccinelle is a program matching and transformation engine which provides the language SmPL (Semantic Patch Language) for specifying desired matches and transformations in C code. Coccinelle was initially targeted towards performing collateral evolutions in Linux. Such evolutions comprise the changes that are needed in client code in response to evolutions in library APIs, and may include modifications such as renaming a function, adding a function argument whose value is somehow context-dependent, and reorganizing a data structure. " As I understand it, the problem with C extensions and 3.1 is the current lack of a "collateral evolution" tool like 2to3 for Python code. Terry Jan Reedy From ncoghlan at gmail.com Wed Nov 18 11:38:06 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 18 Nov 2009 20:38:06 +1000 Subject: [Python-Dev] Add an optional timeout to lock operations In-Reply-To: References: Message-ID: <4B03CE8E.4070303@gmail.com> Antoine Pitrou wrote: > Guido van Rossum python.org> writes: >> Will locks be interruptible with ^C? That is an oft-requested feature >> which also wasn't supported at that time; what's the situation >> nowadays? > > They still aren't interruptible. From what I can read it may be possible to make > them interruptible in the POSIX semaphore-based implementation, not in the POSIX > condition variable-based implementation (which is used as a fallback when POSIX > semaphores are not available, but I don't know whether this fallback is still > useful). I'm pretty sure at least some variants of *BSD still don't have OS level semaphores - their lack is the reason multiprocessing doesn't necessarily work everywhere that the threading module works (since mp needs semaphores in order to work its magic). Jesse would probably know the gory details. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From jnoller at gmail.com Wed Nov 18 13:45:42 2009 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 18 Nov 2009 07:45:42 -0500 Subject: [Python-Dev] Add an optional timeout to lock operations In-Reply-To: <4B03CE8E.4070303@gmail.com> References: <4B03CE8E.4070303@gmail.com> Message-ID: <7D64D4D0-1061-4494-BF98-B3C56B6CCF05@gmail.com> On Nov 18, 2009, at 5:38 AM, Nick Coghlan wrote: > Antoine Pitrou wrote: >> Guido van Rossum python.org> writes: >>> Will locks be interruptible with ^C? That is an oft-requested >>> feature >>> which also wasn't supported at that time; what's the situation >>> nowadays? >> >> They still aren't interruptible. From what I can read it may be >> possible to make >> them interruptible in the POSIX semaphore-based implementation, not >> in the POSIX >> condition variable-based implementation (which is used as a >> fallback when POSIX >> semaphores are not available, but I don't know whether this >> fallback is still >> useful). > > I'm pretty sure at least some variants of *BSD still don't have OS > level > semaphores - their lack is the reason multiprocessing doesn't > necessarily work everywhere that the threading module works (since mp > needs semaphores in order to work its magic). > > Jesse would probably know the gory details. > > Cheers, > Nick. > Nick is right, many of the BSDs and FreeBSD up until fairly recently did not have named shared semaphore support. Still yet, the behavior is broken on some OSes such as OS X which you have to work around. I wouldn't recommend using them for threading right now, there's an assumption that threading "just works" unlike multiprocessing, which people understand has caveats. Jesse From solipsis at pitrou.net Wed Nov 18 14:50:25 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 18 Nov 2009 13:50:25 +0000 (UTC) Subject: [Python-Dev] Add an optional timeout to lock operations References: <4B03CE8E.4070303@gmail.com> <7D64D4D0-1061-4494-BF98-B3C56B6CCF05@gmail.com> Message-ID: Jesse Noller gmail.com> writes: > > Nick is right, many of the BSDs and FreeBSD up until fairly recently > did not have named shared semaphore support. Still yet, the behavior > is broken on some OSes such as OS X which you have to work around. The core locking support only uses anonymous semaphores (they don't need to be shared accross processes obviously). Regards Antoine. From jnoller at gmail.com Wed Nov 18 15:27:36 2009 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 18 Nov 2009 09:27:36 -0500 Subject: [Python-Dev] Add an optional timeout to lock operations In-Reply-To: References: <4B03CE8E.4070303@gmail.com> <7D64D4D0-1061-4494-BF98-B3C56B6CCF05@gmail.com> Message-ID: <4222a8490911180627q432bb2f8o81f711adf396283a@mail.gmail.com> On Wed, Nov 18, 2009 at 8:50 AM, Antoine Pitrou wrote: > Jesse Noller gmail.com> writes: >> >> Nick is right, many of the BSDs and FreeBSD up until fairly recently >> did not have named shared semaphore support. Still yet, the behavior >> is broken on some OSes such as OS X which you have to work around. > > The core locking support only uses anonymous semaphores (they don't need to be > shared accross processes obviously). > > Regards > > Antoine. That's what I get for replying from an iphone over morning coffee. See also http://semanchuk.com/philip/sysv_ipc/ for Philip Semanchuk semaphore/IPC work as well. Given both multiprocessing and Philip's work are concerned about the shared aspects, I think you're right - FBSD and others support the posix semaphores for non-ipc stuff ok AFAIK (recent benchmark http://www.ioremap.net/node/216). jesse From dmalcolm at redhat.com Wed Nov 18 20:09:52 2009 From: dmalcolm at redhat.com (David Malcolm) Date: Wed, 18 Nov 2009 14:09:52 -0500 Subject: [Python-Dev] Static analysis of CPython using coccinelle/spatch In-Reply-To: References: <1258403275.31627.98.camel@brick> Message-ID: <1258571393.3005.28.camel@brick> On Tue, 2009-11-17 at 13:03 -0800, Brett Cannon wrote: > On Mon, Nov 16, 2009 at 12:27, David Malcolm wrote: > > Has anyone else looked at using Coccinelle/spatch[1] on CPython source > > code? [snip] > Running the tool over the code base and reporting the found bugs would > be appreciated. Discounting the false positives, the only issue it finds in python itself (trunk) is the deliberate mistake in Modules/xxmodule.c I also ran it on a random sample of extension modules and found some real bugs (only reported downstream so far, within Fedora's bug tracker): - DBus python bindings assume in one place that "unsigned long" is 32 bits wide: https://bugzilla.redhat.com/show_bug.cgi?id=538225 - MySQL-python assumes in one place that sizeof(int) == sizeof(long): https://bugzilla.redhat.com/show_bug.cgi?id=538234 - rpm.ps.append() uses unrecognized 'N' format specifier: https://bugzilla.redhat.com/show_bug.cgi?id=538218 From dmalcolm at redhat.com Wed Nov 18 20:26:23 2009 From: dmalcolm at redhat.com (David Malcolm) Date: Wed, 18 Nov 2009 14:26:23 -0500 Subject: [Python-Dev] 2to3 for .c code (was Static analysis of CPython using coccinelle/spatch) In-Reply-To: References: <1258403275.31627.98.camel@brick> <20091117214127.GA15423@amk-desktop.matrixgroup.net> Message-ID: <1258572383.3005.38.camel@brick> On Tue, 2009-11-17 at 19:45 -0500, Terry Reedy wrote: > A.M. Kuchling wrote: > > On Mon, Nov 16, 2009 at 03:27:53PM -0500, David Malcolm wrote: > >> Has anyone else looked at using Coccinelle/spatch[1] on CPython source > >> code? > > > > For an excellent explanation of Coccinelle, see > > . > > For those who have not looked, Coccinelle means ladybug (a bug-eating > bug ;-) in French. Its principle use to to take C code and a SmPl file > of high-level patch descriptions (fixers, in 2to3 talk) and produce a > standard diff file. I wonder if this could be used to help people > migrate C extensions to 3.1, by developing a SmPl file with the needed > changes dictated by API changes. This is similar to its motivating > application to Linux. From > > http://coccinelle.lip6.fr/ > > "Coccinelle is a program matching and transformation engine which > provides the language SmPL (Semantic Patch Language) for specifying > desired matches and transformations in C code. Coccinelle was initially > targeted towards performing collateral evolutions in Linux. Such > evolutions comprise the changes that are needed in client code in > response to evolutions in library APIs, and may include modifications > such as renaming a function, adding a function argument whose value is > somehow context-dependent, and reorganizing a data structure. " > > As I understand it, the problem with C extensions and 3.1 is the current > lack of a "collateral evolution" tool like 2to3 for Python code. Indeed; I think it may be possible to use Coccinelle for this. Here's a .cocci semantic patch to convert non-PyObject* dereferences of an "ob_type" field to use Py_TYPE macro instead. @@ PyObject *py_obj_ptr; type T; T non_py_obj_ptr; @@ ( py_obj_ptr->ob_type | - non_py_obj_ptr->ob_type + Py_TYPE(non_py_obj_ptr) ) I was able to use this to generate the attached patch for the DBus python bindings. Note that it leaves dereferences of a PyObject* untouched, and works inside sub-expressions. (There's some noise at the typedef of Server; I don't know why). Hope this is helpful Dave -------------- next part -------------- A non-text attachment was scrubbed... Name: example-resultant-ob_type.patch Type: text/x-patch Size: 1927 bytes Desc: not available URL: From db3l.net at gmail.com Wed Nov 18 22:28:39 2009 From: db3l.net at gmail.com (David Bolen) Date: Wed, 18 Nov 2009 16:28:39 -0500 Subject: [Python-Dev] Add an optional timeout to lock operations References: Message-ID: Antoine Pitrou writes: > I've submitted a patch (*) to add an optional timeout to locking > operations (Lock.acquire() etc.). Since it's a pretty basic > functionality, I would like to know if there was any good reason for > not doing it. I always assumed it was because as a least-common-denominator set of functionality, some platforms didn't have the necessary support. Providing the discussion on this ends up with the an implementation being accepted, I'd absolutely love to see this then leveraged by threading.Condition.wait() rather rather than the current poll-with-timed-sleep approach. The overhead (CPU, but most notably latency) of that approach (which also directly impacts Queues) has historically been my top reason in various applications on Windows to have to implement my own Queue or Condition structures using native Windows calls. -- David From solipsis at pitrou.net Wed Nov 18 23:20:08 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 18 Nov 2009 22:20:08 +0000 (UTC) Subject: [Python-Dev] Add an optional timeout to lock operations References: Message-ID: Hello, > > I've submitted a patch (*) to add an optional timeout to locking > > operations (Lock.acquire() etc.). Since it's a pretty basic > > functionality, I would like to know if there was any good reason for > > not doing it. > > I always assumed it was because as a least-common-denominator set of > functionality, some platforms didn't have the necessary support. Guido's answer says so indeed. Now py3k only needs to support POSIX and Windows (and, provided Andrew MacIntyre maintains the port, OS/2), which both have standard support for waiting-with-timeout. > Providing the discussion on this ends up with the an implementation > being accepted, I'd absolutely love to see this then leveraged by > threading.Condition.wait() rather rather than the current > poll-with-timed-sleep approach. Agreed. The current patch (as proposed on http://bugs.python.org/issue7316) includes exactly that. Poll-with-timed-sleep is especially sub-optimal on laptops where short but frequent wakeups can cause a significant decrease in battery life. (the Linux community has been chasing this using powertop: http://www.lesswatts.org/projects/powertop/) Feel free to test or review if you're interested. Regards Antoine. From ziade.tarek at gmail.com Thu Nov 19 10:30:52 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 19 Nov 2009 10:30:52 +0100 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: <20091116191504.GS25960@clingman.lan> References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <20091116191504.GS25960@clingman.lan> Message-ID: <94bdd2610911190130h5006773exf7c43a166ebe9dd9@mail.gmail.com> On Mon, Nov 16, 2009 at 8:15 PM, Toshio Kuratomi wrote: [..] > I've brought the issue of Makefile and pyconfig.h being needed for distutils > to the attention of every new Fedora python maintainer since the package > split was made. ?The current maintainer, David Malcolm, agrees that > distutils.sysconfig needs to be able to use this data and he has moved the > Makefile and header files into the main python package. ?This doesn't change > the problems with using a Makefile and C header files as a data format for > python. Great News ! Now for the format problem, I agree that it seems more robust to pre-process the variables and inject them in the stdlib when ./configure is run. I am not sure what is the best strategy here, but I would rather not add yet another configuration file (wether its an xml format or an ini-like format). So what I am proposing is to inject those values in a private dict in the new sysconfig.py module, that can be read through the get_config_vars / get_config_var APIs. This means that sysconfig.py will be added as "sysconfig.py.in" Regards Tarek From ziade.tarek at gmail.com Thu Nov 19 10:32:08 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 19 Nov 2009 10:32:08 +0100 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: <1258405773.3763.3.camel@brick> References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <1258405773.3763.3.camel@brick> Message-ID: <94bdd2610911190132n216eb7b2o7542e4a503907b0@mail.gmail.com> On Mon, Nov 16, 2009 at 10:09 PM, David Malcolm wrote: [..] >> >> $ dpkg -S /usr/lib/python2.5/config/Makefile >> python2.5: /usr/lib/python2.5/config/Makefile > For what it's worth, this is fixed in Fedora as of python-2.6.4-2.fc13 ; > the bug tracking it was: > https://bugzilla.redhat.com/show_bug.cgi?id=531901 Thanks David ! From lists at cheimes.de Thu Nov 19 10:55:32 2009 From: lists at cheimes.de (Christian Heimes) Date: Thu, 19 Nov 2009 10:55:32 +0100 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: <94bdd2610911190130h5006773exf7c43a166ebe9dd9@mail.gmail.com> References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <20091116191504.GS25960@clingman.lan> <94bdd2610911190130h5006773exf7c43a166ebe9dd9@mail.gmail.com> Message-ID: <4B051614.3000506@cheimes.de> Tarek Ziad? wrote: > Now for the format problem, I agree that it seems more robust to > pre-process the variables > and inject them in the stdlib when ./configure is run. > > I am not sure what is the best strategy here, but I would rather not > add yet another > configuration file (wether its an xml format or an ini-like format). > > So what I am proposing is to inject those values in a private dict in > the new sysconfig.py module, > that can be read through the get_config_vars / get_config_var APIs. > > This means that sysconfig.py will be added as "sysconfig.py.in" Are you planing to implement the proposal for Windows, too? There may be no need to change the code because we have full control over the official installers. Also the VisualStudio Windows build doesn't use ./configure but hand crafted header files. You'd have to fall back to a hand crafter sysconfig.py. Christian From solipsis at pitrou.net Thu Nov 19 11:04:03 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 19 Nov 2009 10:04:03 +0000 (UTC) Subject: [Python-Dev] buildtime vs runtime in Distutils References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <20091116191504.GS25960@clingman.lan> <94bdd2610911190130h5006773exf7c43a166ebe9dd9@mail.gmail.com> Message-ID: Tarek Ziad? gmail.com> writes: > > So what I am proposing is to inject those values in a private dict in > the new sysconfig.py module, > that can be read through the get_config_vars / get_config_var APIs. > > This means that sysconfig.py will be added as "sysconfig.py.in" This means you have to relaunch the whole configure thing each time you make a change to sysconfig? This doesn't sound like a good idea to me. You could make the private dict a dedicated module instead, e.g. "_sysconfig_data.py.in". As for Windows, if people have the ability to choose the installation directory when installing, I'm not sure how you're gonna handle it. From mal at egenix.com Thu Nov 19 12:04:44 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 19 Nov 2009 12:04:44 +0100 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: <94bdd2610911190130h5006773exf7c43a166ebe9dd9@mail.gmail.com> References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <20091116191504.GS25960@clingman.lan> <94bdd2610911190130h5006773exf7c43a166ebe9dd9@mail.gmail.com> Message-ID: <4B05264C.4030203@egenix.com> Tarek Ziad? wrote: > On Mon, Nov 16, 2009 at 8:15 PM, Toshio Kuratomi wrote: > [..] >> I've brought the issue of Makefile and pyconfig.h being needed for distutils >> to the attention of every new Fedora python maintainer since the package >> split was made. The current maintainer, David Malcolm, agrees that >> distutils.sysconfig needs to be able to use this data and he has moved the >> Makefile and header files into the main python package. This doesn't change >> the problems with using a Makefile and C header files as a data format for >> python. > > Great News ! > > Now for the format problem, I agree that it seems more robust to > pre-process the variables > and inject them in the stdlib when ./configure is run. > > I am not sure what is the best strategy here, but I would rather not > add yet another > configuration file (wether its an xml format or an ini-like format). > > So what I am proposing is to inject those values in a private dict in > the new sysconfig.py module, > that can be read through the get_config_vars / get_config_var APIs. > > This means that sysconfig.py will be added as "sysconfig.py.in" Wouldn't it be cleaner to place the extracted information into a new module _sysconfig_settings.py which gets created by the configure script or the Makefile later during the build ? This new module could then be imported into sysconfig.py at run-time. BTW: There's already a Tools/scripts/h2py.py we could use for converting pyconfig.h into a Python file. For the Makefile we could use the distutils Makefile parser to add the extra values. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 19 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From ziade.tarek at gmail.com Thu Nov 19 17:14:16 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 19 Nov 2009 17:14:16 +0100 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <20091116191504.GS25960@clingman.lan> <94bdd2610911190130h5006773exf7c43a166ebe9dd9@mail.gmail.com> Message-ID: <94bdd2610911190814m710ecc66ve5fd085650b1299a@mail.gmail.com> On Thu, Nov 19, 2009 at 11:04 AM, Antoine Pitrou wrote: > Tarek Ziad? gmail.com> writes: >> >> So what I am proposing is to inject those values in a private dict in >> the new sysconfig.py module, >> that can be read through the get_config_vars / get_config_var APIs. >> >> This means that sysconfig.py will be added as "sysconfig.py.in" > > This means you have to relaunch the whole configure thing each time you make a > change to sysconfig? This doesn't sound like a good idea to me. You could make > the private dict a dedicated module instead, e.g. "_sysconfig_data.py.in". That's much better indeed. > > As for Windows, if people have the ability to choose the installation directory > when installing, I'm not sure how you're gonna handle it. I don't know. I am investigating with what Christian has explained on Windows installers, to see how it can be hooked in the installation process, and a fallback for the win32 build version. If it gets too complex, I'll probably postpone this and finish a first version of sysconfig that does use Makefile. Tarek From ziade.tarek at gmail.com Thu Nov 19 17:19:10 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 19 Nov 2009 17:19:10 +0100 Subject: [Python-Dev] buildtime vs runtime in Distutils In-Reply-To: <4B05264C.4030203@egenix.com> References: <94bdd2610911141721n6700dcds5a489f1972176e48@mail.gmail.com> <20091116191504.GS25960@clingman.lan> <94bdd2610911190130h5006773exf7c43a166ebe9dd9@mail.gmail.com> <4B05264C.4030203@egenix.com> Message-ID: <94bdd2610911190819xad416cl37ed8564d18d529d@mail.gmail.com> On Thu, Nov 19, 2009 at 12:04 PM, M.-A. Lemburg wrote: [..] > BTW: There's already a Tools/scripts/h2py.py we could use for > converting pyconfig.h into a Python file. Good to know, > For the Makefile > we could use the distutils Makefile parser to add the extra > values. Yes, there's also a pyconfig.h parser function. It seems though, for Makefile, that the tools available during the build phase are simpler/better at extracting values. (currently experimenting those) Tarek From benjamin at python.org Fri Nov 20 00:43:30 2009 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 19 Nov 2009 17:43:30 -0600 Subject: [Python-Dev] Meet your 3.2 release manager Message-ID: <1afaf6160911191543g34a746b9xf5395c5278791c7e@mail.gmail.com> I'm pleased to announce that Georg has (naively) volunteered to shepherd the 3.2 release. -- Regards, Benjamin From status at bugs.python.org Fri Nov 20 18:07:15 2009 From: status at bugs.python.org (Python tracker) Date: Fri, 20 Nov 2009 18:07:15 +0100 (CET) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20091120170715.A321478364@psf.upfronthosting.co.za> ACTIVITY SUMMARY (11/13/09 - 11/20/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2506 open (+41) / 16686 closed (+15) / 19192 total (+56) Open issues with patches: 1000 Average duration of open issues: 685 days. Median duration of open issues: 440 days. Open Issues Breakdown open 2471 (+41) pending 34 ( +0) Issues Created Or Reopened (59) _______________________________ textwrap doesn't linebreak on "\n" 11/19/09 http://bugs.python.org/issue1859 reopened gvanrossum float compared to decimal is silently incorrect. 11/14/09 http://bugs.python.org/issue2531 reopened mark.dickinson patch 2to3 execfile conversion changes behavior 11/16/09 http://bugs.python.org/issue7268 reopened benjamin.peterson patch os.path.normpath doesn't normalize ../path/something.py 11/13/09 CLOSED http://bugs.python.org/issue7315 created purpleidea Add a timeout functionality to common locking operations 11/13/09 http://bugs.python.org/issue7316 created pitrou patch Display full tracebacks when an error occurs asynchronously 11/13/09 http://bugs.python.org/issue7317 created pitrou multiprocessing should not wait endlessly 11/13/09 CLOSED http://bugs.python.org/issue7318 created pitrou patch Silence DeprecationWarning by default 11/14/09 http://bugs.python.org/issue7319 created brett.cannon patch Unable to load external modules on build slave with debug python 11/14/09 http://bugs.python.org/issue7320 created db3l PyIter_Check(obj) fails when obj is of type PySetType 11/14/09 CLOSED http://bugs.python.org/issue7321 created damianeads Socket timeout can cause file-like readline() method to lose dat 11/14/09 http://bugs.python.org/issue7322 created beazley decimal.Decimal greater than/less than sometimes gives wrong ans 11/14/09 CLOSED http://bugs.python.org/issue7323 created adamtj Add sanity-check else case to regrtest option parsing 11/14/09 CLOSED http://bugs.python.org/issue7324 created r.david.murray patch, patch tempfile.mkdtemp() does not return absolute pathname when dir is 11/14/09 http://bugs.python.org/issue7325 created roysmith easy SOLUTION pls? /usr/lib/python2.6/dist-packages/visual/__init__.p 11/15/09 CLOSED http://bugs.python.org/issue7326 created pveloz format: minimum width: UTF-8 separators and decimal points 11/15/09 http://bugs.python.org/issue7327 created skrah pydoc doesn't work from the command line 11/15/09 CLOSED http://bugs.python.org/issue7328 created pitrou global statements outside functions/methods should raise SyntaxE 11/15/09 http://bugs.python.org/issue7329 created ezio.melotti PyUnicode_FromFormat segfault when using widths. 11/15/09 http://bugs.python.org/issue7330 created mark.dickinson Command line testing consistency between 2.x and 3.x 11/16/09 http://bugs.python.org/issue7331 created ncoghlan easy python script segment fault at PyMarshal_ReadLastObjectFromFile 11/16/09 http://bugs.python.org/issue7332 created liang Add initgroups to the posix/os modules 11/16/09 http://bugs.python.org/issue7333 created exarkun patch XML file locking in Jython 2.5 (OSError on Windows) 11/16/09 http://bugs.python.org/issue7334 created gdoutch int/long discrepancy when formatting zero with "%.0d" 11/16/09 http://bugs.python.org/issue7335 created mark.dickinson traceback module not properly printing exceptions on interpreter 11/16/09 http://bugs.python.org/issue7336 created rtyler Add lossy queue to queue library module 11/17/09 http://bugs.python.org/issue7337 created bpb recursive __attribute__ -> Fatal Python error: Cannot recover fr 11/17/09 http://bugs.python.org/issue7338 created kaizhu bsddb crashes when transaction limit occurs 11/17/09 http://bugs.python.org/issue7339 created pc Doc for sys.exc_info has warning that is no longer valid 11/17/09 http://bugs.python.org/issue7340 created ghewgill test_tarfile failing (file in use when deleting) on Windows buil 11/17/09 CLOSED http://bugs.python.org/issue7341 created pmoore patch, buildbot str(datetime_obj) doesn't include microseconds if their value is 11/17/09 http://bugs.python.org/issue7342 created ezio.melotti "What's new in 3.0" says % operator will be deprecated in 3.1 11/18/09 CLOSED http://bugs.python.org/issue7343 created jaredgrubb wsgiref tests failing on Windows 7 buildbot 11/18/09 CLOSED http://bugs.python.org/issue7344 created ncoghlan buildbot calendar.TextCalendar().formatyear wrong argument list 11/18/09 http://bugs.python.org/issue7345 created dingle Redirected stdout fires [Errno 9] 11/18/09 http://bugs.python.org/issue7346 created ecir.hana Patch - add RegCreateKeyEx to _winreg 11/18/09 http://bugs.python.org/issue7347 created brian.curtin patch StringIO.StringIO.readline(-1) returns the wrong result compare 11/18/09 http://bugs.python.org/issue7348 created exarkun StringIO.StringIO, io.BytesIO, and io.StringIO accept None in pl 11/18/09 http://bugs.python.org/issue7349 created exarkun csv doesn't handle escaped characters properly 11/18/09 http://bugs.python.org/issue7350 created est_python_tracker Documentation typos found in "zipfile ??? Work with ZIP archives 11/18/09 http://bugs.python.org/issue7351 reopened Retro patch python2.6-config --ldflags out of /usr and missing -L 1 6 days http://bugs.python.org/issue7298 mark.dickinson patch Wrong link to Python Language Mapping Specification 6 days http://bugs.python.org/issue7302 georg.brandl Run some tests in a loop until failure 2 days http://bugs.python.org/issue7312 r.david.murray patch 2to3 fails when source file starts with BOM 0 days http://bugs.python.org/issue7313 benjamin.peterson test_multiprocessing hangs in TestZZZNumberOfObjects after unitt 4 days http://bugs.python.org/issue7314 krisvale buildbot os.path.normpath doesn't normalize ../path/something.py 1 days http://bugs.python.org/issue7315 georg.brandl multiprocessing should not wait endlessly 4 days http://bugs.python.org/issue7318 krisvale patch PyIter_Check(obj) fails when obj is of type PySetType 0 days http://bugs.python.org/issue7321 pitrou decimal.Decimal greater than/less than sometimes gives wrong ans 0 days http://bugs.python.org/issue7323 mark.dickinson Add sanity-check else case to regrtest option parsing 0 days http://bugs.python.org/issue7324 r.david.murray patch, patch SOLUTION pls? /usr/lib/python2.6/dist-packages/visual/__init__.p 0 days http://bugs.python.org/issue7326 Arfrever pydoc doesn't work from the command line 0 days http://bugs.python.org/issue7328 ncoghlan test_tarfile failing (file in use when deleting) on Windows buil 1 days http://bugs.python.org/issue7341 lars.gustaebel patch, buildbot "What's new in 3.0" says % operator will be deprecated in 3.1 1 days http://bugs.python.org/issue7343 rhettinger wsgiref tests failing on Windows 7 buildbot 0 days http://bugs.python.org/issue7344 amaury.forgeotdarc buildbot distutils.tests.test_msvc9compiler - dragfullwindows can have va 0 days http://bugs.python.org/issue7354 tarek patch, patch, buildbot Incorrect error message with def((x)=0) 0 days http://bugs.python.org/issue7362 benjamin.peterson Inconsistent indentation in SocketServer docs 1 days http://bugs.python.org/issue7363 r.david.murray patch Traceback contains incorrect line number for duplicate argument 0 days http://bugs.python.org/issue7364 benjamin.peterson Internal error in threading.py 0 days http://bugs.python.org/issue7368 r.david.murray fix for scheme identification in urllib2? 1448 days http://bugs.python.org/issue1368312 orsenthil easy cp720 encoding map 1066 days http://bugs.python.org/issue1616979 bialix patch, needs review [PATCH] Fix ptcp154 encoding cyrillic_asian alias 851 days http://bugs.python.org/issue1757126 pjenvey patch Top Issues Most Discussed (10) ______________________________ 19 crasher in str(Exception()) 8 days open http://bugs.python.org/issue7309 14 test_msvc9compiler test_reg_class failure on new Windows box 9 days closed http://bugs.python.org/issue7293 13 Provide CPython command line functionality via runpy module 79 days closed http://bugs.python.org/issue6816 12 test_multiprocessing dictionary changed size errors and hang 47 days open http://bugs.python.org/issue7060 11 test_tarfile failing (file in use when deleting) on Windows bui 1 days closed http://bugs.python.org/issue7341 9 reversed(range(x, -1, -1)) is empty when x > 1 6 days closed http://bugs.python.org/issue7298 9 add ftp-tls support to ftplib - RFC 4217 648 days closed http://bugs.python.org/issue2054 7 Struct incorrectly compiles format strings 1 days open http://bugs.python.org/issue7355 7 Documentation typos found in "zipfile ??? Work with ZIP archive 2 days open http://bugs.python.org/issue7351 6 Add initgroups to the posix/os modules 4 days open http://bugs.python.org/issue7333 From glenn at nevcal.com Fri Nov 20 21:31:00 2009 From: glenn at nevcal.com (Glenn Linderman) Date: Fri, 20 Nov 2009 12:31:00 -0800 Subject: [Python-Dev] PyPI comments and ratings, *really*? In-Reply-To: <20091114152921.GA28003@laurie.devork> References: <4AFC9064.20700@v.loewis.de> <20091113221234.GA15732@amk-desktop.matrixgroup.net> <87bpj6t4mp.fsf@benfinney.id.au> <200911141018.13536.steve@pearwood.info> <20091114152921.GA28003@laurie.devork> Message-ID: <4B06FC84.80302@nevcal.com> On approximately 11/14/2009 7:29 AM, came the following characters from the keyboard of Floris Bruynooghe: > Having a "Repository-URL", "Repository-Browse-URL" and a > "Bug-Tracker-URL" field in PyPI would be a lot more usefule then > comments and ratings. > +1 Here's a thought... if the author supplies the above URLs, then comments would be prevented, in favor of using the author-supplied system. If the author doesn't supply the above URL, or someone reports that the URLs don't work, then comments on PyPI would be enabled. -- Glenn ------------------------------------------------------------------------ ?Everyone is entitled to their own opinion, but not their own facts. In turn, everyone is entitled to their own opinions of the facts, but not their own facts based on their opinions.? -- Guy Rocha, retiring NV state archivist From ben+python at benfinney.id.au Sat Nov 21 00:06:56 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 21 Nov 2009 10:06:56 +1100 Subject: [Python-Dev] PyPI comments and ratings, *really*? References: <4AFC9064.20700@v.loewis.de> <20091113221234.GA15732@amk-desktop.matrixgroup.net> <87bpj6t4mp.fsf@benfinney.id.au> <200911141018.13536.steve@pearwood.info> <20091114152921.GA28003@laurie.devork> <4B06FC84.80302@nevcal.com> Message-ID: <87my2gdcdr.fsf@benfinney.id.au> Glenn Linderman writes: > On approximately 11/14/2009 7:29 AM, came the following characters > from the keyboard of Floris Bruynooghe: > > Having a "Repository-URL", "Repository-Browse-URL" and a > > "Bug-Tracker-URL" field in PyPI would be a lot more usefule then > > comments and ratings. > > +1 Agreed, the above would be very useful. > Here's a thought... if the author supplies the above URLs, then > comments would be prevented, in favor of using the author-supplied > system. Others in this thread have pointed out that comments on the package don't serve the same purpose as a bug tracker for the project. I don't think you'll get that ?either/or? suggestion to fly. -- \ ?Programs must be written for people to read, and only | `\ incidentally for machines to execute.? ?Abelson & Sussman, | _o__) _Structure and Interpretation of Computer Programs_ | Ben Finney From regebro at gmail.com Sat Nov 21 13:40:13 2009 From: regebro at gmail.com (Lennart Regebro) Date: Sat, 21 Nov 2009 13:40:13 +0100 Subject: [Python-Dev] standard libraries don't behave like standard 'libraries' In-Reply-To: References: <4AFC6440.1040008@v.loewis.de> <4222a8490911121157i622fbab8q5fe82d32fe9ea8f@mail.gmail.com> <56195B13-39DF-44A3-9A79-5BDAFF244F40@bud.ca> Message-ID: <319e029f0911210440g19472242ld274c66f3940b820@mail.gmail.com> On Fri, Nov 13, 2009 at 09:59, Sriram Srinivasan wrote: > you were thinking wrong. If suppose this feature is introduced it doesn't > mean python will become batteries removed! > you can ship the python release with the 'standard library packages' already > installed. > so what we get here is batteries included and ability to be changed after it > is discharged! ;) You can. You can simply delete any files in the standard library that you don't want. What is your usecase? -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From mal at egenix.com Sat Nov 21 16:06:47 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Sat, 21 Nov 2009 16:06:47 +0100 Subject: [Python-Dev] Removal of intobject.h in 3.1 Message-ID: <4B080207.2020808@egenix.com> The wiki page for porting to 3.x says: http://wiki.python.org/moin/PortingExtensionModulesToPy3k """ long/int Unification In Python 3.0, there is only one integer type. It is called int on the Python level, but actually corresponds to 2.x's long type. In the C-API, PyInt_* functions are replaced by their PyLong_* neighbors. The best course of action here is using the PyInt_* functions aliased to PyLong_* found in intobject.h. """ However, intobject.h is no longer available. The checkin message for the removal says: """ r71697 | mark.dickinson | 2009-04-18 12:12:16 +0200 (Sat, 18 Apr 2009) | 3 lines The comments at the top of intobject.h say that it will be removed in 3.1. Make it so. """ Since package developers are just starting to port things to 3.x and many appear to be considering supporting both 2.7 and 3.1 (including myself), I find it a bit strange that such an import aliasing header was removed in 3.1. IMHO, that's not really a good way to encourage people to try to provide a smooth upgrade to the 3.x branch. Much to the contrary. 3.x should make it easier for developers by providing more standard helpers like the removed intobject.h header file. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 21 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From eric at trueblade.com Sat Nov 21 17:37:54 2009 From: eric at trueblade.com (Eric Smith) Date: Sat, 21 Nov 2009 11:37:54 -0500 Subject: [Python-Dev] Removal of intobject.h in 3.1 In-Reply-To: <4B080207.2020808@egenix.com> References: <4B080207.2020808@egenix.com> Message-ID: <4B081762.8060007@trueblade.com> M.-A. Lemburg wrote: > Since package developers are just starting to port things to 3.x and > many appear to be considering supporting both 2.7 and 3.1 (including > myself), I find it a bit strange that such an import aliasing header > was removed in 3.1. There's some discussion of this at http://bugs.python.org/issue7353 You might want to comment there. From sshock3 at gmail.com Sat Nov 21 19:31:33 2009 From: sshock3 at gmail.com (Phillip Hellewell) Date: Sat, 21 Nov 2009 11:31:33 -0700 Subject: [Python-Dev] Issue 1488943 - differ lib missing tab hinting Message-ID: Is there anything holding up fixing issue 1488943? The bug was found and a patch submitted 3 1/2 years ago. The patch is just a single line of code (and some comment changes): common = min(common, _count_leading(btags[:common], " ")) http://bugs.python.org/issue1488943 Thanks, Phillip Hellewell -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Sat Nov 21 20:05:57 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 21 Nov 2009 20:05:57 +0100 Subject: [Python-Dev] Removal of intobject.h in 3.1 In-Reply-To: <4B080207.2020808@egenix.com> References: <4B080207.2020808@egenix.com> Message-ID: <4B083A15.4020901@v.loewis.de> > IMHO, that's not really a good way to encourage people to try to provide > a smooth upgrade to the 3.x branch. Much to the contrary. 3.x should make > it easier for developers by providing more standard helpers like > the removed intobject.h header file. I think it's better than it sounds. The macros (unfortunately) allowed to make non-obvious mistakes. Now that they are gone, people need to really think of what precisely they want to do. For example, consider if (PyInt_Check(o)){ long val = PyInt_AsLong(o); // process } else if (PyLong_Check(o)) { long long val = PyLong_AsLongLong(o); // check for overflow // process } With intobject.h, this code would continue to compile, but work incorrectly, as the second case will never be executed. It would be better to port this as #if Py2.x if (PyInt_Check(o)){ long val = PyInt_AsLong(o); // process } else #endif if (PyLong_Check(o)) { i.e. eliminating the int case altogether. For another example, long foo = PyInt_AsLong(Foo); has a hidden error in 3.x, with intobject: PyLong_AsLong might overflow, which the 2.x case doesn't. So eliminating intobject.h likely helps avoiding subtle errors. Regards, Martin From casevh at gmail.com Sat Nov 21 21:00:41 2009 From: casevh at gmail.com (Case Vanhorsen) Date: Sat, 21 Nov 2009 12:00:41 -0800 Subject: [Python-Dev] Removal of intobject.h in 3.1 In-Reply-To: <4B083A15.4020901@v.loewis.de> References: <4B080207.2020808@egenix.com> <4B083A15.4020901@v.loewis.de> Message-ID: <99e0b9530911211200g1c4473e1s31597d519184af3@mail.gmail.com> On Sat, Nov 21, 2009 at 11:05 AM, "Martin v. L?wis" wrote: >> IMHO, that's not really a good way to encourage people to try to provide >> a smooth upgrade to the 3.x branch. Much to the contrary. 3.x should make >> it easier for developers by providing more standard helpers like >> the removed intobject.h header file. > > I think it's better than it sounds. The macros (unfortunately) allowed > to make non-obvious mistakes. Now that they are gone, people need to > really think of what precisely they want to do. > > For example, consider > > if (PyInt_Check(o)){ > ?long val = PyInt_AsLong(o); > ?// process > } else if (PyLong_Check(o)) { > ?long long val = PyLong_AsLongLong(o); > ?// check for overflow > ?// process > } > > With intobject.h, this code would continue to compile, but work > incorrectly, as the second case will never be executed. It would > be better to port this as > > #if Py2.x > if (PyInt_Check(o)){ > ?long val = PyInt_AsLong(o); > ?// process > } else > #endif > if (PyLong_Check(o)) { > > i.e. eliminating the int case altogether. For another example, > > long foo = PyInt_AsLong(Foo); > > has a hidden error in 3.x, with intobject: PyLong_AsLong might > overflow, which the 2.x case doesn't. > > So eliminating intobject.h likely helps avoiding subtle errors. FWIW, I ported gmpy to Python 3.x without using intobject.h. I'm now using the #if Py2.x ... #endif approach (almost) everywhere. The same source compiles successfully with Python 2.4 to 3.2. Case > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/casevh%40gmail.com > From s.r at visotech.at Mon Nov 23 09:54:56 2009 From: s.r at visotech.at (Stefan Ring) Date: Mon, 23 Nov 2009 08:54:56 +0000 (UTC) Subject: [Python-Dev] Reworking the GIL References: <1256502140.5621.163.camel@localhost> Message-ID: Hello, I built something very similar for my company last year, and it?s been running flawlessly in production at a few customer sites since, with avg. CPU usage ~50% around the clock. I even posted about it on the Python mailing list [1] where there was almost no resonance at that time. I never posted code, though -- nobody seemed to be too interested. I am well aware that your current work is a lot more far-reaching than what I?ve done, which is basically just a FIFO scheduler. I even added scheduling priorities later which don?t work too great because the amount of time used for a "tick" can vary by several orders of magnitude, as you know. Thought you might be interested. Regards Stefan [1] http://mail.python.org/pipermail/python-dev/2008-March/077814.html [2] http://www.bestinclass.dk/index.php/2009/10/python-vs-clojure-evolving/ [3] www.dabeaz.com/python/GIL.pdf PS On a slightly different note, I came across some Python bashing [2] yesterday and somehow from there to David Beazley?s presentation about the GIL [3]. While I don?t mind the bashing, the observations about the GIL seem quite unfair to me because David?s measurements have been made on Mac OS X with its horribly slow pthreads functions. I was not able to measure any slowdown on Linux. From ncoghlan at gmail.com Mon Nov 23 11:29:58 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 23 Nov 2009 20:29:58 +1000 Subject: [Python-Dev] Reworking the GIL In-Reply-To: References: <1256502140.5621.163.camel@localhost> Message-ID: <4B0A6426.8030103@gmail.com> Stefan Ring wrote: > [2] http://www.bestinclass.dk/index.php/2009/10/python-vs-clojure-evolving/ > [3] www.dabeaz.com/python/GIL.pdf > > PS On a slightly different note, I came across some Python bashing [2] yesterday > and somehow from there to David Beazley?s presentation about the GIL [3]. While > I don?t mind the bashing, the observations about the GIL seem quite unfair to me > because David?s measurements have been made on Mac OS X with its horribly slow > pthreads functions. I was not able to measure any slowdown on Linux. We care about Mac OS X though, so even if the contention wasn't as bad on a different OS, the Mac downsides matter. With the GIL updates in place, it would be interesting to see that analysis redone for 2.7/3.2 though. Regards, Nick. P.S. As far as interest in the idea goes, the GIL is one of those areas where it takes a fairly rare combination of interest, expertise and established credibility to propose a change and get assent to it. You'll notice that even Antoine had to resort to the "if nobody objects soon, I'm checking this in" tactic to garner any responses. It's an area where even those with relevant expertise still have to put aside a fair chunk of time in order to properly review a proposed change :) -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From mal at egenix.com Mon Nov 23 11:44:36 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 23 Nov 2009 11:44:36 +0100 Subject: [Python-Dev] Removal of intobject.h in 3.1 In-Reply-To: <4B081762.8060007@trueblade.com> References: <4B080207.2020808@egenix.com> <4B081762.8060007@trueblade.com> Message-ID: <4B0A6794.9010108@egenix.com> Eric Smith wrote: > M.-A. Lemburg wrote: >> Since package developers are just starting to port things to 3.x and >> many appear to be considering supporting both 2.7 and 3.1 (including >> myself), I find it a bit strange that such an import aliasing header >> was removed in 3.1. > > There's some discussion of this at http://bugs.python.org/issue7353 > > You might want to comment there. Thanks for pointing me to the that ticket. Looks like Guido already commented on this, so intobject.h could be revived in some form. Since this file would only be used by extension modules and possibly also include other helper macros, perhaps it'd be better to to rename the file to py2compat.h or something along those lines ?! We could then also have a py2compat.c to hold corresponding C code, e.g. to provide compatibility wrappers of new APIs that implement different semantics in 3.x. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 23 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mal at egenix.com Mon Nov 23 12:05:51 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 23 Nov 2009 12:05:51 +0100 Subject: [Python-Dev] Removal of intobject.h in 3.1 In-Reply-To: <4B083A15.4020901@v.loewis.de> References: <4B080207.2020808@egenix.com> <4B083A15.4020901@v.loewis.de> Message-ID: <4B0A6C8F.1060803@egenix.com> "Martin v. L?wis" wrote: >> IMHO, that's not really a good way to encourage people to try to provide >> a smooth upgrade to the 3.x branch. Much to the contrary. 3.x should make >> it easier for developers by providing more standard helpers like >> the removed intobject.h header file. > > I think it's better than it sounds. The macros (unfortunately) allowed > to make non-obvious mistakes. Now that they are gone, people need to > really think of what precisely they want to do. > > For example, consider > > if (PyInt_Check(o)){ > long val = PyInt_AsLong(o); > // process > } else if (PyLong_Check(o)) { > long long val = PyLong_AsLongLong(o); > // check for overflow > // process > } > > With intobject.h, this code would continue to compile, but work > incorrectly, as the second case will never be executed. It would > be better to port this as > > #if Py2.x > if (PyInt_Check(o)){ > long val = PyInt_AsLong(o); > // process > } else > #endif > if (PyLong_Check(o)) { > > i.e. eliminating the int case altogether. Sure, but that assumes that the original code already had support for Python longs, which a lot of code doesn't. In an ideal world, developers would add that code to their extensions right away. In the real world, where developers only have limited resources available, you'll get more 3.x ports by making such ports as painless as possible while at the same time not forcing them to alienate their 2.x user base. The long support could then be added in later releases of the extensions, giving the developers more time adapt. > For another example, > > long foo = PyInt_AsLong(Foo); > > has a hidden error in 3.x, with intobject: PyLong_AsLong might > overflow, which the 2.x case doesn't. That's not quite true: PyInt_AsLong(obj) will try the nb_int slot on non-integer objects which can return errors (it returns -1 and sets the error message). > So eliminating intobject.h likely helps avoiding subtle errors. In the long run, yes. In the short run, other criteria are more important, IMHO. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 23 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From s.r at visotech.at Mon Nov 23 12:25:39 2009 From: s.r at visotech.at (Stefan Ring) Date: Mon, 23 Nov 2009 12:25:39 +0100 Subject: [Python-Dev] Reworking the GIL In-Reply-To: References: <1256502140.5621.163.camel@localhost> Message-ID: <53e878390911230325p7da0f767qfdc069c03d0357c2@mail.gmail.com> > I built something very similar for my company last year, and it?s been running > flawlessly in production at a few customer sites since, with avg. CPU usage ~50% > around the clock. I even posted about it on the Python mailing list [1] where > there was almost no resonance at that time. I never posted code, though -- > nobody seemed to be too interested. I've never bothered to make this tidy and nice, especially the function naming (PySpecial_*) leaves some things to be desired. It's not too bad, though; it just doesn't have commit-ready quality. I don't worry about this anymore, so I just post what I have. Maybe someone can make use of it. -------------- next part -------------- A non-text attachment was scrubbed... Name: py252-scheduling.patch Type: text/x-patch Size: 7507 bytes Desc: not available URL: From martin at v.loewis.de Mon Nov 23 13:46:07 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 23 Nov 2009 13:46:07 +0100 Subject: [Python-Dev] Removal of intobject.h in 3.1 In-Reply-To: <4B0A6C8F.1060803@egenix.com> References: <4B080207.2020808@egenix.com> <4B083A15.4020901@v.loewis.de> <4B0A6C8F.1060803@egenix.com> Message-ID: <4B0A840F.5000706@v.loewis.de> > In an ideal world, developers would add that code to their > extensions right away. In the real world, where developers only > have limited resources available, you'll get more 3.x ports > by making such ports as painless as possible while at the > same time not forcing them to alienate their 2.x user base. Unfortunately, such 3.x code would be full of bugs. >> For another example, >> >> long foo = PyInt_AsLong(Foo); >> >> has a hidden error in 3.x, with intobject: PyLong_AsLong might >> overflow, which the 2.x case doesn't. > > That's not quite true: PyInt_AsLong(obj) will try the > nb_int slot on non-integer objects which can return errors > (it returns -1 and sets the error message). However, the 2.x code will typically assume that the object is an int object, in which case PyInt_AsLong can never fail. So with intobject.h, the code will happily compile, but then fail to detect an exception at run-time - causing other difficult-to-find bugs. Regards, Martin From dickinsm at gmail.com Mon Nov 23 18:12:14 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 23 Nov 2009 17:12:14 +0000 Subject: [Python-Dev] Removal of intobject.h in 3.1 In-Reply-To: <4B0A6794.9010108@egenix.com> References: <4B080207.2020808@egenix.com> <4B081762.8060007@trueblade.com> <4B0A6794.9010108@egenix.com> Message-ID: <5c6f2a5d0911230912t6d3560f8m7994d990bf36fcc7@mail.gmail.com> On Mon, Nov 23, 2009 at 10:44 AM, M.-A. Lemburg wrote: > Thanks for pointing me to the that ticket. > > Looks like Guido already commented on this, so intobject.h could > be revived in some form. I'm wondering how a resurrected intobject.h should be used: would Linux distributors (for example) package up intobject.h with the rest of the python-devel RPM/package/whatever so that Python extension modules could just include it directly, or would it be better to encourage the extension writers to make a copy of intobject.h to add to the source for their extension? In the first case, intobject.h would become a requirement for those extensions, so we'd presumably end up committed to either maintaining intobject.h for the lifetime of Python 3.x, or causing some pain when it does eventually get deleted. I quite like Benjamin's suggestion (in the issue tracker) of putting it in Doc/includes. > Since this file would only be used by extension modules and possibly > also include other helper macros, perhaps it'd be better to to rename > the file to py2compat.h or something along those lines ?! > > We could then also have a py2compat.c to hold corresponding > C code, e.g. to provide compatibility wrappers of new APIs that > implement different semantics in 3.x. This is also an interesting idea. Mark From mal at egenix.com Mon Nov 23 19:27:33 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 23 Nov 2009 19:27:33 +0100 Subject: [Python-Dev] Removal of intobject.h in 3.1 In-Reply-To: <5c6f2a5d0911230912t6d3560f8m7994d990bf36fcc7@mail.gmail.com> References: <4B080207.2020808@egenix.com> <4B081762.8060007@trueblade.com> <4B0A6794.9010108@egenix.com> <5c6f2a5d0911230912t6d3560f8m7994d990bf36fcc7@mail.gmail.com> Message-ID: <4B0AD415.6020508@egenix.com> Mark Dickinson wrote: > On Mon, Nov 23, 2009 at 10:44 AM, M.-A. Lemburg wrote: >> Thanks for pointing me to the that ticket. >> >> Looks like Guido already commented on this, so intobject.h could >> be revived in some form. > > I'm wondering how a resurrected intobject.h should be used: would > Linux distributors (for example) package up intobject.h with the rest > of the python-devel RPM/package/whatever so that Python extension > modules could just include it directly, or would it be better to > encourage the extension writers to make a copy of intobject.h to add > to the source for their extension? > > In the first case, intobject.h would become a requirement for those > extensions, so we'd presumably end up committed to either > maintaining intobject.h for the lifetime of Python 3.x, or causing > some pain when it does eventually get deleted. > > I quite like Benjamin's suggestion (in the issue tracker) of putting > it in Doc/includes. I'm not sure whether Doc/ is a good location for such things. In the past, we've put such files in Misc/, e.g. the old Makefile.pre.in based extension mechanism was shipped that way: Extension writers would simply copy the file to their extension directory, create a Setup.in file and then have the users run "make -f Makefile.pre.in boot" to have a Makefile created which then built the extension. However, I'm not sure whether using the copy mechanism really works out that well: it's easier to just have the file(s) in the standard Python include file locations and then have extension writers use #include "py2compat.h" where necessary. Extension writer can then still use a private copy if needed, but most of the time they are probably better served by using the Python provided standard one. >> Since this file would only be used by extension modules and possibly >> also include other helper macros, perhaps it'd be better to to rename >> the file to py2compat.h or something along those lines ?! >> >> We could then also have a py2compat.c to hold corresponding >> C code, e.g. to provide compatibility wrappers of new APIs that >> implement different semantics in 3.x. > > This is also an interesting idea. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 23 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From solipsis at pitrou.net Mon Nov 23 19:47:46 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 23 Nov 2009 18:47:46 +0000 (UTC) Subject: [Python-Dev] Removal of intobject.h in 3.1 References: <4B080207.2020808@egenix.com> <4B081762.8060007@trueblade.com> <4B0A6794.9010108@egenix.com> Message-ID: M.-A. Lemburg egenix.com> writes: > > We could then also have a py2compat.c to hold corresponding > C code, e.g. to provide compatibility wrappers of new APIs that > implement different semantics in 3.x. If the semantic differences are embodied in the builtin object types I wonder how you can make a compatibility wrapper (short of reimplementing the 2.x types). An int is not a long, a bytes is not a str is not a unicode. I'm not sure concealing the differences between 2.x and 3.x behind such a wrapper is a good idea. It would be better if people became aware of / learnt about the new semantics. Regards Antoine. From martin at v.loewis.de Mon Nov 23 21:12:25 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 23 Nov 2009 21:12:25 +0100 Subject: [Python-Dev] Removal of intobject.h in 3.1 In-Reply-To: References: <4B080207.2020808@egenix.com> <4B081762.8060007@trueblade.com> <4B0A6794.9010108@egenix.com> Message-ID: <4B0AECA9.9080109@v.loewis.de> > I'm not sure concealing the differences between 2.x and 3.x behind such a > wrapper is a good idea. It would be better if people became aware of / learnt > about the new semantics. +1. When porting a larger code base, I always came up with a custom set of macros, specific to the way the modules were written in the package I ported. In particular for PyString_, some of the code meant to deal with bytes, other code with text. There was no way to tell, short of reading and understanding the code. As you'll have to review the complete code, anyway, the savings you gain by using macros somebody else wrote are really minor. Regards, Martin From p.f.moore at gmail.com Tue Nov 24 22:00:51 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 24 Nov 2009 21:00:51 +0000 Subject: [Python-Dev] Buildslave gets intermittent errors in the svn step Message-ID: <79990c6b0911241300m5e472b28j819aef69537c0836@mail.gmail.com> I'm getting occasional odd errors in the subversion step on my buildslave (http://www.python.org/dev/buildbot/buildslaves/moore-windows): Traceback (most recent call last): File "/data/buildbot/lib/python/buildbot/process/buildstep.py", line 690, in startStep d.addCallback(self._startStep_2) File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 195, in addCallback callbackKeywords=kw) File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 186, in addCallbacks self._runCallbacks() File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 328, in _runCallbacks self.result = callback(self.result, *args, **kw) --- --- File "/data/buildbot/lib/python/buildbot/process/buildstep.py", line 721, in _startStep_2 skip = self.start() File "/data/buildbot/lib/python/buildbot/steps/source.py", line 184, in start self.startVC(branch, revision, patch) File "/data/buildbot/lib/python/buildbot/steps/source.py", line 445, in startVC raise BuildSlaveTooOldError(m) buildbot.interfaces.BuildSlaveTooOldError: This buildslave (moore-windows) does not know about multiple branches, and using mode=update would probably build the wrong tree. Refusing to build. Please upgrade the buildslave to buildbot-0.7.0 or newer. This is very odd, as (a) a simple rerun fixes the issue, and (b) I'm already on buildbot 0.7.0 or newer (can't recall the exact version). It's a Windows XP system, if that helps. Does anyone have any suggestions? Thanks, Paul. From db3l.net at gmail.com Tue Nov 24 23:35:17 2009 From: db3l.net at gmail.com (David Bolen) Date: Tue, 24 Nov 2009 17:35:17 -0500 Subject: [Python-Dev] Buildslave gets intermittent errors in the svn step References: <79990c6b0911241300m5e472b28j819aef69537c0836@mail.gmail.com> Message-ID: Paul Moore writes: > buildbot.interfaces.BuildSlaveTooOldError: This buildslave > (moore-windows) does not know about multiple branches, and using > mode=update would probably build the wrong tree. Refusing to build. > Please upgrade the buildslave to buildbot-0.7.0 or newer. > > This is very odd, as (a) a simple rerun fixes the issue, and (b) I'm > already on buildbot 0.7.0 or newer (can't recall the exact version). > > It's a Windows XP system, if that helps. > > Does anyone have any suggestions? In my experience this error just means that someone tried to manually trigger a build but put the wrong branch information (or left it blank) on the submission form. -- David From p.f.moore at gmail.com Tue Nov 24 23:45:03 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 24 Nov 2009 22:45:03 +0000 Subject: [Python-Dev] Buildslave gets intermittent errors in the svn step In-Reply-To: References: <79990c6b0911241300m5e472b28j819aef69537c0836@mail.gmail.com> Message-ID: <79990c6b0911241445n61e31007id7e9d19e4303a2e7@mail.gmail.com> 2009/11/24 David Bolen : > Paul Moore writes: > >> buildbot.interfaces.BuildSlaveTooOldError: This buildslave >> (moore-windows) does not know about multiple branches, and using >> mode=update would probably build the wrong tree. Refusing to build. >> Please upgrade the buildslave to buildbot-0.7.0 or newer. >> >> This is very odd, as (a) a simple rerun fixes the issue, and (b) I'm >> already on buildbot 0.7.0 or newer (can't recall the exact version). >> >> It's a Windows XP system, if that helps. >> >> Does anyone have any suggestions? > > In my experience this error just means that someone tried to manually > trigger a build but put the wrong branch information (or left it > blank) on the submission form. Hmm, wasn't me :-) The reason field on the build is blank, so this looks like it's an automatic build. I'm getting odd failures with some of the tests as well, at the moment - test_smtplib and others failing but when "Re-running failed tests in verbose mode" there were no failures. I wonder if it could be a flaky network connection...? Thanks for the pointer. Paul. PS Is there a better list for this type of question? I'm not sure how interesting it is to general python-dev readers. From solipsis at pitrou.net Tue Nov 24 23:54:52 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 24 Nov 2009 22:54:52 +0000 (UTC) Subject: [Python-Dev] Buildslave gets intermittent errors in the svn step References: <79990c6b0911241300m5e472b28j819aef69537c0836@mail.gmail.com> <79990c6b0911241445n61e31007id7e9d19e4303a2e7@mail.gmail.com> Message-ID: Paul Moore gmail.com> writes: > > I'm getting odd failures with some of the tests as well, at the moment > - test_smtplib and others failing but when "Re-running failed tests in > verbose mode" there were no failures. > > I wonder if it could be a flaky network connection...? test_smtplib is notoriously flaky. > PS Is there a better list for this type of question? I'm not sure how > interesting it is to general python-dev readers. IMHO it's ok for python-dev. It's a pity buildbot issues don't generate more interest though. Regards Antoine. From db3l.net at gmail.com Wed Nov 25 01:24:35 2009 From: db3l.net at gmail.com (David Bolen) Date: Tue, 24 Nov 2009 19:24:35 -0500 Subject: [Python-Dev] Buildslave gets intermittent errors in the svn step References: <79990c6b0911241300m5e472b28j819aef69537c0836@mail.gmail.com> <79990c6b0911241445n61e31007id7e9d19e4303a2e7@mail.gmail.com> Message-ID: Paul Moore writes: > Hmm, wasn't me :-) The reason field on the build is blank, so this > looks like it's an automatic build. If it's got an SVN tag indicated, and you can checkout that tag on your slave, then it's probably not the scenario that I was thinking of. In my case, it's almost always me trying to initiate a test build, and forgetting the right branch tag or what its relative to (historically forgetting to put "branches/" in it). > I'm getting odd failures with some of the tests as well, at the moment > - test_smtplib and others failing but when "Re-running failed tests in > verbose mode" there were no failures. > > I wonder if it could be a flaky network connection...? I suppose. Certainly, for this specific error, it's pretty clear that it's not that precise, and various SVN checkout problems can probably be trapped by the error, so it's pretty unlikely it's actually a buildbot version error. What you might try is locating the buildbot log entries (twistd.log) on your slave that correspond to the failed build. It'll show the exact svn command being issued, and you can try it manually in a test tree. If it succeeds then you'll know it was some sort of transient issue - if it fails, it ought to provide useful data as well. > PS Is there a better list for this type of question? I'm not sure how > interesting it is to general python-dev readers. I don't think there's anywhere separate, and generally (recent history being a bit different) buildbot related stuff has been very low volume. So this list is probably appropriate (given what the buildbots are being used for). I expect if it gets too noisy we'll hear about it :-) -- David From martin at v.loewis.de Wed Nov 25 07:45:11 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 25 Nov 2009 07:45:11 +0100 Subject: [Python-Dev] Buildslave gets intermittent errors in the svn step In-Reply-To: <79990c6b0911241300m5e472b28j819aef69537c0836@mail.gmail.com> References: <79990c6b0911241300m5e472b28j819aef69537c0836@mail.gmail.com> Message-ID: <4B0CD277.3090703@v.loewis.de> > Does anyone have any suggestions? I guess that's a buildbot bug. At 2009-11-23 22:14:23, your buildslave disconnected. The master recognized it it as offline. For some reason, it then still decided to start a build, at 22:14:24. With the slave disconnected, all master-side information gets discarded, and it wasn't possible anymore to find out what version of the svn command the slave implements, hence the exception. Regards, Martin From g.brandl at gmx.net Wed Nov 25 09:36:31 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 25 Nov 2009 09:36:31 +0100 Subject: [Python-Dev] Making loggerClass an attribute of the logger manager? Message-ID: Hi, recently I had a use case where I wanted to use logging in two completely separate parts of the same process. One of them needs to create instances a specific Logger subclass, while the other is fine with the default loggers. I got around the problem of the unique root node by using two Managers (and then using Manager.getLogger() instead of getLogger()), but I can only set the loggerClass globally. Making the loggerClass configurable per manager would solve the problem for me, and AFAICS since most applications don't use different managers anyway, there should not be any detrimental effects. What do you think? cheers, Georg From vinay_sajip at yahoo.co.uk Wed Nov 25 11:32:15 2009 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Wed, 25 Nov 2009 10:32:15 +0000 (UTC) Subject: [Python-Dev] Making loggerClass an attribute of the logger manager? References: Message-ID: Georg Brandl gmx.net> writes: > Making the loggerClass configurable per manager would solve the > problem for me, and AFAICS since most applications don't use > different managers anyway, there should not be any detrimental > effects. What do you think? Seems reasonable. Apart from the API to set/get, _loggerClass is only used by the manager when instantiating a new logger. Regards, Vinay Sajip From vinay_sajip at yahoo.co.uk Wed Nov 25 22:00:25 2009 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Wed, 25 Nov 2009 21:00:25 +0000 (GMT) Subject: [Python-Dev] PEP 391 ready for review Message-ID: <904696.84625.qm@web25805.mail.ukl.yahoo.com> Full View PEP 391 is, I believe, ready for review. http://www.python.org/dev/peps/pep-0391/ I've also worked up an implementation, though not yet checked it in: it's available as a module "dictconfig.py" with accompanying unit tests in "test_dictconfig.py", at: http://bitbucket.org/vinay.sajip/dictconfig I believe I have incorporated into the PEP comments from people both here and on python-list. There have been comments in favour of the proposal, no objections to the proposal as a whole, and some questions and objections about specific details. I believe I have addressed these with changes to the PEP. Discussion threads on python-dev: http://mail.python.org/pipermail/python-dev/2009-October/092695.html http://mail.python.org/pipermail/python-dev/2009-October/092782.html http://mail.python.org/pipermail/python-dev/2009-October/093062.html And on python-list: http://mail.python.org/pipermail/python-list/2009-October/1223658.html http://mail.python.org/pipermail/python-list/2009-October/1224228.html There hasn't been a *huge* amount of discussion, but as it's a niche topic I suppose that's fair enough. I'm ready to actually integrate these changes in the core logging package. As I understand it, the next step is to put the PEP forward for review, which is why I'm sending this mail. I've already sent an equivalent mail to the PEP editors: David Goodger suggested I incorporate the above links (done) and Brett Cannon suggested I send this to python-dev - also, now, done. Looking forward to the feedback! Regards, Vinay Sajip From digitalxero at gmail.com Wed Nov 25 23:28:30 2009 From: digitalxero at gmail.com (Dj Gilcrease) Date: Wed, 25 Nov 2009 15:28:30 -0700 Subject: [Python-Dev] PEP 391 ready for review In-Reply-To: <904696.84625.qm@web25805.mail.ukl.yahoo.com> References: <904696.84625.qm@web25805.mail.ukl.yahoo.com> Message-ID: I would recommend removing the class keyword and replacing it with the () as you have in the custom examples or replacing () with class so it is uniform across all config options handlers: file: class : logging.handlers.RotatingFileHandler formatter: precise filename: logconfig.log maxBytes: 1024 backupCount: 3 custom: (): my.package.MyHandler alternate: cfg://handlers.file This just strikes me as odd to have to remember for built in handlers I need to use the class keyword and for my custom handlers I need to use (). My preference would be the class keyword vs the odd (), as that is what I am defining, the class to be used for this handler. The ext:// I think should be py:// when defining objects to be access via pythons normal import mechanisms and ext:// for resolving external processes or system calls From tjreedy at udel.edu Thu Nov 26 00:20:08 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 25 Nov 2009 18:20:08 -0500 Subject: [Python-Dev] PEP 391 ready for review In-Reply-To: <904696.84625.qm@web25805.mail.ukl.yahoo.com> References: <904696.84625.qm@web25805.mail.ukl.yahoo.com> Message-ID: Vinay Sajip wrote: > Full View > PEP 391 is, I believe, ready for review. > > http://www.python.org/dev/peps/pep-0391/ This is my first reading of this. Comments: nit: I believe "both of these serialization formats allow deserialization of Python dictionaries." should be "... to Python dictionaries." "bespoke configuration methods" I had to look up this mostly non-American British word, which I thought meant 'engaged (to be married, as in spoken for)' to guess that you probably mean the much more obscure 'custom configuration methods'. If so, please say so (in plain American ;-) "The logging.config module will have the following additions:" There is currently only one addition. "Nothing will be returned, though exceptions will be raised " Technically incorrect. "Returns None; exceptions ..." or just "Exceptions ...". The doc for print(), for instance, just says what is does and does not mention the default None return. "particular logger has an attached to it a particular handler." delete 'an' " '()' : 'my.package.customFormatterFactory', " This at first appeared to be a typo. You earlier wrote "the user will need to provide a 'factory' - a callable which is called with a configuration dictionary and which returns the instantiated object. This will be signalled by the factory being made available under the special key '()'. " A string is not a callable. So I thought you perhaps meant " '()' : my.package.customFormatterFactory, " without the quotes, or " '()' : <>, ", where <> is understood to be a place filler for an object of the type specified. Then I see "the specified factory callable will be located using normal import mechanisms" If you stick with this, then "This will be signalled by the factory being made available under the special key '()'." would be clearer as "This will be signalled by an absolute import path to the factory being made available under the special key '()'." However, if the code that constructs the dict passed to dictConfig() has the factory in hand, and possibly no absolute import path, why not let it pass the function directly instead of indirectly. (I am here using 'absolute import path' to mean one that logging can resolve, rather than being only resolvable from the dictConfig caller.) Being able to do so is one of the great features of Python. The code for dictConfig could easily wrap the current import. if not hasattr(factory, '__call__'): factory = __import__(factory) "The '()' also serves as a mnemonic that the corresponding value is a callable." More than just that, parentheses preceeded by an expression *are* a call operator, mapped to '__call__', just as the addition operator '+' is mapped to '__add__'. Great choice to me. ... "If serialization is successful, then dictConfig() will be called to process the resulting dictionary" deserialization Terry Jan Reedy From ctb at msu.edu Thu Nov 26 04:39:22 2009 From: ctb at msu.edu (C. Titus Brown) Date: Wed, 25 Nov 2009 19:39:22 -0800 Subject: [Python-Dev] Building a Windows MSI for Python /trunk Message-ID: <20091126033922.GA15605@idyll.org> Hi all, I got an MSI build working on my WinXP VM just now, and I wanted to touch base with whomever it is that is maintaining this (wonderful!) set of scripts... I ran into three problems, and I managed to figure out two of them; the third wasn't fatal. Note, the diff of my fixed checkout is attached. First, the script that finds & builds the external dependencies has two minor problems. * it puts Tcl in tcl-8.*, and Tk in tk-8.*, but msi.py looks for them in tcl8.* and tk8.* to grab the license text. I changed the glob strings appropriately and that seemed to work. * Tix isn't downloaded/installed/built automatically like everything else, and msi.py looks for its license file, too. I just removed the Tix reference. I can't figure out how to build Tix appropriately; any tips? Second, the buildmsi.bat file refers to python26a3.hhp instead of python27a0.hhp. Third, I could not get _tkinter to build properly, although it wasn't fatal to the endeavor. It couldn't find ..\..\tcltk\lib\tcl85.lib, although tcl85g.lib existed. Oh, and there were a bunch of missing commands that (as a non-Windows xpert) I had to figure out with google -- things like nasm/nasmw, for example. Are these documented somewhere, or would it be helpful to document them? I think I had to install: - Microsoft HTML Help Compiler - cygwin with make and python2.5 to build the docs - nasm (and copy nasm.exe to nasmw.exe) - cabarc Errm, and the 'buildmsi.bat' file has 'build' misspelled as 'buold' ;) I'd love to get this build process working completely automatically and 100% correctly, too. Hat tip to Trent Nelson, who helped me figure out where the scripts are and what other things I needed... cheers, --titus -- C. Titus Brown, ctb at msu.edu -------------- next part -------------- A non-text attachment was scrubbed... Name: msi-build.diff Type: text/x-diff Size: 1345 bytes Desc: not available URL: From vinay_sajip at yahoo.co.uk Thu Nov 26 09:01:53 2009 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Thu, 26 Nov 2009 08:01:53 +0000 (UTC) Subject: [Python-Dev] PEP 391 ready for review References: <904696.84625.qm@web25805.mail.ukl.yahoo.com> Message-ID: Terry Reedy udel.edu> writes: > [snip] Terry, Thanks for your helpful comments. I have checked in some changes to the PEP (r76533) which take into account the comments you made. Regards, Vinay Sajip From vinay_sajip at yahoo.co.uk Thu Nov 26 09:09:56 2009 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Thu, 26 Nov 2009 08:09:56 +0000 (UTC) Subject: [Python-Dev] PEP 391 ready for review References: <904696.84625.qm@web25805.mail.ukl.yahoo.com> Message-ID: Dj Gilcrease gmail.com> writes: > > I would recommend removing the class keyword and replacing it with the > () as you have in the custom examples or replacing () with class so it > is uniform across all config options [snip] > > This just strikes me as odd to have to remember for built in handlers > I need to use the class keyword and for my custom handlers I need to > use (). My preference would be the class keyword vs the odd (), as > that is what I am defining, the class to be used for this handler. If you are using the built in handlers, you will be using a class - hence the key 'class' is used. If you are using a custom factory, then the system does not force you to use a class - you can use any callable (e.g. a function which constructs an instance and sets some attributes of it before returning it), and the use of '()' indicates that you're not being forced to use a class. Of course you can, since a class is a callable, but you're not restricted to classes here. > The ext:// I think should be py:// when defining objects to be access > via pythons normal import mechanisms and ext:// for resolving external > processes or system calls > The mechanism I proposed can be extended or changed as you suggest according to the norms of a specific developer environment (e.g. organization policies), but the basic system as I see it doesn't (and shouldn't) care about whether a specific value resolves to an internal (e.g. provided by stdlib) value or an external (e.g. provided by 3rd-party lib) value. The resolution process would be exactly the same in either case. Or perhaps I misunderstood what you meant? From martin at v.loewis.de Thu Nov 26 09:54:35 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Nov 2009 09:54:35 +0100 Subject: [Python-Dev] Building a Windows MSI for Python /trunk In-Reply-To: <20091126033922.GA15605@idyll.org> References: <20091126033922.GA15605@idyll.org> Message-ID: <4B0E424B.9000204@v.loewis.de> > First, the script that finds & builds the external dependencies has two > minor problems. > > * it puts Tcl in tcl-8.*, and Tk in tk-8.*, but msi.py looks for them in > tcl8.* and tk8.* to grab the license text. I changed the glob strings > appropriately and that seemed to work. Not sure what "it" is that put it there; perhaps you are referring to the Tools/buildbot scripts? Unfortunately, the naming convention that these scripts establish doesn't quite work, as Tix would fail to build under these conventions. So in my own checkout, I manually renamed the trees, and that's what msi.py refers to. > * Tix isn't downloaded/installed/built automatically like everything else, > and msi.py looks for its license file, too. I just removed the > Tix reference. I can't figure out how to build Tix appropriately; any > tips? See above. I keep forgetting how to build Tix; one set of command lines is in PCbuild/build_tkinter.py. > Second, the buildmsi.bat file refers to python26a3.hhp instead of > python27a0.hhp. Yes, that script hasn't been updated for a long time, ever since we stopped having automated builds. Ideally, the script would find the Python version on its own. > Third, I could not get _tkinter to build properly, although it wasn't fatal > to the endeavor. It couldn't find ..\..\tcltk\lib\tcl85.lib, although > tcl85g.lib existed. You'll need to build release versions of Tcl/Tk/Tix. > Oh, and there were a bunch of missing commands that (as a non-Windows xpert) I > had to figure out with google -- things like nasm/nasmw, for example. Are > these documented somewhere, or would it be helpful to document them? See PCbuild/readme.txt. > Errm, and the 'buildmsi.bat' file has 'build' misspelled as 'buold' ;) Thanks, fixed. > I'd love to get this build process working completely automatically and > 100% correctly, too. > > Hat tip to Trent Nelson, who helped me figure out where the scripts are > and what other things I needed... Feel free to submit patches. There may be a misunderstanding, though, in that buildmsi.bat isn't actually used for anything, and isn't "meant" to be run by users, but instead by build slaves. One consequence is that these scripts build in debug mode (perhaps except for buildmsi), whereas end users would typically want to build in release mode. Another consequence is that different committers actually use different procedures. Trent created much of the Tools/buildbot setup, so he is obviously in favor of that strategy. Christian Heimes created PCbuild/build_tkinter, so he would probably prefer to use that instead. When I took over Windows builds from Tim Peters, PCbuild/readme.txt was the official reference, and I try to stick to that. Regards, Martin From ncoghlan at gmail.com Thu Nov 26 12:28:49 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 26 Nov 2009 21:28:49 +1000 Subject: [Python-Dev] PEP 391 ready for review In-Reply-To: References: <904696.84625.qm@web25805.mail.ukl.yahoo.com> Message-ID: <4B0E6671.4080604@gmail.com> Terry Reedy wrote: > if not hasattr(factory, '__call__'): factory = __import__(factory) That won't quite work since the string generally isn't referring to a module, and due to the quirk of __import__ returning the top level module since that is what the interpreter needs to bind to a name as part of a normal import statement. This would need to look more like: if not hasattr(factory, '__call__'): mod_name, dot, attr = factory.rpartition('.') if not dot: raise ValueError("") module = imputil.import_module(mod_name) factory = getattr(module, attr) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Thu Nov 26 13:11:46 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 26 Nov 2009 22:11:46 +1000 Subject: [Python-Dev] PEP 391 ready for review In-Reply-To: <4B0E6671.4080604@gmail.com> References: <904696.84625.qm@web25805.mail.ukl.yahoo.com> <4B0E6671.4080604@gmail.com> Message-ID: <4B0E7082.8010704@gmail.com> Nick Coghlan wrote: > Terry Reedy wrote: >> if not hasattr(factory, '__call__'): factory = __import__(factory) > > That won't quite work since the string generally isn't referring to a > module, and due to the quirk of __import__ returning the top level > module since that is what the interpreter needs to bind to a name as > part of a normal import statement. > > This would need to look more like: > > if not hasattr(factory, '__call__'): > mod_name, dot, attr = factory.rpartition('.') > if not dot: > raise ValueError("") > module = imputil.import_module(mod_name) > factory = getattr(module, attr) s/imputil/importlib/ (Oops...) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ctb at msu.edu Thu Nov 26 15:03:27 2009 From: ctb at msu.edu (C. Titus Brown) Date: Thu, 26 Nov 2009 06:03:27 -0800 Subject: [Python-Dev] Building a Windows MSI for Python /trunk In-Reply-To: <4B0E424B.9000204@v.loewis.de> References: <20091126033922.GA15605@idyll.org> <4B0E424B.9000204@v.loewis.de> Message-ID: <20091126140327.GB7370@idyll.org> On Thu, Nov 26, 2009 at 09:54:35AM +0100, "Martin v. L?wis" wrote: > > First, the script that finds & builds the external dependencies has two > > minor problems. > > > > * it puts Tcl in tcl-8.*, and Tk in tk-8.*, but msi.py looks for them in > > tcl8.* and tk8.* to grab the license text. I changed the glob strings > > appropriately and that seemed to work. > > Not sure what "it" is that put it there; perhaps you are referring to > the Tools/buildbot scripts? Yes, sorry, Tools/buildbot/external*.bat, as in the diff. > Unfortunately, the naming convention that these scripts establish > doesn't quite work, as Tix would fail to build under these conventions. > So in my own checkout, I manually renamed the trees, and that's what > msi.py refers to. OK, thanks! > > * Tix isn't downloaded/installed/built automatically like everything else, > > and msi.py looks for its license file, too. I just removed the > > Tix reference. I can't figure out how to build Tix appropriately; any > > tips? > > See above. I keep forgetting how to build Tix; one set of command lines > is in PCbuild/build_tkinter.py. Ok, thanks! > > Second, the buildmsi.bat file refers to python26a3.hhp instead of > > python27a0.hhp. > > Yes, that script hasn't been updated for a long time, ever since we > stopped having automated builds. > > Ideally, the script would find the Python version on its own. Right... > > Third, I could not get _tkinter to build properly, although it wasn't fatal > > to the endeavor. It couldn't find ..\..\tcltk\lib\tcl85.lib, although > > tcl85g.lib existed. > > You'll need to build release versions of Tcl/Tk/Tix. Yes, I saw a reference to that online, but I wasn't sure if that was the problem -- is the naming convention really that 'tcl85g.lib' is the debug lib!? > > Oh, and there were a bunch of missing commands that (as a non-Windows xpert) I > > had to figure out with google -- things like nasm/nasmw, for example. Are > > these documented somewhere, or would it be helpful to document them? > > See PCbuild/readme.txt. OK, thanks. > > I'd love to get this build process working completely automatically and > > 100% correctly, too. > > > > Hat tip to Trent Nelson, who helped me figure out where the scripts are > > and what other things I needed... > > Feel free to submit patches. There may be a misunderstanding, though, in > that buildmsi.bat isn't actually used for anything, and isn't "meant" to > be run by users, but instead by build slaves. Well, yes, although wouldn't we want the scripts to run without any user editing, for the build slaves? > One consequence is that these scripts build in debug mode (perhaps > except for buildmsi), whereas end users would typically want to build in > release mode. Right. Is there a reason the buildbot builds were stopped? I've added binary file uploads to pony-build, so that platform-specific files can be uploaded to build results -- see, e.g. http://lyorn.idyll.org/ctb/pb-dev/p/python/10768/files/ or http://lyorn.idyll.org/ctb/pb-dev/p/pygr/10805/files/ and I thought daily -latest builds for Python latest on Windows and Mac would be a good demo... Incidentally, the pony-build results & file upload scripts can be used from within buildbot, if people are interested. > Another consequence is that different committers actually use different > procedures. Trent created much of the Tools/buildbot setup, so he is > obviously in favor of that strategy. Christian Heimes created > PCbuild/build_tkinter, so he would probably prefer to use that instead. > When I took over Windows builds from Tim Peters, PCbuild/readme.txt > was the official reference, and I try to stick to that. OK, I see. Thanks! So if I can bring the scripts into concordance with that it might be valuable? --titus -- C. Titus Brown, ctb at msu.edu From martin at v.loewis.de Thu Nov 26 16:07:25 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Nov 2009 16:07:25 +0100 Subject: [Python-Dev] Building a Windows MSI for Python /trunk In-Reply-To: <20091126140327.GB7370@idyll.org> References: <20091126033922.GA15605@idyll.org> <4B0E424B.9000204@v.loewis.de> <20091126140327.GB7370@idyll.org> Message-ID: <4B0E99AD.707@v.loewis.de> >> You'll need to build release versions of Tcl/Tk/Tix. > > Yes, I saw a reference to that online, but I wasn't sure if that was the > problem -- is the naming convention really that 'tcl85g.lib' is the debug > lib!? Yes; Tcl apparently follows Unix conventions here rather than Windows conventions (where the debug build would end with "d"). >> Feel free to submit patches. There may be a misunderstanding, though, in >> that buildmsi.bat isn't actually used for anything, and isn't "meant" to >> be run by users, but instead by build slaves. > > Well, yes, although wouldn't we want the scripts to run without any user > editing, for the build slaves? Sure. For the scripts that are in actual use, that's also the case. I stopped working on buildmsi because it was just too tedious to get right. > Is there a reason the buildbot builds were stopped? It was too difficult to get right, plus nobody was interested in using the daily MSI files. At some point, it was broken for several months (IIRC), with nobody reporting that breakage. So when we noticed, we just turned the service off. > and I thought daily -latest builds for Python latest on Windows and Mac would > be a good demo... I'll guess that you will find the same: nobody is really interested in *using* daily builds of Python. > OK, I see. Thanks! So if I can bring the scripts into concordance with > that it might be valuable? Most certainly, yes. Regards, Martin From cesare.di.mauro at gmail.com Thu Nov 26 22:50:42 2009 From: cesare.di.mauro at gmail.com (Cesare Di Mauro) Date: Thu, 26 Nov 2009 22:50:42 +0100 Subject: [Python-Dev] wpython is back Message-ID: Hi Mart I'm back with some news about wpython. I completed all the work that I was committed to do till the end of the year. I made a lot of changes to the code, that I'll report here. First, I added several conditional compilation sections that enable or disable almost every optimization I introduced into the project. Everything is controlled by a new include file, wpython.h, which holds a lot of #DEFINEs for each one of them. Every #DEFINE has a brief explanation, and some report an example with Python code disassembled, showing what happens. It can be useful both to document the code (also to access to the interested parts), and to let people test the effect of all optimizations. There are also a couple of #DEFINEs which are useful to enable or disable all superinstructions, or to make wpython work like CPython (with all new optimizations and superinstructions disabled). Full tracing support required a big effort, due to the missing SETUP_LOOP/POP_BLOCK instructions used in FOR_ITER blocks. It was a pain in the neck to let them work, but I think I have found a good solution fot it. If I remember correctly, Collin asked in the past about performance with testing enabled. I believe that speed is comparable to CPython, since I can trace FOR_ITER blocks enter/exit with very little time spent intercepting them; stack unrolling (for forward jumps cases) is fast too. Restoring Python object model required much of the work. I reverted all the changes that I made to many PyObjects, and just added some accessory code only to a few of them. There are no more hacks, and code is quite polite; only CodeObject required one line of code change in the hash function, to let it calculate hash correctly for the constants tuple (because it can hold lists and dictionaries now, which usally aren't hashable). Every file in Include/ and Objects/ that I modified has only 1 diff (except frameobject.c, for tracing code), so it's easy so see what is changed and the extra helper functions that I added to introduce lists and dictionaries in the consts tuple. In the meanwhile I've added a little optimization for lists and dictionaries used in for loops. Writing this: def f(): for x in ['a', 'b', 'c']: print x generates the following (word)code with the previous wpython: LOAD_CONST (['a', 'b', 'c']) DEEP_LIST_COPY GET_ITER FOR_ITER because ['a', 'b', 'c'] is a mutable object, and a copy must be made before using it. Now it'll be: LOAD_CONST (['a', 'b', 'c']) GET_ITER FOR_ITER So code is reduced and memory consumption too, because there's no need clone the list. The trick works only for lists and dictionaries that holds non-mutable objects, but I found it's a common pattern in Python code. I've also updated the source to the latest Python 2.x version, 2.6.4. All tests pass, both with Debug and Release code, on Visual Studio Express with 32 bit code (I can't compile 64 bits versions with it). There are only a few open issues. test_syntax.py required some changes in the doctest (adding full filesystem path) to let them pass correctly. It's really strange, but... works now! test_compile.py has 2 tests disabled in test_compile_ast: #['', """for n in [1, 2, 3]:\n print n\n"""], #[fname, fcontents], that's because there's no support for constants (except Num_kind and Str_kind) in the current ASTs code. However code compiles well, except that it cannot make use of the new constant folding code. I haven't updated Doc/library/dis.rst, which is exactly the same of CPython. I'll do it when I stop introducing or changing opcodes. Right now wpython requires manual patching of Include/Python-ast.h, with the following lines: enum _expr_kind {BoolOp_kind=1, BinOp_kind=2, UnaryOp_kind=3, Lambda_kind=4, [...] List_kind=18, Tuple_kind=19, Const_kind=20}; enum _expr_const {no_const=0, mutable_const=1, content_const=3, pure_const=7}; struct _expr { enum _expr_kind kind; union { [...] struct { object c; enum _expr_const constant; } Const; } v; int lineno; int col_offset; }; They are need to let ast.c handle constants for the new constant folding code. I greatly appreciate any help to let it be generated automatically with ASDL grammar. That's all about the new code. Now the weird and stupid part. A few days I got a new gmail account, but accidentally I removed the google account that I've used to create the wpython at Google Code. I definitely lost project ownership, so I can't tag the old code and put the new one in trunk. I'll thank very much if someone that works or has contacts with Google can ask for moving ownership from my old account (cesare at pronto do it) to my new (the one which I've using now to write this mail), so I'll commit ASAP. Alternatively, I need to create a new project at Google Code. I hope that the community will appreciate the work (when I'll upload it :-). I know that it's a young project, but I think it's mature enough to take a look at it. Last but not least, think about it like a starting point. I have many ideas on how to optimize several other parts of Python, and the wordcode structure gives me rooms to do it in an elegant and efficient way thanks to the superinstructions (when needed). For the next release I plan to cleanup opcode.h and ceval.c, grouping some instructions into single superinstructions (CALL_FUNCTIONs and IMPORT_NAME), adding a few opcodes, and tweaking a bit the VM main loop (primarily targeted to reduce the jump-table that compilers produce for the big switch statement). Then I'll consider porting it to python 2.7 and/or python 3.1/3.2 if there'll be interest and feedbacks about it. I'm also at your disposal to discuss any detail about wpython source code, since I know that it isn't a simple patch to apply to some files. Cheers, Cesare 2009/11/4 Mart S?mermaa > Thanks for the recap and for the good work on wpython! > > Best, eagerly waiting for the results of your work to land in mainline > python, > MS > -------------- next part -------------- An HTML attachment was scrubbed... URL: From db3l.net at gmail.com Fri Nov 27 00:14:03 2009 From: db3l.net at gmail.com (David Bolen) Date: Thu, 26 Nov 2009 18:14:03 -0500 Subject: [Python-Dev] Building a Windows MSI for Python /trunk References: <20091126033922.GA15605@idyll.org> <4B0E424B.9000204@v.loewis.de> <20091126140327.GB7370@idyll.org> <4B0E99AD.707@v.loewis.de> Message-ID: "Martin v. L?wis" writes: > It was too difficult to get right, plus nobody was interested in > using the daily MSI files. > > At some point, it was broken for several months (IIRC), with nobody > reporting that breakage. So when we noticed, we just turned the service > off. Sounds right - the daily MSIs were built on my slave from Sep 2007 into 2008 somewhere, but it had been failing for 6 months or more when the task was turned off last December. I'm not sure precisely when the failures started (the buildbot status page was also down for a while starting in March), but it had been failing for a long time. I believe the failures at the time were tcl related - maybe some of the MSI buildbot-related scripts getting out of date with ongoing development or something. Sounds like it might be the same stuff Titus is running into. It didn't seem worth the effort to track down at the time, given that nobody had seemed to notice the lack of builds. -- David From g.brandl at gmx.net Fri Nov 27 00:41:56 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 27 Nov 2009 00:41:56 +0100 Subject: [Python-Dev] wpython is back In-Reply-To: References: Message-ID: Cesare Di Mauro schrieb: > Hi Mart > > I'm back with some news about wpython. I completed all the work that I > was committed to do till the end of the year. I made a lot of changes to > the code, that I'll report here.. For those of us without an elephant's memory, could you please also recap what exactly wpython is, and maybe what the goals and intentions are? cheers, Georg From guido at python.org Fri Nov 27 00:55:01 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Nov 2009 15:55:01 -0800 Subject: [Python-Dev] wpython is back In-Reply-To: References: Message-ID: On Thu, Nov 26, 2009 at 3:41 PM, Georg Brandl wrote: > Cesare Di Mauro schrieb: >> Hi Mart >> >> I'm back with some news about wpython. I completed all the work that I >> was committed to do till the end of the year. I made a lot of changes to >> the code, that I'll report here.. > > For those of us without an elephant's memory, could you please also recap > what exactly wpython is, and maybe what the goals and intentions are? It's a Python implementation that uses wordcode instead of bytecode. http://code.google.com/p/wpython/ I don't see any benchmarks though. -- --Guido van Rossum (python.org/~guido) From cesare.di.mauro at gmail.com Fri Nov 27 07:01:17 2009 From: cesare.di.mauro at gmail.com (Cesare Di Mauro) Date: Fri, 27 Nov 2009 07:01:17 +0100 Subject: [Python-Dev] wpython is back In-Reply-To: References: Message-ID: 2009/11/27 Guido van Rossum > It's a Python implementation that uses wordcode instead of bytecode. > > http://code.google.com/p/wpython/ > > I don't see any benchmarks though. > You'll find some at page 28 here . Mart made more interesting oneswith Unladen benchmarks. Cesare -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at cheimes.de Fri Nov 27 15:09:55 2009 From: lists at cheimes.de (Christian Heimes) Date: Fri, 27 Nov 2009 15:09:55 +0100 Subject: [Python-Dev] wpython is back In-Reply-To: References: Message-ID: <4B0FDDB3.9010605@cheimes.de> Cesare Di Mauro wrote: > > You'll find some at page 28 > here > .. > > Mart made more interesting > oneswith > Unladen benchmarks. The PDF document sounded interesting and I was tempted to test WPython. Unfortunately it doesn't compile on my box: $ make gcc -pthread -c -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -DPy_BUILD_CORE -o Python/ast.o Python/ast.c Python/ast.c:30: warning: ?enum _expr_const? declared inside parameter list Python/ast.c:30: warning: its scope is only this definition or declaration, which is probably not what you want Python/ast.c:335: warning: ?enum _expr_const? declared inside parameter list Python/ast.c:335: error: parameter 2 (?constant?) has incomplete type Python/ast.c: In function ?Const?: Python/ast.c:341: error: ?Const_kind? undeclared (first use in this function) Python/ast.c:341: error: (Each undeclared identifier is reported only once Python/ast.c:341: error: for each function it appears in.) Python/ast.c:342: error: ?union ? has no member named ?Const? Python/ast.c:343: error: ?union ? has no member named ?Const? Python/ast.c: In function ?set_context?: Python/ast.c:457: error: ?Const_kind? undeclared (first use in this function) Python/ast.c: At top level: Python/ast.c:591: warning: ?enum _expr_const? declared inside parameter list Python/ast.c:590: error: conflicting types for ?seq_for_testlist? Python/ast.c:29: note: previous declaration of ?seq_for_testlist? was here [...] $ gcc --version gcc (Ubuntu 4.4.1-4ubuntu8) 4.4.1 $ uname -a Linux hamiller 2.6.31-14-generic #48-Ubuntu SMP Fri Oct 16 14:05:01 UTC 2009 x86_64 GNU/Linux From cesare.di.mauro at gmail.com Fri Nov 27 15:18:24 2009 From: cesare.di.mauro at gmail.com (Cesare Di Mauro) Date: Fri, 27 Nov 2009 15:18:24 +0100 Subject: [Python-Dev] wpython is back In-Reply-To: <4B0FDDB3.9010605@cheimes.de> References: <4B0FDDB3.9010605@cheimes.de> Message-ID: 2009/11/27 Christian Heimes > Cesare Di Mauro wrote: > > > > You'll find some at page 28 > > here< > http://wpython.googlecode.com/files/Beyond%20Bytecode%20-%20A%20Wordcode-based%20Python.pdf > > > > .. > > > > Mart made more interesting > > ones >with > > Unladen benchmarks. > > The PDF document sounded interesting and I was tempted to test WPython. > Unfortunately it doesn't compile on my box: > > [...] That's because Include/Python-ast.h file is autogenerated from the ASDL grammar file the first time that you try to compile wpython. You need to replace it with the one bundled with wpython. It's a known problem that I'll address ASAP. Cesare -------------- next part -------------- An HTML attachment was scrubbed... URL: From status at bugs.python.org Fri Nov 27 18:07:14 2009 From: status at bugs.python.org (Python tracker) Date: Fri, 27 Nov 2009 18:07:14 +0100 (CET) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20091127170714.1913878608@psf.upfronthosting.co.za> ACTIVITY SUMMARY (11/20/09 - 11/27/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2506 open (+19) / 16719 closed (+14) / 19225 total (+33) Open issues with patches: 999 Average duration of open issues: 689 days. Median duration of open issues: 444 days. Open Issues Breakdown open 2471 (+19) pending 34 ( +0) Issues Created Or Reopened (34) _______________________________ multiprocessing logging support test 11/24/09 http://bugs.python.org/issue6615 reopened jnoller patch Windows 64-bit build "skips" several projects in VS 2008 11/20/09 CLOSED http://bugs.python.org/issue7371 created srid Regression in pstats 11/21/09 http://bugs.python.org/issue7372 created belopolsky patch Use PyModule_AddIntMacro() in Modules/gcmodule.c 11/21/09 CLOSED http://bugs.python.org/issue7373 created jon patch Property accessor/getter called twice 11/22/09 CLOSED http://bugs.python.org/issue7374 created mliddle 2to3 - does not translate urllib2 to urllib.request correctly fo 11/22/09 CLOSED http://bugs.python.org/issue7375 created orsenthil "python -m doctest" results in FAIL: Doctest: __main__.DebugRunn 11/22/09 http://bugs.python.org/issue7376 created flox patch Slight difference: math.floor returns an Integral 11/22/09 CLOSED http://bugs.python.org/issue7377 created flox unexpected truncation of traceback 11/23/09 http://bugs.python.org/issue7378 created ghewgill patch Docstring for fractions.Fraction.limit_denominator fails 11/23/09 CLOSED http://bugs.python.org/issue7379 created flox uuid.UUID.bytes gives a bytearray() instead of bytes 11/23/09 http://bugs.python.org/issue7380 created flox patch subprocess.check_output: "docstring has inconsistent leading whi 11/23/09 http://bugs.python.org/issue7381 created flox patch bytes.__getnewargs__ is broken; copy.copy() therefore doesn't wo 11/23/09 http://bugs.python.org/issue7382 created sh patch, easy test_multiprocessing leaks 11/23/09 CLOSED http://bugs.python.org/issue7383 created pitrou test_curses crash on FreeBSD buildbots 11/23/09 http://bugs.python.org/issue7384 created mark.dickinson buildbot MemoryView_FromObject crashes if PyBuffer_GetBuffer fails 11/23/09 http://bugs.python.org/issue7385 created pv More precise document on os.path.normpath() 11/24/09 http://bugs.python.org/issue7386 created pengyu.ut More precise document on os.path.normpath() 11/24/09 CLOSED http://bugs.python.org/issue7387 created pengyu.ut Documentation: capitalizations of the word 'python' needed when 11/24/09 http://bugs.python.org/issue7388 created Retro easy Make decimal floating point be default, remove binary floating p 11/24/09 CLOSED http://bugs.python.org/issue7389 created Retro inconsistent type return 11/24/09 CLOSED http://bugs.python.org/issue7390 created surkamp Re-title the "Using Backslash to Continue Statements" anti-idiom 11/24/09 http://bugs.python.org/issue7391 created Horpner cPickle test failure on release26-maint branch 11/24/09 http://bugs.python.org/issue7392 created alexandre.vassalotti Executing Python program of sum of 2 nos. 11/25/09 CLOSED http://bugs.python.org/issue7393 created mayur78 sqlite3: some OperationalError exceptions should be ProgrammingE 11/25/09 http://bugs.python.org/issue7394 created dontbugme pstats add command raises unhandled exception 11/25/09 http://bugs.python.org/issue7395 created exarkun regrtest single: iterator not subscriptable 11/26/09 http://bugs.python.org/issue7396 created jgsack __import__ docs should reference importlib.import_module 11/26/09 http://bugs.python.org/issue7397 created ncoghlan easy Issue with _PyLong_Format while trying to build Python on HP-UX 11/26/09 CLOSED http://bugs.python.org/issue7398 created paulh_irl Possible bug in Python Tutorial 11/26/09 http://bugs.python.org/issue7399 created dh Small typo in Python's library documentation: set 11/26/09 CLOSED http://bugs.python.org/issue7400 created trovao patch os.write and os.close on pipe from separate threads hangs on Mac 11/26/09 http://bugs.python.org/issue7401 created hughsw reduce() is an anti-example in "Idioms and Anti-Idioms" 11/27/09 http://bugs.python.org/issue7402 created cito Race condition in logging._acquireLock()? 11/27/09 CLOSED http://bugs.python.org/issue7403 created allenap Issues Now Closed (34) ______________________ Traceback doesn't run back all the way 412 days http://bugs.python.org/issue4097 benjamin.peterson compilation of sqlite3 fails 400 days http://bugs.python.org/issue4158 brett.cannon urllib2 HTTPS connection failure (BadStatusLine Exception) 307 days http://bugs.python.org/issue5007 orsenthil easy multiprocessing example wrong 224 days http://bugs.python.org/issue5738 jnoller patch datetime.timedelta is inconvenient to use... 222 days http://bugs.python.org/issue5788 pitrou patch tarfile: opening an empty tar file fails 179 days http://bugs.python.org/issue6123 lars.gustaebel patch OS X: python3 from python-3.1.dmg crashes at startup 146 days http://bugs.python.org/issue6393 ronaldoussoren patch, needs review expose setresuid 133 days http://bugs.python.org/issue6508 loewis patch Restart support in binary upload for ftplib 83 days http://bugs.python.org/issue6845 pitrou patch Backport py3k float repr to trunk 43 days http://bugs.python.org/issue7117 mark.dickinson patch cPickle looking for non-existent package copyreg 41 days http://bugs.python.org/issue7128 alexandre.vassalotti patch, easy %lld for PyErr_Format (Modules/_io/bufferedio.c) 28 days http://bugs.python.org/issue7228 mark.dickinson patch global statements outside functions/methods should raise SyntaxE 6 days http://bugs.python.org/issue7329 rhettinger calendar.TextCalendar().formatyear wrong argument list 5 days http://bugs.python.org/issue7345 georg.brandl csv doesn't handle escaped characters properly 2 days http://bugs.python.org/issue7350 tjreedy Fibonacci example does not include 0; section 4.6 3 days http://bugs.python.org/issue7369 mark.dickinson Windows 64-bit build "skips" several projects in VS 2008 5 days http://bugs.python.org/issue7371 srid Use PyModule_AddIntMacro() in Modules/gcmodule.c 1 days http://bugs.python.org/issue7373 benjamin.peterson patch Property accessor/getter called twice 0 days http://bugs.python.org/issue7374 mliddle 2to3 - does not translate urllib2 to urllib.request correctly fo 0 days http://bugs.python.org/issue7375 benjamin.peterson Slight difference: math.floor returns an Integral 0 days http://bugs.python.org/issue7377 r.david.murray Docstring for fractions.Fraction.limit_denominator fails 0 days http://bugs.python.org/issue7379 mark.dickinson test_multiprocessing leaks 1 days http://bugs.python.org/issue7383 r.david.murray More precise document on os.path.normpath() 0 days http://bugs.python.org/issue7387 benjamin.peterson Make decimal floating point be default, remove binary floating p 0 days http://bugs.python.org/issue7389 mark.dickinson inconsistent type return 0 days http://bugs.python.org/issue7390 pitrou Executing Python program of sum of 2 nos. 0 days http://bugs.python.org/issue7393 rhettinger Issue with _PyLong_Format while trying to build Python on HP-UX 0 days http://bugs.python.org/issue7398 eric.smith Small typo in Python's library documentation: set 0 days http://bugs.python.org/issue7400 georg.brandl patch Race condition in logging._acquireLock()? 0 days http://bugs.python.org/issue7403 vinay.sajip Handle ungzipped man pages in rpm builds correctly 1705 days http://bugs.python.org/issue1169193 dmalcolm patch difflib.Differ() doesn't always add hints for tab characters 1288 days http://bugs.python.org/issue1488943 orsenthil patch urllib2 proxy does not work in 2.4.3 1233 days http://bugs.python.org/issue1519816 orsenthil csv module broken for unicode 1091 days http://bugs.python.org/issue1606092 r.david.murray Top Issues Most Discussed (10) ______________________________ 21 multiprocessing logging support test 3 days open http://bugs.python.org/issue6615 8 test_multiprocessing fails consistently with 'signal 12' on Fre 21 days open http://bugs.python.org/issue7272 6 test_multiprocessing leaks 1 days closed http://bugs.python.org/issue7383 6 Cython compiler run crashes CPython 3.1.1 and 3.2 38 days open http://bugs.python.org/issue7173 5 "python -m doctest" results in FAIL: Doctest: __main__.DebugRun 5 days open http://bugs.python.org/issue7376 5 Fibonacci example does not include 0; section 4.6 3 days closed http://bugs.python.org/issue7369 5 cPickle looking for non-existent package copyreg 41 days closed http://bugs.python.org/issue7128 5 Backport py3k float repr to trunk 43 days closed http://bugs.python.org/issue7117 5 bdist_rpm fails when installing man pages 2557 days open http://bugs.python.org/issue644744 5 deepcopy doesn't copy instance methods 730 days open http://bugs.python.org/issue1515 From ctb at msu.edu Sun Nov 29 18:32:59 2009 From: ctb at msu.edu (C. Titus Brown) Date: Sun, 29 Nov 2009 09:32:59 -0800 Subject: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5 Message-ID: <20091129173259.GA26611@idyll.org> Hi all, /trunk test_distutils is failing with the following error on Mac OS X 10.5: Here's the error: test_distutils test test_distutils failed -- Traceback (most recent call last): File "/private/tmp/tmp8UfLPT/python27/Lib/distutils/tests/test_sdist.py", line 342, in test_make_distribution_owner_group self.assertEquals(member.gid, os.getgid()) AssertionError: 0 != 20 It has been a problem for over a week, perhaps longer. I've filed it as: http://bugs.python.org/issue7408 ... So why am I posting this to python-dev? I went to double-check this on the buildbots and noticed that there aren't any Mac OS X buildbots. I would be happy to give one or two people remote access to my Mac OS X 10.5 iMac if someone wanted to set up a buildbot and/or debug this issue further. Tarek, I can give you access immediately through your lyorn account, too. cheers, --titus -- C. Titus Brown, ctb at msu.edu From ziade.tarek at gmail.com Sun Nov 29 18:44:36 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 29 Nov 2009 18:44:36 +0100 Subject: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5 In-Reply-To: <20091129173259.GA26611@idyll.org> References: <20091129173259.GA26611@idyll.org> Message-ID: <94bdd2610911290944v10a41bf1te83b96b3bc6c6ad7@mail.gmail.com> On Sun, Nov 29, 2009 at 6:32 PM, C. Titus Brown wrote: > Hi all, > > /trunk test_distutils is failing with the following error on Mac OS X 10.5: > > Here's the error: > > test_distutils > test test_distutils failed -- Traceback (most recent call last): > File > "/private/tmp/tmp8UfLPT/python27/Lib/distutils/tests/test_sdist.py", > line 342, in test_make_distribution_owner_group > ? ?self.assertEquals(member.gid, os.getgid()) > AssertionError: 0 != 20 > > It has been a problem for over a week, perhaps longer. ?I've filed it as: > > ? ? ? ?http://bugs.python.org/issue7408 I don't have this problem on my own Mac OS 10.5, I am investigating, > > ... So why am I posting this to python-dev? > > I went to double-check this on the buildbots and noticed that there > aren't any Mac OS X buildbots. I thought we had one at some point, >?I would be happy to give one or two > people remote access to my Mac OS X 10.5 iMac if someone wanted to > set up a buildbot and/or debug this issue further. > > Tarek, I can give you access immediately through your lyorn account, too. Please, yes that would be awesome Thank you From martin at v.loewis.de Sun Nov 29 19:28:50 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Nov 2009 19:28:50 +0100 Subject: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5 In-Reply-To: <20091129173259.GA26611@idyll.org> References: <20091129173259.GA26611@idyll.org> Message-ID: <4B12BD62.9040907@v.loewis.de> > I went to double-check this on the buildbots and noticed that there > aren't any Mac OS X buildbots. That's not true, see http://www.python.org/dev/buildbot/builders/x86 osx.5 trunk Regards, Martin From ctb at msu.edu Sun Nov 29 19:36:40 2009 From: ctb at msu.edu (C. Titus Brown) Date: Sun, 29 Nov 2009 10:36:40 -0800 Subject: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5 In-Reply-To: <4B12BD62.9040907@v.loewis.de> References: <20091129173259.GA26611@idyll.org> <4B12BD62.9040907@v.loewis.de> Message-ID: <20091129183636.GA18249@idyll.org> On Sun, Nov 29, 2009 at 07:28:50PM +0100, "Martin v. L?wis" wrote: > > I went to double-check this on the buildbots and noticed that there > > aren't any Mac OS X buildbots. > > That's not true, see > > http://www.python.org/dev/buildbot/builders/x86 osx.5 trunk Ahh! Sorry, was searching for 'mac' somewhere in the string. Those tests are also broken but in different areas from mine. Again, if people would like shell access, just ask. best, --titus -- C. Titus Brown, ctb at msu.edu From dickinsm at gmail.com Sun Nov 29 19:43:50 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Sun, 29 Nov 2009 18:43:50 +0000 Subject: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5 In-Reply-To: <20091129183636.GA18249@idyll.org> References: <20091129173259.GA26611@idyll.org> <4B12BD62.9040907@v.loewis.de> <20091129183636.GA18249@idyll.org> Message-ID: <5c6f2a5d0911291043x95f8a10n24f95926a96f8076@mail.gmail.com> On Sun, Nov 29, 2009 at 6:36 PM, C. Titus Brown wrote: > On Sun, Nov 29, 2009 at 07:28:50PM +0100, "Martin v. L?wis" wrote: >> That's not true, see >> >> http://www.python.org/dev/buildbot/builders/x86 osx.5 trunk > > Ahh! ?Sorry, was searching for 'mac' somewhere in the string. > > Those tests are also broken but in different areas from mine. ?Again, > if people would like shell access, just ask. I think that buildbot is running OS X 10.6 (despite its name). It would definitely be useful to have an OS X 10.5 buildbot as well, since there are significant differences: most obviously, the default build on 10.6 is an LP64 build. I'm not seeing the test_distutils failure you report on my own 10.5 machine, for some reason. Mark From ctb at msu.edu Sun Nov 29 19:47:22 2009 From: ctb at msu.edu (C. Titus Brown) Date: Sun, 29 Nov 2009 10:47:22 -0800 Subject: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5 In-Reply-To: <5c6f2a5d0911291043x95f8a10n24f95926a96f8076@mail.gmail.com> References: <20091129173259.GA26611@idyll.org> <4B12BD62.9040907@v.loewis.de> <20091129183636.GA18249@idyll.org> <5c6f2a5d0911291043x95f8a10n24f95926a96f8076@mail.gmail.com> Message-ID: <20091129184722.GA32263@idyll.org> On Sun, Nov 29, 2009 at 06:43:50PM +0000, Mark Dickinson wrote: > On Sun, Nov 29, 2009 at 6:36 PM, C. Titus Brown wrote: > > On Sun, Nov 29, 2009 at 07:28:50PM +0100, "Martin v. L?wis" wrote: > >> That's not true, see > >> > >> http://www.python.org/dev/buildbot/builders/x86 osx.5 trunk > > > > Ahh! ??Sorry, was searching for 'mac' somewhere in the string. > > > > Those tests are also broken but in different areas from mine. ??Again, > > if people would like shell access, just ask. > > I think that buildbot is running OS X 10.6 (despite its name). It > would definitely be useful to have an OS X 10.5 buildbot as well, > since there are significant differences: most obviously, the > default build on 10.6 is an LP64 build. > > I'm not seeing the test_distutils failure you report on my own > 10.5 machine, for some reason. Yes, neither is Tarek... I'll see if I can track down why. I don't think I installed anything other than the minimum necessary to build Python, but that's a bit worrisome. cheers, --titus -- C. Titus Brown, ctb at msu.edu From nad at acm.org Sun Nov 29 20:55:46 2009 From: nad at acm.org (Ned Deily) Date: Sun, 29 Nov 2009 11:55:46 -0800 Subject: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5 References: <20091129173259.GA26611@idyll.org> <4B12BD62.9040907@v.loewis.de> <20091129183636.GA18249@idyll.org> <5c6f2a5d0911291043x95f8a10n24f95926a96f8076@mail.gmail.com> <20091129184722.GA32263@idyll.org> Message-ID: In article <20091129184722.GA32263 at idyll.org>, "C. Titus Brown" wrote: > On Sun, Nov 29, 2009 at 06:43:50PM +0000, Mark Dickinson wrote: > > I'm not seeing the test_distutils failure you report on my own > > 10.5 machine, for some reason. > > Yes, neither is Tarek... I'll see if I can track down why. I don't > think I installed anything other than the minimum necessary to build > Python, but that's a bit worrisome. I'm also seeing it on 10.5. Investigating... -- Ned Deily, nad at acm.org From ziade.tarek at gmail.com Sun Nov 29 21:33:45 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 29 Nov 2009 21:33:45 +0100 Subject: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5 In-Reply-To: References: <20091129173259.GA26611@idyll.org> <4B12BD62.9040907@v.loewis.de> <20091129183636.GA18249@idyll.org> <5c6f2a5d0911291043x95f8a10n24f95926a96f8076@mail.gmail.com> <20091129184722.GA32263@idyll.org> Message-ID: <94bdd2610911291233h434f129dvcb7976ac87b0ae5b@mail.gmail.com> On Sun, Nov 29, 2009 at 8:55 PM, Ned Deily wrote: > In article <20091129184722.GA32263 at idyll.org>, > ?"C. Titus Brown" wrote: >> On Sun, Nov 29, 2009 at 06:43:50PM +0000, Mark Dickinson wrote: >> > I'm not seeing the test_distutils failure you report on my own >> > 10.5 machine, for some reason. >> >> Yes, neither is Tarek... I'll see if I can track down why. ?I don't >> think I installed anything other than the minimum necessary to build >> Python, but that's a bit worrisome. > > I'm also seeing it on 10.5. ?Investigating... > Following-up in the issue; From db3l.net at gmail.com Sun Nov 29 22:04:06 2009 From: db3l.net at gmail.com (David Bolen) Date: Sun, 29 Nov 2009 16:04:06 -0500 Subject: [Python-Dev] /trunk test_distutils failing on Mac OS X 10.5 References: <20091129173259.GA26611@idyll.org> <4B12BD62.9040907@v.loewis.de> <20091129183636.GA18249@idyll.org> <5c6f2a5d0911291043x95f8a10n24f95926a96f8076@mail.gmail.com> Message-ID: Mark Dickinson writes: > I'm not seeing the test_distutils failure you report on my own > 10.5 machine, for some reason. I don't think it's OSX specific. My FreeBSD slaves (both 6.4 and 7.2) have been getting the same error. It seems related to the sticky bit on /tmp and the fact that the account running the build slave is in the wheel(0) group, so directories/files created beneath /tmp get a group of 0 rather than the user's gid. That affects the temporary files in the distutils test, which is expecting a default of os.getgid(). -- David From aahz at pythoncraft.com Mon Nov 30 17:13:28 2009 From: aahz at pythoncraft.com (Aahz) Date: Mon, 30 Nov 2009 08:13:28 -0800 Subject: [Python-Dev] PyCon 2010 registration open! (Early-bird Jan 6) Message-ID: <20091130161328.GA12794@panix.com> PyCon 2010 registration has opened! Register by January 6 for the best rates! http://us.pycon.org/2010/registration/ Registering early gets you early-bird registration rates, guarantees you the tutorials you want, and helps the PyCon volunteers plan better. Scheduled talk and tutorial lists: http://us.pycon.org/2010/conference/talks/ http://us.pycon.org/2010/tutorials/ We'll see you in Atlanta! Spread the word! -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ The best way to get information on Usenet is not to ask a question, but to post the wrong information.