From ubershmekel at gmail.com Thu Oct 1 03:07:19 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Thu, 1 Oct 2009 04:07:19 +0300 Subject: [Python-ideas] for/else syntax Message-ID: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> On python-dev the subject of for/else statements came up, so I had to mention how "ambiguous" the syntax seems to me. By "ambiguous" I meant that it's not obvious what should happen in the for/else and while/else constructs (the except/else construct is readable and great imo btw). Even though it's documented, for me it's a bad construct because it can and will always be misinterpreted by a non-trivial amount of people seeing it for the first time. It's unreadable and confusing because the "else" isn't referring to the "for" it's in reference to the "break". The construct is only useful in loops with "break" statements, so in pseudo code or human the "else" would have probably been translated to "if didn't break". To make things worse, it's not that newcomers will say "hmm, what does this for/else thing do? lets look at the manual", they will mistakenly assume they understand what the construct means. For evidence, see the for/else construct in the django templating language where "else" means the loop didn't run. The django interpretation IMHO is much more natural because "else" AFAIK means "otherwise" meaning that statements inside an "else" block will never be executed if the statements in the previous block were. To summarize: whatever: A else: B looks like it means either A happened or B happened. In python for/while/else constructs, this doesn't hold. --yuv Original Thread: [Python-Dev] Announcing PEP 3136 ---------- Forwarded message ---------- From: Terry Reedy Date: Thu, Oct 1, 2009 at 2:47 AM Subject: Re: [Python-Dev] Announcing PEP 3136 To: python-dev at python.org Yuvgoog Greenle wrote: > I like how python has a minimalistic and powerful syntax (-1 for the break > ___ PEP). > > Also, I really dislike the for/else ambiguity "butterflies". > Properly understood, no ambiguity. while c: x() is equivalent to hypothetical label z: if c: x() goto: z So while c: x() else: y() is equivalent to label 1: if c: x() goto: 1 else" y() The else clause fires (once) if and when the if/while condition evaluates as false. Break and continue are restricted *unconditional* goto statements, and so *cannot* trigger an else clause. In for loops, the implied condition is 'there is another item in the collection represented by the iterable'. For any more, move to another list. Terry Jan Reedy _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ubershmekel%40gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From wuwei23 at gmail.com Thu Oct 1 03:35:36 2009 From: wuwei23 at gmail.com (alex23) Date: Wed, 30 Sep 2009 18:35:36 -0700 (PDT) Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> Message-ID: <7441c2a3-ec27-4394-a9c1-d52a726db93e@i4g2000prm.googlegroups.com> On Oct 1, 11:07?am, Yuvgoog Greenle wrote: > To make things worse, it's not that newcomers will say "hmm, what does this > for/else thing do? lets look at the manual", they will mistakenly assume > they understand what the construct means. -1 for modifying Python to meet the "intuitive" expectations of those unwilling to learn the language. From ubershmekel at gmail.com Thu Oct 1 03:58:07 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Thu, 1 Oct 2009 04:58:07 +0300 Subject: [Python-ideas] for/else syntax In-Reply-To: <7441c2a3-ec27-4394-a9c1-d52a726db93e@i4g2000prm.googlegroups.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <7441c2a3-ec27-4394-a9c1-d52a726db93e@i4g2000prm.googlegroups.com> Message-ID: <9d153b7c0909301858t5e2c5aagf3df23f92b7db2ac@mail.gmail.com> I wouldn't modify the way for/else behaves btw. I would either:1. rename the "else" to something else for "for" and "while" loops 2. remove the syntax altogether. 2to3 or 3to4 can do this by changing a boolean before the "break" statement for example. expectations of those unwilling to learn the language I can tell you I love python and of course many newcomers will/do too. I'm just noting that this might be a pimple worth popping. One more thing, I have to clear django's name as it doesn't have a for/else construct, it has a for/empty construct that was inspired by a popular for/else django template recipe entitled: > "for" template tag with support for "else" if array is empty (thanks for pointing that out Karen) On Thu, Oct 1, 2009 at 4:35 AM, alex23 wrote: > On Oct 1, 11:07 am, Yuvgoog Greenle wrote: > > To make things worse, it's not that newcomers will say "hmm, what does > this > > for/else thing do? lets look at the manual", they will mistakenly assume > > they understand what the construct means. > > -1 for modifying Python to meet the "intuitive" expectations of those > unwilling to learn the language. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gerald.britton at gmail.com Thu Oct 1 06:23:27 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Thu, 1 Oct 2009 00:23:27 -0400 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0909301858t5e2c5aagf3df23f92b7db2ac@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <7441c2a3-ec27-4394-a9c1-d52a726db93e@i4g2000prm.googlegroups.com> <9d153b7c0909301858t5e2c5aagf3df23f92b7db2ac@mail.gmail.com> Message-ID: <5d1a32000909302123r1998e3daue8bfcd7c2dc8c7b1@mail.gmail.com> Why single out for/while and leave out try? It supports an else statement in a similar fashion. Honestly I think the syntax is fine as it is. Once I grasped what it did and why, I exploited it to great effect. The syntax may be unfamiliar to programmers coming from other languages, but so what? The same can be said of every language on the planet. (I felt like an idiot when I started to learn Haskell). FWIW, nothing in life is intuitive. Absolutely nothing. Everything has to be learned, even the simplest things. A baby has to learn how to nurse and many struggle with that unintuitive process as well. The concept of what is "intuitive" is entirely in the eye of the beholder and is effectively a non-concept. -1. On Wed, Sep 30, 2009 at 9:58 PM, Yuvgoog Greenle wrote: > I wouldn't modify the way for/else behaves btw. I would either: > 1. rename the "else" to something else for "for" and "while" loops > 2. remove the syntax altogether. 2to3 or 3to4 can do this by changing a > boolean before the "break" statement for example. > >> expectations of those unwilling to learn the language > > I can tell you I love python and of course many newcomers will/do too. I'm > just noting that this might be a pimple worth popping. > > One more thing, I have to clear django's name as it doesn't have a for/else > construct, it has a for/empty construct that was inspired by a popular > for/else django template recipe entitled: >> >> "for" template tag with support for "else" if array is empty > > (thanks for pointing that out Karen) > On Thu, Oct 1, 2009 at 4:35 AM, alex23 wrote: >> >> On Oct 1, 11:07?am, Yuvgoog Greenle wrote: >> > To make things worse, it's not that newcomers will say "hmm, what does >> > this >> > for/else thing do? lets look at the manual", they will mistakenly assume >> > they understand what the construct means. >> >> -1 for modifying Python to meet the "intuitive" expectations of those >> unwilling to learn the language. >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > -- Gerald Britton From stephen at xemacs.org Thu Oct 1 07:17:45 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 01 Oct 2009 14:17:45 +0900 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0909301858t5e2c5aagf3df23f92b7db2ac@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <7441c2a3-ec27-4394-a9c1-d52a726db93e@i4g2000prm.googlegroups.com> <9d153b7c0909301858t5e2c5aagf3df23f92b7db2ac@mail.gmail.com> Message-ID: <877hvfk806.fsf@uwakimon.sk.tsukuba.ac.jp> Yuvgoog Greenle writes: > I wouldn't modify the way for/else behaves btw. I would > either:1. rename the "else" to something else for "for" and "while" > loops If you prefer some existing keyword, -0.75, else -1. Note: As a native speaker, since "for ... else" isn't English, I would certainly look up the semantics before using the syntax. This is very probably true for Japanese as well, who are far better at (consciously) remembering English syntax than native speakers are. I don't know about other non-natives. > 2. remove the syntax altogether. -1. It's occasionally useful, and when it is, it's nice and concise. From wuwei23 at gmail.com Thu Oct 1 07:12:19 2009 From: wuwei23 at gmail.com (alex23) Date: Wed, 30 Sep 2009 22:12:19 -0700 (PDT) Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0909301858t5e2c5aagf3df23f92b7db2ac@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <7441c2a3-ec27-4394-a9c1-d52a726db93e@i4g2000prm.googlegroups.com> <9d153b7c0909301858t5e2c5aagf3df23f92b7db2ac@mail.gmail.com> Message-ID: <1ed7251d-5703-4f40-b503-338a8daeb60a@f18g2000prf.googlegroups.com> Yuvgoog Greenle wrote: > I wouldn't modify the way for/else behaves btw. I would [...] > remove the syntax altogether. I'm pretty sure removing the syntax entirely would completely modify the behaviour... From g.brandl at gmx.net Thu Oct 1 09:48:24 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 01 Oct 2009 09:48:24 +0200 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> Message-ID: Yuvgoog Greenle schrieb: > On python-dev the subject of for/else statements came up, so I had to > mention how "ambiguous" the syntax seems to me. By "ambiguous" I meant > that it's not obvious what should happen in the for/else and while/else > constructs (the except/else construct is readable and great imo btw). > > Even though it's documented, for me it's a bad construct because it can > and will always be misinterpreted by a non-trivial amount of people > seeing it for the first time. It's unreadable and confusing because the > "else" isn't referring to the "for" it's in reference to the "break". > The construct is only useful in loops with "break" statements, so in > pseudo code or human the "else" would have probably been translated to > "if didn't break". I agree that the "else" clause is somewhat unintuitive. However, it has its uses and can often replace more awkward constructs. As such, I would be -1 on removing it, and -3 on reusing the else clause for some other purpose. > To make things worse, it's not that newcomers will say "hmm, what does > this for/else thing do? lets look at the manual", they will mistakenly > assume they understand what the construct means. For evidence, see the > for/else construct in the django templating language where "else" means > the loop didn't run. The django interpretation IMHO is much more natural > because "else" AFAIK means "otherwise" meaning that statements inside an > "else" block will never be executed if the statements in the previous > block were. The block in question is called "empty" in the Django template language, not "else". If you look at the discussion that led to its introduction, you can see confusion with Python's else cited, and the tag was named "default"/"empty" because of that: http://code.djangoproject.com/ticket/6398 One template language that uses "else" for this case is Jinja (which is similar in syntax to Django templates, maybe you confused them). However, I know that Armin does know how Python's for-else works :) It is simply not useful for a (template) language that has no "break", while a "default" clause is very convenient there. Georg From ncoghlan at gmail.com Thu Oct 1 12:54:16 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 01 Oct 2009 20:54:16 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> Message-ID: <4AC48A58.1080109@gmail.com> Yuvgoog Greenle wrote: > On python-dev the subject of for/else statements came up, so I had to > mention how "ambiguous" the syntax seems to me. By "ambiguous" I meant > that it's not obvious what should happen in the for/else and while/else > constructs (the except/else construct is readable and great imo btw). > > Even though it's documented, for me it's a bad construct because it can > and will always be misinterpreted by a non-trivial amount of people > seeing it for the first time. It's unreadable and confusing because the > "else" isn't referring to the "for" it's in reference to the "break". > The construct is only useful in loops with "break" statements, so in > pseudo code or human the "else" would have probably been translated to > "if didn't break". You need a much better reason than "it might confuse newbies" to justify breaking working code. In the primary intended context (search loops) the current terminology makes perfect sense: for obj in iterable: if found_it(obj): # Found what we were looking for! break else: # It wasn't there :( Similarly for while loops: while self.more_points_to_check(): if self.found_it(): # Found the desired point break else: # Exhausted the search space :( And once people learn the idiom, it is easy to remember (so long as they learn it properly - i.e. as an easy way to write a search loop with a separate handler for the "not found" case). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ars at iki.fi Thu Oct 1 13:00:21 2009 From: ars at iki.fi (Antti Rasinen) Date: Thu, 1 Oct 2009 14:00:21 +0300 (EEST) Subject: [Python-ideas] for/else syntax In-Reply-To: References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> Message-ID: <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> > Yuvgoog Greenle schrieb: >> On python-dev the subject of for/else statements came up, so I had to >> mention how "ambiguous" the syntax seems to me. By "ambiguous" I meant >> that it's not obvious what should happen in the for/else and while/else >> constructs (the except/else construct is readable and great imo btw). >> >> Even though it's documented, for me it's a bad construct because it can >> and will always be misinterpreted by a non-trivial amount of people >> seeing it for the first time. It's unreadable and confusing because the >> "else" isn't referring to the "for" it's in reference to the "break". >> The construct is only useful in loops with "break" statements, so in >> pseudo code or human the "else" would have probably been translated to >> "if didn't break". > > I agree that the "else" clause is somewhat unintuitive. However, it has > its uses and can often replace more awkward constructs. As such, I would > be -1 on removing it, and -3 on reusing the else clause for some other > purpose. I must say that calling for ... else "somewhat unintuitive" is the understatement of the year. The analogy with if ... else leads further astray than Odysseus. The feature itself is not bad. It's just that it is misnamed. I did a quick poll at the office and at IRC, five people in total, with 2-8 years of Python experience. Two had never heard of the for..else -construct; one had heard that it exists and two know about it but have never used it. Three uninitiated, in other words. All three gave similar replies: if the loop is not executed or the variable is not iterable, then the else-branch is executed. Here are their reactions, when they learned how it really works: * "It should be called for..then" (or for..finally) * "WHAT. No f** way. Completely unintuitive" * "I'd hate whoever wrote that code" These people are actual Python developers, not some newbies. And I completely agree with them. I've only once seen for...else in the wild and never used it myself. To add further insult to the naming injury, the else branch suffers from the fact that 99.9% of the time, the for-loop contains an if: for x in xs: if cond(x): break # stuff else: # more stuff My code pattern matching algorithm reads that as an indentation error. An else matches if so many times more often that decoding for..else requires unnecessary effort. I don't want the feature to be removed or even renamed (at least before Python 4). But the feature *is* badly named and unintuitive. -- [ Antti Rasinen <*> ars at iki.fi ] From ubershmekel at gmail.com Thu Oct 1 14:12:13 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Thu, 1 Oct 2009 15:12:13 +0300 Subject: [Python-ideas] for/else syntax In-Reply-To: <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> Message-ID: <9d153b7c0910010512h4df68169w8f06085b2a4809fa@mail.gmail.com> Thank god for empirical evidence and thanks Antti for the measurement. This construct should have been named "finally", "didnt_break" or "not break": while self.more_points_to_check(): if self.found_it(): # Found the desired point breaknot break: # Exhausted the search space :( On Thu, Oct 1, 2009 at 2:00 PM, Antti Rasinen wrote: > > > Yuvgoog Greenle schrieb: > >> On python-dev the subject of for/else statements came up, so I had to > >> mention how "ambiguous" the syntax seems to me. By "ambiguous" I meant > >> that it's not obvious what should happen in the for/else and while/else > >> constructs (the except/else construct is readable and great imo btw). > >> > >> Even though it's documented, for me it's a bad construct because it can > >> and will always be misinterpreted by a non-trivial amount of people > >> seeing it for the first time. It's unreadable and confusing because the > >> "else" isn't referring to the "for" it's in reference to the "break". > >> The construct is only useful in loops with "break" statements, so in > >> pseudo code or human the "else" would have probably been translated to > >> "if didn't break". > > > > I agree that the "else" clause is somewhat unintuitive. However, it has > > its uses and can often replace more awkward constructs. As such, I would > > be -1 on removing it, and -3 on reusing the else clause for some other > > purpose. > > I must say that calling for ... else "somewhat unintuitive" is the > understatement of the year. The analogy with if ... else leads further > astray than Odysseus. > > The feature itself is not bad. It's just that it is misnamed. > > I did a quick poll at the office and at IRC, five people in total, with > 2-8 years of Python experience. Two had never heard of the for..else > -construct; one had heard that it exists and two know about it but have > never used it. Three uninitiated, in other words. > > All three gave similar replies: if the loop is not executed or the > variable is not iterable, then the else-branch is executed. Here are their > reactions, when they learned how it really works: > > * "It should be called for..then" (or for..finally) > * "WHAT. No f** way. Completely unintuitive" > * "I'd hate whoever wrote that code" > > These people are actual Python developers, not some newbies. And I > completely agree with them. I've only once seen for...else in the wild and > never used it myself. > > To add further insult to the naming injury, the else branch suffers from > the fact that 99.9% of the time, the for-loop contains an if: > > for x in xs: > if cond(x): > break > # stuff > else: > # more stuff > > My code pattern matching algorithm reads that as an indentation error. An > else matches if so many times more often that decoding for..else requires > unnecessary effort. > > I don't want the feature to be removed or even renamed (at least before > Python 4). But the feature *is* badly named and unintuitive. > > -- > [ Antti Rasinen <*> ars at iki.fi ] > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Thu Oct 1 14:36:59 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 01 Oct 2009 22:36:59 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910010512h4df68169w8f06085b2a4809fa@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <9d153b7c0910010512h4df68169w8f06085b2a4809fa@mail.gmail.com> Message-ID: <4AC4A26B.1070603@gmail.com> Yuvgoog Greenle wrote: > Thank god for empirical evidence and thanks Antti for the measurement. The plural of anecdote is not data... Although it seems the "general loop" form in Python may need to be better taught to make these semantics easier to remember :P while loop_cond: # loop body else: # else clause can be translated almost exactly as: while 1: if loop_cond: # Loop body goes here else: # Else clause goes here break The for loop form is a natural extension of that (with the implicit loop condition being "while there is stuff left in the iterable"). I can understand people not knowing what the construct means the first time they see it. But the underlying mechanics really shouldn't be all that difficult to grasp. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From debatem1 at gmail.com Thu Oct 1 15:01:59 2009 From: debatem1 at gmail.com (geremy condra) Date: Thu, 1 Oct 2009 09:01:59 -0400 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> Message-ID: On Wed, Sep 30, 2009 at 9:07 PM, Yuvgoog Greenle wrote: > On python-dev the subject of for/else statements came up, so I had to > mention how "ambiguous" the syntax seems to me. By "ambiguous" I meant that > it's not obvious what should happen in the for/else and while/else > constructs (the except/else construct is readable and great imo btw). > > Even though it's documented, for me it's a bad construct because it can and > will always be misinterpreted by a non-trivial amount of people seeing it > for the first time. It's unreadable and confusing because the "else" isn't > referring to the "for" it's in reference to the "break". The construct is > only useful in loops with "break" statements, so in pseudo code or human the > "else" would have probably been translated to "if didn't break". > > To make things worse, it's not that newcomers will say "hmm, what does this > for/else thing do? lets look at the manual", they will mistakenly assume > they understand what the construct means. For evidence, see the for/else > construct in the django templating language where "else" means the loop > didn't run. The django interpretation IMHO is much more natural because > "else" AFAIK means "otherwise" meaning that statements inside an "else" > block will never be executed if the statements in the previous block were. > > To summarize: > > whatever: > A > else: > B > > looks like it means either A happened or B happened. In python > for/while/else constructs, this doesn't hold. > > --yuv > > Original Thread: [Python-Dev] Announcing PEP 3136 > > ---------- Forwarded message ---------- > From: Terry Reedy > Date: Thu, Oct 1, 2009 at 2:47 AM > Subject: Re: [Python-Dev] Announcing PEP 3136 > To: python-dev at python.org > > > Yuvgoog Greenle wrote: > >> I like how python has a minimalistic and powerful syntax (-1 for the break >> ___ PEP). >> >> Also, I really dislike the for/else ambiguity "butterflies". >> > > Properly understood, no ambiguity. > > while c: > x() > > is equivalent to hypothetical > > label z: > if c: > x() > goto: z > > So > > while c: > x() > else: > y() > > is equivalent to > > label 1: > if c: > x() > goto: 1 > else" > y() > > The else clause fires (once) if and when the if/while condition evaluates > as false. Break and continue are restricted *unconditional* goto statements, > and so *cannot* trigger an else clause. > > In for loops, the implied condition is 'there is another item in the > collection represented by the iterable'. > > For any more, move to another list. > > Terry Jan Reedy > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ubershmekel%40gmail.com > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > I'm pretty sure breaking loop behavior on a point release sentences you to the ninth ring of software development hell. -1 from me. Geremy Condra -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Thu Oct 1 15:37:09 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 01 Oct 2009 22:37:09 +0900 Subject: [Python-ideas] for/else syntax In-Reply-To: <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> Message-ID: <878wfvjkvu.fsf@uwakimon.sk.tsukuba.ac.jp> Antti Rasinen writes: > never used it. Three uninitiated, in other words. Well, of course it's unintuitive, then. In practical programming, "unintuitive" is synonymous with "I've never used it in a real program." > Here are their reactions, when they learned how it really works: > > * "It should be called for..then" (or for..finally) I don't know about the semantics I'd guess for "then", but I think I'd consider that synonymous with "finally". IMO it's non-starter anyway because it's not already a keyword; this pattern is a little too uncommon to justify a new keyword. "finally" is no good because in try blocks it means "no matter what, do this". Having slightly different semantics in try blocks and loops would be a real loser. > To add further insult to the naming injury, the else branch suffers from > the fact that 99.9% of the time, the for-loop contains an if: > > for x in xs: > if cond(x): > break > # stuff > else: > # more stuff > > My code pattern matching algorithm reads that as an indentation error. An > else matches if so many times more often that decoding for..else requires > unnecessary effort. I guess. However, you're a native speaker of a natural language, so I bet you can deal with idioms that are *much* less intuitive than this one, if used in context. Probably even in English (which I guess is not your native language). I find the search idiom very natural, almost intuitive (even though I've never used it myself), now that I've seen it in Nick's post. Don't you? From arnodel at googlemail.com Thu Oct 1 15:54:03 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 1 Oct 2009 14:54:03 +0100 Subject: [Python-ideas] for/else syntax In-Reply-To: <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> Message-ID: <9bfc700a0910010654o622ff379j73bfd31c487c6192@mail.gmail.com> 2009/10/1 Antti Rasinen : > I must say that calling for ... else "somewhat unintuitive" is the > understatement of the year. The analogy with if ... else leads further > astray than Odysseus. > > The feature itself is not bad. It's just that it is misnamed. [...] > I don't want the feature to be removed or even renamed (at least before > Python 4). But the feature *is* badly named and unintuitive. I agree. I use it from time to time and I used to have to look up the docs every time, because I couldn't remember wheteher the 'else:' clause meant 'if did break' or 'if didn't break'. I solved this problem by stopping thinking about it as while-else or for-else, but rather as break-else. -- Arnaud From gerald.britton at gmail.com Thu Oct 1 16:21:18 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Thu, 1 Oct 2009 10:21:18 -0400 Subject: [Python-ideas] for/else syntax In-Reply-To: <9bfc700a0910010654o622ff379j73bfd31c487c6192@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <9bfc700a0910010654o622ff379j73bfd31c487c6192@mail.gmail.com> Message-ID: <5d1a32000910010721w472a7408mdd38699d7928db66@mail.gmail.com> There is consistency between the use of "else" in for, while and try: The "else" suite is executed when the for/while/try statement is exhausted -- that is, is not terminated abnormally with a break, return, exception, etc. Once I got it, I GOT IT! and have found it quite handy. I don't think we should talk about changing this syntax for for/while without including the try statement. I do believe, though, that the idea is a non-starter since it would break existing code. As for the "intuitive" argument, that's a red herring. One person's "intuitive" is another's "obscure." The word is overused and undefined in any objective, measurable sense. Steve Ballmer likes it since he thinks it gives him a leg up on the competition. That should be enough for us to eschew its use. On Thu, Oct 1, 2009 at 9:54 AM, Arnaud Delobelle wrote: > 2009/10/1 Antti Rasinen : >> I must say that calling for ... else "somewhat unintuitive" is the >> understatement of the year. The analogy with if ... else leads further >> astray than Odysseus. >> >> The feature itself is not bad. It's just that it is misnamed. > [...] >> I don't want the feature to be removed or even renamed (at least before >> Python 4). But the feature *is* badly named and unintuitive. > > I agree. ?I use it from time to time and I used to have to look up the docs > every time, because I couldn't remember wheteher the 'else:' clause > meant 'if did break' or 'if didn't break'. > > I solved this problem by stopping thinking about it as while-else or > for-else, but rather as break-else. > > -- > Arnaud > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From guido at python.org Thu Oct 1 16:47:37 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 1 Oct 2009 07:47:37 -0700 Subject: [Python-ideas] for/else syntax In-Reply-To: <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> Message-ID: On Thu, Oct 1, 2009 at 4:00 AM, Antti Rasinen wrote: > I must say that calling for ... else "somewhat unintuitive" is the > understatement of the year. "The nipple is the only thing that's intuitive. Everything else is learned." -- --Guido van Rossum (home page: http://www.python.org/~guido/) From gerald.britton at gmail.com Thu Oct 1 16:49:04 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Thu, 1 Oct 2009 10:49:04 -0400 Subject: [Python-ideas] for/else syntax In-Reply-To: References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> Message-ID: <5d1a32000910010749g428dcd01h4ef2cb5e769fe8b2@mail.gmail.com> Even the nipple is unintuitive. Just ask any new mother trying to get her baby to nurse. On Thu, Oct 1, 2009 at 10:47 AM, Guido van Rossum wrote: > On Thu, Oct 1, 2009 at 4:00 AM, Antti Rasinen wrote: >> I must say that calling for ... else "somewhat unintuitive" is the >> understatement of the year. > > "The nipple is the only thing that's intuitive. Everything else is learned." > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From phd at phd.pp.ru Thu Oct 1 17:04:43 2009 From: phd at phd.pp.ru (Oleg Broytman) Date: Thu, 1 Oct 2009 19:04:43 +0400 Subject: [Python-ideas] for/else syntax In-Reply-To: References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> Message-ID: <20091001150443.GA21693@phd.pp.ru> On Thu, Oct 01, 2009 at 07:47:37AM -0700, Guido van Rossum wrote: > On Thu, Oct 1, 2009 at 4:00 AM, Antti Rasinen wrote: > > I must say that calling for ... else "somewhat unintuitive" is the > > understatement of the year. > > "The nipple is the only thing that's intuitive. Everything else is learned." "There is no intuitive interface, not even the nipple. It's all learned." http://www.greenend.org.uk/rjk/2002/08/nipple.html Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From stephen at xemacs.org Thu Oct 1 18:23:02 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 02 Oct 2009 01:23:02 +0900 Subject: [Python-ideas] for/else syntax In-Reply-To: <20091001150443.GA21693@phd.pp.ru> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <20091001150443.GA21693@phd.pp.ru> Message-ID: <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> Oleg Broytman writes: > On Thu, Oct 01, 2009 at 07:47:37AM -0700, Guido van Rossum wrote: > > "The nipple is the only thing that's intuitive. Everything else is learned." > > "There is no intuitive interface, not even the nipple. It's all learned." "Although that way may not be obvious at first unless you're Dutch." From steve at pearwood.info Fri Oct 2 05:07:25 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 2 Oct 2009 13:07:25 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> Message-ID: <200910021307.25444.steve@pearwood.info> On Thu, 1 Oct 2009 09:00:21 pm Antti Rasinen wrote: > To add further insult to the naming injury, the else branch suffers > from the fact that 99.9% of the time, the for-loop contains an if: Really? You've done a statistically significant survey of code in the wild and come to a figure accurate to one part in a thousand? Not just plucked a number straight out of thin air? In my code, the figure is more like 20-40%. Choosing one representative module at random, only one in three for-loops include an if inside the loop. And when I use for...else, it's closer to 10%. > for x in xs: > if cond(x): > break > # stuff > else: > # more stuff > > My code pattern matching algorithm reads that as an indentation > error. An else matches if so many times more often that decoding > for..else requires unnecessary effort. But then there are these: for x in xs: if cond(x): process() else: process_differently() else: do_something_else() for x in xs: process() else: do_something_else() They don't look anything like an indentation error to me. -- Steven D'Aprano From ubershmekel at gmail.com Fri Oct 2 05:11:22 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Fri, 2 Oct 2009 06:11:22 +0300 Subject: [Python-ideas] for/else syntax In-Reply-To: <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> Let me rephrase - for..else is a _misleading_ construct. It suggests one thing yet does another. This response is way too long, so here's a TOC: 1. The Question Challenge - A way to examine if "for..else" is just unintuitive or misleading. 2. What can be done about for...else. 3. Lets talk about "try/else" because a few people mentioned it as though it's somehow related to the discussion. ---------------- 1. The Question Challenge - A way to examine if "for..else" is just unintuitive or misleading. Antti's experiment got me asking, you can try this as well. Ask a person who knows python the following questions: Question #1: Do you know what happens when an "else" appears after a "for" loop? if they answer yes then say nevermind. otherwise: continue to question #2. Question #2: When would the code in the "else" block execute? If you don't know, it's ok to guess. If they answer "when a break didn't occur", I'm willing to give you, the asker, up to 100 lines of patch code in whatever python project you want.. If they answer "maybe when the for loop didn't run or was an empty list" then you don't owe me anything. ------------- 2. What can be done about for...else. 1. Not allowing a for loop that has no "break" to have an "else". Just like "else" isn't allowed when there isn't an "except" in the "try". There really is no excuse, justification or sense in a "for" loop that has no "break" yet has an "else". 2. Lets not talk about removing the "else" from for/while completely since it does have its fans and cute use cases. 3. Renaming this construct for py4k wouldn't be too crazy if you use the existing reserved words "not" and "break". Look at this code and please note that it doesn't need any comments to make it readable (as opposed to Nick's example, no offence): for item in list_of_stuff: if item.is_awesome(): break not break: print("nothing is awesome :(") And I can even think of a few use cases for having just "break:". This block would execute only upon "break" occurring. That way you could have loops like this: for item in list_of_stuff: if item.is_awesome(): break break: celebrate(list_of_stuff) -------------------------- 3. Lets talk about "try/else" because a few people mentioned it as though it's somehow related to the discussion. "try/else" now _that_ is a red herring. There is no "try/else". OTOH "except/else", as I said before, is a beautiful, pythonic, useful and cool construct. This is the evidence: >>> try: ... print(1) ... else: File "", line 3 else: ^ SyntaxError: invalid syntax >>> try: ... print(2) ... finally: ... print(3) ... else: File "", line 5 else: ^ >>> try: ... print(7) ... except Exception as e: ... print(13) ... else: ... print(6) ... 7 6 -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Fri Oct 2 05:23:27 2009 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 01 Oct 2009 22:23:27 -0500 Subject: [Python-ideas] for/else syntax In-Reply-To: <200910021307.25444.steve@pearwood.info> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <200910021307.25444.steve@pearwood.info> Message-ID: Steven D'Aprano wrote: > On Thu, 1 Oct 2009 09:00:21 pm Antti Rasinen wrote: >> for x in xs: >> if cond(x): >> break >> # stuff >> else: >> # more stuff >> >> My code pattern matching algorithm reads that as an indentation >> error. An else matches if so many times more often that decoding >> for..else requires unnecessary effort. > > But then there are these: > > for x in xs: > if cond(x): > process() > else: > process_differently() > else: > do_something_else() > > > for x in xs: > process() > else: > do_something_else() > > > They don't look anything like an indentation error to me. Am I missing something? Since those examples don't have any break statements in them, why do you use an else:? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From bruce at leapyear.org Fri Oct 2 05:33:18 2009 From: bruce at leapyear.org (Bruce Leban) Date: Thu, 1 Oct 2009 20:33:18 -0700 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> Message-ID: On Thu, Oct 1, 2009 at 8:11 PM, Yuvgoog Greenle wrote: > > Question #2: When would the code in the "else" block execute? If you don't > know, it's ok to guess. > If they answer "when a break didn't occur", I'm willing to give you, > the asker, up to 100 lines of patch code in whatever python project you > want.. > If they answer "maybe when the for loop didn't run or was an empty > list" then you don't owe me anything. > > When I see for/else I think of the second case even though I know it's not that. I think that second pattern is very common. In fact, I wrote code today that used it and I had to store the list in a variable and wrap the for in an if to test whether it was empty. Arguably, either one of these is appropriate for the word "else". I'd rather something like: for x in y: stuff else if no_break: more stuff else if no_elements: more stuff OK. This is ugly, but it's clear which of these is which use case. Surely someone can make this better in time for Python 4. --- Bruce -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Fri Oct 2 06:24:05 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 2 Oct 2009 14:24:05 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910021307.25444.steve@pearwood.info> Message-ID: <200910021424.05666.steve@pearwood.info> On Fri, 2 Oct 2009 01:23:27 pm Robert Kern wrote: > > for x in xs: > > if cond(x): > > process() > > else: > > process_differently() > > else: > > do_something_else() > > > > > > for x in xs: > > process() > > else: > > do_something_else() > > > > > > They don't look anything like an indentation error to me. > > Am I missing something? Since those examples don't have any break > statements in them, why do you use an else:? My bad -- process and process_differently are meant to stand in for a *block*, presumably including a break, not just a function call. I was attempting to show the indent structure. I shouldn't have included the parentheses, which just muddied the water. -- Steven D'Aprano From steve at pearwood.info Fri Oct 2 07:06:17 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 2 Oct 2009 15:06:17 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> Message-ID: <200910021506.17321.steve@pearwood.info> On Thu, 1 Oct 2009 11:07:19 am Yuvgoog Greenle wrote: > On python-dev the subject of for/else statements came up, so I had to > mention how "ambiguous" the syntax seems to me. By "ambiguous" I > meant that it's not obvious what should happen in the for/else and > while/else constructs (the except/else construct is readable and > great imo btw). > > Even though it's documented, for me it's a bad construct because it > can and will always be misinterpreted by a non-trivial amount of > people seeing it for the first time. It's unreadable and confusing > because the "else" isn't referring to the "for" it's in reference to > the "break". That's a very fine distinction you are drawing, and I don't think it is useful. The else block runs when you exit the for-loop normally, by falling off the end. There are three ways to exit the for-loop without falling off the end: break, raise and return. raise and return are special: return exits the function immediately, nothing further gets run, and uncaught exceptions do similar. So the only way to exit a for-loop abnormally and still keep processing is with break. Hence you argue that the else refers to the break. But of course you can't have a break outside of a for loop, so else is equally associated with for. And in fact, it is legal to have for loops without a break but include an else. Is it useful? I don't know, but Python can't forbid them without breaking (pun not intended) code like this: for x in xs: if __debug__: break else: print "whatever" "if __debug__: suite" is special: the Python compiler optimizes the entire suite away when running with the -O flag. So if Python would treat the presence of an else as an error unless there was a break, you could have some code which was, or wasn't, legal according to the presence of the optimize flag. This is clearly a Bad Thing. Hence, even if for...else with no breaks are useless, they must be allowed. > To make things worse, it's not that newcomers will say "hmm, what > does this for/else thing do? lets look at the manual", they will > mistakenly assume they understand what the construct means. For > evidence, see the for/else construct in the django templating > language where "else" means the loop didn't run. That's not evidence of what newcomers will do. It's evidence that other languages can do things differently from Python. -- Steven D'Aprano From stephen at xemacs.org Fri Oct 2 07:47:26 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 02 Oct 2009 14:47:26 +0900 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> Message-ID: <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> Yuvgoog Greenle writes: > 1. Not allowing a for loop that has no "break" to have an "else". Just like > "else" isn't allowed when there isn't an "except" in the "try". There really > is no excuse, justification or sense in a "for" loop that has no "break" yet > has an "else". +1. File an RFE against PyLint for sure, and against Python too. That's a *great* idea! Bikeshedding follows. > Antti's experiment got me asking, you can try this as well. Ask a person who > knows python the following questions: > Question #1: Do you know what happens when an "else" appears after a "for" > loop? > if they answer yes then say nevermind. Er, no. It's more important if what people "know" is wrong than if naive guesses are wrong. > otherwise: continue to question #2. > Question #2: When would the code in the "else" block execute? If you don't > know, it's ok to guess. I would fire a programmer who gave an answer.<0.5 wink> He might do the same in his own code or in a review of someone else's. > 3. Renaming this construct for py4k wouldn't be too crazy if you use the > existing reserved words "not" and "break". Look at this code and please note > that it doesn't need any comments to make it readable (as opposed to Nick's > example, no offence): Nick's example is far more readable IMO. No offense. I can tell you why, too. In his example, the "else" is immediately preceded by the "if ... break" suite. Because "break" can only happen once in a loop, I read the else as the alternative to breaking out. Sloppy, yes, but since this is the common idiom using for-else, it works for me. :-) OTOH, "break" is an imperative; "not break" looks like a prohibition to me, not the target of a branch. You could use "for: ... if not break:" but that looks ugly to me, and worse requires several tokens of lookahead to disambiguate from an ordinary if block. I'll leave it up to GvR to decide whether for-else is "less Pythonic" than try-except-else. > And I can even think of a few use cases for having just "break:". This block > would execute only upon "break" occurring. That way you could have loops > like this: > > for item in list_of_stuff: > if item.is_awesome(): > break > break: > celebrate(list_of_stuff) You'd need multiple breaks for this to make sense. Else: for item in list_of_stuff: if item.is_awesome(): celebrate(list_of_stuff) break Some people think that having multiple breaks is bad style. From ars at iki.fi Fri Oct 2 08:05:00 2009 From: ars at iki.fi (Antti Rasinen) Date: Fri, 2 Oct 2009 09:05:00 +0300 Subject: [Python-ideas] for/else syntax In-Reply-To: <200910021307.25444.steve@pearwood.info> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <200910021307.25444.steve@pearwood.info> Message-ID: On 2009-10-02, at 06:07, Steven D'Aprano wrote: > On Thu, 1 Oct 2009 09:00:21 pm Antti Rasinen wrote: > >> To add further insult to the naming injury, the else branch suffers >> from the fact that 99.9% of the time, the for-loop contains an if: > > Really? You've done a statistically significant survey of code in the > wild and come to a figure accurate to one part in a thousand? Not just > plucked a number straight out of thin air? > > In my code, the figure is more like 20-40%. Choosing one > representative > module at random, only one in three for-loops include an if inside the > loop. And when I use for...else, it's closer to 10%. Oh silly me. I used some sloppy language there. I did not mean that all for-loops have ifs inside them. What does your last statistic mean? Do you have if statements in only 10% of your for..else loops or in 10% of your all for-loops (including for...else)? If you mean the first case, then please ignore the rest of this message and post an example :-) What I meant that most *for...else* structures have an if inside. The 99.9% number is, as you say, plucked from thin air -- but I think it is very close to the truth. There are simpler alternatives to for..else if the loop does not contain an if. Consider the following structure: for x in xs: # body else: # else-branch If you don't have a break in the body of the for-loop, then you don't need the else: at all. You can write the above segment as: for x in xs: # body # else-branch If, on the other hand, you do have a break in the body, then either it is conditional or it is not. If there is no break, then you are either only processing the first element in the iterable or skipping directly to the else branch. In this case it would be natural to use plain if: if xs: # body with xs[0] # else-branch Iterators complicate the matter, but even in that case then I'd rather use a peeking wrapper around xs. It seems to me that the only sensible use case for for..else is when there is an conditional break inside the loop. I'll appreciate any counter examples, of course. Nick Coghlan's had a lovely example about search loops earlier in the thread. It also highlights one of the reasons why I think the for..else construct is so rare. Consider his example: for obj in iterable: if found_it(obj): # Found what we were looking for! break else: # It wasn't there :( There are several ways of writing this without for..else. If the context for the loop is correct, you might just use return True (or obj) during the loop. No need for an else-branch there. Or you might use any: if not any(found_it(obj) for obj in iterable): # It wasn't there :( It seems to me that there often are "more intuitive" ways of writing a for..else-loop. And accordingly, the construct is rare. I'll resort to some newspeak here and define "intuitive" as either "conveys the intent of the programmer better" or as "feels like the more obvious way to do it". Pick either. --Antti -- [ ars at iki.fi <*> Antti Rasinen ] This drone-vessel speaks with the voice and authority of the Ur-Quan. From cmjohnson.mailinglist at gmail.com Fri Oct 2 11:06:46 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Thu, 1 Oct 2009 23:06:46 -1000 Subject: [Python-ideas] for/else syntax In-Reply-To: References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <200910021307.25444.steve@pearwood.info> Message-ID: <3bdda690910020206p1c8cfdbcta434cf7b95700372@mail.gmail.com> I also find for:? else: unintuitive. I know if has something to do with break, but I always forget if it means "there was no break" or "there was a break." I support the following change: for item in items: suite else: #Same as before, executes only if there was no break for item in items: suite else not break: #Executes only if there was no break. #PEP-8 should deprecate the bare else in favor of this. for item in items: suite else break: #Executes only if there was a break for item in items: suite else None: #None is a keyword now, so why not? #Executes only if there were no items or in while-loops the condition is never true I think the latter two language changes would be welcome additions to Python regardless of whether for-else stays the same in the ordinary case of not. Yes, there are ways of telling if the loop never broke or never ran the suite besides adding these grammar changes, but so what? This a fairly common pattern, and I think it would improve readability without hampering the programmer's mental model of Python any more than the existing for-else does today. If anything, this would hamper the mental model *less*. Two-cents-ly-yours, ?Carl M. Johnson From rob.cliffe at btinternet.com Fri Oct 2 12:43:13 2009 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Fri, 2 Oct 2009 11:43:13 +0100 Subject: [Python-ideas] Decorator syntax restriction Message-ID: Last time I wrote in favour of removing the restriction on decorator syntax, an unfortunate offhand remark of mine about something being more Pythonic led to a tangential debate about what was Pythonic and whether concision was always good, etc., with almost everyone ignoring the issue I was actually raising. An honourable exception was Brett Cannon, who answered my points squarely (and did his best to shoot me down in flames). So please, back to the point. I argued in favour of removing the syntax restriction (i.e. allowing any expression after '@') on the following grounds, which I paraphrase from my original: 1(a)) It was inconsistent to impose a restriction on expressions in one particular context [meaning: and not in others]. 1(b)) The restriction was hard to explain [but see below]. 2) The restriction can easily be circumvented. 3) Plausible use cases exist [I gave some]. I would like to answer some of Brett's points here: I wrote: "Guido has said he has a 'gut feeling' against [removing the restriction] but has not as far as I know rationalised it." Brett wrote: "When it comes to Guido's gut, a rationalization isn't needed. Perk of being BDFL. Plus his gut is right so often it tends to not be questioned." I am sure that Guido (may-he-live-forever) would be the first to agree that saying an opinion of his should NEVER be questioned is just, well, silly. When I said in point 1(b)) above that the restriction was hard to explain, this was ambiguous. Did I mean (i) It was hard to explain why there was a restriction at all., or (ii) It was hard to explain exactly what the restriction was. ? Actually I intended (i), so my point 1(b) sort of overlaps with 1(a), but Brett in good faith anwered to (ii) and wrote "It's not difficult to explain; decorators can only be a dotted name w/ an optional method call and its corresponding arguments." OK, but I at first got the wrong idea. I was reading the book "Core Python programming" by Wesley J. Chun. I quote from the book (page 422 section 11.3.6): 'The syntax for decorators uses a leading "at-sign" ( @ ) followed by the decorator function name and optional arguments.' No mention, you see, of the fact that the function name could be a dotted name. Nor AFAIK does a dotted name occur in any of the examples in the book. In fact it didn't occur to me that it could be, until some of the correspondence included examples that used a dotted name. So while it is possible to explain what the syntax restriction is, it is also possible to mis-explain it (or explain it poorly). OK, this is probably more a criticism of the book than anything else, but it shows that an additional potential source of confusion has been introduced. In reply to my IMO plausible use cases, Brett wrote: "Plausible does not equal useful. You need to show that this actually comes up in normal coding for a decent amount of Python code to warrant tweaking the language over." Why? That would be correct if I were proposing adding a new language feature or changing the way an existing feature works and/or causing breakage of existing code. But all I am proposing is lifting a restriction which at least arguably shouldn't have been there in the first place. After all, in other contexts where an expression is expected, say on the RHS of an assignment statement, it is easy to write something nonsensical that nobody could produce a use case for, e.g. f() + f 'a' - 'b' 0/0 but nobody is suggesting that such constructs should be SYNTACTICALLY illegal, and quite right too. Look at it this way: Suppose the restriction on decorator syntax had never been imposed. Everyone who uses decorators would be using them in exactly the same way. Would there be much of a case for then adding a restriction? Would we be having this discussion at all? I doubt it. So: please speak, if you agree with me, whether for the reasons I give or for different ones, or if you disagree with me - please give reasons too. Thanks and best wishes Rob Cliffe -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Oct 2 13:01:20 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 02 Oct 2009 21:01:20 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> Message-ID: <4AC5DD80.6050803@gmail.com> Yuvgoog Greenle wrote: > 1. Not allowing a for loop that has no "break" to have an "else". Just > like "else" isn't allowed when there isn't an "except" in the "try". > There really is no excuse, justification or sense in a "for" loop that > has no "break" yet has an "else". As Stephen said, that's a very good point. Having the compiler issue a SyntaxWarning when compiling such code is perfectly possible and would make a great deal of sense (and probably prevent many cases of naive misuse). > 2. Lets not talk about removing the "else" from for/while completely > since it does have its fans and cute use cases. Good :) > 3. Renaming this construct for py4k wouldn't be too crazy if you use the > existing reserved words "not" and "break". Look at this code and please > note that it doesn't need any comments to make it readable (as opposed > to Nick's example, no offence): > > for item in list_of_stuff: > if item.is_awesome(): > break > not break: > print("nothing is awesome :(") If you want to turn the "else" into something explicit, the phrase you're looking for would be "elif not break". I think it's better just to teach people that's what the "else" means though - no need to make them type it every time. If you want to do things in the break case then that code goes inside the body of the loop (after checking the break condition, but before the break statement itself). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Fri Oct 2 13:06:11 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 02 Oct 2009 21:06:11 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> Message-ID: <4AC5DEA3.3020701@gmail.com> Yuvgoog Greenle wrote: > There is no "try/else". OTOH "except/else", as I said before, is a > beautiful, pythonic, useful and cool construct. This is also a very good point, but perhaps not quite in the way you intended (and it relates back to your first point about the compiler not complaining about redundant else clauses on loops that contain no break statements). Just as there is no "try/else", but only "except/else", so there is no "for/else" or "while/else", but only "break/else". Issuing a SyntaxWarning might be a good way to make that clear. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Fri Oct 2 13:14:00 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 02 Oct 2009 21:14:00 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <200910021307.25444.steve@pearwood.info> Message-ID: <4AC5E078.3010000@gmail.com> Antti Rasinen wrote: > It seems to me that the only sensible use case for for..else is when > there is an conditional break inside the loop. I'll appreciate any > counter examples, of course. Nope, as far as I know, that's what the construct is for - search loops. > It seems to me that there often are "more intuitive" ways of writing a > for..else-loop. And accordingly, the construct is rare. I'll resort to > some newspeak here and define "intuitive" as either "conveys the intent > of the programmer better" or as "feels like the more obvious way to do > it". Pick either. I'd agree. However, given that it is an existing construct with valid (albeit fairly uncommon) use cases, I don't see much point in messing with it. Encouraging people to refer to the idiom as "break/else" and including a SyntaxWarning for dubious usage of loop else clauses (i.e. those without an associated break statement) are probably two fairly low cost educational steps that could be taken though. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Fri Oct 2 13:16:50 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 02 Oct 2009 21:16:50 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <200910021506.17321.steve@pearwood.info> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910021506.17321.steve@pearwood.info> Message-ID: <4AC5E122.9000902@gmail.com> Steven D'Aprano wrote: > And in fact, it is legal to have for loops without a break but include > an else. Is it useful? I don't know, but Python can't forbid them > without breaking (pun not intended) code like this: > > for x in xs: > if __debug__: > break > else: > print "whatever" > > "if __debug__: suite" is special: the Python compiler optimizes the > entire suite away when running with the -O flag. So if Python would > treat the presence of an else as an error unless there was a break, you > could have some code which was, or wasn't, legal according to the > presence of the optimize flag. This is clearly a Bad Thing. > > Hence, even if for...else with no breaks are useless, they must be > allowed. A SyntaxWarning should still be OK though - having an else clause that can only be reached in debug mode is pretty dubious in its own right. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From fuzzyman at gmail.com Fri Oct 2 13:25:24 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Fri, 2 Oct 2009 12:25:24 +0100 Subject: [Python-ideas] for/else syntax In-Reply-To: <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> 2009/10/2 Stephen J. Turnbull > Yuvgoog Greenle writes: > > > 1. Not allowing a for loop that has no "break" to have an "else". Just > like > > "else" isn't allowed when there isn't an "except" in the "try". There > really > > is no excuse, justification or sense in a "for" loop that has no "break" > yet > > has an "else". > > +1. File an RFE against PyLint for sure, and against Python too. > That's a *great* idea! > Nonsense - there are several other ways to break out of a loop. Raising an exception or returning for example. Michael > > Bikeshedding follows. > > > Antti's experiment got me asking, you can try this as well. Ask a person > who > > knows python the following questions: > > Question #1: Do you know what happens when an "else" appears after a > "for" > > loop? > > if they answer yes then say nevermind. > > Er, no. It's more important if what people "know" is wrong than if > naive guesses are wrong. > > > otherwise: continue to question #2. > > Question #2: When would the code in the "else" block execute? If you > don't > > know, it's ok to guess. > > I would fire a programmer who gave an answer.<0.5 wink> He might do > the same in his own code or in a review of someone else's. > > > 3. Renaming this construct for py4k wouldn't be too crazy if you use the > > existing reserved words "not" and "break". Look at this code and please > note > > that it doesn't need any comments to make it readable (as opposed to > Nick's > > example, no offence): > > Nick's example is far more readable IMO. No offense. I can tell you > why, too. In his example, the "else" is immediately preceded by the > "if ... break" suite. Because "break" can only happen once in a loop, > I read the else as the alternative to breaking out. Sloppy, yes, but > since this is the common idiom using for-else, it works for me. :-) > > OTOH, "break" is an imperative; "not break" looks like a prohibition > to me, not the target of a branch. You could use "for: ... if not > break:" but that looks ugly to me, and worse requires several tokens > of lookahead to disambiguate from an ordinary if block. > > I'll leave it up to GvR to decide whether for-else is "less Pythonic" > than try-except-else. > > > And I can even think of a few use cases for having just "break:". This > block > > would execute only upon "break" occurring. That way you could have loops > > like this: > > > > for item in list_of_stuff: > > if item.is_awesome(): > > break > > break: > > celebrate(list_of_stuff) > > You'd need multiple breaks for this to make sense. Else: > > for item in list_of_stuff: > if item.is_awesome(): > celebrate(list_of_stuff) > break > > Some people think that having multiple breaks is bad style. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Fri Oct 2 14:18:03 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 2 Oct 2009 13:18:03 +0100 Subject: [Python-ideas] Decorator syntax restriction In-Reply-To: References: Message-ID: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> 2009/10/2 Rob Cliffe : > So: please speak, if you agree with me, whether for the reasons I give or > for different ones, or if you disagree with me - please give reasons too. I believe that the key reason for there being restrictions is to avoid Python code looking like "line noise". That's not precise - the specific rules as given express a sensible (in my view) balance between this goal and simplicity of implementation/explanation/usability. For example, without restrictions the following would be valid (yes, it's silly, yes it's deliberately a worst case) @(a['b'].fns[1])(1,2,{3,4}) def something(): I contend that's clearly "line noise". Python has a long tradition of not just frowning on code like this, but actively forbidding such code and not implementing constructs that contribute to such. Witness the fact that C's ternary operator (?:) and the pre and post increment operators (++ and --) are not valid Python. So forbidding such code is clearly consistent with Python's traditions. @classmethod def f(...): is equally obviously *not* line noise, and it's entirely reasonable to allow it. So pick a point on the spectrum between the first and second example. That's what the current rules do - you may not agree with the precise point picked, but my contention is that not picking a point at all is contrary to Python's established design. I'm not sure there is a corresponding design principle within Python that all constructs should be fully general (which is basically the principle you're appealing to). I can't immediately find counterexamples, though, so there may be a point there. But certainly, I'd say that such a principle (if it exists) isn't as well-established as the "no line noise" one (witness comments like "if you want Perl, you know where to get it"...) > 1(a)) It was inconsistent to impose a restriction on expressions in one particular context [meaning: and not in others]. > 1(b)) The restriction was hard to explain [but see below]. > 2) The restriction can easily be circumvented. > 3) Plausible use cases exist [I gave some]. The above covers my feelings on 1(a) and 1(b). With regard to (2), circumventing the restriction isn't really the point - the whole decorator construct is a shorthand. @deco def fn(): ... vs def fn(): ... fn = deco(fn) Giving a complex expression a name increases readability, and the assignment can be kept near the decorator if it's a one-off, as follows meaningful_name = (a['b'].fns[1])(1,2,{3,4}) @meaningful_name def f(): ... I'd go so far as to say that was improving readability, rather than "circumvention". As for (3), I agree that plausible use cases exist. They *may* warrant allowing certain extra constructs in ("moving the point" as I said above) but I'm not sure about that, and to my mind they certainly don't warrant completely removing the restriction. As a final point, I should say that personally, I have never written any code which is impacted by the current restrictions on decorator syntax. So, I don't personally have any need for the relaxation you're proposing. I'm not sure if that's a point in your favour (as my view is of limited value since I don't need the feature you're proposing) or against (as I'm an example of why the proposal isn't as generally useful as you claim) but I mention it anyway, in the interests of complete disclosure :-) I hope this helps. Paul. From ubershmekel at gmail.com Fri Oct 2 14:45:56 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Fri, 2 Oct 2009 15:45:56 +0300 Subject: [Python-ideas] for/else syntax In-Reply-To: <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> Message-ID: <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> > > Nonsense - there are several other ways to break out of a loop. Raising an > exception or returning for example I do appreciate your response. I'd like you to examine how the "else" in question isn't relevant to the discussion of return or raise: >>> for i in range(10): ... print(1/0) ... else: ... print(100) ... Traceback (most recent call last): File "", line 2, in ZeroDivisionError: int division or modulo by zero >>> >>> def eggs(): ... for i in range(10): ... return 1234 ... else: ... print(69) ... >>> eggs() 1234 >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at gmail.com Fri Oct 2 14:49:07 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Fri, 2 Oct 2009 13:49:07 +0100 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> Message-ID: <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> 2009/10/2 Yuvgoog Greenle > Nonsense - there are several other ways to break out of a loop. Raising an >> exception or returning for example > > > I do appreciate your response. I'd like you to examine how the "else" in > question isn't relevant to the discussion of return or raise: > > >>> for i in range(10): > ... print(1/0) > ... else: > ... print(100) > ... > Traceback (most recent call last): > File "", line 2, in > ZeroDivisionError: int division or modulo by zero > >>> > The point is that the exception may be handled at an outer level - so the loop is broken out of and control flow is handed to another part of the code. Code that *always* raises an exception to break the loop is not common or interesting, but code that does so under certain circumstances is not uncommon (raising an exception is one way to break out of nested loops - the other way is an early return...) > >>> def eggs(): > ... for i in range(10): > ... return 1234 > ... else: > ... print(69) > ... > >>> eggs() > 1234 > >>> > > The return may be conditional in order to explicitly break out of the loop under certain circumstances. Michael -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ubershmekel at gmail.com Fri Oct 2 14:54:57 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Fri, 2 Oct 2009 15:54:57 +0300 Subject: [Python-ideas] for/else syntax In-Reply-To: <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> Message-ID: <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> I think I understand what you're saying and I just don't understand how it pertains to for..else. Specifically I would like an explanation about why the following quote is nonsense to you. 2009/10/2 Stephen J. Turnbull Yuvgoog Greenle writes: > > > 1. Not allowing a for loop that has no "break" to have an "else". Just > like > > "else" isn't allowed when there isn't an "except" in the "try". There > really > > is no excuse, justification or sense in a "for" loop that has no "break" > yet > > has an "else". > > +1. File an RFE against PyLint for sure, and against Python too. > That's a *great* idea! > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at gmail.com Fri Oct 2 14:58:24 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Fri, 2 Oct 2009 13:58:24 +0100 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> Message-ID: <6f4025010910020558m5741673eka2d3cc284f219985@mail.gmail.com> 2009/10/2 Yuvgoog Greenle > I think I understand what you're saying and I just don't understand how it > pertains to for..else. Specifically I would like an explanation about why > the following quote is nonsense to you. > Hmm... on consideration, breaking out of a loop without a break (i.e. exception or return) wouldn't enter the else clause *anyway*, so there is still no need for the else clause. My apologies, I retract. :-) Michael > > 2009/10/2 Stephen J. Turnbull > > Yuvgoog Greenle writes: >> >> > 1. Not allowing a for loop that has no "break" to have an "else". Just >> like >> > "else" isn't allowed when there isn't an "except" in the "try". There >> really >> > is no excuse, justification or sense in a "for" loop that has no >> "break" yet >> > has an "else". >> >> +1. File an RFE against PyLint for sure, and against Python too. >> That's a *great* idea! >> > > > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Fri Oct 2 15:50:17 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 02 Oct 2009 22:50:17 +0900 Subject: [Python-ideas] for/else syntax In-Reply-To: <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> Message-ID: <87pr95j46e.fsf@uwakimon.sk.tsukuba.ac.jp> Michael Foord writes: > > +1. File an RFE against PyLint for sure, and against Python too. > > That's a *great* idea! > > Nonsense - there are several other ways to break out of a loop. Raising an > exception or returning for example. Sure. Maybe I'm missing something, but consider for x in xs: function_that_may_return_or_raise(x,xs) function_that_should_not_be_executed_in_exceptional_case(xs) Now, a return or raise cannot cause f_t_s_n_b_e_i_e_c to be executed; only a break can.[1] Only a break *needs* else. If you can give a stylistic rule to distinguish when that code should be spelled instead for x in xs: function_that_may_return_or_raise(x,xs) else: function_that_should_not_be_executed_in_exceptional_case(xs) then I agree. However, if you can't, I can't either, and I can offer no reason why we shouldn't warn about breakless for-else loops since they are very often not what the user wants, and if they are desired, they can be spelled equally well without else. Footnotes: [1] StopIteration can, but I thought raising StopIteration in user code is considered quite bad form. From gerald.britton at gmail.com Fri Oct 2 15:46:38 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Fri, 2 Oct 2009 09:46:38 -0400 Subject: [Python-ideas] for/else syntax In-Reply-To: <6f4025010910020558m5741673eka2d3cc284f219985@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> <6f4025010910020558m5741673eka2d3cc284f219985@mail.gmail.com> Message-ID: <5d1a32000910020646i39e20e5fh9e15d40c22444528@mail.gmail.com> I can't imagine why I'm still commenting on this thread, since there is no chance that Python will remove the "else" from for/while or put conditions on it, but here is an example of a use that has no break statement: for i, j in enumerate(something): # suite i += 1 else: i = 0 # i == number of things processed The thing here is, "i" won't be set by the "for" statement if "something" is empty. OTOH, if "something" is non-empty, i >= 1 at the end of the for-loop. Using the "else" clause in this way ensure that "i" has the number of things processed at the end of the loop. To do this without the "else" I have to: i = 0 for i, j in enumerate(something): # suite i += 1 I can't imagine why I'm still commenting on this thread, since there is no chance that Python will remove the "else" from for/while or put conditions on it, but here is an example of a use that has no break statement: for i, j in enumerate(something): # suite i += 1 else: i = 0 # i == number of things processed Does it work? Sure! But it moves the setting of "i" outside the for-construct, which I personally dislike. On Fri, Oct 2, 2009 at 8:58 AM, Michael Foord wrote: > > > 2009/10/2 Yuvgoog Greenle >> >> I think I understand what you're saying and I just don't understand how it >> pertains to for..else. Specifically I would like an explanation about why >> the following quote is nonsense to you. > > Hmm... on consideration, breaking out of a loop without a break (i.e. > exception or return) wouldn't enter the else clause *anyway*, so there is > still no need for the else clause. > > My apologies, I retract. :-) > > Michael > >>> >>> 2009/10/2 Stephen J. Turnbull? >>> >>> Yuvgoog Greenle writes: >>> >>> ?> 1. Not allowing a for loop that has no "break" to have an "else". Just >>> like >>> ?> "else" isn't allowed when there isn't an "except" in the "try". There >>> really >>> ?> is no excuse, justification or sense in a "for" loop that has no >>> "break" yet >>> ?> has an "else". >>> >>> +1. ?File an RFE against PyLint for sure, and against Python too. >>> That's a *great* idea! >> >> > > > > -- > http://www.ironpythoninaction.com/ > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > -- Gerald Britton From ubershmekel at gmail.com Fri Oct 2 15:54:36 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Fri, 2 Oct 2009 16:54:36 +0300 Subject: [Python-ideas] for/else syntax In-Reply-To: <5d1a32000910020646i39e20e5fh9e15d40c22444528@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> <6f4025010910020558m5741673eka2d3cc284f219985@mail.gmail.com> <5d1a32000910020646i39e20e5fh9e15d40c22444528@mail.gmail.com> Message-ID: <9d153b7c0910020654p3590ae7du4d58b5a7a1f18035@mail.gmail.com> On Fri, Oct 2, 2009 at 4:46 PM, Gerald Britton wrote: > I can't imagine why I'm still commenting on this thread, since there > is no chance that Python will remove the "else" from for/while or put > conditions on it, but here is an example of a use that has no break > statement: > > for i, j in enumerate(something): > # suite > i += 1 > else: > i = 0 > > # i == number of things processed > > Sorry, there must be some typo in your code. The "i" is always 0 after the loop unless you break. >>> for i, j in enumerate(range(10)): ... i +=1 ... else: ... i = 0 ... >>> i 0 >>> Also, why would one increment the enumeration index "i" by hand? -------------- next part -------------- An HTML attachment was scrubbed... URL: From phd at phd.pp.ru Fri Oct 2 15:57:25 2009 From: phd at phd.pp.ru (Oleg Broytman) Date: Fri, 2 Oct 2009 17:57:25 +0400 Subject: [Python-ideas] for/else syntax In-Reply-To: <5d1a32000910020646i39e20e5fh9e15d40c22444528@mail.gmail.com> References: <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> <6f4025010910020558m5741673eka2d3cc284f219985@mail.gmail.com> <5d1a32000910020646i39e20e5fh9e15d40c22444528@mail.gmail.com> Message-ID: <20091002135725.GA26788@phd.pp.ru> On Fri, Oct 02, 2009 at 09:46:38AM -0400, Gerald Britton wrote: > for i, j in enumerate(something): > # suite > i += 1 > else: > i = 0 > > # i == number of things processed > > The thing here is, "i" won't be set by the "for" statement if > "something" is empty. OTOH, if "something" is non-empty, i >= 1 at > the end of the for-loop. Wrong, 'i' will always be 0 whether 'something' is empty or not. Without a 'break', 'else' is always executed. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From ubershmekel at gmail.com Fri Oct 2 16:00:36 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Fri, 2 Oct 2009 17:00:36 +0300 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC5E078.3010000@gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <200910021307.25444.steve@pearwood.info> <4AC5E078.3010000@gmail.com> Message-ID: <9d153b7c0910020700w5916fbady63c875080a752d85@mail.gmail.com> On Fri, Oct 2, 2009 at 2:14 PM, Nick Coghlan wrote: > > > Encouraging people to refer to the idiom as "break/else" and including a > SyntaxWarning for dubious usage of loop else clauses (i.e. those without > an associated break statement) are probably two fairly low cost > educational steps that could be taken though. > > A SyntaxError would be more fitting for py4k but it's still not gonna make the for..else code readable. Asking people to call this construct a "break/else" is confusing because: 1. "break/else" insinuates the "else" block is for when a break statement _did_ occur. 2. the "else" is located immediately after the "for" block. Calling this anything but "for..else" is going to cause a tongue-twisting, mind boggling Stroop effect. IMHO the only way to avoid this is to rename/modify "else" to somehow convey it's relation to the break statement. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gerald.britton at gmail.com Fri Oct 2 16:03:12 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Fri, 2 Oct 2009 10:03:12 -0400 Subject: [Python-ideas] for/else syntax In-Reply-To: <87pr95j46e.fsf@uwakimon.sk.tsukuba.ac.jp> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <87pr95j46e.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <5d1a32000910020703o2d461c38y4f7102ab8f38788@mail.gmail.com> > Footnotes: > [1] ?StopIteration can, but I thought raising StopIteration in user > code is considered quite bad form. StopIteration once put forward on this very list as a way to stop a generator. e.g. def stop(): raise StopIteration gen = (x in something if or stop() ) Toy example: Say you want the squares of all natural numbers n such that n <= j. Here's one simple way using the stop function above: from itertools import count squares = (n**2 for n in count() if n <= j or stop() ) Without the "or stop()" the generator goes into an infinite loop when n exceeds 10. Of course you can do this toy example other ways, this is just an illustration. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From jimjjewett at gmail.com Fri Oct 2 16:03:53 2009 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 2 Oct 2009 10:03:53 -0400 Subject: [Python-ideas] for/else syntax In-Reply-To: <877hvfk806.fsf@uwakimon.sk.tsukuba.ac.jp> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <7441c2a3-ec27-4394-a9c1-d52a726db93e@i4g2000prm.googlegroups.com> <9d153b7c0909301858t5e2c5aagf3df23f92b7db2ac@mail.gmail.com> <877hvfk806.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Thu, Oct 1, 2009 at 1:17 AM, Stephen J. Turnbull wrote: > Note: As a native speaker, since "for ... else" isn't English, I would > certainly look up the semantics before using the syntax. ?This is very > probably true for Japanese as well, who are far better at (consciously) > remembering English syntax than native speakers are. ?I don't know > about other non-natives. As another native speaker, I do find it confusing on most for loops -- so I don't use it there. It does make sense on search loops. Normal loop, where it is misleading: for item in source: process(item) else: # yikes, I would wrongly expect this to mean empty. But search loop where it makes sense: for candidate in source: if test(candidate): choice=candidate break else: choice=default (That said, even then, I tend to set the default outside the loop construct, just to avoid the else confusion.) -jJ From gerald.britton at gmail.com Fri Oct 2 16:05:18 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Fri, 2 Oct 2009 10:05:18 -0400 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910020654p3590ae7du4d58b5a7a1f18035@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> <6f4025010910020558m5741673eka2d3cc284f219985@mail.gmail.com> <5d1a32000910020646i39e20e5fh9e15d40c22444528@mail.gmail.com> <9d153b7c0910020654p3590ae7du4d58b5a7a1f18035@mail.gmail.com> Message-ID: <5d1a32000910020705g5e23ebafp665df983f6c5acd3@mail.gmail.com> Nope: >>> l = [] >>> for i,j in enumerate(l): pass ... >>> i Traceback (most recent call last): File "", line 1, in NameError: name 'i' is not defined "i" is not initialized if "l" is empty. On Fri, Oct 2, 2009 at 9:54 AM, Yuvgoog Greenle wrote: > > On Fri, Oct 2, 2009 at 4:46 PM, Gerald > Britton??wrote: >> >> I can't imagine why I'm still commenting on this thread, since there >> is no chance that Python will remove the "else" from for/while or put >> conditions on it, but here is an example of a use that has no break >> statement: >> >> for i, j in enumerate(something): >> ?# suite >> ?i += 1 >> else: >> ?i = 0 >> >> # i == number of things processed >> > > Sorry, there must be some typo in your code. The "i" is always 0 after the > loop unless you break. >>>> for i, j in enumerate(range(10)): > ... ? ? i +=1 > ... else: > ... ? ? i = 0 > ... >>>> i > 0 >>>> > Also, why would one increment the enumeration index "i" by hand? > -- Gerald Britton From stephen at xemacs.org Fri Oct 2 16:16:27 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 02 Oct 2009 23:16:27 +0900 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> Message-ID: <87ocopj2ys.fsf@uwakimon.sk.tsukuba.ac.jp> Yuvgoog Greenle writes: > I think I understand what you're saying and I just don't understand how it > pertains to for..else. Specifically I would like an explanation about why > the following quote is nonsense to you. I didn't say it was nonsense and I didn't mean it was nonsense. I can see no reason at all why PyLint wouldn't warn about "breakless for-else". "raise StopIteration" and "if __debug__: break" *may* be sufficient reason Python itself shouldn't be too noisy, but for sure PyLint should warn about it. From steve at pearwood.info Fri Oct 2 16:35:50 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 3 Oct 2009 00:35:50 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC5DD80.6050803@gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DD80.6050803@gmail.com> Message-ID: <200910030035.51041.steve@pearwood.info> On Fri, 2 Oct 2009 09:01:20 pm Nick Coghlan wrote: > Yuvgoog Greenle wrote: > > 1. Not allowing a for loop that has no "break" to have an "else". > > Just like "else" isn't allowed when there isn't an "except" in the > > "try". There really is no excuse, justification or sense in a "for" > > loop that has no "break" yet has an "else". > > As Stephen said, that's a very good point. Having the compiler issue > a SyntaxWarning when compiling such code is perfectly possible and > would make a great deal of sense (and probably prevent many cases of > naive misuse). I disagree that it makes sense. Why raise a SyntaxWarning for legal code? Shall we raise SyntaxWarning for the following as well? #1 pass pass #2 if False: parrot() #3 for x in []: do_something() #4 try: something() except Exception: raise etc. There are all sorts of code which is legal but does nothing. Why should the language single out one of them for a warning? If that warning is anywhere, it should be in PyLint or equivalent. It's even more problematic when you consider code where the break is inside a "if __debug__" clause. Then you've code which raises a warning when you run with the -O flag, and doesn't when you run without. -- Steven D'Aprano From steve at pearwood.info Fri Oct 2 16:43:38 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 3 Oct 2009 00:43:38 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC5DEA3.3020701@gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DEA3.3020701@gmail.com> Message-ID: <200910030043.38556.steve@pearwood.info> On Fri, 2 Oct 2009 09:06:11 pm Nick Coghlan wrote: > Just as there is no "try/else", Perhaps there should be. Does anyone know why there isn't? It seems an arbitrary restriction to me, because the following pieces of code are not equivalent: # not currently legal try: if cond: raise ValueError else: print "no exception was raised" finally: print "done" versus: try: if cond: raise ValueError finally: print "done" print "no exception was raised" > but only "except/else", so there is > no "for/else" or "while/else", but only "break/else". That's certainly not true. >>> for x in [1, 2, 3]: ... pass ... else: ... print "else without break" ... else without break > Issuing a SyntaxWarning might be a good way to make that clear. Such a warning would be spurious. -- Steven D'Aprano From ubershmekel at gmail.com Fri Oct 2 17:12:35 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Fri, 2 Oct 2009 18:12:35 +0300 Subject: [Python-ideas] for/else syntax In-Reply-To: <200910021506.17321.steve@pearwood.info> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910021506.17321.steve@pearwood.info> Message-ID: <9d153b7c0910020812t5a0eb648q4fb79f6c19598ca7@mail.gmail.com> On Fri, Oct 2, 2009 at 8:06 AM, Steven D'Aprano wrote: > > for x in xs: > if __debug__: > break > else: > print "whatever" > > "if __debug__: suite" is special: the Python compiler optimizes the > entire suite away when running with the -O flag. So if Python would > treat the presence of an else as an error unless there was a break, you > could have some code which was, or wasn't, legal according to the > presence of the optimize flag. This is clearly a Bad Thing. > > Concerning "if __debug__: break", I'm no expert on this subject, but I'm guessing that python does it in the following order:1. Parse 2. Compile to bytecode Also, I'm guessing optimizations occur at compilation and not in the parsing stage. So the "if __debug__: break" would parse normally and for..else won't give a syntax error. The "if __debug__" block would just be deleted at compilation time. As I said, this is just an educated guess, please correct me if I'm wrong. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Fri Oct 2 17:29:56 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 03 Oct 2009 01:29:56 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <87ocopj2ys.fsf@uwakimon.sk.tsukuba.ac.jp> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> <87ocopj2ys.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4AC61C74.70707@gmail.com> Stephen J. Turnbull wrote: > Yuvgoog Greenle writes: > > > I think I understand what you're saying and I just don't understand how it > > pertains to for..else. Specifically I would like an explanation about why > > the following quote is nonsense to you. > > I didn't say it was nonsense and I didn't mean it was nonsense. > > I can see no reason at all why PyLint wouldn't warn about "breakless > for-else". "raise StopIteration" and "if __debug__: break" *may* be > sufficient reason Python itself shouldn't be too noisy, but for sure > PyLint should warn about it. raise StopIteration in the body of the for loop will escape, just like any other suggestion. However, Gerald's point that the else clause can also be used to give the iteration variable a "default" value (with nary a break in sight) in addition to the more traditional search loop idiom is enough to convince me that the compiler should stay out of this one. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Fri Oct 2 17:34:34 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 03 Oct 2009 01:34:34 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <200910030043.38556.steve@pearwood.info> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DEA3.3020701@gmail.com> <200910030043.38556.steve@pearwood.info> Message-ID: <4AC61D8A.5030306@gmail.com> Steven D'Aprano wrote: > On Fri, 2 Oct 2009 09:06:11 pm Nick Coghlan wrote: > >> Just as there is no "try/else", > > Perhaps there should be. Does anyone know why there isn't? It seems an > arbitrary restriction to me, because the following pieces of code are > not equivalent: > > # not currently legal > try: > if cond: > raise ValueError > else: > print "no exception was raised" > finally: > print "done" Just drop the else line and you get the semantics you're after: try: if cond: raise ValueError print "no exception was raised" finally: print "done" The reason except/else is necessary is that the code in the else clause runs: 1. Outside the scope of the exception handler 2. Only if the except clause isn't taken If there is no except clause, then the code that would have been in the else clause can just be moved to the end of the try block (as in the example above). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Fri Oct 2 17:35:19 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 03 Oct 2009 01:35:19 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <200910030035.51041.steve@pearwood.info> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DD80.6050803@gmail.com> <200910030035.51041.steve@pearwood.info> Message-ID: <4AC61DB7.9020305@gmail.com> Steven D'Aprano wrote: > On Fri, 2 Oct 2009 09:01:20 pm Nick Coghlan wrote: >> Yuvgoog Greenle wrote: >>> 1. Not allowing a for loop that has no "break" to have an "else". >>> Just like "else" isn't allowed when there isn't an "except" in the >>> "try". There really is no excuse, justification or sense in a "for" >>> loop that has no "break" yet has an "else". >> As Stephen said, that's a very good point. Having the compiler issue >> a SyntaxWarning when compiling such code is perfectly possible and >> would make a great deal of sense (and probably prevent many cases of >> naive misuse). > > I disagree that it makes sense. Why raise a SyntaxWarning for legal > code? Yeah, I was wrong. Definitely in pylint/pychecker territory. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Fri Oct 2 17:39:01 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 03 Oct 2009 01:39:01 +1000 Subject: [Python-ideas] Decorator syntax restriction In-Reply-To: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> References: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> Message-ID: <4AC61E95.40108@gmail.com> Paul Moore wrote: > 2009/10/2 Rob Cliffe : >> So: please speak, if you agree with me, whether for the reasons I give or >> for different ones, or if you disagree with me - please give reasons too. > > I believe that the key reason for there being restrictions is to avoid > Python code looking like "line noise". That's not precise - the > specific rules as given express a sensible (in my view) balance > between this goal and simplicity of > implementation/explanation/usability. +1 to everything Paul said (including the parts I cut) +1 to a patch that relaxes the restriction on decorations to allow subscript operations in addition to call operations. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ubershmekel at gmail.com Fri Oct 2 17:42:48 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Fri, 2 Oct 2009 18:42:48 +0300 Subject: [Python-ideas] for/else syntax In-Reply-To: <200910030035.51041.steve@pearwood.info> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DD80.6050803@gmail.com> <200910030035.51041.steve@pearwood.info> Message-ID: <9d153b7c0910020842r283aaae5pa415df4eaa59288e@mail.gmail.com> On Fri, Oct 2, 2009 at 5:35 PM, Steven D'Aprano wrote: > > I disagree that it makes sense. Why raise a SyntaxWarning for legal > code? Shall we raise SyntaxWarning for the following as well? > > #1 > pass > pass > > #2 > if False: > ? ?parrot() > > #3 > for x in []: > ? ?do_something() > > #4 > try: > ? ?something() > except Exception: > ? ?raise > All of your examples do exactly what they appear to do. The for...else construct that has no "break" appears to do something, yet does nothing. I beg of you to try out "The Question Challenge" I posted earlier. It's a win-win situation for you. If you're still not convinced, check out this code that is legal, yet appears to do something that it doesn't: try: ?? ?z = math.sqrt(x / y) except?ZeroDivisionError,?ValueError: ?? ?print("Math error, try again") That's a real life mistake many have seen occur. Python 3 fixed this in a very elegant way. Now let's talk python 4. From ubershmekel at gmail.com Fri Oct 2 18:00:08 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Fri, 2 Oct 2009 19:00:08 +0300 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC61D8A.5030306@gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DEA3.3020701@gmail.com> <200910030043.38556.steve@pearwood.info> <4AC61D8A.5030306@gmail.com> Message-ID: <9d153b7c0910020900o3620c548y6000045bf8da2487@mail.gmail.com> On Fri, Oct 2, 2009 at 6:34 PM, Nick Coghlan wrote: > Steven D'Aprano wrote: >> On Fri, 2 Oct 2009 09:06:11 pm Nick Coghlan wrote: >> >>> Just as there is no "try/else", >> >> Perhaps there should be. Does anyone know why there isn't? It seems an >> arbitrary restriction to me, because the following pieces of code are >> not equivalent: >> >> # not currently legal >> try: >> ? ? if cond: >> ? ? ? ? raise ValueError >> else: >> ? ? print "no exception was raised" >> finally: >> ? ? print "done" > > Just drop the else line and you get the semantics you're after: > > try: > ? ?if cond: > ? ? ? ?raise ValueError > ? ?print "no exception was raised" > finally: > ? ?print "done" > Also, in English/programmer/pesudo-code/human language, the following: something: A else: B Conveys that either A occurred or B occurred. This is just something everybody is used to because everybody knows if/else. Consider the following code: except: print("A brave attempt") else: print("Yippee") It's easy to understand that either "except" occurred or "else" occurred. The only time this rule is broken in python is the for/else and while/else. From arnodel at googlemail.com Fri Oct 2 18:00:38 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Fri, 2 Oct 2009 17:00:38 +0100 Subject: [Python-ideas] for/else syntax In-Reply-To: <200910030043.38556.steve@pearwood.info> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DEA3.3020701@gmail.com> <200910030043.38556.steve@pearwood.info> Message-ID: <6C7C5C4E-36DA-4F5F-AA15-48EC648FBAB6@googlemail.com> On 2 Oct 2009, at 15:43, Steven D'Aprano wrote: > > Perhaps there should be. Does anyone know why there isn't? It seems an > arbitrary restriction to me, because the following pieces of code are > not equivalent: > > # not currently legal > try: > if cond: > raise ValueError > else: > print "no exception was raised" > finally: > print "done" > > But the above is the same as: try: if cond: raise ValueError print "no exception was raised" finally: print "done" > versus: [...] -- Arnaud From gerald.britton at gmail.com Fri Oct 2 18:01:05 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Fri, 2 Oct 2009 12:01:05 -0400 Subject: [Python-ideas] Decorator syntax restriction In-Reply-To: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> References: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> Message-ID: <5d1a32000910020901v22998662m25ecd1c4a0ac85d6@mail.gmail.com> fwiw C's ternary operator is implemented in Python like this: x = a if b else c Gotta say I miss pre- and post-fix operators, though. I'd rather write: x = a[b++] than x = a[b] b += 1 since I only have to reference "b" once with a post-fix operator. On Fri, Oct 2, 2009 at 8:18 AM, Paul Moore wrote: > 2009/10/2 Rob Cliffe : >> So: please speak, if you agree with me, whether for the reasons I give or >> for different ones, or if you disagree with me - please give reasons too. > > I believe that the key reason for there being restrictions is to avoid > Python code looking like "line noise". That's not precise - the > specific rules as given express a sensible (in my view) balance > between this goal and simplicity of > implementation/explanation/usability. > > For example, without restrictions the following would be valid (yes, > it's silly, yes it's deliberately a worst case) > > @(a['b'].fns[1])(1,2,{3,4}) > def something(): > > I contend that's clearly "line noise". Python has a long tradition of > not just frowning on code like this, but actively forbidding such code > and not implementing constructs that contribute to such. Witness the > fact that C's ternary operator (?:) and the pre and post increment > operators (++ and --) are not valid Python. So forbidding such code is > clearly consistent with Python's traditions. > > @classmethod > def f(...): > > is equally obviously *not* line noise, and it's entirely reasonable to > allow it. So pick a point on the spectrum between the first and second > example. That's what the current rules do - you may not agree with the > precise point picked, but my contention is that not picking a point at > all is contrary to Python's established design. > > I'm not sure there is a corresponding design principle within Python > that all constructs should be fully general (which is basically the > principle you're appealing to). I can't immediately find > counterexamples, though, so there may be a point there. But certainly, > I'd say that such a principle (if it exists) isn't as well-established > as the "no line noise" one (witness comments like "if you want Perl, > you know where to get it"...) > >> 1(a)) It was inconsistent to impose a restriction on expressions in one particular context [meaning: and not in others]. >> 1(b)) The restriction was hard to explain [but see below]. >> 2) The restriction can easily be circumvented. >> 3) Plausible use cases exist [I gave some]. > > The above covers my feelings on 1(a) and 1(b). > > With regard to (2), circumventing the restriction isn't really the > point - the whole decorator construct is a shorthand. > > @deco > def fn(): > ? ?... > > vs > > def fn(): > ? ?... > fn = deco(fn) > > Giving a complex expression a name increases readability, and the > assignment can be kept near the decorator if it's a one-off, as > follows > > meaningful_name = (a['b'].fns[1])(1,2,{3,4}) > > @meaningful_name > def f(): > ? ?... > > I'd go so far as to say that was improving readability, rather than > "circumvention". > > As for (3), I agree that plausible use cases exist. They *may* warrant > allowing certain extra constructs in ("moving the point" as I said > above) but I'm not sure about that, and to my mind they certainly > don't warrant completely removing the restriction. > > As a final point, I should say that personally, I have never written > any code which is impacted by the current restrictions on decorator > syntax. So, I don't personally have any need for the relaxation you're > proposing. I'm not sure if that's a point in your favour (as my view > is of limited value since I don't need the feature you're proposing) > or against (as I'm an example of why the proposal isn't as generally > useful as you claim) but I mention it anyway, in the interests of > complete disclosure :-) > > I hope this helps. > > Paul. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From debatem1 at gmail.com Fri Oct 2 18:07:01 2009 From: debatem1 at gmail.com (geremy condra) Date: Fri, 2 Oct 2009 12:07:01 -0400 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> Message-ID: On Thu, Oct 1, 2009 at 11:11 PM, Yuvgoog Greenle wrote: > Let me rephrase - for..else is a _misleading_ construct. It suggests one > thing yet does another. > This response is way too long, so here's a TOC: > 1. The Question Challenge - A way to examine if "for..else" is just > unintuitive or misleading. > 2. What can be done about for...else. > 3. Lets talk about "try/else" because a few people mentioned it as though > it's somehow related to the discussion. > > ---------------- > 1. The Question Challenge - A way to examine if "for..else" is just > unintuitive or misleading. > > Antti's experiment got me asking, you can try this as well. Ask a person > who knows python the following questions: > Question #1: Do you know what happens when an "else" appears after a "for" > loop? > if they answer yes then say nevermind. > otherwise: continue to question #2. > Question #2: When would the code in the "else" block execute? If you don't > know, it's ok to guess. > If they answer "when a break didn't occur", I'm willing to give you, > the asker, up to 100 lines of patch code in whatever python project you > want.. > If they answer "maybe when the for loop didn't run or was an empty > list" then you don't owe me anything. > > Head on over to graphine.org- I'm getting requests for database-backed graphs and don't know enough to do it portably. I asked 4 and got 2 correct answers, but I'm happy with the promised 100 lines. If you know as little about databases as I do, take a look at the graphml code- it's out of date and probably doesn't work. Geremy Condra -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at gmail.com Fri Oct 2 18:12:25 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Fri, 2 Oct 2009 17:12:25 +0100 Subject: [Python-ideas] for/else syntax In-Reply-To: <6C7C5C4E-36DA-4F5F-AA15-48EC648FBAB6@googlemail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DEA3.3020701@gmail.com> <200910030043.38556.steve@pearwood.info> <6C7C5C4E-36DA-4F5F-AA15-48EC648FBAB6@googlemail.com> Message-ID: <6f4025010910020912g710f2394p734c1cff6fa1d718@mail.gmail.com> 2009/10/2 Arnaud Delobelle > > On 2 Oct 2009, at 15:43, Steven D'Aprano wrote: > > >> Perhaps there should be. Does anyone know why there isn't? It seems an >> arbitrary restriction to me, because the following pieces of code are >> not equivalent: >> >> # not currently legal >> try: >> if cond: >> raise ValueError >> else: >> print "no exception was raised" >> finally: >> print "done" >> >> >> But the above is the same as: > > try: > if cond: > raise ValueError > print "no exception was raised" > finally: > print "done" > > versus: >> > [...] > > But as always with Python, the code in the else clause is not protected by the except so that you do not incorrectly handle unexpected exceptions. Michael > -- > Arnaud > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From gerald.britton at gmail.com Fri Oct 2 18:18:20 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Fri, 2 Oct 2009 12:18:20 -0400 Subject: [Python-ideas] Decorator syntax restriction In-Reply-To: <79990c6b0910020914n7b5feab3k689fd8e2bb981aa8@mail.gmail.com> References: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> <5d1a32000910020901v22998662m25ecd1c4a0ac85d6@mail.gmail.com> <79990c6b0910020914n7b5feab3k689fd8e2bb981aa8@mail.gmail.com> Message-ID: <5d1a32000910020918g497e775fl87aefce63b81a6ce@mail.gmail.com> Quite right. Though I still miss pre- and post-fix operators. On Fri, Oct 2, 2009 at 12:14 PM, Paul Moore wrote: > 2009/10/2 Gerald Britton : >> fwiw C's ternary operator is implemented in Python like this: >> >> x = a if b else c > > I know. My email was wordy enough already, but that was my point - > Python prefers keywords to punctuation, on the same "no line noise" > principle. > > Paul > -- Gerald Britton From ubershmekel at gmail.com Fri Oct 2 18:18:55 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Fri, 2 Oct 2009 19:18:55 +0300 Subject: [Python-ideas] for/else syntax In-Reply-To: References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> Message-ID: <9d153b7c0910020918i57c9626ev76d5c4cf040fa4b9@mail.gmail.com> On Fri, Oct 2, 2009 at 7:07 PM, geremy condra wrote: > > Head on over to graphine.org- I'm getting requests for database-backed > graphs and don't know enough to do it portably. I asked 4 and got 2 correct > answers, but I'm happy with the promised 100 lines. If you know as little > about databases as I do, take a look at the graphml code- it's out of date > and probably doesn't work. > > Geremy Condra > I'm on it. I don't know much about databases, django spoiled me, but I'll try. Others don't be discouraged, go for the challenge, I'll just stack and schedule your assignments if there are any more crazy hacks that can guess this one. I'm keeping score here, and yes, I know blind faith in strangers is blind, but it's worth it if the point'll come through in the end :-) --yuv From steve at pearwood.info Fri Oct 2 18:23:46 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 3 Oct 2009 02:23:46 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910020842r283aaae5pa415df4eaa59288e@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910030035.51041.steve@pearwood.info> <9d153b7c0910020842r283aaae5pa415df4eaa59288e@mail.gmail.com> Message-ID: <200910030223.46998.steve@pearwood.info> On Sat, 3 Oct 2009 01:42:48 am Yuvgoog Greenle wrote: > On Fri, Oct 2, 2009 at 5:35 PM, Steven D'Aprano wrote: > > I disagree that it makes sense. Why raise a SyntaxWarning for legal > > code? Shall we raise SyntaxWarning for the following as well? > > > > #1 > > pass > > pass > > > > #2 > > if False: > > ? ?parrot() > > > > #3 > > for x in []: > > ? ?do_something() > > > > #4 > > try: > > ? ?something() > > except Exception: > > ? ?raise > > All of your examples do exactly what they appear to do. And so does for..else with no break. > The for...else construct that has no "break" appears to do something, > yet does nothing. Nonsense. >>> for x in []: ... print "Do nothing" ... else: ... print "Do something" ... Do something > I beg of you to try out "The Question Challenge" I > posted earlier. It's a win-win situation for you. Your second question: [quote] Question #2: When would the code in the "else" block execute? If you don't know, it's ok to guess. [...] If they answer "maybe when the for loop didn't run or was an empty list" then you don't owe me anything. [end quote] But the else clause *does* run when the loop didn't run or was an empty list. That second answer would be correct, as far as it goes. It fails to describe all the behaviour of the else clause, but it's not wrong, merely incomplete. > If you're still not convinced, check out this code that is legal, yet > appears to do something that it doesn't: > > try: > ?? ?z = math.sqrt(x / y) > except?ZeroDivisionError,?ValueError: > ?? ?print("Math error, try again") Irrelevant. We're not talking about the syntax of the except statement. Why would confusing syntax there convince me that a *completely different* statement was also confusing? That's like saying, "If you don't agree that chocolate tastes terrible, have a taste of this cow-vomit. Terrible, right? So now we agree, chocolate tastes terrible." -- Steven D'Aprano From brett at python.org Fri Oct 2 19:04:07 2009 From: brett at python.org (Brett Cannon) Date: Fri, 2 Oct 2009 10:04:07 -0700 Subject: [Python-ideas] Decorator syntax restriction In-Reply-To: <4AC61E95.40108@gmail.com> References: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> <4AC61E95.40108@gmail.com> Message-ID: [multi-reply] On Fri, Oct 2, 2009 at 03:43, Rob Cliffe wrote: > Last time I wrote in favour of removing the restriction on decorator syntax, > an unfortunate offhand remark of mine about something being more Pythonic > led to a tangential debate about what was Pythonic and whether concision was > always good, etc., with almost everyone ignoring the issue I was actually > raising. > An honourable exception was Brett Cannon, who answered my points squarely > (and did his best to shoot me down in flames). > =) Glad I was an exception. [snip] > I wrote: "Guido has said he has a 'gut feeling' against [removing the > restriction] but has not as far as I know rationalised it." > Brett wrote: "When it comes to Guido's gut, a rationalization isn't needed. > Perk of being BDFL. Plus his gut is right so often it tends to not be > questioned." > > I am sure that Guido (may-he-live-forever) would be the first to agree that > saying an opinion of his should NEVER be questioned is just, well, silly. Very true, but in this case I agree with Guido's gut. If I didn't I wouldn't have brought up support for it. On Fri, Oct 2, 2009 at 08:39, Nick Coghlan wrote: > Paul Moore wrote: >> 2009/10/2 Rob Cliffe : >>> So: please speak, if you agree with me, whether for the reasons I give or >>> for different ones, or if you disagree with me - please give reasons too. >> >> I believe that the key reason for there being restrictions is to avoid >> Python code looking like "line noise". That's not precise - the >> specific rules as given express a sensible (in my view) balance >> between this goal and simplicity of >> implementation/explanation/usability. > > +1 to everything Paul said (including the parts I cut) > +1 from me as well. > +1 to a patch that relaxes the restriction on decorations to allow > subscript operations in addition to call operations. +0 from me. Like Paul, after all of the decorator usage I have done since its introduction I have never even come up against the restriction (hell, I didn't even realize there was one until this thread). But expanding to index syntax I could live with. -Brett From mwm-keyword-python.b4bdba at mired.org Sat Oct 3 01:42:45 2009 From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer) Date: Fri, 2 Oct 2009 19:42:45 -0400 Subject: [Python-ideas] for/else syntax In-Reply-To: <6f4025010910020912g710f2394p734c1cff6fa1d718@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DEA3.3020701@gmail.com> <200910030043.38556.steve@pearwood.info> <6C7C5C4E-36DA-4F5F-AA15-48EC648FBAB6@googlemail.com> <6f4025010910020912g710f2394p734c1cff6fa1d718@mail.gmail.com> Message-ID: <20091002194245.5ffb2a5b@bhuda.mired.org> On Fri, 2 Oct 2009 17:12:25 +0100 Michael Foord wrote: > 2009/10/2 Arnaud Delobelle > > On 2 Oct 2009, at 15:43, Steven D'Aprano wrote: > >> Perhaps there should be. Does anyone know why there isn't? It seems an > >> arbitrary restriction to me, because the following pieces of code are > >> not equivalent: > >> > >> # not currently legal > >> try: > >> if cond: > >> raise ValueError > >> else: > >> print "no exception was raised" > >> finally: > >> print "done" > >> > >> > >> But the above is the same as: > > > > try: > > if cond: > > raise ValueError > > print "no exception was raised" > > finally: > > print "done" > > > But as always with Python, the code in the else clause is not protected by > the except so that you do not incorrectly handle unexpected exceptions. Um, there's not an except clause to protect anything there. In both cases, the code flows from the try: block to the else: block (even without the else: line) and then the finally: block, with an exception in the try: or else: block skipping the rest of both. Consider the three cases: 1) no exception raised - both cases print "no exception was raised" and "done". 2) exception in original try block - both cases just print "done". 3) exception in original else - both cases print whatever prints from the else block before the exception, and then "done". So the else: might as well be a pass. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From mwm-keyword-python.b4bdba at mired.org Sat Oct 3 01:50:49 2009 From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer) Date: Fri, 2 Oct 2009 19:50:49 -0400 Subject: [Python-ideas] Addition to the Zen of python In-Reply-To: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> References: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> Message-ID: <20091002195049.38481b9d@bhuda.mired.org> On Fri, 2 Oct 2009 13:18:03 +0100 Paul Moore wrote: > I'm not sure there is a corresponding design principle within Python > that all constructs should be fully general (which is basically the > principle you're appealing to). I can't immediately find > counterexamples, though, so there may be a point there. But certainly, > I'd say that such a principle (if it exists) isn't as well-established > as the "no line noise" one (witness comments like "if you want Perl, > you know where to get it"...) This principle is know in Unix circles as "POLA" - the "Principle Of Least Astonishment". I think it's an important part of what makes Python Python - when you do something that seems obvious, it very seldom astonishes you. I'm not sure why Tim didn't include it in the Zen; my gut says that it's because this is an even more fundamental part of the landscape than most of the things on the list, so he overlooked it, but I'd rather he speaks for himself. So I propose amending the Zen of Python (as from "import this") to include it, as so: Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. The obvious way should not produce astonishing results. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those! http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From ubershmekel at gmail.com Sat Oct 3 02:06:57 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Sat, 3 Oct 2009 03:06:57 +0300 Subject: [Python-ideas] Addition to the Zen of python In-Reply-To: <20091002195049.38481b9d@bhuda.mired.org> References: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> <20091002195049.38481b9d@bhuda.mired.org> Message-ID: <9d153b7c0910021706x23f7a8afm1682b5e09a91911b@mail.gmail.com> From?Merriam-Webster's - obvious -?easily discovered, seen, or understood. So I think obvious might imply "not astonishing". On Sat, Oct 3, 2009 at 2:50 AM, Mike Meyer wrote: > > On Fri, 2 Oct 2009 13:18:03 +0100 > Paul Moore wrote: > > I'm not sure there is a corresponding design principle within Python > > that all constructs should be fully general (which is basically the > > principle you're appealing to). I can't immediately find > > counterexamples, though, so there may be a point there. But certainly, > > I'd say that such a principle (if it exists) isn't as well-established > > as the "no line noise" one (witness comments like "if you want Perl, > > you know where to get it"...) > > This principle is know in Unix circles as "POLA" - the "Principle Of > Least Astonishment". > > I think it's an important part of what makes Python Python - when you > do something that seems obvious, it very seldom astonishes you. I'm > not sure why Tim didn't include it in the Zen; my gut says that it's > because this is an even more fundamental part of the landscape than > most of the things on the list, so he overlooked it, but I'd rather he > speaks for himself. > > So I propose amending the Zen of Python (as from "import this") to > include it, as so: > > > Beautiful is better than ugly. > Explicit is better than implicit. > Simple is better than complex. > Complex is better than complicated. > Flat is better than nested. > Sparse is better than dense. > Readability counts. > Special cases aren't special enough to break the rules. > Although practicality beats purity. > Errors should never pass silently. > Unless explicitly silenced. > In the face of ambiguity, refuse the temptation to guess. > There should be one-- and preferably only one --obvious way to do it. > Although that way may not be obvious at first unless you're Dutch. > The obvious way should not produce astonishing results. > Now is better than never. > Although never is often better than *right* now. > If the implementation is hard to explain, it's a bad idea. > If the implementation is easy to explain, it may be a good idea. > Namespaces are one honking great idea -- let's do more of those! > > ? -- > Mike Meyer ? ? ? ? ? ? ?http://www.mired.org/consulting.html > Independent Network/Unix/Perforce consultant, email for more information. > > O< ascii ribbon campaign - stop html mail - www.asciiribbon.org > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas From adam at atlas.st Sat Oct 3 02:08:31 2009 From: adam at atlas.st (Adam Atlas) Date: Fri, 2 Oct 2009 20:08:31 -0400 Subject: [Python-ideas] Addition to the Zen of python In-Reply-To: <20091002195049.38481b9d@bhuda.mired.org> References: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> <20091002195049.38481b9d@bhuda.mired.org> Message-ID: On 2 Oct 2009, at 19:50, Mike Meyer wrote: > There should be one-- and preferably only one --obvious way to do it. > ... > The obvious way should not produce astonishing results. Seems redundant to me. If we're to consider something a "way to do it" (let alone the Obvious Way), I'd presume at the very minimum that it does what you're trying to do. Otherwise, it's just a non-obvious way to do something else. From python at rcn.com Sat Oct 3 02:07:46 2009 From: python at rcn.com (Raymond Hettinger) Date: Fri, 2 Oct 2009 17:07:46 -0700 Subject: [Python-ideas] Addition to the Zen of python References: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> <20091002195049.38481b9d@bhuda.mired.org> Message-ID: <58B4B8E098DE402CB1F7794507410A16@RaymondLaptop1> [Mike Meyer] > This principle is know in Unix circles as "POLA" - the "Principle Of > Least Astonishment". In language design circles, there should be a POGA -- Principle Of Guaranteed Astonishment. No matter what design choices you make, someone, somewhere is going to be astonished. Obviousness is in the eye of the beholder. Raymond From cmjohnson.mailinglist at gmail.com Sat Oct 3 03:18:44 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Fri, 2 Oct 2009 15:18:44 -1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <5d1a32000910020646i39e20e5fh9e15d40c22444528@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> <6f4025010910020558m5741673eka2d3cc284f219985@mail.gmail.com> <5d1a32000910020646i39e20e5fh9e15d40c22444528@mail.gmail.com> Message-ID: <3bdda690910021818u31eac2ebx1feda94dee99d784@mail.gmail.com> Those who believe that the for/else construct is sufficiently clear, please read this email again: Gerald Britton: > I can't imagine why I'm still commenting on this thread, since there > is no chance that Python will remove the "else" from for/while or put > conditions on it, but here is an example of a use that has no break > statement: > > for i, j in enumerate(something): > ?# suite > ?i += 1 > else: > ?i = 0 > > # i == number of things processed > > The thing here is, "i" won't be set by the "for" statement if > "something" is empty. ?OTOH, if "something" is non-empty, i >= 1 at > the end of the for-loop. ?Using the "else" clause in this way ensure > that "i" has the number of things processed at the end of the loop. > > To do this without the "else" I have to: > > i = 0 > for i, j in enumerate(something): > ?# suite > ?i += 1 > I can't imagine why I'm still commenting on this thread, since there > is no chance that Python will remove the "else" from for/while or put > conditions on it, but here is an example of a use that has no break > statement: > > for i, j in enumerate(something): > ?# suite > ?i += 1 > else: > ?i = 0 > > # i == number of things processed > > Does it work? ?Sure! ?But it moves the setting of "i" outside the > for-construct, which I personally dislike. I don't mean to pick on Gerald, but he got the meaning of for/else dead wrong *in the middle of a debate about for/else*. This shows that even experienced Pythoneers who read Python-ideas can be confused about the meaning of for/else. Using it in production code means leaving a trap for the programmers who come after you and read your code to get confused and create a bug. Yes, when the others turn your working use of for/else into a bug, it's their fault for not reading the tutorial more closely so as not get confused by it, but ultimately no matter how much you blame the user, that won't stop the next guy who uses your code from misunderstanding it. The best solution to this inherent confusingness is to leave bare for/else alone for the foreseeable future, but to add a less confusing synonym for this feature. I propose "else not break:" although "elif not break:" is good too. Rewriting Gerald's example as for i, j in enumerate(something): # suite i += 1 else if not break: i = 0 # i == number of things processed makes it 100% smack you in the face obvious that "Oh yeah, this is going to set i to 0 every time, since there's no break." In an unrelated but also good idea, it would be nice to be able to do what Gerald thought he was doing: for i, j in enumerate(something): # suite i += 1 else if None: #or some other suitable phrase for "empty" i = 0 # i == number of things processed ? Carl From cmjohnson.mailinglist at gmail.com Sat Oct 3 03:30:06 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Fri, 2 Oct 2009 15:30:06 -1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <87pr95j46e.fsf@uwakimon.sk.tsukuba.ac.jp> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <87pr95j46e.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <3bdda690910021830p74a7907y39e9efb622d9ea7c@mail.gmail.com> Stephen J. Turnbull: > [1] ?StopIteration can, but I thought raising StopIteration in user > code is considered quite bad form. No, StopIteration doesn't get caught unless it happen in the loop's get the next iter item phase: >>> def stop(): raise StopIteration ... >>> def yields_one(): ... yield 1 ... stop() ... yield 2 ... >>> def yields_two(): ... yield 1 ... yield 2 ... stop() ... >>> for i in yields_one(): #catches the stop ... print(i) ... 1 >>> for i in yields_two(): #catches ... print(i) ... 1 2 >>> for i in yields_two(): ... print(i) ... stop() #goes through ? 1 Traceback (most recent call last): File "", line 3, in File "", line 1, in stop StopIteration >>> ? Carl From steve at pearwood.info Sat Oct 3 03:46:24 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 3 Oct 2009 11:46:24 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <3bdda690910021818u31eac2ebx1feda94dee99d784@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <5d1a32000910020646i39e20e5fh9e15d40c22444528@mail.gmail.com> <3bdda690910021818u31eac2ebx1feda94dee99d784@mail.gmail.com> Message-ID: <200910031146.24683.steve@pearwood.info> On Sat, 3 Oct 2009 11:18:44 am Carl Johnson wrote: > I don't mean to pick on Gerald, but he got the meaning of for/else > dead wrong *in the middle of a debate about for/else*. This shows > that even experienced Pythoneers who read Python-ideas can be > confused about the meaning of for/else. Using it in production code > means leaving a trap for the programmers who come after you and read > your code to get confused and create a bug. Yes, when the others turn > your working use of for/else into a bug, it's their fault for not > reading the tutorial more closely so as not get confused by it, but > ultimately no matter how much you blame the user, that won't stop the > next guy who uses your code from misunderstanding it. Damn straight it's their fault. I hope you're not suggesting that people shouldn't use any feature that might be misunderstood by lazy, ignorant or stupid developers? > The best solution to this inherent confusingness is to leave bare > for/else alone for the foreseeable future, but to add a less > confusing synonym for this feature. I propose "else not break:" > although "elif not break:" is good too. You can already do this today, with no change to the compiler, using only a slightly different syntax to that suggested: for i, j in enumerate(something): # suite i += 1 else: # if not break i = 0 Best of all, it's an extensible syntax, so there's no need for bike-shedding arguments about whether it should be spelled "not break" or "no break" or "didn't break" or "only if loop ended normally". All of these variants, and more, are accepted: for i, j in enumerate(something): # suite i += 1 else: # no break occurred and so the loop ended normally i = 0 -- Steven D'Aprano From ubershmekel at gmail.com Sat Oct 3 05:43:02 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Sat, 3 Oct 2009 06:43:02 +0300 Subject: [Python-ideas] for/else syntax In-Reply-To: <200910030223.46998.steve@pearwood.info> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910030035.51041.steve@pearwood.info> <9d153b7c0910020842r283aaae5pa415df4eaa59288e@mail.gmail.com> <200910030223.46998.steve@pearwood.info> Message-ID: <9d153b7c0910022043k694f39f2r7afe8c1f26a37e7d@mail.gmail.com> *snip* *snip* On Fri, Oct 2, 2009 at 6:23 PM, Steven D'Aprano wrote: > On Sat, 3 Oct 2009 01:42:48 am Yuvgoog Greenle wrote: >> The for...else construct that has no "break" appears to do something, >> yet does nothing. > > Nonsense. > *snip* *snip* On Fri, Oct 2, 2009 at 6:23 PM, Steven D'Aprano wrote: >> try: >> ?? ?z = math.sqrt(x / y) >> except?ZeroDivisionError,?ValueError: >> ?? ?print("Math error, try again") > > Irrelevant. We're not talking about the syntax of the except statement. > > Why would confusing syntax there convince me that a *completely > different* statement was also confusing? That's like saying, "If you > don't agree that chocolate tastes terrible, have a taste of this > cow-vomit. Terrible, right? So now we agree, chocolate tastes > terrible." > You present 2 claims so I present 2 answers, please don't mix the two up: 1. Does "for...else" do what it appears to be doing? 2. Does python allow syntax that's likely to cause confusion or mistakes? A summary of the long answers that follow: 1. In certain cases, no it doesn't. In a few cases showcased here - comments are the only way to make the code readable. 2. Python doesn't impose silly syntax limitations, but it can and has deprecated a "legal" syntax if a more readable, not misleading, alternative can be found/created/used. ---------------- 1. Does "for...else" do what it appears to be doing? This is subjective matter so of course we can agree to disagree. Let me entertain you Steven. First of all I suggest reading through this thread and looking for code examples with for..else that were refuted, retracted, misunderstood or wrong. I can count at least 3-4 instances from here (Carl noted Gerald's but there were others as well). This isn't python-noobs either, it's python-ideas. These are good pythonistas that were mislead by the for..else syntax. Secondly, I recommend anybody interested to go ahead and try "The Question Challenge". I'll rephrase it a bit to accommodate Steven's claims. Ask a person who knows python the following questions: Question #1: Do you know what happens when an "else" appears after a "for" loop? if they answer yes then say never mind. otherwise: continue to question #2. Question #2: When does the code in the "else" block execute? Of course you don't know so it's ok to guess. If they answer "when a break didn't occur", I'm willing to give you, the asker, up to 100 lines of patch code in whatever python project you want. If they answer "only when the for loop didn't run or was an empty list" then you don't owe me anything. To remove any doubt you can ask the follow up question "So either the for loop block runs or the else block runs, but in no way both?" If you want to have some good conversation, tell them what python really does with that "else". Please don't misinterpret this Question Challenge as an attack on Python. I love Python and I have no other language. At least no other language I enjoy using :) ---------------- 2. Does python allow syntax that's likely to cause confusion or mistakes? Short answer - preferably no. It's an issue of practicality vs purity etc. Also, python tries not to have silly limitations. > Steven D'Aprano Wrote > I disagree that it makes sense. Why raise a SyntaxWarning for legal > code? Shall we raise SyntaxWarning for the following as well? > > #1 > pass > pass It would be a silly limitation to not allow: pass pass It would be a silly limitation because there's nothing wrong with "pass-pass". Concerning our specific discussion - "pass-pass" isn't likely to cause any confusion or mistakes. So what's a "syntax that's likely to cause confusion or mistakes"? For example, the old python 2 exception instance catcher: try: # code except Exception, e: # code Sounds ok, but now check out: try: # code except ExceptionClassA, ExceptionClassB: # handle errors of both exception types That usage of the syntax is wrong because ExceptionClassB wouldn't be caught and the exception instance would be named ExceptionClassB. This appears to do one thing (catch A and catch B), yet does something completely different (catch only A as B). Python 3 FTFY with "as". So the same code in the new syntax would be: try: # code except ExceptionClassA as ExceptionClassB: # handle errors of both exception types. The fact that ExceptionClassB is an instance is much more obvious and making mistakes is harder. So yeah, python tries to avoid syntax that may cause mistakes, in other words, the python syntax strives to be as expressive as reasonably possible. Note a few things please: 1. The python 2 syntax was legal 2. For python 3 it was declared an illegal syntax. 3. It was very easy to learn the correct python 2 syntax and not make mistakes but still it was fixed for py3k. 4. Doing crazy things that made no sense became a lot harder. I know I'm happy for that. I hope it's clearer how this example is relevant to "for..else" now. The point is if the "except" syntax can change from 2to3 to make it more readable and exact, so can the for..else syntax change from 3to4. Changing for..else would be done for very similar reasons. And don't get me wrong, I like nipples, chocolate and cow vomit as much as the next guy. It's just I would really love to discuss if and/or how this problem can be resolved. From ncoghlan at gmail.com Sat Oct 3 07:08:49 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 03 Oct 2009 15:08:49 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC61DB7.9020305@gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DD80.6050803@gmail.com> <200910030035.51041.steve@pearwood.info> <4AC61DB7.9020305@gmail.com> Message-ID: <4AC6DC61.8040607@gmail.com> Nick Coghlan wrote: > Steven D'Aprano wrote: >> On Fri, 2 Oct 2009 09:01:20 pm Nick Coghlan wrote: >>> Yuvgoog Greenle wrote: >>>> 1. Not allowing a for loop that has no "break" to have an "else". >>>> Just like "else" isn't allowed when there isn't an "except" in the >>>> "try". There really is no excuse, justification or sense in a "for" >>>> loop that has no "break" yet has an "else". >>> As Stephen said, that's a very good point. Having the compiler issue >>> a SyntaxWarning when compiling such code is perfectly possible and >>> would make a great deal of sense (and probably prevent many cases of >>> naive misuse). >> I disagree that it makes sense. Why raise a SyntaxWarning for legal >> code? > > Yeah, I was wrong. Definitely in pylint/pychecker territory. After the additional posts in the discussion, I'm back to my previous opinion - the construct should be referred to as break/else (or for/break/else and while/break/else) and we should at least investigate having the compiler itself raise at least a SyntaxWarning if it encounters a for/else or while/else construct without also encountering a break statement inside the loop. The situation is *exactly* analagous to try/except/else: if there is no except clause, then using the "else" is redundant - just put the code inside the try block (if there is also a finally clause) or ditch the try statement altogether (if there is no finally clause). The compiler enforces this by treating try/else with no except clause as a SyntaxError. For a loop with no break statements the else clause will *always* be executed, hence using the clause is completely redundant - the code it contains might as well be dedented and placed inline at the same level as the loop header. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From debatem1 at gmail.com Sat Oct 3 07:18:23 2009 From: debatem1 at gmail.com (geremy condra) Date: Sat, 3 Oct 2009 01:18:23 -0400 Subject: [Python-ideas] for/else syntax In-Reply-To: <3bdda690910021818u31eac2ebx1feda94dee99d784@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> <6f4025010910020558m5741673eka2d3cc284f219985@mail.gmail.com> <5d1a32000910020646i39e20e5fh9e15d40c22444528@mail.gmail.com> <3bdda690910021818u31eac2ebx1feda94dee99d784@mail.gmail.com> Message-ID: On Fri, Oct 2, 2009 at 9:18 PM, Carl Johnson < cmjohnson.mailinglist at gmail.com> wrote: > Those who believe that the for/else construct is sufficiently clear, > please read this email again: > > Gerald Britton: > > > I can't imagine why I'm still commenting on this thread, since there > > is no chance that Python will remove the "else" from for/while or put > > conditions on it, but here is an example of a use that has no break > > statement: > > > > for i, j in enumerate(something): > > # suite > > i += 1 > > else: > > i = 0 > > > > # i == number of things processed > > > > The thing here is, "i" won't be set by the "for" statement if > > "something" is empty. OTOH, if "something" is non-empty, i >= 1 at > > the end of the for-loop. Using the "else" clause in this way ensure > > that "i" has the number of things processed at the end of the loop. > > > > To do this without the "else" I have to: > > > > i = 0 > > for i, j in enumerate(something): > > # suite > > i += 1 > > I can't imagine why I'm still commenting on this thread, since there > > is no chance that Python will remove the "else" from for/while or put > > conditions on it, but here is an example of a use that has no break > > statement: > > > > for i, j in enumerate(something): > > # suite > > i += 1 > > else: > > i = 0 > > > > # i == number of things processed > > > > Does it work? Sure! But it moves the setting of "i" outside the > > for-construct, which I personally dislike. > > I don't mean to pick on Gerald, but he got the meaning of for/else > dead wrong *in the middle of a debate about for/else*. This shows that > even experienced Pythoneers who read Python-ideas can be confused > about the meaning of for/else. Using it in production code means > leaving a trap for the programmers who come after you and read your > code to get confused and create a bug. Yes, when the others turn your > working use of for/else into a bug, it's their fault for not reading > the tutorial more closely so as not get confused by it, but ultimately > no matter how much you blame the user, that won't stop the next guy > who uses your code from misunderstanding it. > This seems to be an argument for better documenting this behavior, rather than an argument against the behavior per se. > The best solution to this inherent confusingness is to leave bare > for/else alone for the foreseeable future, but to add a less confusing > synonym for this feature. I propose "else not break:" although "elif > not break:" is good too. Rewriting Gerald's example as > > for i, j in enumerate(something): > # suite > i += 1 > else if not break: > i = 0 > > # i == number of things processed > > To me its this is even less intuitive than the bare else- first you keep that, then you commandeer "if not break" to mean "if this didn't break". There may be a better syntax out there, but no offense, I don't see this as being it. > makes it 100% smack you in the face obvious that "Oh yeah, this is > going to set i to 0 every time, since there's no break." > > In an unrelated but also good idea, it would be nice to be able to do > what Gerald thought he was doing: > > for i, j in enumerate(something): > # suite > i += 1 > else if None: #or some other suitable phrase for "empty" > i = 0 > > # i == number of things processed > > ? Carl > > Again, not my cup of tea. Others will doubtless disagree. Geremy Condra -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmjohnson.mailinglist at gmail.com Sat Oct 3 08:26:57 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Fri, 2 Oct 2009 20:26:57 -1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <200910031146.24683.steve@pearwood.info> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <5d1a32000910020646i39e20e5fh9e15d40c22444528@mail.gmail.com> <3bdda690910021818u31eac2ebx1feda94dee99d784@mail.gmail.com> <200910031146.24683.steve@pearwood.info> Message-ID: <3bdda690910022326h304277b3j1a920e2f878177e6@mail.gmail.com> Steven D'Aprano: > You can already do this today, with no change to the compiler, using > only a slightly different syntax to that suggested: > > for i, j in enumerate(something): > ? ?# suite > ? ?i += 1 > else: ?# if not break > ? ?i = 0 > > > Best of all, it's an extensible syntax, so there's no need for > bike-shedding arguments about whether it should be spelled "not break" > or "no break" or "didn't break" or "only if loop ended normally". All > of these variants, and more, are accepted: Hoisted by my own petard! :-D This was also my suggestion for adding a do-while to Python a few weeks back: while True: #do { stuff # } until { if cond: break # } ? Carl From steve at pearwood.info Sat Oct 3 08:39:47 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 3 Oct 2009 16:39:47 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC6DC61.8040607@gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <4AC61DB7.9020305@gmail.com> <4AC6DC61.8040607@gmail.com> Message-ID: <200910031639.48088.steve@pearwood.info> On Sat, 3 Oct 2009 03:08:49 pm Nick Coghlan wrote: > [...] we should at least investigate > having the compiler itself raise at least a SyntaxWarning if it > encounters a for/else or while/else construct without also > encountering a break statement inside the loop. [...] > For a loop with no break statements the else clause will *always* be > executed, Modulo returns and exceptions. > hence using the clause is completely redundant True. But there is an effectively infinite amount of completely redundant code possible in Python. The performance hit and maintenance burden of raising warnings for redundant code would be immense, to say nothing of the nuisance value for the developer. Such warnings don't belong in the language, they belong in PyLint or equivalent. > - the code > it contains might as well be dedented and placed inline at the same > level as the loop header. True, except that the else clause clearly flags it as part of the semantic unit together with the for loop. Dedenting the code flags it as separate from the for loop. Apart from free style comments, we don't have a way of flagging code blobs finer than the function -- Python doesn't work that way. For instance, we're reduced to marking logical groups of code smaller than a function with comments. # Initialise the values. x = 2 y = 3 + x z = x*y + 1 # Process the sequence. value = 0 for o in seq: value = o - value + z value = 1.0/value # Do something else with it. value = parrot(value) return cheeseshop(value) It would be interesting to hypothesise about languages that naturally grouped (e.g.) the for loop and its pre-entry and post-exit code together into one obvious code unit. Python isn't that language, but (by accident, I'm sure) the else clause comes close. -- Steven D'Aprano From steve at pearwood.info Sat Oct 3 09:30:42 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 3 Oct 2009 17:30:42 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC61D8A.5030306@gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910030043.38556.steve@pearwood.info> <4AC61D8A.5030306@gmail.com> Message-ID: <200910031730.43151.steve@pearwood.info> On Sat, 3 Oct 2009 01:34:34 am Nick Coghlan wrote: > The reason except/else is necessary is that the code in the else > clause runs: > > 1. Outside the scope of the exception handler > 2. Only if the except clause isn't taken > > If there is no except clause, then the code that would have been in > the else clause can just be moved to the end of the try block (as in > the example above). It's quite surprising to me that the following two pieces of code are equivalent: try: print "Catch exceptions here" except AttributeError: # Keep the compiler happy, raise # avoid SyntaxError. else: print "Don't catch exceptions here." finally: print "Done" and: try: print "Catch exceptions here" print "Don't catch exceptions here." finally: print "Done" I've spent some time playing around with various cases, and I can't find any example where they behaviour differently, but I am still not convinced that there's not some unusual corner case where they differ. -- Steven D'Aprano From jared.grubb at gmail.com Sat Oct 3 08:15:53 2009 From: jared.grubb at gmail.com (Jared Grubb) Date: Fri, 2 Oct 2009 23:15:53 -0700 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910022043k694f39f2r7afe8c1f26a37e7d@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910030035.51041.steve@pearwood.info> <9d153b7c0910020842r283aaae5pa415df4eaa59288e@mail.gmail.com> <200910030223.46998.steve@pearwood.info> <9d153b7c0910022043k694f39f2r7afe8c1f26a37e7d@mail.gmail.com> Message-ID: On 2 Oct 2009, at 20:43, Yuvgoog Greenle wrote: > Ask a person who knows python the following questions: > Question #1: Do you know what happens when an "else" appears after a > "for" loop? > if they answer yes then say never mind. > otherwise: continue to question #2. > Question #2: When does the code in the "else" block execute? Of course > you don't know so it's ok to guess. > If they answer "when a break didn't occur", I'm willing to give > you, the asker, up to 100 lines of patch code in whatever python > project you want. > If they answer "only when the for loop didn't run or was an empty > list" then you don't owe me anything. And then ask those same people if they know what a "data descriptor" is. There's some features of a language you just have to learn. And frankly, the for/else statement is very simple. It's surprising the first time you see it, but that's because few other languages have it. But, once you learn it, it's VERY handy. On 2 Oct 2009, at 22:08, Nick Coghlan wrote: > After the additional posts in the discussion, I'm back to my previous > opinion - the construct should be referred to as break/else (or > for/break/else and while/break/else) and we should at least > investigate > having the compiler itself raise at least a SyntaxWarning if it > encounters a for/else or while/else construct without also > encountering > a break statement inside the loop. I would disagree. Imagine, while trying to debug code, you comment out an if statement inside the for loop, and now suddenly the whole loop becomes syntactically invalid. Plus, it's not only a break that could potentially make a for/else a valid construct. Return and yield are both legitimate, and technically anything that can throw an exception is too. Trying to implement the lexer to scan for all the possible combinations does not seem to be worth the effort. Jared From masklinn at masklinn.net Sat Oct 3 13:44:11 2009 From: masklinn at masklinn.net (Masklinn) Date: Sat, 3 Oct 2009 13:44:11 +0200 Subject: [Python-ideas] for/else syntax In-Reply-To: References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910030035.51041.steve@pearwood.info> <9d153b7c0910020842r283aaae5pa415df4eaa59288e@mail.gmail.com> <200910030223.46998.steve@pearwood.info> <9d153b7c0910022043k694f39f2r7afe8c1f26a37e7d@mail.gmail.com> Message-ID: <207E3417-4731-4BDA-BD83-CFBB6E96811C@masklinn.net> On 3 Oct 2009, at 08:15 , Jared Grubb wrote: > > On 2 Oct 2009, at 22:08, Nick Coghlan wrote: >> After the additional posts in the discussion, I'm back to my previous >> opinion - the construct should be referred to as break/else (or >> for/break/else and while/break/else) and we should at least >> investigate >> having the compiler itself raise at least a SyntaxWarning if it >> encounters a for/else or while/else construct without also >> encountering >> a break statement inside the loop. > > I would disagree. Imagine, while trying to debug code, you comment > out an if statement inside the for loop, and now suddenly the whole > loop becomes syntactically invalid. Plus, it's not only a break that > could potentially make a for/else a valid construct. I'm sure that's why Nick suggests a Warning, not an Error. There's no reason to make a break-less (loop) else invalid, but there are plenty of reasons to warn the programmer of such a pattern, as it's usually either pointless or erroneous. > Return and yield are both legitimate, and technically anything that > can throw an exception is too. > Yes, but these are irrelevant to the for:else: or while:else: cases. From ubershmekel at gmail.com Sat Oct 3 15:30:55 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Sat, 3 Oct 2009 16:30:55 +0300 Subject: [Python-ideas] for/else syntax In-Reply-To: <207E3417-4731-4BDA-BD83-CFBB6E96811C@masklinn.net> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910030035.51041.steve@pearwood.info> <9d153b7c0910020842r283aaae5pa415df4eaa59288e@mail.gmail.com> <200910030223.46998.steve@pearwood.info> <9d153b7c0910022043k694f39f2r7afe8c1f26a37e7d@mail.gmail.com> <207E3417-4731-4BDA-BD83-CFBB6E96811C@masklinn.net> Message-ID: <9d153b7c0910030630w74a5b2a8mea9ab13a51614346@mail.gmail.com> It just hit me. The for...else is so cryptic because of one big issue - the "else". Consider the following code: for obj in iterable: if found_it(obj): # Found what we were looking for! break if not break: # It wasn't there :( Similarly for while loops: while self.more_points_to_check(): if self.found_it(): # Found the desired point break if not break: # Exhausted the search space :( Notice how readable and beautiful the code has become. We have 2 implementation options: 1. an "if" that comes immediately after a for/while can have the special keywords "not" or "break". 2. the for/while syntax has an optional "if break" or "if not break". I'd like to hear if people like this idea and if so, which of the 2 options do you like better. Check it out, there's no need for a syntax error because it's so obviously wrong: for item in list_of_items: process(item) if not break: do_something() Also, now that the syntax is generic, you can add on to it whatever you like. For example: for item in list_of_items: process(item) if item.is_gold(): break if item.is_silver(): break if break and searching_for_metals: celebrate(list_of_items) --yuv From rrr at ronadam.com Sat Oct 3 16:26:18 2009 From: rrr at ronadam.com (Ron Adam) Date: Sat, 03 Oct 2009 09:26:18 -0500 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC6DC61.8040607@gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DD80.6050803@gmail.com> <200910030035.51041.steve@pearwood.info> <4AC61DB7.9020305@gmail.com> <4AC6DC61.8040607@gmail.com> Message-ID: <4AC75F0A.6080504@ronadam.com> Nick Coghlan wrote: > Nick Coghlan wrote: > For a loop with no break statements the else clause will *always* be > executed, hence using the clause is completely redundant - the code it > contains might as well be dedented and placed inline at the same level > as the loop header. The loop could have a "return", "raise", or be inside a "try/except" instead of having a "break". Cheers, Ron From ncoghlan at gmail.com Sat Oct 3 17:04:17 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 04 Oct 2009 01:04:17 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910030630w74a5b2a8mea9ab13a51614346@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910030035.51041.steve@pearwood.info> <9d153b7c0910020842r283aaae5pa415df4eaa59288e@mail.gmail.com> <200910030223.46998.steve@pearwood.info> <9d153b7c0910022043k694f39f2r7afe8c1f26a37e7d@mail.gmail.com> <207E3417-4731-4BDA-BD83-CFBB6E96811C@masklinn.net> <9d153b7c0910030630w74a5b2a8mea9ab13a51614346@mail.gmail.com> Message-ID: <4AC767F1.1080608@gmail.com> Yuvgoog Greenle wrote: > I'd like to hear if people like this idea and if so, which of the 2 > options do you like better. The compiler can't handle it - it will see the "if" as the start of a new statement. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Sat Oct 3 17:08:15 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 04 Oct 2009 01:08:15 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910030035.51041.steve@pearwood.info> <9d153b7c0910020842r283aaae5pa415df4eaa59288e@mail.gmail.com> <200910030223.46998.steve@pearwood.info> <9d153b7c0910022043k694f39f2r7afe8c1f26a37e7d@mail.gmail.com> Message-ID: <4AC768DF.5060609@gmail.com> Jared Grubb wrote: > Return and yield are both > legitimate, and technically anything that can throw an exception is too. Nope, break is unique in that it will execute the code immediately after the loop while not executing the code in the body of the else clause. "yield" and "continue" won't terminate the loop at all, while "return" and a raised exception will skip all of the code after the loop whether it is in an else clause or not. The *only* way to skip the else clause without also skipping the code immediately following the loop is to use a break statement. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From gerald.britton at gmail.com Sat Oct 3 13:49:46 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Sat, 3 Oct 2009 07:49:46 -0400 Subject: [Python-ideas] for/else syntax In-Reply-To: <3bdda690910021818u31eac2ebx1feda94dee99d784@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> <6f4025010910020558m5741673eka2d3cc284f219985@mail.gmail.com> <5d1a32000910020646i39e20e5fh9e15d40c22444528@mail.gmail.com> <3bdda690910021818u31eac2ebx1feda94dee99d784@mail.gmail.com> Message-ID: <5d1a32000910030449q795c6765j940bda7d61d05279@mail.gmail.com> Sory Carl, I did NOT get it dead wrong. The else is executed if the for loop falls through. That's the whole idea. Probably I should have set i to -1 in the else clause of the toy example though: for i, j in enumerate(something): # do something i += 1 else: i = -1 if i > 0: # we did something If you don't think the else is executed when the for-loop falls through, then you should re-read the documentation, here: http://docs.python.org/reference/compound_stmts.html#the-for-statement Where it says (just as I did, but in other words): "When the items are exhausted (which is immediately when the sequence is empty), the suite in the else clause, if present, is executed, and the loop terminates." On Fri, Oct 2, 2009 at 9:18 PM, Carl Johnson wrote: > Those who believe that the for/else construct is sufficiently clear, > please read this email again: > > Gerald Britton: > >> I can't imagine why I'm still commenting on this thread, since there >> is no chance that Python will remove the "else" from for/while or put >> conditions on it, but here is an example of a use that has no break >> statement: >> >> for i, j in enumerate(something): >> ?# suite >> ?i += 1 >> else: >> ?i = 0 >> >> # i == number of things processed >> >> The thing here is, "i" won't be set by the "for" statement if >> "something" is empty. ?OTOH, if "something" is non-empty, i >= 1 at >> the end of the for-loop. ?Using the "else" clause in this way ensure >> that "i" has the number of things processed at the end of the loop. >> >> To do this without the "else" I have to: >> >> i = 0 >> for i, j in enumerate(something): >> ?# suite >> ?i += 1 >> I can't imagine why I'm still commenting on this thread, since there >> is no chance that Python will remove the "else" from for/while or put >> conditions on it, but here is an example of a use that has no break >> statement: >> >> for i, j in enumerate(something): >> ?# suite >> ?i += 1 >> else: >> ?i = 0 >> >> # i == number of things processed >> >> Does it work? ?Sure! ?But it moves the setting of "i" outside the >> for-construct, which I personally dislike. > > I don't mean to pick on Gerald, but he got the meaning of for/else > dead wrong *in the middle of a debate about for/else*. This shows that > even experienced Pythoneers who read Python-ideas can be confused > about the meaning of for/else. Using it in production code means > leaving a trap for the programmers who come after you and read your > code to get confused and create a bug. Yes, when the others turn your > working use of for/else into a bug, it's their fault for not reading > the tutorial more closely so as not get confused by it, but ultimately > no matter how much you blame the user, that won't stop the next guy > who uses your code from misunderstanding it. > > The best solution to this inherent confusingness is to leave bare > for/else alone for the foreseeable future, but to add a less confusing > synonym for this feature. I propose "else not break:" although "elif > not break:" is good too. Rewriting Gerald's example as > > for i, j in enumerate(something): > ?# suite > ?i += 1 > else if not break: > ?i = 0 > > # i == number of things processed > > makes it 100% smack you in the face obvious that "Oh yeah, this is > going to set i to 0 every time, since there's no break." > > In an unrelated but also good idea, it would be nice to be able to do > what Gerald thought he was doing: > > for i, j in enumerate(something): > ?# suite > ?i += 1 > else if None: #or some other suitable phrase for "empty" > ?i = 0 > > # i == number of things processed > > ? Carl > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From ncoghlan at gmail.com Sat Oct 3 17:12:51 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 04 Oct 2009 01:12:51 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC75F0A.6080504@ronadam.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DD80.6050803@gmail.com> <200910030035.51041.steve@pearwood.info> <4AC61DB7.9020305@gmail.com> <4AC6DC61.8040607@gmail.com> <4AC75F0A.6080504@ronadam.com> Message-ID: <4AC769F3.6060808@gmail.com> Ron Adam wrote: > > > Nick Coghlan wrote: >> Nick Coghlan wrote: > > >> For a loop with no break statements the else clause will *always* be >> executed, hence using the clause is completely redundant - the code it >> contains might as well be dedented and placed inline at the same level >> as the loop header. > > The loop could have a "return", "raise", or be inside a "try/except" > instead of having a "break". As I've said elsewhere in this thread, while the else clause won't be executed in those cases, neither will any code that occurs immediately after the loop statement. Accordingly, using the else clause is unnecessary and rather misleading - just writing the extra code after the loop would be much clearer. The number one thing I have learned from this thread is the fact that the else clause only makes sense when used in conjunction with a break statement and the fact that this construct isn't regularly documented and explained as for/break/else and while/break/else is the major barrier to understanding what it does. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Sat Oct 3 17:17:35 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 04 Oct 2009 01:17:35 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC76720.80603@ronadam.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910021506.17321.steve@pearwood.info> <4AC5E122.9000902@gmail.com> <4AC76720.80603@ronadam.com> Message-ID: <4AC76B0F.6030609@gmail.com> Ron Adam wrote: > How about this? > > try: > for x in xs: > foo() > esle: > Print "foo didn't raise an exception." > except: > print "foo riased an exception" > > Using exceptions for flow control is very common in python. The else is adding no value here. That example would be better written as: try: for x in xs: foo() print "foo didn't raise an exception." except: print "foo riased an exception" > Or this: > > for x in xs: > y = foo() > if y is True: > return > else: > Print "y was never True" > > This is perfectly acceptable in my opinion. Again, using else here is redundant and misleading. The code is clearer if it is left out entirely: for x in xs: y = foo() if y is True: return print "y was never True" The *only* time an else clause is a better alternative to just writing the code after the loop is when there is a break statement present that may skip over it. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From rrr at ronadam.com Sat Oct 3 17:00:48 2009 From: rrr at ronadam.com (Ron Adam) Date: Sat, 03 Oct 2009 10:00:48 -0500 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC5E122.9000902@gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910021506.17321.steve@pearwood.info> <4AC5E122.9000902@gmail.com> Message-ID: <4AC76720.80603@ronadam.com> Nick Coghlan wrote: > Steven D'Aprano wrote: >> And in fact, it is legal to have for loops without a break but include >> an else. Is it useful? I don't know, but Python can't forbid them >> without breaking (pun not intended) code like this: >> >> for x in xs: >> if __debug__: >> break >> else: >> print "whatever" >> >> "if __debug__: suite" is special: the Python compiler optimizes the >> entire suite away when running with the -O flag. So if Python would >> treat the presence of an else as an error unless there was a break, you >> could have some code which was, or wasn't, legal according to the >> presence of the optimize flag. This is clearly a Bad Thing. >> >> Hence, even if for...else with no breaks are useless, they must be >> allowed. > > A SyntaxWarning should still be OK though - having an else clause that > can only be reached in debug mode is pretty dubious in its own right. > > Cheers, > Nick. How about this? try: for x in xs: foo() esle: Print "foo didn't raise an exception." except: print "foo riased an exception" Using exceptions for flow control is very common in python. Or this: for x in xs: y = foo() if y is True: return else: Print "y was never True" This is perfectly acceptable in my opinion. The explicit spelling of the "else in a "for/else" is closer to. for x in xs: do something elif x == xs[-1]: do something more But that isn't reliable if x or xs is modified inside the loop. So a better way to say it is: for x in xs: do something elif for.finished: do something more But of course since a for loop isn't an object with a name, we can't look at it directly to see what it's status is. I think we just need a really nice tutorial on using for-else's. Ron From ncoghlan at gmail.com Sat Oct 3 17:22:25 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 04 Oct 2009 01:22:25 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <5d1a32000910030449q795c6765j940bda7d61d05279@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> <6f4025010910020558m5741673eka2d3cc284f219985@mail.gmail.com> <5d1a32000910020646i39e20e5fh9e15d40c22444528@mail.gmail.com> <3bdda690910021818u31eac2ebx1feda94dee99d784@mail.gmail.com> <5d1a32000910030449q795c6765j940bda7d61d05279@mail.gmail.com> Message-ID: <4AC76C31.5090700@gmail.com> Gerald Britton wrote: > Sory Carl, I did NOT get it dead wrong. The else is executed if the > for loop falls through. That's the whole idea. Probably I should > have set i to -1 in the else clause of the toy example though: > > for i, j in enumerate(something): > # do something > i += 1 > else: > i = -1 > > if i > 0: > # we did something > > If you don't think the else is executed when the for-loop falls > through, then you should re-read the documentation, here: > > http://docs.python.org/reference/compound_stmts.html#the-for-statement > > Where it says (just as I did, but in other words): > > "When the items are exhausted (which is immediately when the sequence > is empty), the suite in the else clause, if present, is executed, and > the loop terminates." The code in your example is semantically identical to the following: for i, j in enumerate(something): # do something i += 1 i = -1 if i > 0: # This block is unreachable... Without a break statement in the body of the loop the else will by executed unconditionally, hence it is exactly the same as if the code it contains was just inline after the loop body. The semantics you describe can be obtained only by initialising i before the loop: i = -1 for i, j in enumerate(something): # do something i += 1 if i >= 0: # We did something Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From stephen at xemacs.org Sat Oct 3 17:34:18 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 04 Oct 2009 00:34:18 +0900 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910030630w74a5b2a8mea9ab13a51614346@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910030035.51041.steve@pearwood.info> <9d153b7c0910020842r283aaae5pa415df4eaa59288e@mail.gmail.com> <200910030223.46998.steve@pearwood.info> <9d153b7c0910022043k694f39f2r7afe8c1f26a37e7d@mail.gmail.com> <207E3417-4731-4BDA-BD83-CFBB6E96811C@masklinn.net> <9d153b7c0910030630w74a5b2a8mea9ab13a51614346@mail.gmail.com> Message-ID: <871vlkij9h.fsf@uwakimon.sk.tsukuba.ac.jp> Since we're voting, Yuvgoog Greenle writes: > I'd like to hear if people like this idea and if so, which of the 2 > options do you like better. -1 on both. In one liners (ie, comprehensions and generator expressions) I think "for ... if" is readable, but it means something else! > Also, now that the syntax is generic, you can add on to it whatever > you like. With the wind chill factor, -2. The only other syntax I've seen so far that makes any sense to me at all is for x in xs: # suite containing break then: # suite to execute after processing all xs but that would require a new keyword, so that's -1, too. From gerald.britton at gmail.com Sat Oct 3 17:29:44 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Sat, 3 Oct 2009 11:29:44 -0400 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC76B0F.6030609@gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910021506.17321.steve@pearwood.info> <4AC5E122.9000902@gmail.com> <4AC76720.80603@ronadam.com> <4AC76B0F.6030609@gmail.com> Message-ID: <5d1a32000910030829j1b6db39ct85b2218b36f9b26f@mail.gmail.com> More completely: The else clause is useful when the for-loop is expected to _not_ exit normally, which could be by break, return or raise. So: for something in somewhere: # do something if some_condition: break elif some_other_condition: return elif some_exceptional_condition: raise SomeException else: #only execute this if the for loop falls through So the else clause will be skipped in all three early exits from the for-loop, but would run if the loop fell-through or never executed. This is useful since the alternative usually involves setting some flag before the loop and again in the body of the loop and the n testing it afterwards -- a construct I, for one, really don't like. On Sat, Oct 3, 2009 at 11:17 AM, Nick Coghlan wrote: > Ron Adam wrote: >> How about this? >> >> ? ?try: >> ? ? ? for x in xs: >> ? ? ? ? ? foo() >> ? ? ? esle: >> ? ? ? ? ? Print "foo didn't raise an exception." >> ? ?except: >> ? ? ? print "foo riased an exception" >> >> Using exceptions for flow control is very common in python. > > The else is adding no value here. That example would be better written as: > > ? try: > ? ? ?for x in xs: > ? ? ? ? ?foo() > ? ? ?print "foo didn't raise an exception." > ? except: > ? ? ?print "foo riased an exception" > >> Or this: >> >> ? ?for x in xs: >> ? ? ? ?y = foo() >> ? ? ? ?if y is True: >> ? ? ? ? ? return >> ? ?else: >> ? ? ? ?Print "y was never True" >> >> This is perfectly acceptable in my opinion. > > Again, using else here is redundant and misleading. The code is clearer > if it is left out entirely: > > ? for x in xs: > ? ? ? y = foo() > ? ? ? if y is True: > ? ? ? ? ?return > ? print "y was never True" > > The *only* time an else clause is a better alternative to just writing > the code after the loop is when there is a break statement present that > may skip over it. > > Cheers, > Nick. > > -- > Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia > --------------------------------------------------------------- > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From masklinn at masklinn.net Sat Oct 3 17:31:25 2009 From: masklinn at masklinn.net (Masklinn) Date: Sat, 3 Oct 2009 17:31:25 +0200 Subject: [Python-ideas] for/else syntax In-Reply-To: <5d1a32000910030449q795c6765j940bda7d61d05279@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> <6f4025010910020558m5741673eka2d3cc284f219985@mail.gmail.com> <5d1a32000910020646i39e20e5fh9e15d40c22444528@mail.gmail.com> <3bdda690910021818u31eac2ebx1feda94dee99d784@mail.gmail.com> <5d1a32000910030449q795c6765j940bda7d61d05279@mail.gmail.com> Message-ID: On 3 Oct 2009, at 13:49 , Gerald Britton wrote: > Sory Carl, I did NOT get it dead wrong. The else is executed if the > for loop falls through. That's the whole idea. Probably I should > have set i to -1 in the else clause of the toy example though: > > for i, j in enumerate(something): > # do something > i += 1 > else: > i = -1 > > if i > 0: > # we did something > > If you don't think the else is executed when the for-loop falls > through, then you should re-read the documentation, here: Carl didn't dispute that point. In fact that's exactly the issue with your code: since you don't have any `break` statement in your `for:` loop, the `else:` clause will *always* be executed, whether `something` is an empty collection or not. Thus, the following `if i > 0:` clause cannot under any circumstance be executed. Your code (unless there is a break in the 'do something' part) is equivalent to: for i, j in enumerate(something): # do something i += 1 i = -1 if i > 0: # can never happen From gerald.britton at gmail.com Sat Oct 3 17:32:33 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Sat, 3 Oct 2009 11:32:33 -0400 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC76C31.5090700@gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> <6f4025010910020558m5741673eka2d3cc284f219985@mail.gmail.com> <5d1a32000910020646i39e20e5fh9e15d40c22444528@mail.gmail.com> <3bdda690910021818u31eac2ebx1feda94dee99d784@mail.gmail.com> <5d1a32000910030449q795c6765j940bda7d61d05279@mail.gmail.com> <4AC76C31.5090700@gmail.com> Message-ID: <5d1a32000910030832qaf1410dlc146d664c7a65810@mail.gmail.com> Yes, quite right. Another way would be: for i, j in enumerate(x): # do something i += 1 else: if "i" not in vars(): i = 0 Is it better than initializing before? Not sure. I guess it's a matter of taste. On Sat, Oct 3, 2009 at 11:22 AM, Nick Coghlan wrote: > Gerald Britton wrote: >> Sory Carl, I did NOT get it dead wrong. ?The else is executed if the >> for loop falls through. ?That's the whole idea. ?Probably I should >> have set i to -1 in the else clause of the toy example though: >> >> for i, j in enumerate(something): >> ? # do something >> ? i += 1 >> else: >> ?i = -1 >> >> if i > 0: >> ? # we did something >> >> If you don't think the else is executed when the for-loop falls >> through, then you should re-read the documentation, here: >> >> http://docs.python.org/reference/compound_stmts.html#the-for-statement >> >> Where it says (just as I did, but in other words): >> >> "When the items are exhausted (which is immediately when the sequence >> is empty), the suite in the else clause, if present, is executed, and >> the loop terminates." > > The code in your example is semantically identical to the following: > > ?for i, j in enumerate(something): > ? ?# do something > ? ?i += 1 > ?i = -1 > > ?if i > 0: > ? ?# This block is unreachable... > > Without a break statement in the body of the loop the else will by > executed unconditionally, hence it is exactly the same as if the code it > contains was just inline after the loop body. > > The semantics you describe can be obtained only by initialising i before > the loop: > > ?i = -1 > ?for i, j in enumerate(something): > ? ?# do something > ? ?i += 1 > > ?if i >= 0: > ? ?# We did something > > Cheers, > Nick. > > -- > Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia > --------------------------------------------------------------- > -- Gerald Britton From gerald.britton at gmail.com Sat Oct 3 17:34:21 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Sat, 3 Oct 2009 11:34:21 -0400 Subject: [Python-ideas] for/else syntax In-Reply-To: References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> <6f4025010910020558m5741673eka2d3cc284f219985@mail.gmail.com> <5d1a32000910020646i39e20e5fh9e15d40c22444528@mail.gmail.com> <3bdda690910021818u31eac2ebx1feda94dee99d784@mail.gmail.com> <5d1a32000910030449q795c6765j940bda7d61d05279@mail.gmail.com> Message-ID: <5d1a32000910030834x1358629do77b2b63d25f7d094@mail.gmail.com> Yes, you need to have some early exit in the body of the loop. Anyway, it's just a toy example. Not too hard to come with better ones. On Sat, Oct 3, 2009 at 11:31 AM, Masklinn wrote: > On 3 Oct 2009, at 13:49 , Gerald Britton wrote: >> >> Sory Carl, I did NOT get it dead wrong. ?The else is executed if the >> for loop falls through. ?That's the whole idea. ?Probably I should >> have set i to -1 in the else clause of the toy example though: >> >> for i, j in enumerate(something): >> ?# do something >> ?i += 1 >> else: >> i = -1 >> >> if i > 0: >> ?# we did something >> >> If you don't think the else is executed when the for-loop falls >> through, then you should re-read the documentation, here: > > Carl didn't dispute that point. In fact that's exactly the issue with your > code: since you don't have any `break` statement in your `for:` loop, the > `else:` clause will *always* be executed, whether `something` is an empty > collection or not. > > Thus, the following `if i > 0:` clause cannot under any circumstance be > executed. > > Your code (unless there is a break in the 'do something' part) is equivalent > to: > > ? ?for i, j in enumerate(something): > ? ? ? ?# do something > ? ? ? ?i += 1 > ? ?i = -1 > > ? ?if i > 0: > ? ? ? ?# can never happen > -- Gerald Britton From rrr at ronadam.com Sat Oct 3 17:27:45 2009 From: rrr at ronadam.com (Ron Adam) Date: Sat, 03 Oct 2009 10:27:45 -0500 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC769F3.6060808@gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DD80.6050803@gmail.com> <200910030035.51041.steve@pearwood.info> <4AC61DB7.9020305@gmail.com> <4AC6DC61.8040607@gmail.com> <4AC75F0A.6080504@ronadam.com> <4AC769F3.6060808@gmail.com> Message-ID: <4AC76D71.1080305@ronadam.com> Nick Coghlan wrote: > Ron Adam wrote: >> >> Nick Coghlan wrote: >>> Nick Coghlan wrote: >> >>> For a loop with no break statements the else clause will *always* be >>> executed, hence using the clause is completely redundant - the code it >>> contains might as well be dedented and placed inline at the same level >>> as the loop header. >> The loop could have a "return", "raise", or be inside a "try/except" >> instead of having a "break". > > As I've said elsewhere in this thread, while the else clause won't be > executed in those cases, neither will any code that occurs immediately > after the loop statement. Accordingly, using the else clause is > unnecessary and rather misleading - just writing the extra code after > the loop would be much clearer. > > The number one thing I have learned from this thread is the fact that > the else clause only makes sense when used in conjunction with a break > statement and the fact that this construct isn't regularly documented > and explained as for/break/else and while/break/else is the major > barrier to understanding what it does. > > Cheers, > Nick. Ahh, Ok, I agree. Documenting them as for/break/else and while/break/else would probably reduce 80% of the confusion people have with it. Ron From ubershmekel at gmail.com Sat Oct 3 17:39:53 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Sat, 3 Oct 2009 18:39:53 +0300 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC76D71.1080305@ronadam.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DD80.6050803@gmail.com> <200910030035.51041.steve@pearwood.info> <4AC61DB7.9020305@gmail.com> <4AC6DC61.8040607@gmail.com> <4AC75F0A.6080504@ronadam.com> <4AC769F3.6060808@gmail.com> <4AC76D71.1080305@ronadam.com> Message-ID: <9d153b7c0910030839l3e9737aaw6ee19f0fabb35834@mail.gmail.com> Document it as much as you want but the "else" doesn't hint or clue as to whether it catches breaks or is skipped by them. The next problem's gonna be at midnight when you're reading code and asking yourself "hmmm, for, blah blah, else, hmm oh yeah, else-break, so this else occurs whenever there's a break." On Sat, Oct 3, 2009 at 6:27 PM, Ron Adam wrote: > > > Nick Coghlan wrote: >> >> Ron Adam wrote: >>> >>> Nick Coghlan wrote: >>>> >>>> Nick Coghlan wrote: >>> >>>> For a loop with no break statements the else clause will *always* be >>>> executed, hence using the clause is completely redundant - the code it >>>> contains might as well be dedented and placed inline at the same level >>>> as the loop header. >>> >>> The loop could have a "return", "raise", or be inside a "try/except" >>> instead of having a "break". >> >> As I've said elsewhere in this thread, while the else clause won't be >> executed in those cases, neither will any code that occurs immediately >> after the loop statement. Accordingly, using the else clause is >> unnecessary and rather misleading - just writing the extra code after >> the loop would be much clearer. >> >> The number one thing I have learned from this thread is the fact that >> the else clause only makes sense when used in conjunction with a break >> statement and the fact that this construct isn't regularly documented >> and explained as for/break/else and while/break/else is the major >> barrier to understanding what it does. >> >> Cheers, >> Nick. > > Ahh, Ok, I agree. ?Documenting them as for/break/else and while/break/else > would probably reduce 80% of the confusion people have with it. > > Ron > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From stephen at xemacs.org Sat Oct 3 17:49:47 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 04 Oct 2009 00:49:47 +0900 Subject: [Python-ideas] for/else syntax In-Reply-To: <5d1a32000910030449q795c6765j940bda7d61d05279@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> <6f4025010910020558m5741673eka2d3cc284f219985@mail.gmail.com> <5d1a32000910020646i39e20e5fh9e15d40c22444528@mail.gmail.com> <3bdda690910021818u31eac2ebx1feda94dee99d784@mail.gmail.com> <5d1a32000910030449q795c6765j940bda7d61d05279@mail.gmail.com> Message-ID: <87y6nsh3z8.fsf@uwakimon.sk.tsukuba.ac.jp> Gerald Britton writes: > Sory Carl, I did NOT get it dead wrong. The else is executed if the > for loop falls through. > for i, j in enumerate(something): > # do something > i += 1 > else: > i = -1 Carl's point (revised to correspond to the new example) is assert(i == -1) so > if i > 0: > # we did something is dead code. From ubershmekel at gmail.com Sat Oct 3 17:52:24 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Sat, 3 Oct 2009 18:52:24 +0300 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC767F1.1080608@gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910030035.51041.steve@pearwood.info> <9d153b7c0910020842r283aaae5pa415df4eaa59288e@mail.gmail.com> <200910030223.46998.steve@pearwood.info> <9d153b7c0910022043k694f39f2r7afe8c1f26a37e7d@mail.gmail.com> <207E3417-4731-4BDA-BD83-CFBB6E96811C@masklinn.net> <9d153b7c0910030630w74a5b2a8mea9ab13a51614346@mail.gmail.com> <4AC767F1.1080608@gmail.com> Message-ID: <9d153b7c0910030852m7d5115b6i2f264ad61d8921e4@mail.gmail.com> Ok, in theory, it's possible to flag a boolean after a for/while when compiling, but I think the "special syntax" option might be simpler to implement. So let's talk grammar... The old grammar was: while_stmt ::= "while" expression ":" suite ["else" ":" suite] for_stmt ::= "for" target_list "in" expression_list ":" suite ["else" ":" suite] And the new one I propose looks like this: while_stmt ::= "while" expression ":" suite ["if" expression ":" suite] for_stmt ::= "for" target_list "in" expression_list ":" suite ["if" expression ":" suite] The "if" expression would be evaluated regularly but would have a magic boolean "break" that exists only in the if statement's expression eval. Note a few things: 1. the "if" would have been evaluated either way after the loop (exceptions/returns aside). 2. the only thing that's changed is the existence of the magic boolean "break". 3. You don't have to use it as the old "for...else" but you can and it's perfectly readable. I'm no expert on the compiler, but I have a strong feeling this is doable. On Sat, Oct 3, 2009 at 6:04 PM, Nick Coghlan wrote: > Yuvgoog Greenle wrote: >> I'd like to hear if people like this idea and if so, which of the 2 >> options do you like better. > > The compiler can't handle it - it will see the "if" as the start of a > new statement. > > Cheers, > Nick. > > -- > Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia > --------------------------------------------------------------- > From rrr at ronadam.com Sat Oct 3 18:09:38 2009 From: rrr at ronadam.com (Ron Adam) Date: Sat, 03 Oct 2009 11:09:38 -0500 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC76B0F.6030609@gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910021506.17321.steve@pearwood.info> <4AC5E122.9000902@gmail.com> <4AC76720.80603@ronadam.com> <4AC76B0F.6030609@gmail.com> Message-ID: <4AC77742.1070507@ronadam.com> Nick Coghlan wrote: > Ron Adam wrote: >> How about this? >> >> try: >> for x in xs: >> foo() >> esle: >> Print "foo didn't raise an exception." >> except: >> print "foo riased an exception" >> >> Using exceptions for flow control is very common in python. > > The else is adding no value here. That example would be better written as: > > try: > for x in xs: > foo() > print "foo didn't raise an exception." > except: > print "foo riased an exception" > >> Or this: >> >> for x in xs: >> y = foo() >> if y is True: >> return >> else: >> Print "y was never True" >> >> This is perfectly acceptable in my opinion. > > Again, using else here is redundant and misleading. The code is clearer > if it is left out entirely: > > for x in xs: > y = foo() > if y is True: > return > print "y was never True" > > The *only* time an else clause is a better alternative to just writing > the code after the loop is when there is a break statement present that > may skip over it. All good points. I need another cup of coffee. ;-) There may be some obscure exception to the break-else pattern but I can't think of any at the moment. Considering the following... >>> break File "", line 1 SyntaxError: 'break' outside loop Documenting for/break/else patterns seems very appropriate. +1 I suppose that it would have been clearer if it was for/then and while/then. With a break stopping the loop, and skipping over the then block. While it's much easer to explain that way, It adds a new keyword and isn't something we could or should change now I suppose. Cheers, Ron From rrr at ronadam.com Sat Oct 3 18:19:17 2009 From: rrr at ronadam.com (Ron Adam) Date: Sat, 03 Oct 2009 11:19:17 -0500 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910030839l3e9737aaw6ee19f0fabb35834@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DD80.6050803@gmail.com> <200910030035.51041.steve@pearwood.info> <4AC61DB7.9020305@gmail.com> <4AC6DC61.8040607@gmail.com> <4AC75F0A.6080504@ronadam.com> <4AC769F3.6060808@gmail.com> <4AC76D71.1080305@ronadam.com> <9d153b7c0910030839l3e9737aaw6ee19f0fabb35834@mail.gmail.com> Message-ID: <4AC77985.6000008@ronadam.com> Yuvgoog Greenle wrote: > Document it as much as you want but the "else" doesn't hint or clue as > to whether it catches breaks or is skipped by them. > > The next problem's gonna be at midnight when you're reading code and > asking yourself "hmmm, for, blah blah, else, hmm oh yeah, else-break, > so this else occurs whenever there's a break." True, and that is part of the other 20% I expect the documentation would't help. The only solution to that that I can think of is to change the esle's in for and while loops to for/then and while/then in Python 5.0. It might be doable, and if enough people want it, then just maybe... > On Sat, Oct 3, 2009 at 6:27 PM, Ron Adam wrote: >> >> Nick Coghlan wrote: >>> Ron Adam wrote: >>>> Nick Coghlan wrote: >>>>> Nick Coghlan wrote: >>>>> For a loop with no break statements the else clause will *always* be >>>>> executed, hence using the clause is completely redundant - the code it >>>>> contains might as well be dedented and placed inline at the same level >>>>> as the loop header. >>>> The loop could have a "return", "raise", or be inside a "try/except" >>>> instead of having a "break". >>> As I've said elsewhere in this thread, while the else clause won't be >>> executed in those cases, neither will any code that occurs immediately >>> after the loop statement. Accordingly, using the else clause is >>> unnecessary and rather misleading - just writing the extra code after >>> the loop would be much clearer. >>> >>> The number one thing I have learned from this thread is the fact that >>> the else clause only makes sense when used in conjunction with a break >>> statement and the fact that this construct isn't regularly documented >>> and explained as for/break/else and while/break/else is the major >>> barrier to understanding what it does. >>> >>> Cheers, >>> Nick. >> Ahh, Ok, I agree. Documenting them as for/break/else and while/break/else >> would probably reduce 80% of the confusion people have with it. >> >> Ron >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> > From ubershmekel at gmail.com Sat Oct 3 18:27:45 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Sat, 3 Oct 2009 19:27:45 +0300 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC77985.6000008@ronadam.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <4AC5DD80.6050803@gmail.com> <200910030035.51041.steve@pearwood.info> <4AC61DB7.9020305@gmail.com> <4AC6DC61.8040607@gmail.com> <4AC75F0A.6080504@ronadam.com> <4AC769F3.6060808@gmail.com> <4AC76D71.1080305@ronadam.com> <9d153b7c0910030839l3e9737aaw6ee19f0fabb35834@mail.gmail.com> <4AC77985.6000008@ronadam.com> Message-ID: <9d153b7c0910030927v82107f2ve6d767ef77f561b3@mail.gmail.com> On Sat, Oct 3, 2009 at 7:19 PM, Ron Adam wrote: > > True, and that is part of the other 20% I expect the documentation would't > help. > > The only solution to that that I can think of is to change the esle's in for > and while loops to for/then and while/then in Python 5.0. > > It might be doable, and if enough people want it, then just maybe... > Documenting this syntax with the appropriate warnings and clear explanations is very important. But I don't think the amount of code mishaps on this thread are properly addressed just by better documentation. The syntax I proposed is harmless, it won't break any existing code, and it's something people in python 3 can use right now. Deprecation etc can wait for python 4. Don't tea5e me :) Of course people here would have to say if my idea sounds good or not. while still_more_items(): if found_it(): break if not break: # exhausted search space From rrr at ronadam.com Sat Oct 3 18:51:25 2009 From: rrr at ronadam.com (Ron Adam) Date: Sat, 03 Oct 2009 11:51:25 -0500 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910030927v82107f2ve6d767ef77f561b3@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <4AC5DD80.6050803@gmail.com> <200910030035.51041.steve@pearwood.info> <4AC61DB7.9020305@gmail.com> <4AC6DC61.8040607@gmail.com> <4AC75F0A.6080504@ronadam.com> <4AC769F3.6060808@gmail.com> <4AC76D71.1080305@ronadam.com> <9d153b7c0910030839l3e9737aaw6ee19f0fabb35834@mail.gmail.com> <4AC77985.6000008@ronadam.com> <9d153b7c0910030927v82107f2ve6d767ef77f561b3@mail.gmail.com> Message-ID: <4AC7810D.3080603@ronadam.com> Yuvgoog Greenle wrote: > On Sat, Oct 3, 2009 at 7:19 PM, Ron Adam wrote: >> True, and that is part of the other 20% I expect the documentation would't >> help. >> >> The only solution to that that I can think of is to change the esle's in for >> and while loops to for/then and while/then in Python 5.0. >> >> It might be doable, and if enough people want it, then just maybe... >> > > Documenting this syntax with the appropriate warnings and clear > explanations is very important. > > But I don't think the amount of code mishaps on this thread are > properly addressed just by better documentation. The syntax I proposed > is harmless, it won't break any existing code, and it's something > people in python 3 can use right now. Deprecation etc can wait for > python 4. Don't tea5e me :) Lol, Sorry about that. ;-), but I do think it's too late to change for/else and while/else to for/then and while/then in python 4.0 already. > Of course people here would have to say if my idea sounds good or not. > > while still_more_items(): > if found_it(): > break > if not break: > # exhausted search space > I suppose most people won't like the way break is used in two entirely different ways depending on where it is. When you see a break, two things become really clear: 1: you are inside a loop 2: we are exiting a loop When you use it in the other context it makes those things a bit less obvious especially if you consider loops inside of loops. We then have to consider the closeness these too statements are to each other. if not condition: break if not break: condition we aren't exiting a loop in the second case, even though it may be inside another loop. Cheers, Ron From steve at pearwood.info Sat Oct 3 19:06:17 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 4 Oct 2009 03:06:17 +1000 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910030630w74a5b2a8mea9ab13a51614346@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <207E3417-4731-4BDA-BD83-CFBB6E96811C@masklinn.net> <9d153b7c0910030630w74a5b2a8mea9ab13a51614346@mail.gmail.com> Message-ID: <200910040406.18417.steve@pearwood.info> On Sat, 3 Oct 2009 11:30:55 pm Yuvgoog Greenle wrote: > if not break: [...] > Notice how readable and beautiful the code has become. No I don't. It looks like you're testing against a boolean flag called "break". > We have 2 implementation options: > 1. an "if" that comes immediately after a for/while can have the > special keywords "not" or "break". > 2. the for/while syntax has an optional "if break" or "if not break". Or #3: don't change a thing. > I'd like to hear if people like this idea and if so, which of the 2 > options do you like better. No. Neither of them. > Check it out, there's no need for a syntax error That's right. > because it's so obviously wrong: Your reasoning is backwards. If it were obviously wrong, there *would* be need to have a syntax error: >>> 4+*3 File "", line 1 4+*3 ^ SyntaxError: invalid syntax Syntax errors are for code which can't be compiled, not for code that does something unexpected, or unintuitive, or pointless. -- Steven D'Aprano From g.brandl at gmx.net Sat Oct 3 19:04:56 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 03 Oct 2009 19:04:56 +0200 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910030852m7d5115b6i2f264ad61d8921e4@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910030035.51041.steve@pearwood.info> <9d153b7c0910020842r283aaae5pa415df4eaa59288e@mail.gmail.com> <200910030223.46998.steve@pearwood.info> <9d153b7c0910022043k694f39f2r7afe8c1f26a37e7d@mail.gmail.com> <207E3417-4731-4BDA-BD83-CFBB6E96811C@masklinn.net> <9d153b7c0910030630w74a5b2a8mea9ab13a51614346@mail.gmail.com> <4AC767F1.1080608@gmail.com> <9d153b7c0910030852m7d5115b6i2f264ad61d8921e4@mail.gmail.com> Message-ID: Yuvgoog Greenle schrieb: > Ok, in theory, it's possible to flag a boolean after a for/while when > compiling, but I think the "special syntax" option might be simpler to > implement. So let's talk grammar... > > The old grammar was: > > while_stmt ::= "while" expression ":" suite > ["else" ":" suite] > > for_stmt ::= "for" target_list "in" expression_list ":" suite > ["else" ":" suite] > > > And the new one I propose looks like this: > > while_stmt ::= "while" expression ":" suite > ["if" expression ":" suite] > > for_stmt ::= "for" target_list "in" expression_list ":" suite > ["if" expression ":" suite] > > > The "if" expression would be evaluated regularly but would have a > magic boolean "break" that exists only in the if statement's > expression eval. Note a few things: > 1. the "if" would have been evaluated either way after the loop > (exceptions/returns aside). > 2. the only thing that's changed is the existence of the magic boolean "break". > 3. You don't have to use it as the old "for...else" but you can and > it's perfectly readable. > > I'm no expert on the compiler, but I have a strong feeling this is doable. Sure, almost any language change is doable if you bend over backwards far enough and break all rules of consistency. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Sat Oct 3 19:07:13 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 03 Oct 2009 19:07:13 +0200 Subject: [Python-ideas] for/else syntax In-Reply-To: <5d1a32000910030832qaf1410dlc146d664c7a65810@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> <9d153b7c0910020545h45882a37j9b4b0a1c305ef1de@mail.gmail.com> <6f4025010910020549r50959b24m6eef1b9a4e15bc11@mail.gmail.com> <9d153b7c0910020554u90d0696g40635f987ec3717c@mail.gmail.com> <6f4025010910020558m5741673eka2d3cc284f219985@mail.gmail.com> <5d1a32000910020646i39e20e5fh9e15d40c22444528@mail.gmail.com> <3bdda690910021818u31eac2ebx1feda94dee99d784@mail.gmail.com> <5d1a32000910030449q795c6765j940bda7d61d05279@mail.gmail.com> <4AC76C31.5090700@gmail.com> <5d1a32000910030832qaf1410dlc146d664c7a65810@mail.gmail.com> Message-ID: Gerald Britton schrieb: > Yes, quite right. Another way would be: > > for i, j in enumerate(x): > # do something > i += 1 > else: > if "i" not in vars(): > i = 0 And there is STILL no need for using an else clause here. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From python at mrabarnett.plus.com Sat Oct 3 19:11:21 2009 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 03 Oct 2009 18:11:21 +0100 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910030852m7d5115b6i2f264ad61d8921e4@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910030035.51041.steve@pearwood.info> <9d153b7c0910020842r283aaae5pa415df4eaa59288e@mail.gmail.com> <200910030223.46998.steve@pearwood.info> <9d153b7c0910022043k694f39f2r7afe8c1f26a37e7d@mail.gmail.com> <207E3417-4731-4BDA-BD83-CFBB6E96811C@masklinn.net> <9d153b7c0910030630w74a5b2a8mea9ab13a51614346@mail.gmail.com> <4AC767F1.1080608@gmail.com> <9d153b7c0910030852m7d5115b6i2f264ad61d8921e4@mail.gmail.com> Message-ID: <4AC785B9.2050609@mrabarnett.plus.com> Yuvgoog Greenle wrote: > Ok, in theory, it's possible to flag a boolean after a for/while when > compiling, but I think the "special syntax" option might be simpler to > implement. So let's talk grammar... > > The old grammar was: > > while_stmt ::= "while" expression ":" suite > ["else" ":" suite] > > for_stmt ::= "for" target_list "in" expression_list ":" suite > ["else" ":" suite] > > > And the new one I propose looks like this: > > while_stmt ::= "while" expression ":" suite > ["if" expression ":" suite] > > for_stmt ::= "for" target_list "in" expression_list ":" suite > ["if" expression ":" suite] > > > The "if" expression would be evaluated regularly but would have a > magic boolean "break" that exists only in the if statement's > expression eval. Note a few things: > 1. the "if" would have been evaluated either way after the loop > (exceptions/returns aside). > 2. the only thing that's changed is the existence of the magic boolean "break". > 3. You don't have to use it as the old "for...else" but you can and > it's perfectly readable. > > I'm no expert on the compiler, but I have a strong feeling this is doable. > It means having to several tokens ahead before being able to tell whether you're continuing the current structure or starting another. Perhaps 'elif' instead? while still_more_items(): if found_it(): break elif not break: # exhausted search space If the body of loop wasn't executed at all: while still_more_items(): if found_it(): break elif pass: # empty search space > > On Sat, Oct 3, 2009 at 6:04 PM, Nick Coghlan wrote: >> Yuvgoog Greenle wrote: >>> I'd like to hear if people like this idea and if so, which of the 2 >>> options do you like better. >> The compiler can't handle it - it will see the "if" as the start of a >> new statement. >> From masklinn at masklinn.net Sat Oct 3 19:13:49 2009 From: masklinn at masklinn.net (Masklinn) Date: Sat, 3 Oct 2009 19:13:49 +0200 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC77985.6000008@ronadam.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DD80.6050803@gmail.com> <200910030035.51041.steve@pearwood.info> <4AC61DB7.9020305@gmail.com> <4AC6DC61.8040607@gmail.com> <4AC75F0A.6080504@ronadam.com> <4AC769F3.6060808@gmail.com> <4AC76D71.1080305@ronadam.com> <9d153b7c0910030839l3e9737aaw6ee19f0fabb35834@mail.gmail.com> <4AC77985.6000008@ronadam.com> Message-ID: <0B98F67F-B891-4F38-A32C-F563E28FF2F2@masklinn.net> On 3 Oct 2009, at 18:19 , Ron Adam wrote: > Yuvgoog Greenle wrote: >> Document it as much as you want but the "else" doesn't hint or clue >> as >> to whether it catches breaks or is skipped by them. >> The next problem's gonna be at midnight when you're reading code and >> asking yourself "hmmm, for, blah blah, else, hmm oh yeah, else-break, >> so this else occurs whenever there's a break." > True, and that is part of the other 20% I expect the documentation > would't help. > > The only solution to that that I can think of is to change the > esle's in for and while loops to for/then and while/then in Python > 5.0. That would yield for val in vals: if cond(val): break # processing then: # more stuff how does it make the "alternate" clause being skipped in case of break any clearer? From gerald.britton at gmail.com Sat Oct 3 20:35:58 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Sat, 3 Oct 2009 14:35:58 -0400 Subject: [Python-ideas] for/else syntax In-Reply-To: <0B98F67F-B891-4F38-A32C-F563E28FF2F2@masklinn.net> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910030035.51041.steve@pearwood.info> <4AC61DB7.9020305@gmail.com> <4AC6DC61.8040607@gmail.com> <4AC75F0A.6080504@ronadam.com> <4AC769F3.6060808@gmail.com> <4AC76D71.1080305@ronadam.com> <9d153b7c0910030839l3e9737aaw6ee19f0fabb35834@mail.gmail.com> <4AC77985.6000008@ronadam.com> <0B98F67F-B891-4F38-A32C-F563E28FF2F2@masklinn.net> Message-ID: <5d1a32000910031135w2a683adei5eafad1553a937c1@mail.gmail.com> Looking over this all again, it seems to me that the main request would be for a more explicit statement in the documentation. Something like: "When the items are exhausted (which is immediately when the sequence is empty), the suite in the else clause, if present, is executed, and the loop terminates. Note: The suite in the else clause will be executed unless the iteration is terminated with a break or return, or raises an exception that is not handled or does not return." Probably it could use some word-smithing, but is that the basic idea? One thing I DO like about the construct, though, is it makes it more apparent what code belongs together. e.g. for x in y: # suite 1 else: # suite 2 It should be obvious that suite 2 belongs with the for statement in some way. OTOH: for x in y: # suite 1 # suite 2 You can't immediately see that suite 2 is related to the for statement. You might think it is completely independent which could lead to erroneous results if you modified it without keeping the for-loop context in mind. On Sat, Oct 3, 2009 at 1:13 PM, Masklinn wrote: > On 3 Oct 2009, at 18:19 , Ron Adam wrote: >> >> Yuvgoog Greenle wrote: >>> >>> Document it as much as you want but the "else" doesn't hint or clue as >>> to whether it catches breaks or is skipped by them. >>> The next problem's gonna be at midnight when you're reading code and >>> asking yourself "hmmm, for, blah blah, else, hmm oh yeah, else-break, >>> so this else occurs whenever there's a break." >> >> True, and that is part of the other 20% I expect the documentation would't >> help. >> >> The only solution to that that I can think of is to change the esle's in >> for and while loops to for/then and while/then in Python 5.0. > > That would yield > > ? ?for val in vals: > ? ? ? ?if cond(val): > ? ? ? ? ? ?break > ? ? ? ?# processing > ? ?then: > ? ? ? ?# more stuff > > how does it make the "alternate" clause being skipped in case of break any > clearer? > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From rrr at ronadam.com Sat Oct 3 20:50:19 2009 From: rrr at ronadam.com (Ron Adam) Date: Sat, 03 Oct 2009 13:50:19 -0500 Subject: [Python-ideas] for/else syntax In-Reply-To: <0B98F67F-B891-4F38-A32C-F563E28FF2F2@masklinn.net> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DD80.6050803@gmail.com> <200910030035.51041.steve@pearwood.info> <4AC61DB7.9020305@gmail.com> <4AC6DC61.8040607@gmail.com> <4AC75F0A.6080504@ronadam.com> <4AC769F3.6060808@gmail.com> <4AC76D71.1080305@ronadam.com> <9d153b7c0910030839l3e9737aaw6ee19f0fabb35834@mail.gmail.com> <4AC77985.6000008@ronadam.com> <0B98F67F-B891-4F38-A32C-F563E28FF2F2@masklinn.net> Message-ID: <4AC79CEB.5070406@ronadam.com> Masklinn wrote: > On 3 Oct 2009, at 18:19 , Ron Adam wrote: >> Yuvgoog Greenle wrote: >>> Document it as much as you want but the "else" doesn't hint or clue as >>> to whether it catches breaks or is skipped by them. >>> The next problem's gonna be at midnight when you're reading code and >>> asking yourself "hmmm, for, blah blah, else, hmm oh yeah, else-break, >>> so this else occurs whenever there's a break." >> True, and that is part of the other 20% I expect the documentation >> would't help. >> >> The only solution to that that I can think of is to change the esle's >> in for and while loops to for/then and while/then in Python 5.0. > > That would yield > > for val in vals: > if cond(val): > break > # processing > then: > # more stuff > > how does it make the "alternate" clause being skipped in case of break > any clearer? I didn't say the behavior would change. for val in vals: if cond(val): break # processing val then: #all values processed # do more stuff The break would brake out of the entire for/then block. for val in vals: if found(val): searchval = val break then: # if not found searchval = default # breaks jumps to here I think that reads very well. :-) On the other hand "else" with the same comments really isn't all that much worse once you understand how break works in for/while loops. It's just a bit more of a hurdle for people new to python to get because of it's counter intuitive behavior compared else's use in if/else blocks. In those other languages that use case blocks, break is used to escape out of the entire case block in the same way break escapes out of the entire for/else blocks in python. Cheers, Ron From ubershmekel at gmail.com Sat Oct 3 19:05:53 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Sat, 3 Oct 2009 20:05:53 +0300 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC7810D.3080603@ronadam.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <4AC61DB7.9020305@gmail.com> <4AC6DC61.8040607@gmail.com> <4AC75F0A.6080504@ronadam.com> <4AC769F3.6060808@gmail.com> <4AC76D71.1080305@ronadam.com> <9d153b7c0910030839l3e9737aaw6ee19f0fabb35834@mail.gmail.com> <4AC77985.6000008@ronadam.com> <9d153b7c0910030927v82107f2ve6d767ef77f561b3@mail.gmail.com> <4AC7810D.3080603@ronadam.com> Message-ID: <9d153b7c0910031005t72786f97pb161a4f39d0f87ef@mail.gmail.com> You do have a point and I'll meditate on it. But I do think the situation is better than you made it appear. There will usually be a linebreak. We can't help the readability of code written by people that compress code. So a more fair representation would be: if not condition: break if not break: condition (note: I'm not sure what you meant by "if not break: condition") The bad case is when you have nested loops so it would look like this... for item in list_of_items: while condition: if reason_to_stop(): break if break: # do something after the while broke if other_condition: break On Sat, Oct 3, 2009 at 7:51 PM, Ron Adam wrote: > > > Yuvgoog Greenle wrote: >> >> On Sat, Oct 3, 2009 at 7:19 PM, Ron Adam wrote: >>> >>> True, and that is part of the other 20% I expect the documentation >>> would't >>> help. >>> >>> The only solution to that that I can think of is to change the esle's in >>> for >>> and while loops to for/then and while/then in Python 5.0. >>> >>> It might be doable, and if enough people want it, then just maybe... >>> >> >> Documenting this syntax with the appropriate warnings and clear >> explanations is very important. >> >> But I don't think the amount of code mishaps on this thread are >> properly addressed just by better documentation. The syntax I proposed >> is harmless, it won't break any existing code, and it's something >> people in python 3 can use right now. Deprecation etc can wait for >> python 4. Don't tea5e me :) > > Lol, Sorry about that. ;-), ?but I do think it's too late to change for/else > and while/else to for/then and while/then in python 4.0 already. > > >> Of course people here would have to say if my idea sounds good or not. >> >> while still_more_items(): >> ? ?if found_it(): >> ? ? ? ?break >> if not break: >> ? ?# exhausted search space >> > > I suppose most people won't like the way break is used in two entirely > different ways depending on where it is. > > When you see a break, two things become really clear: > > ? 1: you are inside a loop > > ? 2: we are exiting a loop > > > When you use it in the other context it makes those things a bit less > obvious especially if you consider loops inside of loops. ?We then have to > consider the closeness these too statements are to each other. > > ? if not condition: break > > ? if not break: condition > > we aren't exiting a loop in the second case, even though it may be inside > another loop. > > > Cheers, > ? Ron > > > > > > > > > > > > > > > > > > > > > > > > > From m.lenzen at gmail.com Sun Oct 4 02:40:52 2009 From: m.lenzen at gmail.com (Michael Lenzen) Date: Sat, 3 Oct 2009 19:40:52 -0500 Subject: [Python-ideas] collections_extended - bags/multisets and setlists Message-ID: <420d28400910031740vbb876b6u3a76e64feca1c45a@mail.gmail.com> A few months ago I proposed extending collections to include bags/multisets and setlists (ordered sets/unique lists). If anyone is still interested I have created a PyPI module here: http://pypi.python.org/pypi/data-structures If you want to join the project to file a bug or add something, the project homepage is here: http://code.google.com/p/python-data-structures/ This is the first module I've created so if you could point me to the right list to "promote" it on or point out errors or best practices it would be much appreciated. I know this isn't really the place to be posting about *actual code* but I thought I'd follow up. Thanks, -LenzM From stephen at xemacs.org Sun Oct 4 15:08:18 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 04 Oct 2009 22:08:18 +0900 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC77742.1070507@ronadam.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910021506.17321.steve@pearwood.info> <4AC5E122.9000902@gmail.com> <4AC76720.80603@ronadam.com> <4AC76B0F.6030609@gmail.com> <4AC77742.1070507@ronadam.com> Message-ID: <87ws3bgvct.fsf@uwakimon.sk.tsukuba.ac.jp> Ron Adam writes: > All good points. I need another cup of coffee. ;-) > > There may be some obscure exception to the break-else pattern but I can't > think of any at the moment. May I suggest reading my posts. I've made at least two attempts at obscure exceptions and been corrected within minutes every time. At least you can avoid those! :-) From rrr at ronadam.com Mon Oct 5 01:54:27 2009 From: rrr at ronadam.com (Ron Adam) Date: Sun, 04 Oct 2009 18:54:27 -0500 Subject: [Python-ideas] for/else syntax In-Reply-To: <87ws3bgvct.fsf@uwakimon.sk.tsukuba.ac.jp> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <200910021506.17321.steve@pearwood.info> <4AC5E122.9000902@gmail.com> <4AC76720.80603@ronadam.com> <4AC76B0F.6030609@gmail.com> <4AC77742.1070507@ronadam.com> <87ws3bgvct.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4AC935B3.1020800@ronadam.com> Stephen J. Turnbull wrote: > Ron Adam writes: > > > All good points. I need another cup of coffee. ;-) > > > > There may be some obscure exception to the break-else pattern but I can't > > think of any at the moment. > > May I suggest reading my posts. I've made at least two attempts at > obscure exceptions and been corrected within minutes every time. At > least you can avoid those! :-) I'm not even going to try. :-) I looked at the grammer for each of the items in question and am wondering if that could be improved in any way so that they indicate how break and continue are used in loops. Here's the current grammer for Python 3.0 loops. Break and Continue are next to each other but no where near for and while in the documentation. while_stmt ::= "while" expression ":" suite ["else" ":" suite] for_stmt ::= "for" target_list "in" expression_list ":" suite ["else" ":" suite] break_stmt ::= "break" continue_stmt ::= "continue" Cheers, Ron From greg.ewing at canterbury.ac.nz Sat Oct 3 03:33:29 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 03 Oct 2009 13:33:29 +1200 Subject: [Python-ideas] for/else syntax In-Reply-To: <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <20091001150443.GA21693@phd.pp.ru> <873a63jd7d.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <87ske2ibyp.fsf@uwakimon.sk.tsukuba.ac.jp> <6f4025010910020425s3a2364ecye8c1d23817d68641@mail.gmail.com> Message-ID: <4AC6A9E9.3050209@canterbury.ac.nz> Michael Foord wrote: > Nonsense - there are several other ways to break out of a loop. Raising > an exception or returning for example. But both of those bypass any code following the for-loop, so there is no need for an else-clause in those cases. -- Greg From greg.ewing at canterbury.ac.nz Sat Oct 3 03:40:18 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 03 Oct 2009 13:40:18 +1200 Subject: [Python-ideas] Decorator syntax restriction In-Reply-To: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> References: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> Message-ID: <4AC6AB82.40002@canterbury.ac.nz> Paul Moore wrote: > @(a['b'].fns[1])(1,2,{3,4}) > def something(): > > I contend that's clearly "line noise". But there's nothing to stop you from writing f = (a['b'].fns[1])(1,2,{3,4}) @f def something(): If the first version is unreadable, the second one isn't going to be significantly better. -- Greg From greg.ewing at canterbury.ac.nz Sat Oct 3 03:51:13 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 03 Oct 2009 13:51:13 +1200 Subject: [Python-ideas] for/else syntax In-Reply-To: <200910030035.51041.steve@pearwood.info> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DD80.6050803@gmail.com> <200910030035.51041.steve@pearwood.info> Message-ID: <4AC6AE11.4090602@canterbury.ac.nz> Steven D'Aprano wrote: > It's even more problematic when you consider code where the break is > inside a "if __debug__" clause. Then you've code which raises a warning > when you run with the -O flag, and doesn't when you run without. Not necessarily -- the decision to issue the warning could be made at an earlier stage of processing than the one that optimises away the if. -- Greg From greg.ewing at canterbury.ac.nz Sat Oct 3 04:07:04 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 03 Oct 2009 14:07:04 +1200 Subject: [Python-ideas] for/else syntax In-Reply-To: <200910030043.38556.steve@pearwood.info> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <9d153b7c0910012011h70fead0cp17ccb22b2272d2f7@mail.gmail.com> <4AC5DEA3.3020701@gmail.com> <200910030043.38556.steve@pearwood.info> Message-ID: <4AC6B1C8.5090400@canterbury.ac.nz> Steven D'Aprano wrote: > try: > if cond: > raise ValueError > else: > print "no exception was raised" > finally: > print "done" According to the Language Reference, the finally clause protects all of the preceding clauses in the try statement, including the else clause. So this would be equivalent to try: if cond: raise ValueError print "no exception was raised" finally: print "done" -- Greg From greg.ewing at canterbury.ac.nz Fri Oct 2 03:35:54 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 02 Oct 2009 14:35:54 +1300 Subject: [Python-ideas] for/else syntax In-Reply-To: <9d153b7c0910010512h4df68169w8f06085b2a4809fa@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <9d153b7c0910010512h4df68169w8f06085b2a4809fa@mail.gmail.com> Message-ID: <4AC558FA.7010103@canterbury.ac.nz> Yuvgoog Greenle wrote: > This > construct should have been named "finally", "didnt_break" or "not break": "finally" would be at least as confusing as "else", since by analogy with try-finally it suggests that it will *always* be executed, which isn't the case. -- Greg From bruce at leapyear.org Wed Oct 7 05:46:58 2009 From: bruce at leapyear.org (Bruce Leban) Date: Tue, 6 Oct 2009 20:46:58 -0700 Subject: [Python-ideas] for/except/else syntax Message-ID: Switching gears a bit, here's what I want: *for_stmt* ::= "for" target_list "in" expression_list ":" suite_loop_body["except" ":" suite_if_loop_body_is_not_executed] ["else" ":" suite_if_loop_is_not_broken] The except and else appear in this order to make it clear that except takes precedence. If the loop body is not executed at all then both conditions are true but it obviously is only useful if except takes precedence. Comments? --- Bruce -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Wed Oct 7 05:53:49 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 6 Oct 2009 20:53:49 -0700 Subject: [Python-ideas] for/except/else syntax In-Reply-To: References: Message-ID: On Tue, Oct 6, 2009 at 8:46 PM, Bruce Leban wrote: > Switching gears a bit, here's what I want: > > for_stmt ::= "for" target_list "in" expression_list ":" suite_loop_body > ["except" ":" suite_if_loop_body_is_not_executed] ["else" ":" > suite_if_loop_is_not_broken] > > The except and else appear in this order to make it clear that except takes > precedence. If the loop body is not executed at all then both conditions are > true but it obviously is only useful if except takes precedence. > Comments? I'm sure it's obvious to you what the except clause should do, but it isn't to me. :-( Assuming you're not proposing we change the meaning of else in the absence of except, and that its meaning together with except is a logical extension of this, can you show how we should translate a for-loop with an except clause into current Python? Say, what would I be writing today to get the exact effect of your proposed for i in SEQ: A except: B else: C ? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From bruce at leapyear.org Wed Oct 7 06:06:59 2009 From: bruce at leapyear.org (Bruce Leban) Date: Tue, 6 Oct 2009 21:06:59 -0700 Subject: [Python-ideas] for/except/else syntax In-Reply-To: References: Message-ID: Here you go: do_except = True for i in SEQ: do_except = False A else: if not do_except: C # suite_if_loop_is_not_broken if do_except: B # suite_if_loop_body_is_not_executed --- Bruce On Tue, Oct 6, 2009 at 8:53 PM, Guido van Rossum wrote: > On Tue, Oct 6, 2009 at 8:46 PM, Bruce Leban wrote: > > Switching gears a bit, here's what I want: > > > > for_stmt ::= "for" target_list "in" expression_list ":" suite_loop_body > > ["except" ":" suite_if_loop_body_is_not_executed] ["else" ":" > > suite_if_loop_is_not_broken] > > > > The except and else appear in this order to make it clear that except > takes > > precedence. If the loop body is not executed at all then both conditions > are > > true but it obviously is only useful if except takes precedence. > > Comments? > > I'm sure it's obvious to you what the except clause should do, but it > isn't to me. :-( Assuming you're not proposing we change the meaning > of else in the absence of except, and that its meaning together with > except is a logical extension of this, can you show how we should > translate a for-loop with an except clause into current Python? Say, > what would I be writing today to get the exact effect of your proposed > > for i in SEQ: > A > except: > B > else: > C > > ? > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ubershmekel at gmail.com Wed Oct 7 09:00:05 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Wed, 7 Oct 2009 09:00:05 +0200 Subject: [Python-ideas] for/except/else syntax In-Reply-To: References: Message-ID: <9d153b7c0910070000q6c1f8c11g83cb0d2ec01eba87@mail.gmail.com> I don't like your "except" and "else" only because it's not obvious when they execute. btw your example breaks the current "for..else" search idiom. C, suite_if_loop_is_not_broken, should be labelled suite_if_loop_is_not_broken_and_not_empty in your example code. As I understand it, pythoneers need a way to know what happened with a previous loop. Did it break? Did it execute at all? The answer to both questions is either True or False and developers need these answers only after the loop finished. If python wasn't in over it's head with reserved words I would suggest adding "loop": for i in SEQ: A if not loop.broke: C # suite_if_loop_is_not_broken elif loop.empty: B # suite_if_loop_body_is_not_executed For optimization I would suggest the "loop" variable only be instantiated if it's referenced. Also, there may be better names for "loop", "broke" and "empty". Two drawbacks: 1. python's in over it's head with reserved words. 2. developers might want to use the "loop" variable a mile down from the loop which would be ugly but their fault. I'm writing a summary of the previous for/else thread and hope that by the time I finish that (a week+), someone will invent a readable, low-cost syntax (optional but awesome if it can handle all 4: break/not break/empty/not empty). On Wed, Oct 7, 2009 at 6:06 AM, Bruce Leban wrote: > Here you go: > do_except = True > for i in SEQ: > ??do_except = False > ??A > else: > ??if not do_except: > ?? ?C ?#?suite_if_loop_is_not_broken > if do_except: > ??B ?#?suite_if_loop_body_is_not_executed > --- Bruce > > > On Tue, Oct 6, 2009 at 8:53 PM, Guido van Rossum wrote: >> >> On Tue, Oct 6, 2009 at 8:46 PM, Bruce Leban wrote: >> > Switching gears a bit, here's what I want: >> > >> > for_stmt ::= "for" target_list "in" expression_list ":" suite_loop_body >> > ["except" ":" suite_if_loop_body_is_not_executed] ["else" ":" >> > suite_if_loop_is_not_broken] >> > >> > The except and else appear in this order to make it clear that except >> > takes >> > precedence. If the loop body is not executed at all then both conditions >> > are >> > true but it obviously is only useful if except takes precedence. >> > Comments? >> >> I'm sure it's obvious to you what the except clause should do, but it >> isn't to me. :-( Assuming you're not proposing we change the meaning >> of else in the absence of except, and that its meaning together with >> except is a logical extension of this, can you show how we should >> translate a for-loop with an except clause into current Python? Say, >> what would I be writing today to get the exact effect of your proposed >> >> for i in SEQ: >> ?A >> except: >> ?B >> else: >> ?C >> >> ? >> >> -- >> --Guido van Rossum (home page: http://www.python.org/~guido/) > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > From arnodel at googlemail.com Wed Oct 7 11:15:48 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Wed, 7 Oct 2009 10:15:48 +0100 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <9d153b7c0910070000q6c1f8c11g83cb0d2ec01eba87@mail.gmail.com> References: <9d153b7c0910070000q6c1f8c11g83cb0d2ec01eba87@mail.gmail.com> Message-ID: <9bfc700a0910070215h17bea49cx89898b35f0615a11@mail.gmail.com> 2009/10/7 Yuvgoog Greenle : > I don't like your "except" and "else" only because it's not obvious > when they execute. > > btw your example breaks the current "for..else" search idiom. C, > suite_if_loop_is_not_broken, should be labelled > suite_if_loop_is_not_broken_and_not_empty in your example code. > > As I understand it, pythoneers need a way to know what happened with a > previous loop. Did it break? Did it execute at all? The answer to both > questions is either True or False and developers need these answers > only after the loop finished. If python wasn't in over it's head with > reserved words I would suggest adding "loop": > > for i in SEQ: > ? ?A > if not loop.broke: > ? ?C # suite_if_loop_is_not_broken > elif loop.empty: > ? ?B ?# suite_if_loop_body_is_not_executed > > For optimization I would suggest the "loop" variable only be > instantiated if it's referenced. Also, there may be better names for > "loop", "broke" and "empty". > > Two drawbacks: > 1. python's in over it's head with reserved words. > 2. developers might want to use the "loop" variable a mile down from > the loop which would be ugly but their fault. > > I'm writing a summary of the previous for/else thread and hope that by > the time I finish that (a week+), someone will invent a readable, > low-cost syntax (optional but awesome if it can handle all 4: > break/not break/empty/not empty). Here's an idea with current syntax (python < 3): class Loop(object): def __init__(self, iterable): self.iterable = iterable def __iter__(self): self.broken = True self.empty = True for i in self.iterable: self.empty = False yield i self.broken = False # Example: for string in 'spam', 'bot', '': loop = Loop(string) for i in loop: if i == 'a': break print repr(string), ':' if loop.broken: print 'broken', if loop.empty: print 'empty', print # Output: 'spam' : broken 'bot' : '' : empty -- Arnaud From rob.cliffe at btinternet.com Wed Oct 7 11:24:56 2009 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Wed, 7 Oct 2009 10:24:56 +0100 Subject: [Python-ideas] for/except/else syntax References: <9d153b7c0910070000q6c1f8c11g83cb0d2ec01eba87@mail.gmail.com> <9bfc700a0910070215h17bea49cx89898b35f0615a11@mail.gmail.com> Message-ID: I'm coming to this debate rather late but ..... I did find the for ... else syntax confusing when I learned Python, and I still do a little. I would favour keeping for-loops exactly the same except allowing a more intelligible keyword instead of 'else', e.g. 'ifnobreak'. Then moving towards deprecating and ultimately forbidding the use of 'else'. Rob Cliffe ----- Original Message ----- From: "Arnaud Delobelle" To: "Yuvgoog Greenle" Cc: Sent: Wednesday, October 07, 2009 10:15 AM Subject: Re: [Python-ideas] for/except/else syntax 2009/10/7 Yuvgoog Greenle : > I don't like your "except" and "else" only because it's not obvious > when they execute. > > btw your example breaks the current "for..else" search idiom. C, > suite_if_loop_is_not_broken, should be labelled > suite_if_loop_is_not_broken_and_not_empty in your example code. > > As I understand it, pythoneers need a way to know what happened with a > previous loop. Did it break? Did it execute at all? The answer to both > questions is either True or False and developers need these answers > only after the loop finished. If python wasn't in over it's head with > reserved words I would suggest adding "loop": > > for i in SEQ: > A > if not loop.broke: > C # suite_if_loop_is_not_broken > elif loop.empty: > B # suite_if_loop_body_is_not_executed > > For optimization I would suggest the "loop" variable only be > instantiated if it's referenced. Also, there may be better names for > "loop", "broke" and "empty". > > Two drawbacks: > 1. python's in over it's head with reserved words. > 2. developers might want to use the "loop" variable a mile down from > the loop which would be ugly but their fault. > > I'm writing a summary of the previous for/else thread and hope that by > the time I finish that (a week+), someone will invent a readable, > low-cost syntax (optional but awesome if it can handle all 4: > break/not break/empty/not empty). Here's an idea with current syntax (python < 3): class Loop(object): def __init__(self, iterable): self.iterable = iterable def __iter__(self): self.broken = True self.empty = True for i in self.iterable: self.empty = False yield i self.broken = False # Example: for string in 'spam', 'bot', '': loop = Loop(string) for i in loop: if i == 'a': break print repr(string), ':' if loop.broken: print 'broken', if loop.empty: print 'empty', print # Output: 'spam' : broken 'bot' : '' : empty -- Arnaud _______________________________________________ Python-ideas mailing list Python-ideas at python.org http://mail.python.org/mailman/listinfo/python-ideas From rob.cliffe at btinternet.com Wed Oct 7 11:31:26 2009 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Wed, 7 Oct 2009 10:31:26 +0100 Subject: [Python-ideas] Decorator syntax restriction References: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> <4AC6AB82.40002@canterbury.ac.nz> Message-ID: <8829BBB7E9584E00A3EB0E92218A4BBB@robslaptop> Well, if the syntax was loosened to allow a subscript, i.e. @a.b.c[x]( ...args... ) that would be something. And allowing a dictionary ley as well - with the same syntax - would be even better; it seems hard to justify allowing one without the other. On the other hand, maybe in future someone will come up with another 'must-have' loosening of the syntax. Eventually, don't you get to the point where it's simpler to allow anything? Best wishes Rob Cliffe ----- Original Message ----- From: "Greg Ewing" To: Sent: Saturday, October 03, 2009 2:40 AM Subject: Re: [Python-ideas] Decorator syntax restriction > Paul Moore wrote: > >> @(a['b'].fns[1])(1,2,{3,4}) >> def something(): >> >> I contend that's clearly "line noise". > > But there's nothing to stop you from writing > > f = (a['b'].fns[1])(1,2,{3,4}) > @f > def something(): > > If the first version is unreadable, the second one isn't going > to be significantly better. > > -- > Greg > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas From p.f.moore at gmail.com Wed Oct 7 10:19:47 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 7 Oct 2009 09:19:47 +0100 Subject: [Python-ideas] Decorator syntax restriction In-Reply-To: <4AC6AB82.40002@canterbury.ac.nz> References: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> <4AC6AB82.40002@canterbury.ac.nz> Message-ID: <79990c6b0910070119ge96f77aj4013d181b77e01b8@mail.gmail.com> 2009/10/3 Greg Ewing : > Paul Moore wrote: > >> @(a['b'].fns[1])(1,2,{3,4}) >> def something(): >> >> I contend that's clearly "line noise". > > But there's nothing to stop you from writing > > f = (a['b'].fns[1])(1,2,{3,4}) > @f > def something(): > > If the first version is unreadable, the second one isn't going > to be significantly better. Absolutely. Unless you give f a meaningful name... I'm not trying to say that you can't write unreadable line noise in Python (look at any of my code for examples :-)) just that the design principle is to avoid actively encouraging it. There's no doubt that this is a grey area. And I'm not against allowing subscripts in decorators (I've no personal need for it but I see how it might help, so +0) so it may be that I already support allowing the monstrosity above :-) (I refuse to go and check if it violates any other syntax rules, as my point is basically that if I need to check, I don't think it should be used, whether or not it's allowed). Paul From gerald.britton at gmail.com Wed Oct 7 15:10:00 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Wed, 7 Oct 2009 09:10:00 -0400 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC558FA.7010103@canterbury.ac.nz> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <9d153b7c0910010512h4df68169w8f06085b2a4809fa@mail.gmail.com> <4AC558FA.7010103@canterbury.ac.nz> Message-ID: <5d1a32000910070610h75ff5bd4t76c0e6ddc0427b40@mail.gmail.com> Are you sure? Under what conditions would the "finally" clause be bypassed? On Thu, Oct 1, 2009 at 9:35 PM, Greg Ewing wrote: > Yuvgoog Greenle wrote: >> >> This >> construct should have been named "finally", "didnt_break" or "not break": > > "finally" would be at least as confusing as "else", > since by analogy with try-finally it suggests that > it will *always* be executed, which isn't the case. > > -- > Greg > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From steve at pearwood.info Wed Oct 7 16:09:26 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 8 Oct 2009 01:09:26 +1100 Subject: [Python-ideas] for/else syntax In-Reply-To: <5d1a32000910070610h75ff5bd4t76c0e6ddc0427b40@mail.gmail.com> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <4AC558FA.7010103@canterbury.ac.nz> <5d1a32000910070610h75ff5bd4t76c0e6ddc0427b40@mail.gmail.com> Message-ID: <200910080109.27464.steve@pearwood.info> On Thu, 8 Oct 2009 12:10:00 am Gerald Britton wrote: > Are you sure? Under what conditions would the "finally" clause be > bypassed? Are you talking about the existing try...finally clause, or the hypothetical "let's rename for...else to for...finally" clause? If the second, then it would be bypassed under exactly the same conditions as the else in for...else is bypassed. E.g.: for x in [1, 2, 3]: raise Exception else: print "This is never printed" def test(): for x in [1, 2, 3]: return x else: print "This is never printed" Renaming 'for...else' to 'for...finally' doesn't turn it into a try block. The semantics remain the same. -- Steven D'Aprano From masklinn at masklinn.net Wed Oct 7 16:18:02 2009 From: masklinn at masklinn.net (Masklinn) Date: Wed, 7 Oct 2009 16:18:02 +0200 Subject: [Python-ideas] for/else syntax In-Reply-To: <4AC558FA.7010103@canterbury.ac.nz> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <9d153b7c0910010512h4df68169w8f06085b2a4809fa@mail.gmail.com> <4AC558FA.7010103@canterbury.ac.nz> Message-ID: <4147EC63-BA05-4705-8005-B6895A2860E1@masklinn.net> On 2 Oct 2009, at 03:35 , Greg Ewing wrote: > Yuvgoog Greenle wrote: >> This >> construct should have been named "finally", "didnt_break" or "not >> break": > > "finally" would be at least as confusing as "else", > since by analogy with try-finally it suggests that > it will *always* be executed, which isn't the case. Then again, a try's finally isn't *always* executed. It's *almost always* executed, but if the runtime (or just the thread the block is in) crashes, is killed with kill -9 or the machine is shut down, it won't be. Interestingly, because sys.exit throws an exception instead of killing the runtime, the finally block does run if you call sys.exit in the try block (in Java, it doesn't) From guido at python.org Wed Oct 7 16:44:53 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 7 Oct 2009 07:44:53 -0700 Subject: [Python-ideas] for/except/else syntax In-Reply-To: References: Message-ID: On Tue, Oct 6, 2009 at 9:06 PM, Bruce Leban wrote: > Here you go: > do_except = True > for i in SEQ: > ??do_except = False > ??A > else: > ??if not do_except: > ?? ?C ?#?suite_if_loop_is_not_broken > if do_except: > ??B ?#?suite_if_loop_body_is_not_executed This doesn't keep the else semantics the same; currently in a for-loop over an empty list, the else clause is still executed, but not in your case, since do_except remains true. You may have meant this intentionally but it means that adding 'except: pass' would change the semantics of a for+else loop, which I find unexpected. I now understand that you want the except clause executed when there were zero iterations (your wording "not executed" didn't mean anything to me). Naming it 'except' is a really bad idea though because it makes people thing of exception handling. I can somewhat sympathize with this desire, but I think there are already plenty of ways to do this without adding new syntax. In general the addition of new syntax ought to be a truly rare event, and with the still pending transition to Python 3.x, doubly so. > --- Bruce > > > On Tue, Oct 6, 2009 at 8:53 PM, Guido van Rossum wrote: >> >> On Tue, Oct 6, 2009 at 8:46 PM, Bruce Leban wrote: >> > Switching gears a bit, here's what I want: >> > >> > for_stmt ::= "for" target_list "in" expression_list ":" suite_loop_body >> > ["except" ":" suite_if_loop_body_is_not_executed] ["else" ":" >> > suite_if_loop_is_not_broken] >> > >> > The except and else appear in this order to make it clear that except >> > takes >> > precedence. If the loop body is not executed at all then both conditions >> > are >> > true but it obviously is only useful if except takes precedence. >> > Comments? >> >> I'm sure it's obvious to you what the except clause should do, but it >> isn't to me. :-( Assuming you're not proposing we change the meaning >> of else in the absence of except, and that its meaning together with >> except is a logical extension of this, can you show how we should >> translate a for-loop with an except clause into current Python? Say, >> what would I be writing today to get the exact effect of your proposed >> >> for i in SEQ: >> ?A >> except: >> ?B >> else: >> ?C >> >> ? >> >> -- >> --Guido van Rossum (home page: http://www.python.org/~guido/) > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From debatem1 at gmail.com Wed Oct 7 17:06:46 2009 From: debatem1 at gmail.com (geremy condra) Date: Wed, 7 Oct 2009 11:06:46 -0400 Subject: [Python-ideas] Decorator syntax restriction In-Reply-To: <4AC6AB82.40002@canterbury.ac.nz> References: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> <4AC6AB82.40002@canterbury.ac.nz> Message-ID: On Fri, Oct 2, 2009 at 9:40 PM, Greg Ewing wrote: > Paul Moore wrote: > > @(a['b'].fns[1])(1,2,{3,4}) >> def something(): >> >> I contend that's clearly "line noise". >> > > But there's nothing to stop you from writing > > f = (a['b'].fns[1])(1,2,{3,4}) > @f > def something(): > > If the first version is unreadable, the second one isn't going > to be significantly better. > > -- > Greg I'm +0 on the whole thing, but this to me is the strongest argument advanced thus far. Seems silly to make people run otherwise reasonable, readable code through the manglifier in order to satisfy a rule that doesn't work anyway. Geremy Condra -------------- next part -------------- An HTML attachment was scrubbed... URL: From anfedorov at gmail.com Wed Oct 7 17:17:38 2009 From: anfedorov at gmail.com (Andrey Fedorov) Date: Wed, 7 Oct 2009 11:17:38 -0400 Subject: [Python-ideas] for/except/else syntax In-Reply-To: References: Message-ID: <7659cab30910070817u78094ddele0b8a08b0176cb8b@mail.gmail.com> Agree with Rob that the "else" keyword confusing in the context of a for loop. In my mind, "for each pebble in the bag, give it to Ben, or else ..." has no clear semantic meaning. What do you mean, "or else"? In my mind, better words for what "else" currently does seem to be "afterwards", "atend", "ending", "thereafter", or "subsequently". - Andrey On Wed, Oct 7, 2009 at 10:44 AM, Guido van Rossum wrote: > On Tue, Oct 6, 2009 at 9:06 PM, Bruce Leban wrote: > > Here you go: > > do_except = True > > for i in SEQ: > > do_except = False > > A > > else: > > if not do_except: > > C # suite_if_loop_is_not_broken > > if do_except: > > B # suite_if_loop_body_is_not_executed > > This doesn't keep the else semantics the same; currently in a for-loop > over an empty list, the else clause is still executed, but not in your > case, since do_except remains true. You may have meant this > intentionally but it means that adding 'except: pass' would change the > semantics of a for+else loop, which I find unexpected. > > I now understand that you want the except clause executed when there > were zero iterations (your wording "not executed" didn't mean anything > to me). Naming it 'except' is a really bad idea though because it > makes people thing of exception handling. > > I can somewhat sympathize with this desire, but I think there are > already plenty of ways to do this without adding new syntax. In > general the addition of new syntax ought to be a truly rare event, and > with the still pending transition to Python 3.x, doubly so. > > > --- Bruce > > > > > > On Tue, Oct 6, 2009 at 8:53 PM, Guido van Rossum > wrote: > >> > >> On Tue, Oct 6, 2009 at 8:46 PM, Bruce Leban wrote: > >> > Switching gears a bit, here's what I want: > >> > > >> > for_stmt ::= "for" target_list "in" expression_list ":" > suite_loop_body > >> > ["except" ":" suite_if_loop_body_is_not_executed] ["else" ":" > >> > suite_if_loop_is_not_broken] > >> > > >> > The except and else appear in this order to make it clear that except > >> > takes > >> > precedence. If the loop body is not executed at all then both > conditions > >> > are > >> > true but it obviously is only useful if except takes precedence. > >> > Comments? > >> > >> I'm sure it's obvious to you what the except clause should do, but it > >> isn't to me. :-( Assuming you're not proposing we change the meaning > >> of else in the absence of except, and that its meaning together with > >> except is a logical extension of this, can you show how we should > >> translate a for-loop with an except clause into current Python? Say, > >> what would I be writing today to get the exact effect of your proposed > >> > >> for i in SEQ: > >> A > >> except: > >> B > >> else: > >> C > >> > >> ? > >> > >> -- > >> --Guido van Rossum (home page: http://www.python.org/~guido/) > > > > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Wed Oct 7 19:27:22 2009 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 07 Oct 2009 18:27:22 +0100 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <9d153b7c0910070000q6c1f8c11g83cb0d2ec01eba87@mail.gmail.com> References: <9d153b7c0910070000q6c1f8c11g83cb0d2ec01eba87@mail.gmail.com> Message-ID: <4ACCCF7A.8090508@mrabarnett.plus.com> Yuvgoog Greenle wrote: > I don't like your "except" and "else" only because it's not obvious > when they execute. > > btw your example breaks the current "for..else" search idiom. C, > suite_if_loop_is_not_broken, should be labelled > suite_if_loop_is_not_broken_and_not_empty in your example code. > > As I understand it, pythoneers need a way to know what happened with a > previous loop. Did it break? Did it execute at all? The answer to both > questions is either True or False and developers need these answers > only after the loop finished. If python wasn't in over it's head with > reserved words I would suggest adding "loop": > > for i in SEQ: > A > if not loop.broke: > C # suite_if_loop_is_not_broken > elif loop.empty: > B # suite_if_loop_body_is_not_executed > [snip] Perhaps: for i in SEQ: A elif not break: C # suite_if_loop_is_not_broken elif pass: B # suite_if_loop_body_is_not_executed or: for i in SEQ: A elif pass: B # suite_if_loop_body_is_not_executed else: C # suite_if_loop_is_not_broken The first possibility suggests that you could test whether there _was_ a break and well as whether there wasn't: for item in item_list: if item == desired_item: break elif break: print "Found the item" elif pass: print "No items" else: # or elif not break print "Didn't find the item" This would cause a problem in that "else" wouldn't be the default for when no elif matched, but would always mean "elif not break"! (Unless it defaulted to "elif not break" if it occurred alone.) From gerald.britton at gmail.com Wed Oct 7 19:40:16 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Wed, 7 Oct 2009 13:40:16 -0400 Subject: [Python-ideas] for/else syntax In-Reply-To: <4147EC63-BA05-4705-8005-B6895A2860E1@masklinn.net> References: <9d153b7c0909301807r5c5f0a14r79cca8689575dc5f@mail.gmail.com> <39378.195.197.209.254.1254394821.squirrel@webmail.kapsi.fi> <9d153b7c0910010512h4df68169w8f06085b2a4809fa@mail.gmail.com> <4AC558FA.7010103@canterbury.ac.nz> <4147EC63-BA05-4705-8005-B6895A2860E1@masklinn.net> Message-ID: <5d1a32000910071040v41a07d19y89ce156fdf28edc9@mail.gmail.com> That's what I was getting at. As long as Python doesn't crash or the process is not killed by some external force (e.g. power failure), the finally clause of a try/except/finally will always be executed. On Wed, Oct 7, 2009 at 10:18 AM, Masklinn wrote: > On 2 Oct 2009, at 03:35 , Greg Ewing wrote: >> >> Yuvgoog Greenle wrote: >>> >>> This >>> construct should have been named "finally", "didnt_break" or "not break": >> >> "finally" would be at least as confusing as "else", >> since by analogy with try-finally it suggests that >> it will *always* be executed, which isn't the case. > > Then again, a try's finally isn't *always* executed. It's *almost always* > executed, but if the runtime (or just the thread the block is in) crashes, > is killed with kill -9 or the machine is shut down, it won't be. > > Interestingly, because sys.exit throws an exception instead of killing the > runtime, the finally block does run if you call sys.exit in the try block > (in Java, it doesn't) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From bruce at leapyear.org Wed Oct 7 20:05:04 2009 From: bruce at leapyear.org (Bruce Leban) Date: Wed, 7 Oct 2009 11:05:04 -0700 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <4ACCCF7A.8090508@mrabarnett.plus.com> References: <9d153b7c0910070000q6c1f8c11g83cb0d2ec01eba87@mail.gmail.com> <4ACCCF7A.8090508@mrabarnett.plus.com> Message-ID: On Wed, Oct 7, 2009 at 10:27 AM, MRAB wrote: > Yuvgoog Greenle wrote: > >> I don't like your "except" and "else" only because it's not obvious >> when they execute. >> >> btw your example breaks the current "for..else" search idiom. C, >> suite_if_loop_is_not_broken, should be labelled >> suite_if_loop_is_not_broken_and_not_empty in your example code. >> > Some of the discussion makes it clear that it wasn't clear that my translation was for the for/except/else case and would not change the for/else case we have today. Here's a translation that covers both cases: do_except = True do_else = False for i in SEQ: do_except = False A else: do_else = True if do_except: B # suite_if_loop_body_is_not_executed elif do_else: C Regarding Guido's comment about the use of 'except' here being it makes people think of exceptions, I think it makes people think of exceptional cases too. If I could commandeer the Python time machine and rollback for/else, what I'd do is: for i in SEQ: suite except None: suite # executed if SEQ is empty except NoBreak: suite # executed if loop did not break excluding except None if present NoBreak is not a keyword; it's a special value. -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam at atlas.st Wed Oct 7 20:26:39 2009 From: adam at atlas.st (Adam Atlas) Date: Wed, 7 Oct 2009 14:26:39 -0400 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <7659cab30910070817u78094ddele0b8a08b0176cb8b@mail.gmail.com> References: <7659cab30910070817u78094ddele0b8a08b0176cb8b@mail.gmail.com> Message-ID: On 7 Oct 2009, at 11:17, Andrey Fedorov wrote: > Agree with Rob that the "else" keyword confusing in the context of a > for loop. In my mind, "for each pebble in the bag, give it to Ben, > or else ..." has no clear semantic meaning. What do you mean, "or > else"? That wouldn't have any particular meaning in Python either (in that the "else" clause would never execute). If you had an example where there *was* some use for the "else" clause, it might make more sense in an English interpretation too. Maybe something like, "for each pebble in the bag, when you get to a blue one, give it to Ben and stop, or else...". Not the best way to phrase it, but it's comprehensible. > In my mind, better words for what "else" currently does seem to be > "afterwards", "atend", "ending", "thereafter", or "subsequently". I don't think any of those really sound like what the "else" clause does... they all just sound like something to execute when the loop is done, which is what happens after the loop body anyway. Switching to any of those words would probably only make it more unintuitive. - Adam From ilya.nikokoshev at gmail.com Wed Oct 7 21:18:59 2009 From: ilya.nikokoshev at gmail.com (ilya) Date: Wed, 7 Oct 2009 23:18:59 +0400 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <9d153b7c0910070000q6c1f8c11g83cb0d2ec01eba87@mail.gmail.com> References: <9d153b7c0910070000q6c1f8c11g83cb0d2ec01eba87@mail.gmail.com> Message-ID: I was actually surprise to learn that for/else is already part of Python. I would be more happy with 'if not break', but anyway. As for `loop.empty`, this is probably what I would reserve if I was designing language from scratch, but it does seem very unpythonic to me. Note though that you can do empty = object() # sentinel ... i = empty for i in SEQ: A if i is empty: B # suite_if_loop_body_is_not_executed This requires just one more line (i=empty) per loop compared to other proposals and is completely unambiguos to both human and computer. Moreover, you can put `empty` into builtins without breaking existing programs (however, this idiom breaks in a weird case of `SEQ[-1]==empty`). On Wed, Oct 7, 2009 at 11:00 AM, Yuvgoog Greenle wrote: > I don't like your "except" and "else" only because it's not obvious > when they execute. > > btw your example breaks the current "for..else" search idiom. C, > suite_if_loop_is_not_broken, should be labelled > suite_if_loop_is_not_broken_and_not_empty in your example code. > > As I understand it, pythoneers need a way to know what happened with a > previous loop. Did it break? Did it execute at all? The answer to both > questions is either True or False and developers need these answers > only after the loop finished. If python wasn't in over it's head with > reserved words I would suggest adding "loop": > > for i in SEQ: > ? ?A > if not loop.broke: > ? ?C # suite_if_loop_is_not_broken > elif loop.empty: > ? ?B ?# suite_if_loop_body_is_not_executed > > For optimization I would suggest the "loop" variable only be > instantiated if it's referenced. Also, there may be better names for > "loop", "broke" and "empty". > > Two drawbacks: > 1. python's in over it's head with reserved words. > 2. developers might want to use the "loop" variable a mile down from > the loop which would be ugly but their fault. > > I'm writing a summary of the previous for/else thread and hope that by > the time I finish that (a week+), someone will invent a readable, > low-cost syntax (optional but awesome if it can handle all 4: > break/not break/empty/not empty). > > On Wed, Oct 7, 2009 at 6:06 AM, Bruce Leban wrote: >> Here you go: >> do_except = True >> for i in SEQ: >> ??do_except = False >> ??A >> else: >> ??if not do_except: >> ?? ?C ?#?suite_if_loop_is_not_broken >> if do_except: >> ??B ?#?suite_if_loop_body_is_not_executed >> --- Bruce >> >> >> On Tue, Oct 6, 2009 at 8:53 PM, Guido van Rossum wrote: >>> >>> On Tue, Oct 6, 2009 at 8:46 PM, Bruce Leban wrote: >>> > Switching gears a bit, here's what I want: >>> > >>> > for_stmt ::= "for" target_list "in" expression_list ":" suite_loop_body >>> > ["except" ":" suite_if_loop_body_is_not_executed] ["else" ":" >>> > suite_if_loop_is_not_broken] >>> > >>> > The except and else appear in this order to make it clear that except >>> > takes >>> > precedence. If the loop body is not executed at all then both conditions >>> > are >>> > true but it obviously is only useful if except takes precedence. >>> > Comments? >>> >>> I'm sure it's obvious to you what the except clause should do, but it >>> isn't to me. :-( Assuming you're not proposing we change the meaning >>> of else in the absence of except, and that its meaning together with >>> except is a logical extension of this, can you show how we should >>> translate a for-loop with an except clause into current Python? Say, >>> what would I be writing today to get the exact effect of your proposed >>> >>> for i in SEQ: >>> ?A >>> except: >>> ?B >>> else: >>> ?C >>> >>> ? >>> >>> -- >>> --Guido van Rossum (home page: http://www.python.org/~guido/) >> >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> >> > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From steve at pearwood.info Wed Oct 7 23:58:11 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 8 Oct 2009 08:58:11 +1100 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <4ACCCF7A.8090508@mrabarnett.plus.com> References: <9d153b7c0910070000q6c1f8c11g83cb0d2ec01eba87@mail.gmail.com> <4ACCCF7A.8090508@mrabarnett.plus.com> Message-ID: <200910080858.12407.steve@pearwood.info> On Thu, 8 Oct 2009 04:27:22 am MRAB wrote: > Perhaps: > > for i in SEQ: > A > elif not break: > C # suite_if_loop_is_not_broken > elif pass: > B # suite_if_loop_body_is_not_executed Here's my executive summary, for those in a hurry: -1 on any variation like "else not break" or "elif not break". +1 on aliasing "for...else" to "for...then", with identical semantics, with eventual deprecation and removal of for...else some time in the indefinite future. (Likewise for while...else.) -0 on introducing a clause that executes if the loop is never executed at all. And here are the details: Any variation like "else not break" or "elif not break" makes it look that you are testing a flag. If you think for...else is confusing now, I guarantee that this will be nothing compared to the confusion people will feel when they discover this (proposed) thing that looks like testing a flag but isn't. People will wonder where this break flag is set, and why they can't set it themselves, or even access it like any other object. (Before anyone suggests that perhaps we should introduce a global variable called "break", and allow the caller to read/write to it outside of the for loop, remember that break is a statement. It would be too abominable for words to have break a global variable and a statement at the same time.) Assuming there is consensus that for...else is confusing, my suggestion is to add an alias "then": for x in seq: ... then: suite The behaviour of "then" is identical to the current behaviour of "else", it's just another way of spelling it. Assuming consensus, eventually else would be deprecated and then removed. This has the advantage of a more natural interpretation. It seems more natural to say: run the loop, then execute suite (unless there was a break) instead of run the loop, optionally break, else execute suite. If you need to distinguish between "loop ran" and "loop didn't run at all", you don't need syntax for that, you can use the sentinel trick Ilya suggested. (You don't even need a special sentinel, just use None, provided you know None is not in the sequence you're iterating over.) But if this was so common a use-case that we needed syntax for it (and I don't believe it is), then I'd suggest: for x in seq: suite A then: suite B otherwise: suite C I'm not really enamoured of "otherwise". I'd prefer to use "else" for the case of "the loop was never entered at all", but of course that is impossible given that it currently has another meaning. -- Steven D'Aprano Operations Manager Cybersource Pty Ltd, ABN 13 053 904 082 Level 1, 130-132 Stawell St, Richmond VIC 3121 Tel: +61 3 9428 6922 Fax: +61 3 9428 6944 Web: http://www.cybersource.com.au From debatem1 at gmail.com Thu Oct 8 02:57:49 2009 From: debatem1 at gmail.com (geremy condra) Date: Wed, 7 Oct 2009 20:57:49 -0400 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <200910080858.12407.steve@pearwood.info> References: <9d153b7c0910070000q6c1f8c11g83cb0d2ec01eba87@mail.gmail.com> <4ACCCF7A.8090508@mrabarnett.plus.com> <200910080858.12407.steve@pearwood.info> Message-ID: On Wed, Oct 7, 2009 at 5:58 PM, Steven D'Aprano wrote: > On Thu, 8 Oct 2009 04:27:22 am MRAB wrote: > > > Perhaps: > > > > for i in SEQ: > > A > > elif not break: > > C # suite_if_loop_is_not_broken > > elif pass: > > B # suite_if_loop_body_is_not_executed > > Here's my executive summary, for those in a hurry: > > -1 on any variation like "else not break" or "elif not break". > > +1 on aliasing "for...else" to "for...then", with identical semantics, > with eventual deprecation and removal of for...else some time in the > indefinite future. (Likewise for while...else.) > > -0 on introducing a clause that executes if the loop is never executed > at all. > > > And here are the details: > > Any variation like "else not break" or "elif not break" makes it look > that you are testing a flag. If you think for...else is confusing > now, I guarantee that this will be nothing compared to the confusion > people will feel when they discover this (proposed) thing that looks > like testing a flag but isn't. People will wonder where this break > flag is set, and why they can't set it themselves, or even access it > like any other object. > > (Before anyone suggests that perhaps we should introduce a global > variable called "break", and allow the caller to read/write to it > outside of the for loop, remember that break is a statement. It would > be too abominable for words to have break a global variable and a > statement at the same time.) > > Assuming there is consensus that for...else is confusing, my > suggestion is to add an alias "then": > > for x in seq: > ... > then: > suite > > The behaviour of "then" is identical to the current behaviour > of "else", it's just another way of spelling it. Assuming consensus, > eventually else would be deprecated and then removed. > > This has the advantage of a more natural interpretation. It seems more > natural to say: > > run the loop, then execute suite (unless there was a break) > > instead of > > run the loop, optionally break, else execute suite. > > > > If you need to distinguish between "loop ran" and "loop didn't run at > all", you don't need syntax for that, you can use the sentinel trick > Ilya suggested. (You don't even need a special sentinel, just use > None, provided you know None is not in the sequence you're iterating > over.) But if this was so common a use-case that we needed syntax for > it (and I don't believe it is), then I'd suggest: > > for x in seq: > suite A > then: > suite B > otherwise: > suite C > > I'm not really enamoured of "otherwise". I'd prefer to use "else" for > the case of "the loop was never entered at all", but of course that > is impossible given that it currently has another meaning. > > > > -- > Steven D'Aprano > Sounds good to me. Geremy Condra -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Thu Oct 8 08:29:34 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 08 Oct 2009 15:29:34 +0900 Subject: [Python-ideas] for/except/else syntax In-Reply-To: References: <7659cab30910070817u78094ddele0b8a08b0176cb8b@mail.gmail.com> Message-ID: <87ws36qtyp.fsf@uwakimon.sk.tsukuba.ac.jp> Adam Atlas writes: > > On 7 Oct 2009, at 11:17, Andrey Fedorov wrote: > > Agree with Rob that the "else" keyword confusing in the context of a > > for loop. In my mind, "for each pebble in the bag, give it to Ben, > > or else ..." has no clear semantic meaning. What do you mean, "or > > else"? > > That wouldn't have any particular meaning in Python either (in that > the "else" clause would never execute). No, the else clause (as currently defined) *always* executes. > > In my mind, better words for what "else" currently does seem to be > > "afterwards", "atend", "ending", "thereafter", or "subsequently". > > I don't think any of those really sound like what the "else" clause > does... they all just sound like something to execute when the loop is > done, which is what happens after the loop body anyway. Switching to > any of those words would probably only make it more unintuitive. Since your intuition as expressed above is 100% wrong, I don't see how switching to any of those words could possibly hurt. :-) See Steven d'Aprano's post advocating an alternative spelling of "else" as "then" (which is a much more compact way of saying "subsequently"). Since IIRC we *still* have only one at all common use case for this facility, which looks like for thing in iterable: if thing.looks_good(): break else: thing = default_thing I'm -1 on all extensions, -1 on any multiword respelling of "else", -1 on any spelling of "else" that includes the substring "break", and -0 on renaming "else" to "then" (or something like it). From ubershmekel at gmail.com Thu Oct 8 10:52:04 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Thu, 8 Oct 2009 10:52:04 +0200 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <87ws36qtyp.fsf@uwakimon.sk.tsukuba.ac.jp> References: <7659cab30910070817u78094ddele0b8a08b0176cb8b@mail.gmail.com> <87ws36qtyp.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <9d153b7c0910080152u2b012e62scddfb1b466c88b6b@mail.gmail.com> -1 on for...then Everything after the for loop is a "then": for i in SEQ: A # then # code follows The special thing about "else" is that it's skipped upon break. That's the *one and only* use case. Since the current "else" clause only tests for "if not break", I think spelling it out somehow could extremely improve "else" readability. +1 for improving the readability of "for..break..else" and not just putting the confusion under a "then" rug. --yuv -------------- next part -------------- An HTML attachment was scrubbed... URL: From list-ener at strank.info Thu Oct 8 10:57:38 2009 From: list-ener at strank.info (Stefan Rank) Date: Thu, 08 Oct 2009 10:57:38 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? Message-ID: <4ACDA982.3070604@strank.info> Hi, a suggestion/question related to the discussion about renaming for/else and while/else: Is it easy to generate a SyntaxWarning if you use these constructs without a break or return statement in the loop? AFAICS, any such usage would be either wrong or unnecessary. (right?) This would both help prevent wrong usage because of a false intuition and, if adequately and prominently documented, would help those interested in learning about for/else while/else. Easily doable? Maybe even make it a SyntaxError? I guess it would involve using a flag while parsing loops, but that's why I am asking, since I don't have experience with Python's parser... cheers, stefan From ubershmekel at gmail.com Thu Oct 8 11:24:46 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Thu, 8 Oct 2009 11:24:46 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <4ACDA982.3070604@strank.info> References: <4ACDA982.3070604@strank.info> Message-ID: <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> On Thu, Oct 8, 2009 at 10:57 AM, Stefan Rank wrote: > a suggestion/question related to the discussion about renaming for/else and > while/else: > Is it easy to generate a SyntaxWarning if you use these constructs without > a break or return statement in the loop? AFAICS, any such usage would be > either wrong or unnecessary. (right?) > > Your question is good and valid except that "return" has nothing to do with it. In order to understand the "else" clause it's better that you think of it as an "if not break" after the loop. Any way you look at it "return" behaves as expected - skips the "else" AND skips all the code that follows (much like "raise" would have). "break" is the only way to skip the "else" clause whilst still executing the code that follows the for/while loop. So the only way to the "else" clause is useful is when a "break" can skip it. If there is no "break", you might as well remove the "else" and deindent. --yuv -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmjohnson.mailinglist at gmail.com Thu Oct 8 11:51:20 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Wed, 7 Oct 2009 23:51:20 -1000 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <9d153b7c0910080152u2b012e62scddfb1b466c88b6b@mail.gmail.com> References: <7659cab30910070817u78094ddele0b8a08b0176cb8b@mail.gmail.com> <87ws36qtyp.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910080152u2b012e62scddfb1b466c88b6b@mail.gmail.com> Message-ID: <3bdda690910080251w1047d812uf2f01270e5c2e2ef@mail.gmail.com> Yuvgoog Greenle: > -1 on for...then > Everything after the for loop is a "then": > for i in SEQ: > ?? ?A > # then > # code follows > > The special thing about "else" is that it's skipped upon break. That's the > *one and only* use case. > Since the current "else" clause only tests for "if not break", I think > spelling it out somehow could extremely improve "else" readability. > +1 for improving the readability of "for..break..else" and not just putting > the confusion under a "then" rug. Agreed. "Then" is not only a new keyword (=undesirable), it's not any more clear than what it replaces. It might lead to less misunderstanding of what the for-else clause does, but only because it's so opaque that those encountering it would be forced to look it up in the documentation rather than guess. But if that's the only advantage, you may as well name it "squazzlefritz" or something in Dutch. Or just do broke = False for item in items: if cond(item) broke = True break if not broke: #Pseudo else suite so that it's perfectly clear to the yahoo who comes after you what you're doing and they avoid the temptation to guess. In fact, that's what I would advise for most Python programmers to do today, unless they're completely confident that their code won't fall into the hands of yahoos at some point in the future. ?Carl From list-ener at strank.info Thu Oct 8 11:59:21 2009 From: list-ener at strank.info (Stefan Rank) Date: Thu, 08 Oct 2009 11:59:21 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> Message-ID: <4ACDB7F9.6030908@strank.info> on 2009-10-08 11:24 Yuvgoog Greenle said the following: > On Thu, Oct 8, 2009 at 10:57 AM, Stefan Rank > > wrote: > > a suggestion/question related to the discussion about renaming > for/else and while/else: > Is it easy to generate a SyntaxWarning if you use these constructs > without a break or return statement in the loop? AFAICS, any such > usage would be either wrong or unnecessary. (right?) > > > Your question is good and valid except that "return" has nothing to do > with it. > > In order to understand the "else" clause it's better that you think of > it as an "if not break" after the loop. Any way you look at it "return" > behaves as expected - skips the "else" AND skips all the code that > follows (much like "raise" would have). "break" is the only way to skip > the "else" clause whilst still executing the code that follows the > for/while loop. > > So the only way to the "else" clause is useful is when a "break" can > skip it. If there is no "break", you might as well remove the "else" and > deindent. You're right. In the return-but-no-break-example I had in mind:: def findit(bag): for elem in bag: if isgreat(elem): return elem else: return default the use of else, though correct and arguably readable, is still unnecessary. --strank From masklinn at masklinn.net Thu Oct 8 12:16:00 2009 From: masklinn at masklinn.net (Masklinn) Date: Thu, 8 Oct 2009 12:16:00 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <4ACDB7F9.6030908@strank.info> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDB7F9.6030908@strank.info> Message-ID: On 8 Oct 2009, at 11:59 , Stefan Rank wrote: > on 2009-10-08 11:24 Yuvgoog Greenle said the following: >> On Thu, Oct 8, 2009 at 10:57 AM, Stefan Rank > > wrote: >> a suggestion/question related to the discussion about renaming >> for/else and while/else: >> Is it easy to generate a SyntaxWarning if you use these constructs >> without a break or return statement in the loop? AFAICS, any such >> usage would be either wrong or unnecessary. (right?) >> Your question is good and valid except that "return" has nothing to >> do with it. >> In order to understand the "else" clause it's better that you think >> of it as an "if not break" after the loop. Any way you look at it >> "return" behaves as expected - skips the "else" AND skips all the >> code that follows (much like "raise" would have). "break" is the >> only way to skip the "else" clause whilst still executing the code >> that follows the for/while loop. >> So the only way to the "else" clause is useful is when a "break" >> can skip it. If there is no "break", you might as well remove the >> "else" and deindent. > > You're right. In the return-but-no-break-example I had in mind:: > > def findit(bag): > for elem in bag: > if isgreat(elem): > return elem > else: > return default > > the use of else, though correct and arguably readable, is still > unnecessary. So is it in the following cases: def find_it(bag): for elem in bag: if is_great(elem): raise Whatever(elem) else: return default or def find_it(bag): flag = False for elem in bag: if is_great(elem): flag = True else: if flag: blow_up() the point of the SyntaxWarning proposal, as far as I read it, was to point out that the `else:` clause is only ever useful in the `for:break else:` case. So there's no reason to *not* generate a SyntaxWarning if there is a `return` in the loop if you generate one when there's a `raise`. From ncoghlan at gmail.com Thu Oct 8 12:19:06 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 08 Oct 2009 20:19:06 +1000 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> Message-ID: <4ACDBC9A.2010908@gmail.com> Yuvgoog Greenle wrote: > So the only way to the "else" clause is useful is when a "break" can > skip it. If there is no "break", you might as well remove the "else" and > deindent. Yup. As to the original question in this thread - definitely possible, and shouldn't be particularly difficult. The AST and the symbol table generation don't really keep track of loop nesting (as they don't care all that much), so you would defer picking this situation up to the actual compilation stage (Python/compile.c in the source tree). Conveniently, the compiler already searches for a LOOP fblock in order to generate the syntax error for using break outside a loop, so adding a "fb_break" field to the fblock struct would be a simple way to keep track of whether or not a loop contains a break statement (although I suspect you would have to reverse the current fblock search order to find the innermost loop instead of the outermost one). Then in compiler_for and compiler_while, the fb_break field on the relevant LOOP fblock could be checked whenever an OrElse clause was present in the AST. A SyntaxWarning would then be emitted whenever the else clause was present but the break flag was not set on the relevant LOOP fblock. That's just an implementation sketch obviously, and not actually tested in any way shape or form. Still, it should be fairly straightforward to add the warning. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From gerald.britton at gmail.com Thu Oct 8 15:08:51 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Thu, 8 Oct 2009 09:08:51 -0400 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <4ACDBC9A.2010908@gmail.com> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> Message-ID: <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> I'm still pretty much against the whole idea of changing this syntax or issuing warnings. I like writing: # for loop using else clause for x in y: do_something_with(x) if something_happened_with(x): break else: # only get here if nothing happened nothing_happened() #end of for loop since it visually ties the "nothing happened" to the "for" There's a logical context that one can see. An equivalent without an else clause: #for loop without else clause something_happened = False for x in y: do_something_with(x) if something_happened_with(x): something_happened = True break # end of for loop. We get here whether something happened or not if not something_happened: nothing_happened() #end of logical context lacks such visual clues (and is more verbose and introduces another variable) and has the potential for future problems, since another programmer may add something else between the end of the loop and the conditional call to nothing_happened. OTOH if said programmer put it in the else clause, he or she would hopefully think twice if it belonged there (which it might not). If the syntax were changed (snowball's chance in hades, I bet!) it would break existing programs; if a warning were issued instead, working programs would suddenly spout spurious messages. -1 On Thu, Oct 8, 2009 at 6:19 AM, Nick Coghlan wrote: > Yuvgoog Greenle wrote: >> So the only way to the "else" clause is useful is when a "break" can >> skip it. If there is no "break", you might as well remove the "else" and >> deindent. > > Yup. > > As to the original question in this thread - definitely possible, and > shouldn't be particularly difficult. > > The AST and the symbol table generation don't really keep track of loop > nesting (as they don't care all that much), so you would defer picking > this situation up to the actual compilation stage (Python/compile.c in > the source tree). > > Conveniently, the compiler already searches for a LOOP fblock in order > to generate the syntax error for using break outside a loop, so adding a > "fb_break" field to the fblock struct would be a simple way to keep > track of whether or not a loop contains a break statement (although I > suspect you would have to reverse the current fblock search order to > find the innermost loop instead of the outermost one). > > Then in compiler_for and compiler_while, the fb_break field on the > relevant LOOP fblock could be checked whenever an OrElse clause was > present in the AST. A SyntaxWarning would then be emitted whenever the > else clause was present but the break flag was not set on the relevant > LOOP fblock. > > That's just an implementation sketch obviously, and not actually tested > in any way shape or form. Still, it should be fairly straightforward to > add the warning. > > Cheers, > Nick. > > -- > Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia > --------------------------------------------------------------- > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From masklinn at masklinn.net Thu Oct 8 15:15:10 2009 From: masklinn at masklinn.net (Masklinn) Date: Thu, 8 Oct 2009 15:15:10 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> Message-ID: <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> On 8 Oct 2009, at 15:08 , Gerald Britton wrote: > I'm still pretty much against the whole idea of changing this syntax > or issuing warnings. I like writing: > > # for loop using else clause > for x in y: > do_something_with(x) > if something_happened_with(x): > break > else: # only get here if nothing happened > nothing_happened() > #end of for loop > [snip] > > if a warning were issued instead, > working programs would suddenly spout spurious messages. None of the proposals (as far as I can see) would emit a warning on this snippet: there is a `break` going with the `else:` clause, which is (as far as I can say from the discussion) the normal use case for `for: else:`. The SyntaxWarning proposed would only be emitted on a `for: else:` *without* a break as it's entirely equivalent to just deleting the `else:` clause and dedenting the code it contains. As far as outputting "spurious" messages in working programs, that's the point of a warning isn't it? Tell the programmer that something works, but might not do what one would like (or does nothing at all). From gerald.britton at gmail.com Thu Oct 8 15:44:40 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Thu, 8 Oct 2009 09:44:40 -0400 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> Message-ID: <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> re warnings: The compiler is simply not smart enough to know if a program is doing what a programmer wants. If a given program is working quietly according to its specifications, we should do nothing to disturb the peace. re syntax: There are at least three ways to exit a for-loop early: break, return and raise (explicit or implicit). Would this code generate a warning? (I hope not) for x in y: if found_what_I_want(x): return True else: return False or this? for a in b: if bogus(a): raise Bogus, "Found a bad one" else: nothing_bogus_found() On Thu, Oct 8, 2009 at 9:15 AM, Masklinn wrote: > On 8 Oct 2009, at 15:08 , Gerald Britton wrote: >> >> I'm still pretty much against the whole idea of changing this syntax >> or issuing warnings. ?I like writing: >> >> # for loop using else clause >> for x in y: >> ?do_something_with(x) >> ?if something_happened_with(x): >> ? ?break >> else: # only get here if nothing happened >> ?nothing_happened() >> #end of for loop >> > [snip] >> >> if a warning were issued instead, >> working programs would suddenly spout spurious messages. > > None of the proposals (as far as I can see) would emit a warning on this > snippet: there is a `break` going with the `else:` clause, which is (as far > as I can say from the discussion) the normal use case for `for: else:`. > > The SyntaxWarning proposed would only be emitted on a `for: else:` *without* > a break as it's entirely equivalent to just deleting the `else:` clause and > dedenting the code it contains. > > As far as outputting "spurious" messages in working programs, that's the > point of a warning isn't it? Tell the programmer that something works, but > might not do what one would like (or does nothing at all). > -- Gerald Britton From masklinn at masklinn.net Thu Oct 8 16:06:40 2009 From: masklinn at masklinn.net (Masklinn) Date: Thu, 8 Oct 2009 16:06:40 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> Message-ID: <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> On 8 Oct 2009, at 15:44 , Gerald Britton wrote: > re warnings: The compiler is simply not smart enough to know if a > program is doing what a programmer wants. There are cases in which what's written doesn't make sense or serve any purpose (even if it isn't actively harmful). This includes variables which are created but never used, side-effect free functions whose results are not collected or branches which cannot be taken. Compilers or runtimes typically warn about such cases, this allows the programmer to spot potential bugs and/or simplify their code. And having a `for: else:` statement without a `break` is one such case which, according to Nick Coghlan, can be recognized. So why not warn about it? > If a given program is > working quietly according to its specifications, we should do nothing > to disturb the peace. > If the program is running quietly on its current runtime, it'll keep doing so as long as it isn't switched to a runtime implementing that warning won't it? > re syntax: There are at least three ways to exit a for-loop early: > break, return and raise (explicit or implicit). Would this code > generate a warning? (I hope not) > Both would, because in both cases the `else:` clause serves no purpose whatsoever. It's noise. From george.sakkis at gmail.com Thu Oct 8 16:07:02 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Thu, 8 Oct 2009 10:07:02 -0400 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <3bdda690910080251w1047d812uf2f01270e5c2e2ef@mail.gmail.com> References: <7659cab30910070817u78094ddele0b8a08b0176cb8b@mail.gmail.com> <87ws36qtyp.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910080152u2b012e62scddfb1b466c88b6b@mail.gmail.com> <3bdda690910080251w1047d812uf2f01270e5c2e2ef@mail.gmail.com> Message-ID: <91ad5bf80910080707qba1491fv8250efa4384dc5c5@mail.gmail.com> On Thu, Oct 8, 2009 at 5:51 AM, Carl Johnson wrote: > Yuvgoog Greenle: > >> -1 on for...then >> Everything after the for loop is a "then": >> for i in SEQ: >> ?? ?A >> # then >> # code follows >> >> The special thing about "else" is that it's skipped upon break. That's the >> *one and only* use case. >> Since the current "else" clause only tests for "if not break", I think >> spelling it out somehow could extremely improve "else" readability. >> +1 for improving the readability of "for..break..else" and not just putting >> the confusion under a "then" rug. > > Agreed. "Then" is not only a new keyword (=undesirable), it's not any > more clear than what it replaces. It might lead to less > misunderstanding of what the for-else clause does, but only because > it's so opaque that those encountering it would be forced to look it > up in the documentation rather than guess. But if that's the only > advantage, you may as well name it "squazzlefritz" or something in > Dutch. Or just do > > broke = False > for item in items: > ? ?if cond(item) > ? ? ? ?broke = True > ? ? ? ?break > > if not broke: > ? ?#Pseudo else suite > > so that it's perfectly clear to the yahoo who comes after you what > you're doing and they avoid the temptation to guess. In fact, that's > what I would advise for most Python programmers to do today, unless > they're completely confident that their code won't fall into the hands > of yahoos at some point in the future. Agreed. The more people disagree on how "for/else" should be spelled, the more I think there is no keyword that can encapsulate what it does unambiguously. Certainly "then" leaves me as clueless as "else" for the reasons above. I was thinking of something verbose like "elifnotbreak", but not only it is ugly, it is not 100% accurate; the correct spelling would be "elifnotbreakandnotreturnandnotexception" ;) George From debatem1 at gmail.com Thu Oct 8 16:10:53 2009 From: debatem1 at gmail.com (geremy condra) Date: Thu, 8 Oct 2009 10:10:53 -0400 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> Message-ID: On Thu, Oct 8, 2009 at 9:44 AM, Gerald Britton wrote: > re warnings: The compiler is simply not smart enough to know if a > program is doing what a programmer wants. If a given program is > working quietly according to its specifications, we should do nothing > to disturb the peace. > I grant you that compilers *shouldn't* be smarter than programmers, but I highly doubt that any program including an extraneous else clause is "working according to specifications", and that's the point of this proposal. Geremy Condra -------------- next part -------------- An HTML attachment was scrubbed... URL: From jh at improva.dk Thu Oct 8 16:10:20 2009 From: jh at improva.dk (Jacob Holm) Date: Thu, 08 Oct 2009 16:10:20 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> Message-ID: <4ACDF2CC.5020703@improva.dk> Gerald Britton wrote: > re warnings: The compiler is simply not smart enough to know if a > program is doing what a programmer wants. If a given program is > working quietly according to its specifications, we should do nothing > to disturb the peace. If the code is written using a style that is misleading, I think it makes sense to issue a warning. > > re syntax: There are at least three ways to exit a for-loop early: > break, return and raise (explicit or implicit). Would this code > generate a warning? (I hope not) > > for x in y: > if found_what_I_want(x): > return True > else: > return False This would generate a warning, because the 'else' is unnecessary and potentially confusing here. The above code is exactly equivalent to: for x in y: if found_what_I_want(x): return True return False > > or this? > > for a in b: > if bogus(a): > raise Bogus, "Found a bad one" > else: > nothing_bogus_found() > Would also generate a warning. Again, the version without 'else': for a in b: if bogus(a): raise Bogus, "Found a bad one" nothing_bogus_found() In both cases, I find the version without 'else' to be easier to understand. +1 to making for/else and while/else without break generate a warning. +0 to making it an error. -1 to removing or renaming the 'else' of for and while loops. - Jacob From stephen at xemacs.org Thu Oct 8 18:17:38 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 09 Oct 2009 01:17:38 +0900 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> Message-ID: <8763aprhb1.fsf@uwakimon.sk.tsukuba.ac.jp> Gerald Britton writes: > re warnings: The compiler is simply not smart enough to know if a > program is doing what a programmer wants. If a given program is > working quietly according to its specifications, we should do nothing > to disturb the peace. That *may* be true for Python, but my face has been saved more times than I like to think by the warning on if (x = f(y)) in many C compilers. I think I would probably be saved at least once in my remaining years of Python programming by Nick's proposal, too. From stephen at xemacs.org Thu Oct 8 18:19:35 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 09 Oct 2009 01:19:35 +0900 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> Message-ID: <874oq9rh7s.fsf@uwakimon.sk.tsukuba.ac.jp> Masklinn writes: > Both would, because in both cases the `else:` clause serves no purpose > whatsoever. It's noise. Tut, tut. In Nick's proposal, both would raise because there is an "else:" but there is no break. I doubt the Python compiler would add the editorial comment about "noise," though. From stephen at xemacs.org Thu Oct 8 18:33:12 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 09 Oct 2009 01:33:12 +0900 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> Message-ID: <873a5trgl3.fsf@uwakimon.sk.tsukuba.ac.jp> geremy condra writes: > I grant you that compilers *shouldn't* be smarter than programmers, > but I highly doubt that any program including an extraneous else > clause is "working according to specifications", and that's the point > of this proposal. Actually, I can imagine someone developing a style where a redundant else was used to indicate "this is part of/cleanup of the for loop, and should go with it in any refactoring". Say, in a program that speaks SMTP: for line in line_factory: smtp_send(line + "\n") else: smtp_send(".\n") From masklinn at masklinn.net Thu Oct 8 18:24:55 2009 From: masklinn at masklinn.net (Masklinn) Date: Thu, 8 Oct 2009 18:24:55 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <874oq9rh7s.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> <874oq9rh7s.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On 8 Oct 2009, at 18:19 , Stephen J. Turnbull wrote: > Masklinn writes: >> Both would, because in both cases the `else:` clause serves no >> purpose >> whatsoever. It's noise. > > Tut, tut. In Nick's proposal, both would raise because there is an > "else:" but there is no break. Well yeah but that was what I was saying: in both cases (because there is no break) the `else:` clause has no purpose, the exact same behavior would be obtained by moving the code inside it to just behind the `for:` clause and removing the `else:` one. Therefore the "else:" part of the `else:` clause is noise in the system, it's not signal and it's not absent. > I doubt the Python compiler would add > the editorial comment about "noise," though. Oh absolutely. From guido at python.org Thu Oct 8 18:27:09 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 8 Oct 2009 09:27:09 -0700 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <91ad5bf80910080707qba1491fv8250efa4384dc5c5@mail.gmail.com> References: <7659cab30910070817u78094ddele0b8a08b0176cb8b@mail.gmail.com> <87ws36qtyp.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910080152u2b012e62scddfb1b466c88b6b@mail.gmail.com> <3bdda690910080251w1047d812uf2f01270e5c2e2ef@mail.gmail.com> <91ad5bf80910080707qba1491fv8250efa4384dc5c5@mail.gmail.com> Message-ID: On Thu, Oct 8, 2009 at 7:07 AM, George Sakkis wrote: > The more people disagree on how "for/else" should be spelled, > the more I think there is no keyword that can encapsulate what it does > unambiguously. Certainly "then" leaves me as clueless as "else" for > the reasons above. I was thinking of something verbose like > "elifnotbreak", but not only it is ugly, it is not 100% accurate; the > correct spelling would be "elifnotbreakandnotreturnandnotexception" ;) It is something that has to be learned. Now, lots of things have to be learned -- the behavior of range(1, 10) does not come naturally to many people either. But apparently for-else is used rarely enough that, given the way most people learn (by trying and by reading others' code, not by reading docs), many people don't know about it. That's a flaw, and I don't quite know what to do about it. It's about 20 years too late to remove or rename it. But we probably should not do more of these. That's a lesson. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From gerald.britton at gmail.com Thu Oct 8 18:32:11 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Thu, 8 Oct 2009 12:32:11 -0400 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <874oq9rh7s.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> <874oq9rh7s.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <5d1a32000910080932x197fe67h983184674b39e20@mail.gmail.com> In all my use cases, the else serves the purpose of keeping the code in a logical group. For me at least, it is a structural aid. I vote strongly against any change that would flag for/while loops with else clauses but no breaks, especially since there are at least two other ways for early exit. On Thu, Oct 8, 2009 at 12:19 PM, Stephen J. Turnbull wrote: > Masklinn writes: > > ?> Both would, because in both cases the `else:` clause serves no purpose > ?> whatsoever. It's noise. > > Tut, tut. ?In Nick's proposal, both would raise because there is an > "else:" but there is no break. ?I doubt the Python compiler would add > the editorial comment about "noise," though. > -- Gerald Britton From george.sakkis at gmail.com Thu Oct 8 19:22:06 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Thu, 8 Oct 2009 13:22:06 -0400 Subject: [Python-ideas] for/except/else syntax In-Reply-To: References: <7659cab30910070817u78094ddele0b8a08b0176cb8b@mail.gmail.com> <87ws36qtyp.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910080152u2b012e62scddfb1b466c88b6b@mail.gmail.com> <3bdda690910080251w1047d812uf2f01270e5c2e2ef@mail.gmail.com> <91ad5bf80910080707qba1491fv8250efa4384dc5c5@mail.gmail.com> Message-ID: <91ad5bf80910081022hf5e0c68o212feb07f99a3d1@mail.gmail.com> On Thu, Oct 8, 2009 at 12:27 PM, Guido van Rossum wrote: > That's a flaw, and I don't quite know what to do about it. It's about > 20 years too late to remove or rename it. But we probably should not > do more of these. That's a lesson. +1 George From jared.grubb at gmail.com Thu Oct 8 21:09:19 2009 From: jared.grubb at gmail.com (Jared Grubb) Date: Thu, 8 Oct 2009 12:09:19 -0700 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <5d1a32000910080932x197fe67h983184674b39e20@mail.gmail.com> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> <874oq9rh7s.fsf@uwakimon.sk.tsukuba.ac.jp> <5d1a32000910080932x197fe67h983184674b39e20@mail.gmail.com> Message-ID: <5FB60732-27DD-44DC-B9A2-DD725E0F87A9@gmail.com> On 8 Oct 2009, at 09:32, Gerald Britton wrote: > In all my use cases, the else serves the purpose of keeping the code > in a logical group. For me at least, it is a structural aid. I vote > strongly against any change that would flag for/while loops with else > clauses but no breaks, especially since there are at least two other > ways for early exit. I am also against this. It's a PyLint problem, if anything. (Just for completeness, I suppose 'yield' is a third way; but like return/raise, it makes the else unnecessary) Jared From tjreedy at udel.edu Thu Oct 8 21:44:17 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 08 Oct 2009 15:44:17 -0400 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> Message-ID: Masklinn wrote: > On 8 Oct 2009, at 15:44 , Gerald Britton wrote: >> re syntax: There are at least three ways to exit a for-loop early: >> break, return and raise (explicit or implicit). Would this code >> generate a warning? (I hope not) >> > Both would, because in both cases the `else:` clause serves no purpose > whatsoever. It's noise. To you, but not to Gerald or me. To me, this is a formatting style issue that the *compiler* should keeps its hands off of. As I said at the begining of this thread, this issue is appropriate for separate and optional code/style checkers like pylint/pychecker. Suppose I have for i is s: do_a(): if c(i): break else: do_b do_c I decide *maybe* I do not want do_c in the function, so I comment out the last line and change 'break' to 'return'. You and the OP would force ME to also comment out or delete the 'else:' and dedent 'do_b' for no functional reason but merely to satisy YOUR esthetic style preference. To me, that attitude is pretty obnoxious. Terry Jan Reedy From g.brandl at gmx.net Thu Oct 8 21:50:05 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 08 Oct 2009 21:50:05 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <873a5trgl3.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <873a5trgl3.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: Stephen J. Turnbull schrieb: > geremy condra writes: > > > I grant you that compilers *shouldn't* be smarter than programmers, > > but I highly doubt that any program including an extraneous else > > clause is "working according to specifications", and that's the point > > of this proposal. > > Actually, I can imagine someone developing a style where a redundant > else was used to indicate "this is part of/cleanup of the for loop, > and should go with it in any refactoring". > > Say, in a program that speaks SMTP: > > for line in line_factory: > smtp_send(line + "\n") > else: > smtp_send(".\n") Usually, that effect can be created by newlines: for line in line_factory: smtp_send(line + "\n") smtp_send(".\n") more code goes here I'm not sure myself whether I like generating warnings for valid syntax, but in the often-misunderstood case of for/else, I think it would be reasonable. Georg From masklinn at masklinn.net Thu Oct 8 22:31:22 2009 From: masklinn at masklinn.net (Masklinn) Date: Thu, 8 Oct 2009 22:31:22 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> Message-ID: <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> On 8 Oct 2009, at 21:44 , Terry Reedy wrote: > You and the OP would force ME to also comment out or delete the > 'else:' and dedent 'do_b' for no functional reason but merely to > satisy YOUR esthetic style preference. That is not the case, and that's the difference between an *error* and a *warning*. A warning doesn't *force* anything, it merely? warns. If you want to ignore the warning, you're free to do so. Most of the time, there's even a switch to disable (or enable, if it's not enabled by default) the aforementioned warning. The point of a warning is very much that the code works and is legal, but *in most case* is not what people would want. If it's what you want, you're free to go ahead and keep the code as is, ignoring the warning (or even filtering it out). From gerald.britton at gmail.com Thu Oct 8 23:03:40 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Thu, 8 Oct 2009 17:03:40 -0400 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> Message-ID: <5d1a32000910081403o56ca3183t2c44a615dbd00cd2@mail.gmail.com> Based upon Guido's post in this thread, I think the chances of getting such a warning are between slim and none. And, I'm very happy about that. In fact, I would be extremely unhappy if a program that has worked for 20 years (Guido's number, not mine) would suddenly start to spurt warnings just because I upgraded Python. Bad idea all round. OTOH, if you really would like such a warning, put it in pylint. Seems to me to be by far the best option. On Thu, Oct 8, 2009 at 4:31 PM, Masklinn wrote: > On 8 Oct 2009, at 21:44 , Terry Reedy wrote: >> >> You and the OP would force ME to also comment out or delete the 'else:' >> and dedent 'do_b' for no functional reason but merely to satisy YOUR >> esthetic style preference. > > That is not the case, and that's the difference between an *error* and a > *warning*. > > A warning doesn't *force* anything, it merely? warns. If you want to ignore > the warning, you're free to do so. Most of the time, there's even a switch > to disable (or enable, if it's not enabled by default) the aforementioned > warning. > > The point of a warning is very much that the code works and is legal, but > *in most case* is not what people would want. If it's what you want, you're > free to go ahead and keep the code as is, ignoring the warning (or even > filtering it out). > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From ubershmekel at gmail.com Fri Oct 9 00:52:55 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Fri, 9 Oct 2009 00:52:55 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <5d1a32000910081403o56ca3183t2c44a615dbd00cd2@mail.gmail.com> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> <5d1a32000910081403o56ca3183t2c44a615dbd00cd2@mail.gmail.com> Message-ID: <9d153b7c0910081552s5aaad10brfaa58ed9a5f9e191@mail.gmail.com> Gerald, if this warning would have been generated you might have saved yourself the mistakes on the previous thread. Gerald wrote as an example of "for..else" being useful without a break: > for i, j in enumerate(something): > ?# suite >??i += 1 >?else: >??i = 0 > >?# i == number of things processed If "# suite" has no break, "i" will always be 0. Either way after the loop "i" isn't going to have the number of things processed (unless "something" was empty). After being corrected Gerald wrote the following remedied example: > for i, j in enumerate(something): > ?# do something > ?i += 1 > else: > ?i = -1 > > if i > 0: > ?# we did something The last block "# we did something" is unreachable if "#do something" has no break. This isn't meant to go against Gerald here btw. I'm simply saying that if even the best Pythonistas can cause havoc with "for...else" then think about the noobs. Personally I would suggest not using the "for...break...else" idiom at all but the least we can do is give a warning when "for..else" is so obviously misused. You could allow "break" outside of loops and simply ignore it, but you don't allow it because it helps avoid bugs. Really, a warning is the LEAST python can do here. --yuv From gerald.britton at gmail.com Fri Oct 9 01:35:51 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Thu, 8 Oct 2009 19:35:51 -0400 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <9d153b7c0910081552s5aaad10brfaa58ed9a5f9e191@mail.gmail.com> References: <4ACDA982.3070604@strank.info> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> <5d1a32000910081403o56ca3183t2c44a615dbd00cd2@mail.gmail.com> <9d153b7c0910081552s5aaad10brfaa58ed9a5f9e191@mail.gmail.com> Message-ID: <5d1a32000910081635l24cf5d1ex28d46e8c97bf11b1@mail.gmail.com> Ach, I just whipped it up in a hurry. I didn't pull it from live code (or even test it). A better (working) version: >>> x = [] >>> del i >>> for i, j in x: ... do_something() ... else: ... i = vars().get('i',-1) ... >>> i -1 >>> And before someone says _again_ that the else is not necessary here, that's not the reason I use it. I use it to structure the code so that it is obvious that the stuff in the else clause belongs to the for loop, not to code that follows it. The point is, there are valid reasons to use this construct. That's just one of many, many more. I would fight tooth-and-nail against a warning message, if I thought there was even the faintest chance of it being considered. As it is, there is essentially no chance that it will be considered for the compiler (pylint is another matter, I think that it _should_ be there). On Thu, Oct 8, 2009 at 6:52 PM, Yuvgoog Greenle wrote: > Gerald, if this warning would have been generated you might have saved > yourself the mistakes on the previous thread. > > Gerald wrote as an example of "for..else" being useful without a break: >> for i, j in enumerate(something): >> ?# suite >>??i += 1 >>?else: >>??i = 0 >> >>?# i == number of things processed > > If "# suite" has no break, "i" will always be 0. Either way after the > loop "i" isn't going to have the number of things processed (unless > "something" was empty). After being corrected Gerald wrote the > following remedied example: > >> for i, j in enumerate(something): >> ?# do something >> ?i += 1 >> else: >> ?i = -1 >> >> if i > 0: >> ?# we did something > > The last block "# we did something" is unreachable if "#do something" > has no break. > > This isn't meant to go against Gerald here btw. I'm simply saying that > if even the best Pythonistas can cause havoc with "for...else" then > think about the noobs. Personally I would suggest not using the > "for...break...else" idiom at all but the least we can do is give a > warning when "for..else" is so obviously misused. > > You could allow "break" outside of loops and simply ignore it, but you > don't allow it because it helps avoid bugs. Really, a warning is the > LEAST python can do here. > > --yuv > -- Gerald Britton From ubershmekel at gmail.com Fri Oct 9 02:09:18 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Fri, 9 Oct 2009 02:09:18 +0200 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <91ad5bf80910080707qba1491fv8250efa4384dc5c5@mail.gmail.com> References: <7659cab30910070817u78094ddele0b8a08b0176cb8b@mail.gmail.com> <87ws36qtyp.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910080152u2b012e62scddfb1b466c88b6b@mail.gmail.com> <3bdda690910080251w1047d812uf2f01270e5c2e2ef@mail.gmail.com> <91ad5bf80910080707qba1491fv8250efa4384dc5c5@mail.gmail.com> Message-ID: <9d153b7c0910081709u76410704q3befbad25ce9a54d@mail.gmail.com> On Thu, Oct 8, 2009 at 4:07 PM, George Sakkis wrote: > Agreed. The more people disagree on how "for/else" should be spelled, > the more I think there is no keyword that can encapsulate what it does > unambiguously. Certainly "then" leaves me as clueless as "else" for > the reasons above. I was thinking of something verbose like > "elifnotbreak", but not only it is ugly, it is not 100% accurate; the > correct spelling would be "elifnotbreakandnotreturnandnotexception" ;) I agree with you on "then" giving no clues but I think you're wrong about the spelling. The correct spelling would be "if not break". x = y / z if condition: print(1,2,3) The above code translated to "George Sakkis Language" should be spelled: x = y / x if condition and notreturnandnotexception: print(1,2,3) So because it's obvious that everything in python is under the condition of "notreturnandnotexception", your point is moot. The "else" in "for...break...else" is very simply a condition tested after the loop finished (exactly like any other "if" that would immediately follow the loop). The condition that's tested is simple: "if there was a break - skip this". So your spelling exaggeration doesn't worry me :). The fact that python-ideas can't agree on a better syntax for python 4/5 does. I appreciate the lesson taught, as Guido put it, but I think python can have a long term plan concerning this old, rare and misleading syntax. --yuv From debatem1 at gmail.com Fri Oct 9 02:12:39 2009 From: debatem1 at gmail.com (geremy condra) Date: Thu, 8 Oct 2009 20:12:39 -0400 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <5d1a32000910081635l24cf5d1ex28d46e8c97bf11b1@mail.gmail.com> References: <4ACDA982.3070604@strank.info> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> <5d1a32000910081403o56ca3183t2c44a615dbd00cd2@mail.gmail.com> <9d153b7c0910081552s5aaad10brfaa58ed9a5f9e191@mail.gmail.com> <5d1a32000910081635l24cf5d1ex28d46e8c97bf11b1@mail.gmail.com> Message-ID: On Thu, Oct 8, 2009 at 7:35 PM, Gerald Britton wrote: > Ach, I just whipped it up in a hurry. I didn't pull it from live > code (or even test it). A better (working) version: > > >>> x = [] > >>> del i > >>> for i, j in x: > ... do_something() > ... else: > ... i = vars().get('i',-1) > ... > >>> i > -1 > >>> > > And before someone says _again_ that the else is not necessary here, > that's not the reason I use it. I use it to structure the code so > that it is obvious that the stuff in the else clause belongs to the > for loop, not to code that follows it. > > The point is, there are valid reasons to use this construct. That's > just one of many, many more. I would fight tooth-and-nail against a > warning message, if I thought there was even the faintest chance of it > being considered. As it is, there is essentially no chance that it > will be considered for the compiler (pylint is another matter, I > think that it _should_ be there). > I'm not clear- you're saying its a code smell to use it that way? Geremy Condra > > On Thu, Oct 8, 2009 at 6:52 PM, Yuvgoog Greenle > wrote: > > Gerald, if this warning would have been generated you might have saved > > yourself the mistakes on the previous thread. > > > > Gerald wrote as an example of "for..else" being useful without a break: > >> for i, j in enumerate(something): > >> # suite > >> i += 1 > >> else: > >> i = 0 > >> > >> # i == number of things processed > > > > If "# suite" has no break, "i" will always be 0. Either way after the > > loop "i" isn't going to have the number of things processed (unless > > "something" was empty). After being corrected Gerald wrote the > > following remedied example: > > > >> for i, j in enumerate(something): > >> # do something > >> i += 1 > >> else: > >> i = -1 > >> > >> if i > 0: > >> # we did something > > > > The last block "# we did something" is unreachable if "#do something" > > has no break. > > > > This isn't meant to go against Gerald here btw. I'm simply saying that > > if even the best Pythonistas can cause havoc with "for...else" then > > think about the noobs. Personally I would suggest not using the > > "for...break...else" idiom at all but the least we can do is give a > > warning when "for..else" is so obviously misused. > > > > You could allow "break" outside of loops and simply ignore it, but you > > don't allow it because it helps avoid bugs. Really, a warning is the > > LEAST python can do here. > > > > --yuv > > > > > > -- > Gerald Britton > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gerald.britton at gmail.com Fri Oct 9 02:12:31 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Thu, 8 Oct 2009 20:12:31 -0400 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <9d153b7c0910081709u76410704q3befbad25ce9a54d@mail.gmail.com> References: <7659cab30910070817u78094ddele0b8a08b0176cb8b@mail.gmail.com> <87ws36qtyp.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910080152u2b012e62scddfb1b466c88b6b@mail.gmail.com> <3bdda690910080251w1047d812uf2f01270e5c2e2ef@mail.gmail.com> <91ad5bf80910080707qba1491fv8250efa4384dc5c5@mail.gmail.com> <9d153b7c0910081709u76410704q3befbad25ce9a54d@mail.gmail.com> Message-ID: <5d1a32000910081712n7afb1ffcw6cee9e8e67695226@mail.gmail.com> The thing is, if you read the documentation, it is not misleading. It does just what it says it does. The problem Guido identified, is that too many folks don't read the documentation, but try to figure it out by reading examples. Small wonder they get confused! On Thu, Oct 8, 2009 at 8:09 PM, Yuvgoog Greenle wrote: > On Thu, Oct 8, 2009 at 4:07 PM, George Sakkis wrote: >> Agreed. The more people disagree on how "for/else" should be spelled, >> the more I think there is no keyword that can encapsulate what it does >> unambiguously. Certainly "then" leaves me as clueless as "else" for >> the reasons above. I was thinking of something verbose like >> "elifnotbreak", but not only it is ugly, it is not 100% accurate; the >> correct spelling would be "elifnotbreakandnotreturnandnotexception" ;) > > I agree with you on "then" giving no clues but I think you're wrong > about the spelling. The correct spelling would be "if not break". > > ? ?x = y / z > ? ?if condition: > ? ? ? ?print(1,2,3) > > The above code translated to "George Sakkis Language" should be spelled: > > ? ?x = y / x > ? ?if condition and notreturnandnotexception: > ? ? ? ?print(1,2,3) > > So because it's obvious that everything in python is under the > condition of "notreturnandnotexception", your point is moot. > > The "else" in "for...break...else" is very simply a condition tested > after the loop finished (exactly like any other "if" that would > immediately follow the loop). The condition that's tested is simple: > "if there was a break - skip this". > > So your spelling exaggeration doesn't worry me :). The fact that > python-ideas can't agree on a better syntax for python 4/5 does. I > appreciate the lesson taught, as Guido put it, but I think python can > have a long term plan concerning this old, rare and misleading syntax. > > --yuv > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From ubershmekel at gmail.com Fri Oct 9 02:21:18 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Fri, 9 Oct 2009 02:21:18 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <5d1a32000910081635l24cf5d1ex28d46e8c97bf11b1@mail.gmail.com> References: <4ACDA982.3070604@strank.info> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> <5d1a32000910081403o56ca3183t2c44a615dbd00cd2@mail.gmail.com> <9d153b7c0910081552s5aaad10brfaa58ed9a5f9e191@mail.gmail.com> <5d1a32000910081635l24cf5d1ex28d46e8c97bf11b1@mail.gmail.com> Message-ID: <9d153b7c0910081721j1521a3bav6c7e102b4e87edcf@mail.gmail.com> On Fri, Oct 9, 2009 at 1:35 AM, Gerald Britton wrote: > The point is, there are valid reasons to use this construct. ?That's > just one of many, many more. ?I would fight tooth-and-nail against a > warning message, if I thought there was even the faintest chance of it > being considered. ?As it is, there is essentially no chance that it > will be considered for the compiler (pylint is another matter, ?I > think that it _should_ be there). > So you say using "else" to structure the code is a use case, to me it's just an obfuscation, we can agree to disagree on this. I would really like to hear your other "many" reasons for using the "for..else" construct where there are no "break" statements. --yuv From debatem1 at gmail.com Fri Oct 9 02:21:32 2009 From: debatem1 at gmail.com (geremy condra) Date: Thu, 8 Oct 2009 20:21:32 -0400 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <5d1a32000910081712n7afb1ffcw6cee9e8e67695226@mail.gmail.com> References: <7659cab30910070817u78094ddele0b8a08b0176cb8b@mail.gmail.com> <87ws36qtyp.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910080152u2b012e62scddfb1b466c88b6b@mail.gmail.com> <3bdda690910080251w1047d812uf2f01270e5c2e2ef@mail.gmail.com> <91ad5bf80910080707qba1491fv8250efa4384dc5c5@mail.gmail.com> <9d153b7c0910081709u76410704q3befbad25ce9a54d@mail.gmail.com> <5d1a32000910081712n7afb1ffcw6cee9e8e67695226@mail.gmail.com> Message-ID: On Thu, Oct 8, 2009 at 8:12 PM, Gerald Britton wrote: > The thing is, if you read the documentation, it is not misleading. It > does just what it says it does. The problem Guido identified, is that > too many folks don't read the documentation, but try to figure it out > by reading examples. Small wonder they get confused! > Did you read the documentation before posting your examples? Given that you use it often enough to "fight [...] tooth and nail" over these proposals, I'm beginning to think that the construct is confusing even with the benefit of regular use. Geremy Condra > On Thu, Oct 8, 2009 at 8:09 PM, Yuvgoog Greenle > wrote: > > On Thu, Oct 8, 2009 at 4:07 PM, George Sakkis > wrote: > >> Agreed. The more people disagree on how "for/else" should be spelled, > >> the more I think there is no keyword that can encapsulate what it does > >> unambiguously. Certainly "then" leaves me as clueless as "else" for > >> the reasons above. I was thinking of something verbose like > >> "elifnotbreak", but not only it is ugly, it is not 100% accurate; the > >> correct spelling would be "elifnotbreakandnotreturnandnotexception" ;) > > > > I agree with you on "then" giving no clues but I think you're wrong > > about the spelling. The correct spelling would be "if not break". > > > > x = y / z > > if condition: > > print(1,2,3) > > > > The above code translated to "George Sakkis Language" should be spelled: > > > > x = y / x > > if condition and notreturnandnotexception: > > print(1,2,3) > > > > So because it's obvious that everything in python is under the > > condition of "notreturnandnotexception", your point is moot. > > > > The "else" in "for...break...else" is very simply a condition tested > > after the loop finished (exactly like any other "if" that would > > immediately follow the loop). The condition that's tested is simple: > > "if there was a break - skip this". > > > > So your spelling exaggeration doesn't worry me :). The fact that > > python-ideas can't agree on a better syntax for python 4/5 does. I > > appreciate the lesson taught, as Guido put it, but I think python can > > have a long term plan concerning this old, rare and misleading syntax. > > > > --yuv > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas at python.org > > http://mail.python.org/mailman/listinfo/python-ideas > > > > > > -- > Gerald Britton > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wuwei23 at gmail.com Fri Oct 9 03:58:50 2009 From: wuwei23 at gmail.com (alex23) Date: Thu, 8 Oct 2009 18:58:50 -0700 (PDT) Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> Message-ID: <9aaefe81-dbf1-4b54-bfa2-b2db7b27f268@m7g2000prd.googlegroups.com> Masklinn wrote: > The point of a warning is very much that the code works and is legal, ? > but *in most case* is not what people would want. If it's what you ? > want, you're free to go ahead and keep the code as is, ignoring the ? > warning (or even filtering it out). You could make the same case for issuing a warning every time someone uses a mutable object as a default value in a function definition. This has _certainly_ caused a lot more confusion than the loop-else construct. But recourse to naive realism isn't a valid argument here; you're in effect punishing those who have taken their time to familiarise themselves with the language to spare those who haven't from ever having to do so. The effort should be in _learning_ the language, not in continually bypassing the training wheels. From adam at atlas.st Fri Oct 9 04:02:03 2009 From: adam at atlas.st (Adam Atlas) Date: Thu, 8 Oct 2009 22:02:03 -0400 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <87ws36qtyp.fsf@uwakimon.sk.tsukuba.ac.jp> References: <7659cab30910070817u78094ddele0b8a08b0176cb8b@mail.gmail.com> <87ws36qtyp.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <0E8F3BFE-EB22-4571-B3C1-B38F5F2A271C@atlas.st> On 8 Oct 2009, at 02:29, Stephen J. Turnbull wrote: > Adam Atlas writes: >> >> On 7 Oct 2009, at 11:17, Andrey Fedorov wrote: >>> Agree with Rob that the "else" keyword confusing in the context of a >>> for loop. In my mind, "for each pebble in the bag, give it to Ben, >>> or else ..." has no clear semantic meaning. What do you mean, "or >>> else"? >> >> That wouldn't have any particular meaning in Python either (in that >> the "else" clause would never execute). > > No, the else clause (as currently defined) *always* executes. Whoops, got it backwards. And I thought I had gotten used to its meaning. :P (I swear I usually remember what it means!) From steve at pearwood.info Fri Oct 9 04:08:02 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 9 Oct 2009 13:08:02 +1100 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <9d153b7c0910080152u2b012e62scddfb1b466c88b6b@mail.gmail.com> References: <87ws36qtyp.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910080152u2b012e62scddfb1b466c88b6b@mail.gmail.com> Message-ID: <200910091308.02729.steve@pearwood.info> On Thu, 8 Oct 2009 07:52:04 pm Yuvgoog Greenle wrote: > -1 on for...then > Everything after the for loop is a "then": What "everything after"? We're talking about a block construct. It's a self-contained syntactic unit: a for line, a code suite, an optional else line + another code suite. We're not talking about additional code (which may or may not exist) outside of the block. If I say: for x in seq: pass print "spam spam spam" print is not part of the for block. It's an independent piece of code. You can move it somewhere else: print "spam spam spam" for x in seq: pass and it's still legal. But you can't do the same with a for...else loop -- you can't move the "else" before the "for". They *have* to be executed in order: first the loop, THEN the else-suite. And break jumps to the end of the entire block: both the loop itself, and the else-suite. That's all it does. > for i in SEQ: > A > # then > # code follows > > The special thing about "else" is that it's skipped upon break. > That's the *one and only* use case. > > Since the current "else" clause only tests for "if not break", You are incorrect. The else clause doesn't test *anything* -- it is an unconditional jump, to the end of the entire for-block. for...else is a *single* unit, as far as Python is concerned, it just happens that the else part is optional. >>> import dis >>> code = compile("""for x in seq: ... break ... else: ... print "no break" ... """, '', 'exec') >>> dis.dis(code) 1 0 SETUP_LOOP 20 (to 23) 3 LOAD_NAME 0 (seq) 6 GET_ITER >> 7 FOR_ITER 7 (to 17) 10 STORE_NAME 1 (x) 2 13 BREAK_LOOP 14 JUMP_ABSOLUTE 7 >> 17 POP_BLOCK 4 18 LOAD_CONST 0 ('no break') 21 PRINT_ITEM 22 PRINT_NEWLINE >> 23 LOAD_CONST 1 (None) 26 RETURN_VALUE You will notice SETUP_LOOP line. The end of the loop is the end of the else clause, not the end of the for clause. > I > think spelling it out somehow could extremely improve "else" > readability. I've gradually come to agree that "else" is an unfortunate choice of word for this keyword. It's misleading because it sounds like it executes if the loop is empty, and that's not what it does. But consider your proposal: for x in seq: ... else no break: ... It's needlessly verbose -- three words instead of one. It's easy to get confused with variants like "else break" (execute only if you *did* break), "if no break", "elif no break", "else not break", etc. If the intention is to come up with wording that people will remember, this isn't it. And lastly, it implies the existence of some magical global variable "break" which doesn't exist, and people will wonder why they can't do this: for x in seq: ... do_something() do_something_else() # now check how we exited the for loop earlier if break: print "found!" else: # or else no break: print "not found!" I guarantee that if your proposal goes ahead, folks will ask why Python doesn't expose the "break" flag outside of the for loop. The real answer is that there is no such flag, but your suggestion makes it seem like there is. -- Steven D'Aprano From steve at pearwood.info Fri Oct 9 04:10:01 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 9 Oct 2009 13:10:01 +1100 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <4ACDA982.3070604@strank.info> References: <4ACDA982.3070604@strank.info> Message-ID: <200910091310.01401.steve@pearwood.info> On Thu, 8 Oct 2009 07:57:38 pm Stefan Rank wrote: > Hi, > > a suggestion/question related to the discussion about renaming > for/else and while/else: > Is it easy to generate a SyntaxWarning if you use these constructs > without a break or return statement in the loop? AFAICS, any such > usage would be either wrong or unnecessary. (right?) -1 on such a warning. This sort of check belongs in PyLint or equivalent, there is no need to complicate the compiler and make it part of the language. The compiler should be nicely lean and simple and not try to guess what the user's intention is. for...else with no break is legal code, if there's no break inside it, it is still legal code. > This would both help prevent wrong usage because of a false intuition > and, if adequately and prominently documented, would help those > interested in learning about for/else while/else. > > Easily doable? Maybe even make it a SyntaxError? -1000 on making it a SyntaxError. SyntaxErrors are for things which are WRONG, not for things which are unnecessary or redundant or pointless. Should this be prohibited too? for x in seq: if 0: break else: print "whatever" What happens when the break is under a __debug__ test? The same source compiles to two different byte-codes, one of which has break, the other doesn't. Given this: code = compile("""for x in seq: if __debug__: break else: print "no break" """, '', 'exec') this is the byte-code you get running normally: >>> dis.dis(code) 1 0 SETUP_LOOP 20 (to 23) 3 LOAD_NAME 0 (seq) 6 GET_ITER >> 7 FOR_ITER 7 (to 17) 10 STORE_NAME 1 (x) 2 13 BREAK_LOOP 14 JUMP_ABSOLUTE 7 >> 17 POP_BLOCK 4 18 LOAD_CONST 0 ('no break') 21 PRINT_ITEM 22 PRINT_NEWLINE >> 23 LOAD_CONST 1 (None) 26 RETURN_VALUE and this is what you get running under python -O: >>> dis.dis(code) 1 0 SETUP_LOOP 19 (to 22) 3 LOAD_NAME 0 (seq) 6 GET_ITER >> 7 FOR_ITER 6 (to 16) 10 STORE_NAME 1 (x) 2 13 JUMP_ABSOLUTE 7 >> 16 POP_BLOCK 4 17 LOAD_CONST 0 ('no break') 20 PRINT_ITEM 21 PRINT_NEWLINE >> 22 LOAD_CONST 1 (None) 25 RETURN_VALUE No break in the second case. So now we have choices: * Code which runs fine normally becomes a SyntaxError when running with -O. * for...else does not require a break when running with -O, but does require it without -O. * We pile complication on top of complication and make an explicit exception for tests like if __debug__: break -- which means that we still have for loops with no break that run perfectly fine! * Stop optimizing if __debug__ tests. All four of these alternatives are unacceptable. -- Steven D'Aprano From steve at pearwood.info Fri Oct 9 04:23:24 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 9 Oct 2009 13:23:24 +1100 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> Message-ID: <200910091323.24922.steve@pearwood.info> On Thu, 8 Oct 2009 08:24:46 pm Yuvgoog Greenle wrote: > In order to understand the "else" clause it's better that you think > of it as an "if not break" after the loop. That is completely wrong. for...else is a single block, and break jumps to the end of the block. See my previous reply for details. for x in seq: whatever else: suite is a single block. It's not the same as: for x in seq: whatever suite even if they happen to execute the same way. In the first case, the entire block is one single lump of code. In the second case, you have two independent lumps of code, a for-loop with no else, followed by an independent bunch of code following it. Lumps of code in this sense are a weaker grouping than functions, classes or modules, but they are a grouping. When you see for...else you KNOW that the else-suite belongs with the for without needing to read the suite in detail. You can't pick it up and move it somewhere else and expect it to work, they belong together, in that order. The presence of the else tells the reader that the entire for...else block is one single conceptual unit. This extra information is missing in the second case, where there is nothing suggesting that suite belongs with the for-loop: they look independent. To learn that they're not, you have to read the code in enough detail to understand it fully. -- Steven D'Aprano From steve at pearwood.info Fri Oct 9 04:28:09 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 9 Oct 2009 13:28:09 +1100 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <4ACDB7F9.6030908@strank.info> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDB7F9.6030908@strank.info> Message-ID: <200910091328.09509.steve@pearwood.info> On Thu, 8 Oct 2009 08:59:21 pm Stefan Rank wrote: > In the return-but-no-break-example I had in mind:: > > ? ?def findit(bag): > ? ? ? ?for elem in bag: > ? ? ? ? ? ?if isgreat(elem): > ? ? ? ? ? ? ? ?return elem > ? ? ? ?else: > ? ? ? ? ? ?return default > > the use of else, though correct and arguably readable, is still > unnecessary. We should raise UnnecessaryNotActuallyAnError then. There is lots of code which is correct and readable while still unnecessary. Should we ban these too? mylist.extend([]) flag = True and flag if (flag is True) is True: ... while False: ... if True: ... elif False: ... [item for item in seq if True] alist.sort(key=lambda x: x) and so forth. Just think, with a bit of imagination we can slow the compiler down by a factor of two, three or even ten, by prohibiting all sorts of perfectly legal code! -- Steven D'Aprano From steve at pearwood.info Fri Oct 9 04:34:55 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 9 Oct 2009 13:34:55 +1100 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> References: <4ACDA982.3070604@strank.info> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> Message-ID: <200910091334.55398.steve@pearwood.info> On Fri, 9 Oct 2009 12:15:10 am Masklinn wrote: > The SyntaxWarning proposed would only be emitted on a `for: else:` ? > *without* a break as it's entirely equivalent to just deleting the ? > `else:` clause and dedenting the code it contains. No entirely -- the compiler generates different byte-code. > As far as outputting "spurious" messages in working programs, that's > the point of a warning isn't it? Tell the programmer that something > works, but might not do what one would like (or does nothing at all). Heavens no!!! What on earth gives you that idea? How can the compiler possibly know what you would like? And even if it could, it's the compiler's job to enforce the rules of the language, not to satisfy what people would like. Compiler warnings should never be spurious. Python warnings are used for meaningful warnings, such as deprecation warnings, not for stylistic ones like "by the way, you've used pass twice in a row, that's pointless". I don't want the compiler wasting my time with spurious warnings. If I want to run PyLint, I'll run PyLint. I don't want the compiler telling me I've defined a name and never used it, or declared a function that doesn't get called, or have code branches that can't possibly be taken, or anything else. I want it to compile, as fast as possible. The edit-compile-run cycle in Python is *very* lightweight, as it should be. When I want extra checking, I run PyLint, I don't want it every single time. -- Steven D'Aprano From steve at pearwood.info Fri Oct 9 04:44:43 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 9 Oct 2009 13:44:43 +1100 Subject: [Python-ideas] =?iso-8859-1?q?SyntaxWarning_for_for/while/else_wi?= =?iso-8859-1?q?thout_break=09or_return=3F?= In-Reply-To: <8763aprhb1.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4ACDA982.3070604@strank.info> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <8763aprhb1.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <200910091344.43676.steve@pearwood.info> On Fri, 9 Oct 2009 03:17:38 am Stephen J. Turnbull wrote: > I think I would probably be saved at least once > in my remaining years of Python programming by Nick's proposal, too. That makes no sense. If there's no difference between: for x in seq: pass else: whatever() and for x in seq: pass whatever() then what could the warning save you from? Embarrassment when people laugh at your coding style? Perhaps you're thinking of the following: for x in seq: if flag: whatever() else: # oops, I wanted this to be inside the loop something() But Nick's warning is insufficient, because it is defeated by the presence of a break *anywhere* in the loop: for x in seq: if today == tuesday: break if flag: whatever() else: # oops, I wanted this to be inside the loop something() The fact that it, occasionally, will catch a completely unrelated error (you outdented the else clause in if...else) is a fluke. -- Steven D'Aprano From steve at pearwood.info Fri Oct 9 04:53:12 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 9 Oct 2009 13:53:12 +1100 Subject: [Python-ideas] =?iso-8859-1?q?SyntaxWarning_for_for/while/else_wi?= =?iso-8859-1?q?thout_break=09or_return=3F?= In-Reply-To: <874oq9rh7s.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4ACDA982.3070604@strank.info> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> <874oq9rh7s.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <200910091353.12294.steve@pearwood.info> On Fri, 9 Oct 2009 03:19:35 am Stephen J. Turnbull wrote: > Masklinn writes: > > Both would, because in both cases the `else:` clause serves no > > purpose whatsoever. It's noise. > > Tut, tut. In Nick's proposal, both would raise because there is an > "else:" but there is no break. I doubt the Python compiler would add > the editorial comment about "noise," though. But that's exactly what it is doing, even if it doesn't use the word "noise". It's making an editorial judgement that perfectly legal code that does exactly what it is meant to do is somehow "bad" and requires a warning. If I see a message SyntaxWarning('for...else with no break'), my immediate response is "Yes, so what?". The code runs, it runs correctly. People write code incrementally. A for...else block that ends up including a break may go through many edit-compile-run cycles before the break actually gets inserted. I shouldn't have to add a spurious "if False: break" to shut the compiler up during those incremental edits. -- Steven D'Aprano From steve at pearwood.info Fri Oct 9 05:00:25 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 9 Oct 2009 14:00:25 +1100 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <5FB60732-27DD-44DC-B9A2-DD725E0F87A9@gmail.com> References: <4ACDA982.3070604@strank.info> <5d1a32000910080932x197fe67h983184674b39e20@mail.gmail.com> <5FB60732-27DD-44DC-B9A2-DD725E0F87A9@gmail.com> Message-ID: <200910091400.25423.steve@pearwood.info> On Fri, 9 Oct 2009 06:09:19 am Jared Grubb wrote: > (Just for completeness, I suppose 'yield' is a third way; but like ? > return/raise, it makes the else unnecessary) That's incorrect. Code after a yield will (in general) be executed, when you call the iterator again. E.g.: def gen(y): for x in [1, 2, 3]: yield x if y == 0: break else: yield 0 yield -1 gen(1) will yield 1, 2, 3, 0, -1. -- Steven D'Aprano From steve at pearwood.info Fri Oct 9 05:06:48 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 9 Oct 2009 14:06:48 +1100 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> References: <4ACDA982.3070604@strank.info> <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> Message-ID: <200910091406.49238.steve@pearwood.info> On Fri, 9 Oct 2009 07:31:22 am Masklinn wrote: > A warning doesn't *force* anything, it merely? warns. If you want to > ? ignore the warning, you're free to do so. Most of the time, there's > even a switch to disable (or enable, if it's not enabled by default) > the aforementioned warning. It forces me to read it. It forces me to take the time to filter out in my head real warnings for real problems, and pretend warnings for non-problems, or to run Python with some switch to disable warnings -- which means I lose the ability to get real warnings I care about. Excessive warnings trains me to ignore them. "Oh, it's some stupid compiler warning, don't worry about it, it's complaining for no good reason". (Sometimes I think some coders haven't heard of the story of The Boy Who Cried Wolf.) > The point of a warning is very much that the code works and is legal, > ? but *in most case* is not what people would want. Can you give any examples in the Python standard library where code raises a warning on such a basis? I would be shocked if you can give any examples. -- Steven D'Aprano From steve at pearwood.info Fri Oct 9 05:17:06 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 9 Oct 2009 14:17:06 +1100 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <9d153b7c0910081709u76410704q3befbad25ce9a54d@mail.gmail.com> References: <91ad5bf80910080707qba1491fv8250efa4384dc5c5@mail.gmail.com> <9d153b7c0910081709u76410704q3befbad25ce9a54d@mail.gmail.com> Message-ID: <200910091417.06410.steve@pearwood.info> On Fri, 9 Oct 2009 11:09:18 am Yuvgoog Greenle wrote: > The "else" in "for...break...else" is very simply a condition tested > after the loop finished (exactly like any other "if" that would > immediately follow the loop). The condition that's tested is simple: > "if there was a break - skip this". That is completely incorrect. That's not what the code does. break is an unconditional jump to the end of the loop block, which includes the else suite. The following Python code: for x in seq: print "ham" if x: break else: print "spam" print "done" compiles to something like the following pseudo-code: 10: x = next item in seq 20: print "ham" 30: if x: GOTO 60 40: GOTO 10 50: print "spam" 60: print "done" The only conditional test is the test "if x". -- Steven D'Aprano From george.sakkis at gmail.com Fri Oct 9 05:28:29 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Thu, 8 Oct 2009 23:28:29 -0400 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <9d153b7c0910081709u76410704q3befbad25ce9a54d@mail.gmail.com> References: <7659cab30910070817u78094ddele0b8a08b0176cb8b@mail.gmail.com> <87ws36qtyp.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910080152u2b012e62scddfb1b466c88b6b@mail.gmail.com> <3bdda690910080251w1047d812uf2f01270e5c2e2ef@mail.gmail.com> <91ad5bf80910080707qba1491fv8250efa4384dc5c5@mail.gmail.com> <9d153b7c0910081709u76410704q3befbad25ce9a54d@mail.gmail.com> Message-ID: <91ad5bf80910082028u4c88b4a2ne45a1f458a69d3c@mail.gmail.com> On Thu, Oct 8, 2009 at 8:09 PM, Yuvgoog Greenle wrote: > I agree with you on "then" giving no clues but I think you're wrong > about the spelling. The correct spelling would be "if not break". > > ? ?x = y / z > ? ?if condition: > ? ? ? ?print(1,2,3) > > The above code translated to "George Sakkis Language" should be spelled: > > ? ?x = y / x > ? ?if condition and notreturnandnotexception: > ? ? ? ?print(1,2,3) Um, false analogy. The "condition" expression either doesn't raise an expression and it is evaluated in boolean context, or it does raise an exception and there is no value to evaluate. Or to put it otherwise, if "condition" evaluates to True, the hypothetical "notreturnandnotexception" is also True by definition. Regardless, even if I accepted that we don't need to emphasize the exception/return case, I'd be -1 on abusing the "break" keyword to allow "if not break". Using three keywords to express a single case is a bad design smell IMO. > So your spelling exaggeration doesn't worry me :). The fact that > python-ideas can't agree on a better syntax for python 4/5 does. The only improvement I could hope for python 4/5 was to drop this feature altogether. It's more trouble than it's worth; it saves a couple of lines at the cost of error-proneness and head scratching when reading other people's code and wondering "did he really mean this or did he misunderstand what for/else does ?". People often reject adding a do/while syntax with the argument that it can be already expressed as a loop-and-a-half, and I bet do/while is at least as common as for/else, probably more. George From debatem1 at gmail.com Fri Oct 9 06:00:57 2009 From: debatem1 at gmail.com (geremy condra) Date: Fri, 9 Oct 2009 00:00:57 -0400 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <200910091353.12294.steve@pearwood.info> References: <4ACDA982.3070604@strank.info> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> <874oq9rh7s.fsf@uwakimon.sk.tsukuba.ac.jp> <200910091353.12294.steve@pearwood.info> Message-ID: On Thu, Oct 8, 2009 at 10:53 PM, Steven D'Aprano wrote: > On Fri, 9 Oct 2009 03:19:35 am Stephen J. Turnbull wrote: > > Masklinn writes: > > > Both would, because in both cases the `else:` clause serves no > > > purpose whatsoever. It's noise. > > > > Tut, tut. In Nick's proposal, both would raise because there is an > > "else:" but there is no break. I doubt the Python compiler would add > > the editorial comment about "noise," though. > > But that's exactly what it is doing, even if it doesn't use the > word "noise". It's making an editorial judgement that perfectly legal > code that does exactly what it is meant to do is somehow "bad" and > requires a warning. > The fact that code is currently legal (or illegal) does not stop us from making judgments about whether it should or should not be legal- see the current discussion about relaxing the restriction on decorators if you want an example. The only difference here is that instead of attempting to outright forbid that behavior, the proposal is to issue a warning notifying the programmer that they have probably screwed up, like pretty much everybody providing code examples in the course of this discussion. If I see a message SyntaxWarning('for...else with no break'), my > immediate response is "Yes, so what?". The code runs, it runs > correctly. > You're switching definitions of "what it is meant to do" from one based on the idea that the compiler is always right to one based on the idea that there's some higher definition of correctness. > People write code incrementally. A for...else block that ends up > including a break may go through many edit-compile-run cycles before > the break actually gets inserted. I shouldn't have to add a > spurious "if False: break" to shut the compiler up during those > incremental edits. > So what? Suppress the warning if you want, otherwise, listen to it, it's telling you that your code probably isn't doing what you intended it to do. That's the sort of thing I'd generally like to know while in the middle of the dev cycle. Geremy Condra -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Fri Oct 9 06:09:51 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 9 Oct 2009 15:09:51 +1100 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: References: <4ACDA982.3070604@strank.info> <200910091353.12294.steve@pearwood.info> Message-ID: <200910091509.51833.steve@pearwood.info> On Fri, 9 Oct 2009 03:00:57 pm geremy condra wrote: > The fact that code is currently legal (or illegal) does not stop us > from making judgments about whether it should or should not be > legal- Of course. *We* do. The compiler shouldn't. It should shut up and compile, not comment on my coding style. That's what PyLint is for. [...] > So what? Suppress the warning if you want, otherwise, listen > to it, it's telling you that your code probably isn't doing what > you intended it to do. I reject the position that the compiler should be guessing what I intended. The compiler should do what I say, exactly as I say it, and not editorialise with spurious warnings about things which aren't errors. Again, that's what PyLint is for. -- Steven D'Aprano From ubershmekel at gmail.com Fri Oct 9 06:58:03 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Fri, 9 Oct 2009 06:58:03 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <200910091509.51833.steve@pearwood.info> References: <4ACDA982.3070604@strank.info> <200910091353.12294.steve@pearwood.info> <200910091509.51833.steve@pearwood.info> Message-ID: <9d153b7c0910082158n77f3a6edv2ac9fbcf075338f9@mail.gmail.com> On Fri, Oct 9, 2009 at 6:09 AM, Steven D'Aprano wrote: > I reject the position that the compiler should be guessing what I > intended. The compiler should do what I say, exactly as I say it, and > not editorialise with spurious warnings about things which aren't > errors. Again, that's what PyLint is for. > So the compiler shouldn't guess you were wrong when you wrote a "break" not in a loop? Please do remember we prefer no silly limitations as much as we are against silly mistakes. And btw, concerning your "__debug__" example: that code won't raise a warning because the "__debug__: break" will pass the parsing stage and be optimized out in the code generation stage, so don't worry, it's taken care of. --yuv From ubershmekel at gmail.com Fri Oct 9 06:58:56 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Fri, 9 Oct 2009 06:58:56 +0200 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <91ad5bf80910082028u4c88b4a2ne45a1f458a69d3c@mail.gmail.com> References: <7659cab30910070817u78094ddele0b8a08b0176cb8b@mail.gmail.com> <87ws36qtyp.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910080152u2b012e62scddfb1b466c88b6b@mail.gmail.com> <3bdda690910080251w1047d812uf2f01270e5c2e2ef@mail.gmail.com> <91ad5bf80910080707qba1491fv8250efa4384dc5c5@mail.gmail.com> <9d153b7c0910081709u76410704q3befbad25ce9a54d@mail.gmail.com> <91ad5bf80910082028u4c88b4a2ne45a1f458a69d3c@mail.gmail.com> Message-ID: <9d153b7c0910082158n48957501pb3642e68f84c500c@mail.gmail.com> I'm willing to drop it as well btw. On Fri, Oct 9, 2009 at 5:28 AM, George Sakkis wrote: > On Thu, Oct 8, 2009 at 8:09 PM, Yuvgoog Greenle wrote: > >> I agree with you on "then" giving no clues but I think you're wrong >> about the spelling. The correct spelling would be "if not break". >> >> ? ?x = y / z >> ? ?if condition: >> ? ? ? ?print(1,2,3) >> >> The above code translated to "George Sakkis Language" should be spelled: >> >> ? ?x = y / x >> ? ?if condition and notreturnandnotexception: >> ? ? ? ?print(1,2,3) > > Um, false analogy. The "condition" expression either doesn't raise an > expression and it is evaluated in boolean context, or it does raise an > exception and there is no value to evaluate. Or to put it otherwise, > if "condition" evaluates to True, the hypothetical > "notreturnandnotexception" is also True by definition. > > Regardless, even if I accepted that we don't need to emphasize the > exception/return case, ?I'd be -1 on abusing the "break" keyword to > allow "if not break". Using three keywords to express a single case is > a bad design smell IMO. > >> So your spelling exaggeration doesn't worry me :). The fact that >> python-ideas can't agree on a better syntax for python 4/5 does. > > The only improvement I could hope for python 4/5 was to drop this > feature altogether. It's more trouble than it's worth; it saves a > couple of lines at the cost of error-proneness and head scratching > when reading other people's code and wondering "did he really mean > this or did he misunderstand what for/else does ?". People often > reject adding a do/while syntax with the argument that it can be > already expressed as a loop-and-a-half, and I bet do/while is at least > as common as for/else, probably more. > > George > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From zac256 at gmail.com Fri Oct 9 07:19:19 2009 From: zac256 at gmail.com (Zac Burns) Date: Thu, 8 Oct 2009 22:19:19 -0700 Subject: [Python-ideas] Decorator syntax restriction In-Reply-To: References: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> <4AC6AB82.40002@canterbury.ac.nz> Message-ID: <333edbe80910082219h28deb897vc3a61fa4add0558f@mail.gmail.com> > > I'm +0 on the whole thing, but this to me is the strongest > argument advanced thus far. Seems silly to make people > run otherwise reasonable, readable code through the > manglifier in order to satisfy a rule that doesn't work > anyway. > > Geremy Condra > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > Use case: @noDecorator if not debug else debug I'm +1 only because I tend to believe that smaller descriptions (in this case of the language) are more elegant. -- Zachary Burns (407)590-4814 Aim - Zac256FL Production Engineer (Digital Overlord) Zindagi Games From jh at improva.dk Fri Oct 9 08:24:47 2009 From: jh at improva.dk (Jacob Holm) Date: Fri, 09 Oct 2009 08:24:47 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <200910091310.01401.steve@pearwood.info> References: <4ACDA982.3070604@strank.info> <200910091310.01401.steve@pearwood.info> Message-ID: <4ACED72F.1030007@improva.dk> Steven D'Aprano wrote: > On Thu, 8 Oct 2009 07:57:38 pm Stefan Rank wrote: >> This would both help prevent wrong usage because of a false intuition >> and, if adequately and prominently documented, would help those >> interested in learning about for/else while/else. >> >> Easily doable? Maybe even make it a SyntaxError? > > -1000 on making it a SyntaxError. SyntaxErrors are for things which are > WRONG, not for things which are unnecessary or redundant or pointless. > > Should this be prohibited too? > > for x in seq: > if 0: break > else: > print "whatever" > No, see below. > > > What happens when the break is under a __debug__ test? The same source > compiles to two different byte-codes, one of which has break, the other > doesn't. If I understand Nick correctly, this is irrelevant. The warning/error generation would be handled by an earlier stage than the optimization that removes the break. - Jacob From arnodel at googlemail.com Fri Oct 9 09:22:58 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Fri, 9 Oct 2009 08:22:58 +0100 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <5d1a32000910081712n7afb1ffcw6cee9e8e67695226@mail.gmail.com> References: <7659cab30910070817u78094ddele0b8a08b0176cb8b@mail.gmail.com> <87ws36qtyp.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910080152u2b012e62scddfb1b466c88b6b@mail.gmail.com> <3bdda690910080251w1047d812uf2f01270e5c2e2ef@mail.gmail.com> <91ad5bf80910080707qba1491fv8250efa4384dc5c5@mail.gmail.com> <9d153b7c0910081709u76410704q3befbad25ce9a54d@mail.gmail.com> <5d1a32000910081712n7afb1ffcw6cee9e8e67695226@mail.gmail.com> Message-ID: <9bfc700a0910090022o4753be8eic96126f53e147745@mail.gmail.com> 2009/10/9 Gerald Britton : > The thing is, if you read the documentation, it is not misleading. ?It > does just what it says it does. ?The problem Guido identified, is that > too many folks don't read the documentation, but try to figure it out > by reading examples. ?Small wonder they get confused! I read the docs as well. In fact, I used to read the docs *every time* I used the for-else construct. To me, for-else used to be confusing because although I knew very well that the else clause was about whether the loop was broken or not, I could never remember which of the two it was. It was one of these things that I couldn't learn because although I knew the dichotomy, I couldn't figure out which half of it the 'else:' referred to. It's a bit like the difference between the noun 'bow' which describes the device that throws arrows and the noun 'bow' which refers to a movement you make in the presence of royalty. I know how to pronounce them both, I just can't remember which one is which (I'm not a native english speaker)! That's why I've come to think of for-else as break-else. It makes it clear to me that the else clause is executed when the loop *didn't* break. -- Arnaud From stephen at xemacs.org Fri Oct 9 14:13:59 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 09 Oct 2009 21:13:59 +0900 Subject: [Python-ideas] SyntaxWarning for for/while/else wi thout break or return? In-Reply-To: <200910091344.43676.steve@pearwood.info> References: <4ACDA982.3070604@strank.info> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <8763aprhb1.fsf@uwakimon.sk.tsukuba.ac.jp> <200910091344.43676.steve@pearwood.info> Message-ID: <8763aon4s8.fsf@uwakimon.sk.tsukuba.ac.jp> Steven D'Aprano writes: > That makes no sense. If there's no difference between: > > for x in seq: > pass > else: > whatever() > > and > > for x in seq: > pass > whatever() > > then what could the warning save you from? The warning can save me from the brain fart that has been repeated over and over in this thread: thinking that there *is* a difference. This thread is ample evidence that people can and do misread and miswrite that construct *even in a context where they are forewarned that they are discussing an easily misused construct*. > Perhaps you're thinking of the following: Stop trying to read my mind. What you should be reading is some of the posts in this thread explaining that for x in seq: pass else: whatever() means if seq: for x in seq: pass else: whatever() or similar ... *after* Nick's explanation that the for/break/else idiom for search is the only semantically interesting form. From stephen at xemacs.org Fri Oct 9 14:25:09 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 09 Oct 2009 21:25:09 +0900 Subject: [Python-ideas] SyntaxWarning for for/while/else wi thout break or return? In-Reply-To: <200910091353.12294.steve@pearwood.info> References: <4ACDA982.3070604@strank.info> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> <874oq9rh7s.fsf@uwakimon.sk.tsukuba.ac.jp> <200910091353.12294.steve@pearwood.info> Message-ID: <874oq8n49m.fsf@uwakimon.sk.tsukuba.ac.jp> Steven D'Aprano writes: > But that's exactly what it is doing, even if it doesn't use the > word "noise". It's making an editorial judgement that perfectly legal > code that does exactly what it is meant to do is somehow "bad" and > requires a warning. Tut, tut. You are confused. It is true that this perfectly legal code does exactly what *Guido* meant it to do. That is not a problem, as you correctly and oh so tediously insist. The problem is that it does *not* do what any number of Pythonistas in the street, as well as several posters to this very thread, meant it to do. That is a real problem, even if you do not suffer from it personally. > I shouldn't have to add a spurious "if False: break" to shut the > compiler up during those incremental edits. People who hire otherwise competent Python programmers who think they know what for...else does -- but are wrong -- shouldn't have to suffer buggy programs either. That's what warnings are for. From gerald.britton at gmail.com Fri Oct 9 15:03:14 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Fri, 9 Oct 2009 09:03:14 -0400 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <200910091323.24922.steve@pearwood.info> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <200910091323.24922.steve@pearwood.info> Message-ID: <5d1a32000910090603o6faf7179r24655644dc03c4c9@mail.gmail.com> Thanks Steve. This is what I've been trying to say all along, but you have done it more eloquently than I. On Thu, Oct 8, 2009 at 10:23 PM, Steven D'Aprano wrote: > On Thu, 8 Oct 2009 08:24:46 pm Yuvgoog Greenle wrote: >> In order to understand the "else" clause it's better that you think >> of it as an "if not break" after the loop. > > That is completely wrong. for...else is a single block, and break jumps > to the end of the block. See my previous reply for details. > > for x in seq: > ? ?whatever > else: > ? ?suite > > > is a single block. It's not the same as: > > for x in seq: > ? ?whatever > suite > > even if they happen to execute the same way. In the first case, the > entire block is one single lump of code. In the second case, you have > two independent lumps of code, a for-loop with no else, followed by an > independent bunch of code following it. > > Lumps of code in this sense are a weaker grouping than functions, > classes or modules, but they are a grouping. When you see for...else > you KNOW that the else-suite belongs with the for without needing to > read the suite in detail. You can't pick it up and move it somewhere > else and expect it to work, they belong together, in that order. The > presence of the else tells the reader that the entire for...else block > is one single conceptual unit. > > This extra information is missing in the second case, where there is > nothing suggesting that suite belongs with the for-loop: they look > independent. To learn that they're not, you have to read the code in > enough detail to understand it fully. > > > > -- > Steven D'Aprano > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From g.brandl at gmx.net Fri Oct 9 23:07:30 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 09 Oct 2009 23:07:30 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <5d1a32000910081403o56ca3183t2c44a615dbd00cd2@mail.gmail.com> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> <5d1a32000910081403o56ca3183t2c44a615dbd00cd2@mail.gmail.com> Message-ID: Gerald Britton schrieb: > Based upon Guido's post in this thread, I think the chances of getting > such a warning are between slim and none. I don't read the post that way. Changing for/else's semantics, or removing the else clause entirely won't happen, yes. But > And, I'm very happy about > that. In fact, I would be extremely unhappy if a program that has > worked for 20 years (Guido's number, not mine) would suddenly start to > spurt warnings just because I upgraded Python. Sorry, but that happens often enough, e.g. when new keywords are added. > Bad idea all round. That's a very presumptuous statement. There is quite a heavy argument in favor, namely prevention of bugs for people who misunderstand how for/else works, see your initial examples. These bugs can be nasty, because they are triggered by a corner case. An example from a different language is mixing of && and || without parens, which GCC warns about. The syntax is valid C, but I assume the warning exists because the bugs it prevents weigh more than the irritation it causes in some developers. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From masklinn at masklinn.net Fri Oct 9 23:46:42 2009 From: masklinn at masklinn.net (Masklinn) Date: Fri, 9 Oct 2009 23:46:42 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> <5d1a32000910081403o56ca3183t2c44a615dbd00cd2@mail.gmail.com> Message-ID: On 9 Oct 2009, at 23:07 , Georg Brandl wrote: > An example from a different language is mixing of && and || without > parens, > which GCC warns about. The syntax is valid C, but I assume the > warning exists > because the bugs it prevents weigh more than the irritation it > causes in > some developers. Indeed, there are numerous examples of such warnings in compilers: warning about assignments in conditional contexts (which Python forbids entirely, the ultimate hand-holding), warning about mismatches between the format specifiers and the arguments of a `printf` or `scanf`, warning about two-digit year formats, warning about incomplete enumerations, warning about storing values but never reading them, warning about allocated objects which are might not get freed, warning about potential null-pointer dereferencings, warning about overlapping patterns, warning about incomplete patterns, warning about mixing tabs and spaces? Python even has the last one already. In all of those cases, the code is legal, and it might even have been written that way on purpose. From gerald.britton at gmail.com Sat Oct 10 01:04:32 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Fri, 9 Oct 2009 19:04:32 -0400 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: References: <4ACDA982.3070604@strank.info> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> <5d1a32000910081403o56ca3183t2c44a615dbd00cd2@mail.gmail.com> Message-ID: <5d1a32000910091604w6c52d144v6ebf8ba5992c3250@mail.gmail.com> Well I guess you can read the post how you like. Based on the way Python works today, I'm sure you won't get a warning for people failing to read the documentation. In fact, Python issues few warnings at all, ever where many other compilers do. Without a significant change in philosophy, it never will. OTOH if Python did start warnings, I'd most like to see warnings for things like: 1. variables assigned but never read 2. dead code (like if False: do_something()) 3. meaningless code (like x = True or False or x == True or False) 4. try/except without any exceptions to catch 5. unused imports Still, I think the best place for these is pylint. From greg.ewing at canterbury.ac.nz Sat Oct 10 01:20:33 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 10 Oct 2009 12:20:33 +1300 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <9bfc700a0910090022o4753be8eic96126f53e147745@mail.gmail.com> References: <7659cab30910070817u78094ddele0b8a08b0176cb8b@mail.gmail.com> <87ws36qtyp.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910080152u2b012e62scddfb1b466c88b6b@mail.gmail.com> <3bdda690910080251w1047d812uf2f01270e5c2e2ef@mail.gmail.com> <91ad5bf80910080707qba1491fv8250efa4384dc5c5@mail.gmail.com> <9d153b7c0910081709u76410704q3befbad25ce9a54d@mail.gmail.com> <5d1a32000910081712n7afb1ffcw6cee9e8e67695226@mail.gmail.com> <9bfc700a0910090022o4753be8eic96126f53e147745@mail.gmail.com> Message-ID: <4ACFC541.9090108@canterbury.ac.nz> Arnaud Delobelle wrote: > It's a bit like the difference between the noun 'bow' > which describes the device that throws arrows and the noun 'bow' which > refers to a movement you make in the presence of royalty. I know how > to pronounce them both, I just can't remember which one is which (I'm > not a native english speaker)! Can you remember how to pronounce "arrow"? If so, then just remember that it's fired by something that rhymes with its last syllable. -- Greg From debatem1 at gmail.com Sat Oct 10 06:45:53 2009 From: debatem1 at gmail.com (geremy condra) Date: Sat, 10 Oct 2009 00:45:53 -0400 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <5d1a32000910091604w6c52d144v6ebf8ba5992c3250@mail.gmail.com> References: <4ACDA982.3070604@strank.info> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> <5d1a32000910081403o56ca3183t2c44a615dbd00cd2@mail.gmail.com> <5d1a32000910091604w6c52d144v6ebf8ba5992c3250@mail.gmail.com> Message-ID: On Fri, Oct 9, 2009 at 7:04 PM, Gerald Britton wrote: > Well I guess you can read the post how you like. Based on the way > Python works today, I'm sure you won't get a warning for people > failing to read the documentation. In fact, Python issues few > warnings at all, ever where many other compilers do. Without a > significant change in philosophy, it never will. > You'd do well to consider the fact that you CC'd this comment to someone with more than 2500 commits to core, and who happens to have authored our doc system to boot. He may be qualified to talk about what is or is not pythonic. Geremy Condra -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmjohnson.mailinglist at gmail.com Sat Oct 10 08:12:44 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Fri, 9 Oct 2009 20:12:44 -1000 Subject: [Python-ideas] Decorator syntax restriction In-Reply-To: <333edbe80910082219h28deb897vc3a61fa4add0558f@mail.gmail.com> References: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> <4AC6AB82.40002@canterbury.ac.nz> <333edbe80910082219h28deb897vc3a61fa4add0558f@mail.gmail.com> Message-ID: <3bdda690910092312ka23a8ccu325ef20c591a8768@mail.gmail.com> 2009/10/8 Zac Burns: > Use case: > > @noDecorator if not debug else debug > One might also write: @debug if debug else lambda f: f def f.... Is that too confusing though? My instinct says it's fine, but maybe others disagree. Still, I think the restriction should be taken off. We're all adults, etc. ? Carl From debatem1 at gmail.com Sat Oct 10 08:40:54 2009 From: debatem1 at gmail.com (geremy condra) Date: Sat, 10 Oct 2009 02:40:54 -0400 Subject: [Python-ideas] Decorator syntax restriction In-Reply-To: <3bdda690910092312ka23a8ccu325ef20c591a8768@mail.gmail.com> References: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> <4AC6AB82.40002@canterbury.ac.nz> <333edbe80910082219h28deb897vc3a61fa4add0558f@mail.gmail.com> <3bdda690910092312ka23a8ccu325ef20c591a8768@mail.gmail.com> Message-ID: On Sat, Oct 10, 2009 at 2:12 AM, Carl Johnson wrote: > 2009/10/8 Zac Burns: > >> Use case: >> >> @noDecorator if not debug else debug >> > > One might also write: > > @debug if debug else lambda f: f > def f.... > > Is that too confusing though? My instinct says it's fine, but maybe > others disagree. Still, I think the restriction should be taken off. > We're all adults, etc. > > ? Carl I'm moving towards +1 on this- the above looks very useful to me. Geremy Condra From stephen at xemacs.org Sat Oct 10 10:57:14 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 10 Oct 2009 17:57:14 +0900 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <5d1a32000910091604w6c52d144v6ebf8ba5992c3250@mail.gmail.com> References: <4ACDA982.3070604@strank.info> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> <5d1a32000910081403o56ca3183t2c44a615dbd00cd2@mail.gmail.com> <5d1a32000910091604w6c52d144v6ebf8ba5992c3250@mail.gmail.com> Message-ID: <87pr8vlj85.fsf@uwakimon.sk.tsukuba.ac.jp> Gerald Britton writes: > Well I guess you can read the post how you like. Based on the way > Python works today, I'm sure you won't get a warning for people > failing to read the documentation. The point is giving warnings for constructs that are sufficiently confusing, even having read the documentation, or typoable (in Emacs, the difference between for thing in iterable: if thing.is_good(): break else: thing = default_thing() and for thing in iterable: if thing.is_good(): break else: thing = default_thing() is one DEL when typing the code). Whether it's worth doing it for this construct is marginal to me, but I find those who advocate it to be giving plausible arguments. And you really ought to stop posting and let Steven d'Aprano carry the flag. Given *your* record of repeatedly explaining the semantics incorrectly when you have clearly read the documentation, for-else qualifies on the criterion above. The others on your list don't, especially since most of them have nearly identical semantics to the preferred form, and at least one (#4) is not currently possible to implement. From stephen at xemacs.org Sat Oct 10 13:15:26 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 10 Oct 2009 20:15:26 +0900 Subject: [Python-ideas] Decorator syntax restriction In-Reply-To: References: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> <4AC6AB82.40002@canterbury.ac.nz> <333edbe80910082219h28deb897vc3a61fa4add0558f@mail.gmail.com> <3bdda690910092312ka23a8ccu325ef20c591a8768@mail.gmail.com> Message-ID: <87my3zlctt.fsf@uwakimon.sk.tsukuba.ac.jp> geremy condra writes: > On Sat, Oct 10, 2009 at 2:12 AM, Carl Johnson > wrote: > > One might also write: > > > > @debug if debug else lambda f: f > > def f.... > > > > Is that too confusing though? My instinct says it's fine, but maybe > > others disagree. Still, I think the restriction should be taken off. > > We're all adults, etc. > > > > --Carl > > I'm moving towards +1 on this- the above looks very useful to > me. Does either of the suggestions for debug actually "work", stylistically? Specifically, consider the preparatory code required. This looks awful to me: def debug (f): return decorate_for_debug(f) debug = False # gag, retch, puke @debug if debug else lambda f: f def foo(): pass The comment may be *little* extreme, but surely that's not very beautiful in your eyes? Isn't debug = False def instrument (f): return decorate_for_debug(f) if debug else f @instrument def foo(): pass preferable? Caveat: I'm of the school that lambda isn't very beautiful itself, not in Python. From ncoghlan at gmail.com Sat Oct 10 13:22:11 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 10 Oct 2009 21:22:11 +1000 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <9aaefe81-dbf1-4b54-bfa2-b2db7b27f268@m7g2000prd.googlegroups.com> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <4ACDBC9A.2010908@gmail.com> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <5d1a32000910080644k506e1f1ci3572b9c9809f3bdf@mail.gmail.com> <325C1961-A0B6-4C58-B91F-0B1588227834@masklinn.net> <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> <9aaefe81-dbf1-4b54-bfa2-b2db7b27f268@m7g2000prd.googlegroups.com> Message-ID: <4AD06E63.2070507@gmail.com> alex23 wrote: > Masklinn wrote: >> The point of a warning is very much that the code works and is legal, >> but *in most case* is not what people would want. If it's what you >> want, you're free to go ahead and keep the code as is, ignoring the >> warning (or even filtering it out). > > You could make the same case for issuing a warning every time someone > uses a mutable object as a default value in a function definition. > This has _certainly_ caused a lot more confusion than the loop-else > construct. But recourse to naive realism isn't a valid argument here; > you're in effect punishing those who have taken their time to > familiarise themselves with the language to spare those who haven't > from ever having to do so. > > The effort should be in _learning_ the language, not in continually > bypassing the training wheels. We don't warn about mutable default arguments because in a lot of cases the compiler doesn't know if the default argument is mutable or not, and there are multiple well-established legitimate use cases that look identical to noobish mistakes. Warning about for-else usage without break is an entirely different scenario. The comparison would hold water if we were proposing to issue a warning for *any* use of for-else. We aren't - we're only proposing to issue it for cases where the lack of a break statement suggests that the programmer has misunderstood what the clause is for. Will that give the occasional false positive due to code that is in the process of being modified? Yeah, of course it will, otherwise I'd be proposing a SyntaxError. Are those occasional false positives worth the benefit of fewer buggy programs due to misunderstandings of the for/break/else construct? In my opinion, yes. For the record, there are three cases (not counting Py3k warnings) where we currently issue syntax warnings (but generate valid code anyway). 1. Using "import *" at function level (as it disables local variable optimisations): >>> def f(): ... from sys import * ... print version ... print "version" in locals() ... :1: SyntaxWarning: import * only allowed at module level >>> f() 2.5.2 (r252:60911, Jul 22 2009, 15:35:03) [GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu3)] True 2. Referencing a global variable before declaring it as such: >>> def f(): ... print x ... global x ... :3: SyntaxWarning: name 'x' is used prior to global declaration >>> f() Traceback (most recent call last): File "", line 1, in File "", line 2, in f NameError: global name 'x' is not defined 3. Assigning to a global variable before declaring it as such: >>> def f(): ... x = 1 ... global x ... :3: SyntaxWarning: name 'x' is assigned to before global declaration >>> f() >>> x 1 All 3 could easily be left to pylint/pychecker style tools, but were deemed worthy of being warned about by the compiler itself. Keep in mind that using a pylint/pychecker style tool at all is a comparatively advanced Python technique. For cases like this where misunderstandings are prevalent amongst both experienced and novice Pythoneers, getting the compiler itself to offer a helping hand still strikes me as a good idea. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Sat Oct 10 13:32:55 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 10 Oct 2009 21:32:55 +1000 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <200910091406.49238.steve@pearwood.info> References: <4ACDA982.3070604@strank.info> <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> <200910091406.49238.steve@pearwood.info> Message-ID: <4AD070E7.1010207@gmail.com> Steven D'Aprano wrote: >> The point of a warning is very much that the code works and is legal, >> but *in most case* is not what people would want. > > Can you give any examples in the Python standard library where code > raises a warning on such a basis? > > I would be shocked if you can give any examples. See my other post - there are currently 3 in the 2.x branch. Two of them are pure style warnings strongly encouraging people to put their global statements at the beginning of the function, while the third is a warning issued due to the fact that using "import *" inside a function is a really bad idea (at the very least, it forces the compiler to disable all of its normal local variable optimisations, as it has no idea what local variable names are going to exist after that import statement executes). I just checked the Py3k branch, and that adds a few more: - A warning for assert statements with a non-empty tuple as the first argument (i.e. cases where the assert will never fail) - Another two extending the global statement warnings to cover nonlocal statements as well. All 6 are cases where the code is legal and the interpreter will do the right thing, but the core developers feel people should be encouraged to write their program differently. In this thread, the proposal is that we encourage people to only use the else clause on loops containing at least one break statement. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Sat Oct 10 13:42:50 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 10 Oct 2009 21:42:50 +1000 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <200910091334.55398.steve@pearwood.info> References: <4ACDA982.3070604@strank.info> <5d1a32000910080608s902799fpb85ff2554b162598@mail.gmail.com> <479B5BB7-0189-4A8E-83F3-6F4A6D44A1BE@masklinn.net> <200910091334.55398.steve@pearwood.info> Message-ID: <4AD0733A.2000103@gmail.com> Steven D'Aprano wrote: > On Fri, 9 Oct 2009 12:15:10 am Masklinn wrote: >> The SyntaxWarning proposed would only be emitted on a `for: else:` >> *without* a break as it's entirely equivalent to just deleting the >> `else:` clause and dedenting the code it contains. > > No entirely -- the compiler generates different byte-code. I would advise against putting money on that prospect. >>> def f1(): ... for x in seq: ... pass ... else: ... 1 ... >>> def f2(): ... for x in seq: ... pass ... ... 1 ... >>> from dis import dis >>> dis(f1) 2 0 SETUP_LOOP 14 (to 17) 3 LOAD_GLOBAL 0 (seq) 6 GET_ITER >> 7 FOR_ITER 6 (to 16) 10 STORE_FAST 0 (x) 3 13 JUMP_ABSOLUTE 7 >> 16 POP_BLOCK 5 >> 17 LOAD_CONST 0 (None) 20 RETURN_VALUE >>> dis(f2) 2 0 SETUP_LOOP 14 (to 17) 3 LOAD_GLOBAL 0 (seq) 6 GET_ITER >> 7 FOR_ITER 6 (to 16) 10 STORE_FAST 0 (x) 3 13 JUMP_ABSOLUTE 7 >> 16 POP_BLOCK 5 >> 17 LOAD_CONST 0 (None) 20 RETURN_VALUE If you leave out the newline after the loop in f2 then the final line number would change from a 5 to a 4, but the bytecode would otherwise remain identical. With the extra newline in there, the bytecode *is* identical. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Sat Oct 10 13:45:46 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 10 Oct 2009 21:45:46 +1000 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <200910091323.24922.steve@pearwood.info> References: <4ACDA982.3070604@strank.info> <9d153b7c0910080224m354edb3dy7d6967370533471@mail.gmail.com> <200910091323.24922.steve@pearwood.info> Message-ID: <4AD073EA.3040004@gmail.com> Steven D'Aprano wrote: > This extra information is missing in the second case, where there is > nothing suggesting that suite belongs with the for-loop: they look > independent. To learn that they're not, you have to read the code in > enough detail to understand it fully. I can see your point, but the confusion with the naive "if seq is contains at least 1 element do the loop, else do the else clause" outweighs it in my mind. That may have something to do with my personal preference for using vertical whitespace and comment lines to obtain logical groupings below the function level. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Sat Oct 10 14:02:35 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 10 Oct 2009 22:02:35 +1000 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <4ACED72F.1030007@improva.dk> References: <4ACDA982.3070604@strank.info> <200910091310.01401.steve@pearwood.info> <4ACED72F.1030007@improva.dk> Message-ID: <4AD077DB.8070506@gmail.com> Jacob Holm wrote: > If I understand Nick correctly, this is irrelevant. The warning/error > generation would be handled by an earlier stage than the optimization > that removes the break. I never promised that - someone else said that would happen, but it's actually implementation dependent. In the case of CPython, the compiler stage checks for recognised constants (such as 0, non-zero integers, True, False and __debug__) in if statement test expressions and doesn't even bother visiting any unreachable code. Accordingly, Steven is quite correct that having your only break statement inside "if __debug__:" would lead to the natural implementation approach for CPython issuing a SyntaxWarning when running under the optimiser. Make no mistake, the whole point of adding this warning would be to cause some currently legal code to generate a warning. It would be a policy decision, stating that "for/break/else" and "while/break/else" were the only accepted use cases for the else clause and that loop cleanup code should merely be written after the loop with either comments or vertical whitespace used for logical grouping, rather than having the else clause available for that purpose. The choice to be made is whether the potential for improvement in understanding of the nature and purpose of the else clause amongst Python programmers as a whole (even those that don't use static analysis tools like pylint or pychecker) would be worth the mild aggravation to folks like Steven and Gerald that currently use the else clause to logically group cleanup code with the relevant loop. (That said, I don't think the question should even be asked of python-dev without a patch on the tracker that actually adds the new warning. I personally have several other Python issues higher on my to-do list that I have yet to find the roundtuits to address, so I'm not likely to be producing such a patch anytime soon...) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From steve at pearwood.info Sat Oct 10 14:15:07 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 10 Oct 2009 23:15:07 +1100 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <4AD06E63.2070507@gmail.com> References: <4ACDA982.3070604@strank.info> <9aaefe81-dbf1-4b54-bfa2-b2db7b27f268@m7g2000prd.googlegroups.com> <4AD06E63.2070507@gmail.com> Message-ID: <200910102315.08302.steve@pearwood.info> On Sat, 10 Oct 2009 10:22:11 pm Nick Coghlan wrote: > For the record, there are three cases (not counting Py3k warnings) > where we currently issue syntax warnings (but generate valid code > anyway). > > 1. Using "import *" at function level (as it disables local variable > optimisations): `import *` inside a function is illegal in Python3. It's a warning in Python2.x only because of backwards compatibility. This was the primary use-case for the warning module in the first place: to warn when functionality is being deprecated and will be removed in the future. Even though `import *` at the module level is a frequent source of bugs and confusion even for experienced coders, and the usual advice is Just Don't Do It, Python does not generate a warning for that case. > 2. Referencing a global variable before declaring it as such: > 3. Assigning to a global variable before declaring it as such: These are insignificant variations of the same act: using a global before declaring it. They may have two different error messages, but the underlying problem is the same. > All 3 could easily be left to pylint/pychecker style tools, but were > deemed worthy of being warned about by the compiler itself. The first could not, because it is functionality being removed. The second and third are warning about the same thing, and this is a genuine case of Python warning about a stylistic issue. -- Steven D'Aprano From cmjohnson.mailinglist at gmail.com Sat Oct 10 14:25:15 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Sat, 10 Oct 2009 02:25:15 -1000 Subject: [Python-ideas] Decorator syntax restriction In-Reply-To: <87my3zlctt.fsf@uwakimon.sk.tsukuba.ac.jp> References: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> <4AC6AB82.40002@canterbury.ac.nz> <333edbe80910082219h28deb897vc3a61fa4add0558f@mail.gmail.com> <3bdda690910092312ka23a8ccu325ef20c591a8768@mail.gmail.com> <87my3zlctt.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <3bdda690910100525n19507965he1566246ebb1cfb@mail.gmail.com> Stephen J. Turnbull: > Does either of the suggestions for debug actually "work", > stylistically? ?Specifically, consider the preparatory code required. I had some questions about that myself, but I don't think they're that deep. One could easily write something like: from debugger import debug_dec debug_flag = True #Or False @debug_dec if debug_flag else lambda f: f def my_func(arg1): ? I was only using "debug" as both a flag and a function because it was used that way in the previously given example. I see a certain elegance in using "debug" as both a flag and a function, but there's also something a little unsettling about it that others might not like. Now, a legitimate follow up question might be, why not do this, which is legal today: from debugger import debug_dec debug_flag = True #Or False if not debug_flag: debug_dec = lambda f: f @debug_dec def my_func(arg1): ? I think the counter-objection here will be, "Why should I have @debug_dec (or in Stephen's example @instrument) in front of my function if it's only sometimes a debug decorator and other times an identity function?" And I think that's a good counter-objection. The reason that we have decorators at the top of functions instead of the bottom is to improve readability. But when @debug_dec only means "sometimes this will apply a debug decorator but not necessarily" that seems very misleading to me. Why not keep the "if" at the top so it doesn't get passed over because the next programmer only read the function declaration but not the actual definition of debug_dec/instrument? At this point though, instead of debating subjective stylistic questions I really do think that something like the consenting adults clause should come into play. Why enforce debatable style decisions through arbitrary grammar restrictions? ? Carl From steve at pearwood.info Sat Oct 10 14:36:13 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 10 Oct 2009 23:36:13 +1100 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <4AD070E7.1010207@gmail.com> References: <4ACDA982.3070604@strank.info> <200910091406.49238.steve@pearwood.info> <4AD070E7.1010207@gmail.com> Message-ID: <200910102336.13786.steve@pearwood.info> On Sat, 10 Oct 2009 10:32:55 pm Nick Coghlan wrote: > Steven D'Aprano wrote: > >> The point of a warning is very much that the code works and is > >> legal, but *in most case* is not what people would want. > > > > Can you give any examples in the Python standard library where code > > raises a warning on such a basis? > > > > I would be shocked if you can give any examples. > > See my other post - there are currently 3 in the 2.x branch. See my previous response. > Two of > them are pure style warnings strongly encouraging people to put their > global statements at the beginning of the function, That's fundamentally the same warning: don't access a global before you declare it global. Whether that access is an assignment or a lookup is trivial. > while the third > is a warning issued due to the fact that using "import *" inside a > function is a really bad idea (at the very least, it forces the > compiler to disable all of its normal local variable optimisations, > as it has no idea what local variable names are going to exist after > that import statement executes). It's actually a warning that the functionality is in the process of being removed. > I just checked the Py3k branch, and that adds a few more: > > - A warning for assert statements with a non-empty tuple as the first > argument (i.e. cases where the assert will never fail) But only if the first argument is a tuple: >>> assert 1 >>> assert 1, "error" >>> assert (1, "error") :1: SyntaxWarning: assertion is always true, perhaps remove parentheses? And ridiculously: >>> assert (1, 2), "error" :1: SyntaxWarning: assertion is always true, perhaps remove parentheses? Why single out always-true tuples for a warning, when other always-true items are considered legitimate, even when there is no ambiguity about what the programmer meant? This sort of inconsistency makes little sense and should stand as a cautionary tale for people putting warnings in the compiler: sometimes the warning is *bad advice*: >>> assert 1, 2, "error" # remove the () like the compiler said File "", line 1 assert 1, 2, "error" # remove the () like the compiler said ^ SyntaxError: invalid syntax > - Another two extending the global statement warnings to cover > nonlocal statements as well. Given that Python warns about global, it makes sense to warn about nonlocal. Its all the same thing really, just the details change: whether the declaration is global or nonlocal, whether it is an assignment or a lookup, it's the same fundamental warning: don't use a non-local variable before declaring it as such. -- Steven D'Aprano From stephen at xemacs.org Sat Oct 10 16:33:34 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 10 Oct 2009 23:33:34 +0900 Subject: [Python-ideas] Decorator syntax restriction In-Reply-To: <3bdda690910100525n19507965he1566246ebb1cfb@mail.gmail.com> References: <79990c6b0910020518g1f0bc8e2ob1e2db32297b0a57@mail.gmail.com> <4AC6AB82.40002@canterbury.ac.nz> <333edbe80910082219h28deb897vc3a61fa4add0558f@mail.gmail.com> <3bdda690910092312ka23a8ccu325ef20c591a8768@mail.gmail.com> <87my3zlctt.fsf@uwakimon.sk.tsukuba.ac.jp> <3bdda690910100525n19507965he1566246ebb1cfb@mail.gmail.com> Message-ID: <87hbu7l3nl.fsf@uwakimon.sk.tsukuba.ac.jp> Carl Johnson writes: > At this point though, instead of debating subjective stylistic > questions I really do think that something like the consenting > adults clause should come into play. Why enforce debatable style > decisions through arbitrary grammar restrictions? Well, that's not the point. The grammar restriction is no longer arbitrary, it's historical. The question is there enough here to warrant relaxing the restrictions, and by how much. Since decorators can accept arguments, I don't really see a need to relax it at all. TOOWTDI, and it may as well stay that way. But that's just a -0, since I don't understand the use cases for extensions that have been presented. From ncoghlan at gmail.com Sat Oct 10 17:30:51 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 11 Oct 2009 01:30:51 +1000 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <200910102336.13786.steve@pearwood.info> References: <4ACDA982.3070604@strank.info> <200910091406.49238.steve@pearwood.info> <4AD070E7.1010207@gmail.com> <200910102336.13786.steve@pearwood.info> Message-ID: <4AD0A8AB.5060800@gmail.com> Steven D'Aprano wrote: >> while the third >> is a warning issued due to the fact that using "import *" inside a >> function is a really bad idea (at the very least, it forces the >> compiler to disable all of its normal local variable optimisations, >> as it has no idea what local variable names are going to exist after >> that import statement executes). > > It's actually a warning that the functionality is in the process of > being removed. > `import *` inside a function is illegal in Python3. It's a warning in > Python2.x only because of backwards compatibility. This was the primary > use-case for the warning module in the first place: to warn when > functionality is being deprecated and will be removed in the future. However, this particular warning has nothing to do with Py3k, as it has been present at least since the AST branch was merged back in 2005. import * at function level is genuinely harmful because of the effect it has on the compiler on top of the effect it has on readability. It makes sense to warn about it. This was a case of generating a warning for legal syntax that the compiler could handle because the developers considered it dubious. It was just considered so dubious that the warning was upgraded to a SyntaxError for Py3k. > Even though `import *` at the module level is a frequent source of bugs > and confusion even for experienced coders, and the usual advice is Just > Don't Do It, Python does not generate a warning for that case. Actually, we don't generate a warning for this because there are significant legitimate use cases for import * at the module level that can't be handled any other way (the two that immediately come to mind are for convenience at the interactive prompt and for bringing in methods and classes from an optional acceleration module as happens in a number of standard library modules). We can hardly generate a warning for constructs that we use ourselves. A visual grouping technique that can easily be handled with vertical whitespace or comments isn't even close to being in the same league as the legitimate use cases for import * at module level. > Why single out always-true tuples for a warning, when other always-true > items are considered legitimate, even when there is no ambiguity about > what the programmer meant? This sort of inconsistency makes little > sense and should stand as a cautionary tale for people putting warnings > in the compiler: sometimes the warning is *bad advice*: That's why it says "perhaps remove" rather than "remove". That hint also indicates the developer error that it is actually trying to warn against: assert (longAndComplicatedAssertionCondition, "My assertion message") That's not an assert statement using parentheses purely for line continuation - it's an assertion of a 2-tuple that will never fail, and a naive user may not realise what is happening (thus getting very confused when their assert statement appears to pass, but subsequent code relying on that assertion fails anyway). You have to either use a backslash for the line continuation here, or else ensure the comma is outside the parentheses: assert longAndComplicatedAssertionCondition, \ "My assertion message" assert longAndComplicatedAssertionCondition, ( "My assertion message") In that context, the syntax warning turns what could be a very obscure and hard to debug problem into something fairly obvious (as the only parentheses present will be the ones that are causing the comma to be misinterpreted). The warning could probably be refined such that it only triggers for a 2-tuple as the sole argument to an assert statement, but how often is such a false triggering actually going to happen in real code rather than dummy examples? For the loop else clauses, the users who are going to get themselves into trouble are the ones who *think* they know what it means, but actually don't (i.e. not the sort of people that are likely to be regularly running pylint or pychecker over their code). A SyntaxWarning built directly into the compiler would hopefully give them the hint they need in order to realise that they don't actually know what is going on. ("break? What does break have to do with using else on a loop? Hmm, maybe I better look this up or ask someone about it..."). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From arnodel at googlemail.com Sat Oct 10 18:57:44 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 10 Oct 2009 17:57:44 +0100 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <4AD0A8AB.5060800@gmail.com> References: <4ACDA982.3070604@strank.info> <200910091406.49238.steve@pearwood.info> <4AD070E7.1010207@gmail.com> <200910102336.13786.steve@pearwood.info> <4AD0A8AB.5060800@gmail.com> Message-ID: <8C59A5D2-A514-4C7E-B368-5631D4AB33AF@googlemail.com> On 10 Oct 2009, at 16:30, Nick Coghlan wrote: > > Actually, we don't generate a warning for this because there are > significant legitimate use cases for import * at the module level that > can't be handled any other way (the two that immediately come to mind > are for convenience at the interactive prompt and for bringing in > methods and classes from an optional acceleration module as happens > in a > number of standard library modules). We can hardly generate a warning > for constructs that we use ourselves. Also, it would be odd to have a language feature that generates a warning whenever it is used :) -- Arnaud From stephen at xemacs.org Sat Oct 10 20:36:07 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 11 Oct 2009 03:36:07 +0900 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <8C59A5D2-A514-4C7E-B368-5631D4AB33AF@googlemail.com> References: <4ACDA982.3070604@strank.info> <200910091406.49238.steve@pearwood.info> <4AD070E7.1010207@gmail.com> <200910102336.13786.steve@pearwood.info> <4AD0A8AB.5060800@gmail.com> <8C59A5D2-A514-4C7E-B368-5631D4AB33AF@googlemail.com> Message-ID: <87bpkfksfc.fsf@uwakimon.sk.tsukuba.ac.jp> Arnaud Delobelle writes: > Also, it would be odd to have a language feature that generates a > warning whenever it is used :) Not really. Deprecation warnings behave that way. AFAICS, a deprecation warning is just the extreme case of a syntax warning. From ubershmekel at gmail.com Sat Oct 10 22:40:02 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Sat, 10 Oct 2009 22:40:02 +0200 Subject: [Python-ideas] for/except/else syntax In-Reply-To: <4ACFC541.9090108@canterbury.ac.nz> References: <87ws36qtyp.fsf@uwakimon.sk.tsukuba.ac.jp> <9d153b7c0910080152u2b012e62scddfb1b466c88b6b@mail.gmail.com> <3bdda690910080251w1047d812uf2f01270e5c2e2ef@mail.gmail.com> <91ad5bf80910080707qba1491fv8250efa4384dc5c5@mail.gmail.com> <9d153b7c0910081709u76410704q3befbad25ce9a54d@mail.gmail.com> <5d1a32000910081712n7afb1ffcw6cee9e8e67695226@mail.gmail.com> <9bfc700a0910090022o4753be8eic96126f53e147745@mail.gmail.com> <4ACFC541.9090108@canterbury.ac.nz> Message-ID: <9d153b7c0910101340h4dd3e309xd18f4c6e3c992c53@mail.gmail.com> On Sat, Oct 10, 2009 at 1:20 AM, Greg Ewing wrote: [ snip ] > something that rhymes [ snip ] Well, we all know that poetry is a pythonic way of explaining things... After a loop, a "for" or a "while", runs the "else" along with a smile. To skip the "else" without breaking a leg, use a "break", don't boil an egg. --yuv (ps, I'd prefer the for/break/else be completely removed or renamed to something readable if-not-break or whatever. I just had a strange muse for this...) From ubershmekel at gmail.com Sat Oct 10 22:56:03 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Sat, 10 Oct 2009 22:56:03 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <4AD077DB.8070506@gmail.com> References: <4ACDA982.3070604@strank.info> <200910091310.01401.steve@pearwood.info> <4ACED72F.1030007@improva.dk> <4AD077DB.8070506@gmail.com> Message-ID: <9d153b7c0910101356m6fb5d681od7c54311cfe36364@mail.gmail.com> On Sat, Oct 10, 2009 at 2:02 PM, Nick Coghlan wrote: > (That said, I don't think the question should even be asked of > python-dev without a patch on the tracker that actually adds the new > warning. I personally have several other Python issues higher on my > to-do list that I have yet to find the roundtuits to address, so I'm not > likely to be producing such a patch anytime soon...) > I'll have time to start working on a patch within 2 weeks. I've never touched the compiler but it sounds like an interesting subject. --yuv From greg.ewing at canterbury.ac.nz Sun Oct 11 02:26:08 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 11 Oct 2009 13:26:08 +1300 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <200910102336.13786.steve@pearwood.info> References: <4ACDA982.3070604@strank.info> <200910091406.49238.steve@pearwood.info> <4AD070E7.1010207@gmail.com> <200910102336.13786.steve@pearwood.info> Message-ID: <4AD12620.3010103@canterbury.ac.nz> Steven D'Aprano wrote: >>>>assert (1, 2), "error" > > :1: SyntaxWarning: assertion is always true, perhaps remove > parentheses? > > Why single out always-true tuples for a warning, when other always-true > items are considered legitimate, even when there is no ambiguity about > what the programmer meant? I think it's a side effect of the way the check is implemented. It just looks at whether the first expression is a tuple constructor, and ignores the second one. While theoretically it's not ideal, it doesn't do any harm in practice, because it's so unlikely that anyone would intentionally write such an assert statement. There's no use for it whatsoever. >sometimes the warning is *bad advice*: > >>>>assert 1, 2, "error" # remove the () like the compiler said > > File "", line 1 > assert 1, 2, "error" # remove the () like the compiler said > ^ > SyntaxError: invalid syntax Hm, yes... well, at least you get an error rather than incorrect results! -- Greg From steve at pearwood.info Sun Oct 11 04:56:39 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 11 Oct 2009 13:56:39 +1100 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <4AD0A8AB.5060800@gmail.com> References: <4ACDA982.3070604@strank.info> <200910102336.13786.steve@pearwood.info> <4AD0A8AB.5060800@gmail.com> Message-ID: <200910111356.40784.steve@pearwood.info> On Sun, 11 Oct 2009 02:30:51 am Nick Coghlan wrote: > import * at function level is genuinely harmful because of the effect > it has on the compiler on top of the effect it has on readability. It > makes sense to warn about it. Right. It's so dangerous that it's becoming illegal. You can't just turn language features illegal overnight, you have to go through a period of deprecation. This makes perfect sense, and is the primary use-case for warnings in Python. > > Even though `import *` at the module level is a frequent source of > > bugs and confusion even for experienced coders, and the usual > > advice is Just Don't Do It, Python does not generate a warning for > > that case. > > Actually, we don't generate a warning for this because there are > significant legitimate use cases for import * at the module level > that can't be handled any other way (the two that immediately come to > mind are for convenience at the interactive prompt and for bringing > in methods and classes from an optional acceleration module as > happens in a number of standard library modules). We can hardly > generate a warning for constructs that we use ourselves. Of course we can. But we shouldn't, it would be silly. The point is that there are many language features that are pointless, silly or troublesome, and we don't -- and shouldn't -- raise warnings for them. I don't see for...else to be exceptional enough to deserve a language warning. [...] > That's not an assert statement using parentheses purely for line > continuation - it's an assertion of a 2-tuple that will never fail, > and a naive user may not realise what is happening (thus getting very > confused when their assert statement appears to pass, but subsequent > code relying on that assertion fails anyway). I don't believe the language should be hand-holding the naive user to that extent. What do we do, fill the compiler up with a thousand warnings for things which might confuse some naive user? What ever happened to "We're all adults here"? The solution to naive users is for the user to learn more about the language and be less naive, not for the language to hold his hand. The compiler isn't your babysitter, and Python isn't meant to be a teaching language for kiddies. The proliferation of warnings leads to complacently among naive users ("I don't need to understand this feature to use it, the compiler will warn me if I do something wrong") and, much worse, overload among non-naive users ("it's just another damn warning, ignore it, the compiler is just complaining about nothing"). In my opinion, we've already taken too many steps down this path, lets not go any further. > The warning could probably be refined such that it only triggers for > a 2-tuple as the sole argument to an assert statement, but how often > is such a false triggering actually going to happen in real code > rather than dummy examples? It's a bug -- it is output that is generated unnecessarily. Defending it on the basis that it's a rare bug is not a wise idea. > For the loop else clauses, the users who are going to get themselves > into trouble are the ones who *think* they know what it means, but > actually don't (i.e. not the sort of people that are likely to be > regularly running pylint or pychecker over their code). You can't protect ignorant, naive and downright stupid users against themselves. It's a never-ending problem. Perhaps we should raise a warning when the user says: for in in range(len(sequence)): SyntaxWarning: There's probably a better way to do this. or for list.insert? PerformanceWarning: This may be slow for large lists. You may think I'm exaggerating, but Python already does something similar, only worse. The built-in sum() prohibits summing strings because it may be O(N**2) -- even though in recent versions of CPython, it may not be, even though for small numbers of small strings it makes no practical difference, and even though it allows O(N**2) summation of lists! A naive user will be lulled into a false sense of security that sum(list_of_lists, []) is fine, and an advanced user who wants to use sum() as a teaching aid to show how repeated string concatenation is slow is frustrated. -- Steven D'Aprano From tjreedy at udel.edu Sun Oct 11 06:09:40 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 11 Oct 2009 00:09:40 -0400 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <4AD070E7.1010207@gmail.com> References: <4ACDA982.3070604@strank.info> <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> <200910091406.49238.steve@pearwood.info> <4AD070E7.1010207@gmail.com> Message-ID: Nick Coghlan wrote: > Steven D'Aprano wrote: >>> The point of a warning is very much that the code works and is legal, >>> but *in most case* is not what people would want. >> Can you give any examples in the Python standard library where code >> raises a warning on such a basis? >> >> I would be shocked if you can give any examples. > > See my other post - there are currently 3 in the 2.x branch. Two of them > are pure style warnings strongly encouraging people to put their global > statements at the beginning of the function, If it is perfectly legal to put global statements anywhere in the function, and there is no intention to tighten the rule to disallow that (which would be ok with me), then I am against the warning. It suggests that one has done something wrong, when one has not. A global statement as module level has no effect, even though some newbies think it does (else why would they write it when they do?). Perhaps we should add a warning for *that*, especially given that it is more likely to indicate a real error of thinking rather and simply making the 'wrong' stylistic choice. > In this thread, the proposal is that we encourage people to only use the > else clause on loops containing at least one break statement. Whereas I want to be able to use while/else and for/else in didactic code without having to warn people against a spurious warning. Terry Jan Reedy From stephen at xemacs.org Sun Oct 11 06:36:45 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 11 Oct 2009 13:36:45 +0900 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <200910111356.40784.steve@pearwood.info> References: <4ACDA982.3070604@strank.info> <200910102336.13786.steve@pearwood.info> <4AD0A8AB.5060800@gmail.com> <200910111356.40784.steve@pearwood.info> Message-ID: <871vlalf6q.fsf@uwakimon.sk.tsukuba.ac.jp> Steven D'Aprano writes: > I don't believe the language should be hand-holding the naive user to > that extent. What do we do, fill the compiler up with a thousand > warnings for things which might confuse some naive user? Nobody (except you and Gerald) is talking about things that "might" confuse "some naive user", nor does anybody advocate proliferating warnings or even getting within a light-year of that slippery slope. We're talking about one construct that *demonstrably* confuses *people who have taken it upon themselves to explain correct usage in this thread*, including at least one who considers himself above needing such warnings. We're talking about a single construct that the BDFL has deigned to deprecate as an unfortunate choice of keywords. Please stop wandering off topic, and address the arguments that have been presented. From debatem1 at gmail.com Sun Oct 11 06:58:18 2009 From: debatem1 at gmail.com (geremy condra) Date: Sun, 11 Oct 2009 00:58:18 -0400 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: References: <4ACDA982.3070604@strank.info> <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> <200910091406.49238.steve@pearwood.info> <4AD070E7.1010207@gmail.com> Message-ID: On Sun, Oct 11, 2009 at 12:09 AM, Terry Reedy wrote: > Nick Coghlan wrote: >> >> Steven D'Aprano wrote: >>>> >>>> The point of a warning is very much that the code works and is legal, >>>> ?but *in most case* is not what people would want. >>> >>> Can you give any examples in the Python standard library where code >>> raises a warning on such a basis? >>> >>> I would be shocked if you can give any examples. >> >> See my other post - there are currently 3 in the 2.x branch. Two of them >> are pure style warnings strongly encouraging people to put their global >> statements at the beginning of the function, > > If it is perfectly legal to put global statements anywhere in the function, > and there is no intention to tighten the rule to disallow that (which would > be ok with me), then I am against the warning. It suggests that one has done > something wrong, when one has not. > > A global statement as module level has no effect, even though some newbies > think it does (else why would they write it when they do?). Perhaps we > should add a warning for *that*, especially given that it is more likely to > indicate a real error of thinking rather and simply making the 'wrong' > stylistic choice. > >> In this thread, the proposal is that we encourage people to only use the >> else clause on loops containing at least one break statement. > > Whereas I want to be able to use while/else and for/else in didactic code > without having to warn people against a spurious warning. Nobody is holding a gun to your head. Use it if you must- but when a construct is obviously confusing the heck out of people who are demonstrably good programmers, it's time to think seriously about warning people who are using it in ways that *do not make good sense*. All kinds of decisions in python have been made explicitly to discourage buggy, hard to read code, and the discussion here, complete with numerous mistakes and misunderstandings, constitutes abject proof that for-else is both of those things and hard to write correctly to boot. If we can't make it less dangerous, the least we can do is make it more obvious when you're screwing it up. Geremy Condra From debatem1 at gmail.com Sun Oct 11 07:05:33 2009 From: debatem1 at gmail.com (geremy condra) Date: Sun, 11 Oct 2009 01:05:33 -0400 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <200910111356.40784.steve@pearwood.info> References: <4ACDA982.3070604@strank.info> <200910102336.13786.steve@pearwood.info> <4AD0A8AB.5060800@gmail.com> <200910111356.40784.steve@pearwood.info> Message-ID: On Sat, Oct 10, 2009 at 10:56 PM, Steven D'Aprano wrote: > On Sun, 11 Oct 2009 02:30:51 am Nick Coghlan wrote: > >> import * at function level is genuinely harmful because of the effect >> it has on the compiler on top of the effect it has on readability. It >> makes sense to warn about it. > > Right. It's so dangerous that it's becoming illegal. You can't just turn > language features illegal overnight, you have to go through a period of > deprecation. This makes perfect sense, and is the primary use-case for > warnings in Python. > > >> > Even though `import *` at the module level is a frequent source of >> > bugs and confusion even for experienced coders, and the usual >> > advice is Just Don't Do It, Python does not generate a warning for >> > that case. >> >> Actually, we don't generate a warning for this because there are >> significant legitimate use cases for import * at the module level >> that can't be handled any other way (the two that immediately come to >> mind are for convenience at the interactive prompt and for bringing >> in methods and classes from an optional acceleration module as >> happens in a number of standard library modules). We can hardly >> generate a warning for constructs that we use ourselves. > > Of course we can. But we shouldn't, it would be silly. > > The point is that there are many language features that are pointless, > silly or troublesome, and we don't -- and shouldn't -- raise warnings > for them. I don't see for...else to be exceptional enough to deserve a > language warning. > > [...] >> That's not an assert statement using parentheses purely for line >> continuation - it's an assertion of a 2-tuple that will never fail, >> and a naive user may not realise what is happening (thus getting very >> confused when their assert statement appears to pass, but subsequent >> code relying on that assertion fails anyway). > > I don't believe the language should be hand-holding the naive user to > that extent. What do we do, fill the compiler up with a thousand > warnings for things which might confuse some naive user? What ever > happened to "We're all adults here"? The solution to naive users is for > the user to learn more about the language and be less naive, not for > the language to hold his hand. The compiler isn't your babysitter, and > Python isn't meant to be a teaching language for kiddies. > > The proliferation of warnings leads to complacently among naive users > ("I don't need to understand this feature to use it, the compiler will > warn me if I do something wrong") and, much worse, overload among > non-naive users ("it's just another damn warning, ignore it, the > compiler is just complaining about nothing"). In my opinion, we've > already taken too many steps down this path, lets not go any further. > > >> The warning could probably be refined such that it only triggers for >> a 2-tuple as the sole argument to an assert statement, but how often >> is such a false triggering actually going to happen in real code >> rather than dummy examples? > > It's a bug -- it is output that is generated unnecessarily. Defending it > on the basis that it's a rare bug is not a wise idea. > > >> For the loop else clauses, the users who are going to get themselves >> into trouble are the ones who *think* they know what it means, but >> actually don't (i.e. not the sort of people that are likely to be >> regularly running pylint or pychecker over their code). > > You can't protect ignorant, naive and downright stupid users against > themselves. It's a never-ending problem. > > Perhaps we should raise a warning when the user says: > > for in in range(len(sequence)): > > SyntaxWarning: There's probably a better way to do this. > > > or for list.insert? > > PerformanceWarning: This may be slow for large lists. > > > You may think I'm exaggerating, but Python already does something > similar, only worse. The built-in sum() prohibits summing strings > because it may be O(N**2) -- even though in recent versions of CPython, > it may not be, even though for small numbers of small strings it makes > no practical difference, and even though it allows O(N**2) summation of > lists! A naive user will be lulled into a false sense of security that > sum(list_of_lists, []) is fine, and an advanced user who wants to use > sum() as a teaching aid to show how repeated string concatenation is > slow is frustrated. > > > > -- > Steven D'Aprano > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > I realize that this will strike many here as heresy, but yelling "RTFM, n00b!" is not actually a productive way to resolve what is clearly a point of difficulty for a sizeable number of competent developers. Geremy Condra From g.brandl at gmx.net Sun Oct 11 10:06:42 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 11 Oct 2009 10:06:42 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <200910102336.13786.steve@pearwood.info> References: <4ACDA982.3070604@strank.info> <200910091406.49238.steve@pearwood.info> <4AD070E7.1010207@gmail.com> <200910102336.13786.steve@pearwood.info> Message-ID: Steven D'Aprano schrieb: > And ridiculously: > >>>> assert (1, 2), "error" > :1: SyntaxWarning: assertion is always true, perhaps remove > parentheses? > > Why single out always-true tuples for a warning, when other always-true > items are considered legitimate, even when there is no ambiguity about > what the programmer meant? This sort of inconsistency makes little > sense and should stand as a cautionary tale for people putting warnings > in the compiler: sometimes the warning is *bad advice*: > >>>> assert 1, 2, "error" # remove the () like the compiler said > File "", line 1 > assert 1, 2, "error" # remove the () like the compiler said > ^ > SyntaxError: invalid syntax This would be a problem indeed if programmers were computers. Last time I looked they were humans, and capable of not blindly following any instruction the computer gives them. [1] And even if they do, they obviously don't know how the assert statement works, and need to revisit the docs. Georg [1] Note, only speaking of programmers here ;) -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Sun Oct 11 10:07:38 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 11 Oct 2009 10:07:38 +0200 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: References: <4ACDA982.3070604@strank.info> <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> <200910091406.49238.steve@pearwood.info> <4AD070E7.1010207@gmail.com> Message-ID: Terry Reedy schrieb: >> In this thread, the proposal is that we encourage people to only use the >> else clause on loops containing at least one break statement. > > Whereas I want to be able to use while/else and for/else in didactic > code without having to warn people against a spurious warning. Why would you want to do that, except to confuse a new generation of Python programmers about its semantics? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From stephen at xemacs.org Sun Oct 11 11:23:13 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 11 Oct 2009 18:23:13 +0900 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: References: <4ACDA982.3070604@strank.info> <7507480B-17A8-4D59-96D9-F63D6582E713@masklinn.net> <200910091406.49238.steve@pearwood.info> <4AD070E7.1010207@gmail.com> Message-ID: <87vdimjncu.fsf@uwakimon.sk.tsukuba.ac.jp> Georg Brandl writes: > Terry Reedy schrieb: > > Whereas I want to be able to use while/else and for/else in didactic > > code without having to warn people against a spurious warning. > Why would you want to do that, except to confuse a new generation of > Python programmers about its semantics? Sheesh. Now *you*'re doing it. He and Steven have explained their use case at length, and it's valid. Please keep to the point, which is the relative importance of the stylistic and didactic value of placing associated code inside the for-else block rather than outside it, versus the attractive nuisance that this construct is for many programmers. From steve at pearwood.info Sun Oct 11 12:01:05 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 11 Oct 2009 21:01:05 +1100 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <871vlalf6q.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4ACDA982.3070604@strank.info> <200910111356.40784.steve@pearwood.info> <871vlalf6q.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <200910112101.05991.steve@pearwood.info> On Sun, 11 Oct 2009 03:36:45 pm Stephen J. Turnbull wrote: > Steven D'Aprano writes: > > I don't believe the language should be hand-holding the naive user > > to that extent. What do we do, fill the compiler up with a > > thousand warnings for things which might confuse some naive user? > > Nobody (except you and Gerald) is talking about things that "might" > confuse "some naive user", nor does anybody advocate proliferating > warnings or even getting within a light-year of that slippery slope. > > We're talking about one construct that *demonstrably* confuses > *people who have taken it upon themselves to explain correct usage in > this thread*, including at least one who considers himself above > needing such warnings. With all respect to those people, just because somebody takes it upon themselves to explain correct usage, doesn't mean they know the correct usage. This isn't meant as a dig at anyone here, but sadly the least competent people tend to be the most sure of themselves: http://www.ncbi.nlm.nih.gov/pubmed/10626367 http://en.wikipedia.org/wiki/Illusory_superiority To quote Charles Darwin: "ignorance more frequently begets confidence than does knowledge". I don't exclude myself in this -- we all make mistakes, and those who are on comp.lang.python may have noticed me make a clanger just yesterday. Nobody here has demonstrated that for...else is a significant problem. I'd describe it as a gotcha, and Python, like every computer language, has a number of gotchas. There's no need have the compiler defensively warn people about them. > We're talking about a single construct that > the BDFL has deigned to deprecate as an unfortunate choice of > keywords. > > Please stop wandering off topic, and address the arguments that have > been presented. You're kidding, right? Or is anything but acquiescence to the idea of giving a warning "off-topic"? Just because somebody proposes a change, doesn't mean that opposition to that change is off-topic. -- Steven D'Aprano From steve at pearwood.info Sun Oct 11 13:12:59 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 11 Oct 2009 22:12:59 +1100 Subject: [Python-ideas] =?iso-8859-1?q?SyntaxWarning_for_for/while/else_wi?= =?iso-8859-1?q?thout_break=09or_return=3F?= In-Reply-To: <87bpkfksfc.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4ACDA982.3070604@strank.info> <8C59A5D2-A514-4C7E-B368-5631D4AB33AF@googlemail.com> <87bpkfksfc.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <200910112213.00141.steve@pearwood.info> On Sun, 11 Oct 2009 05:36:07 am Stephen J. Turnbull wrote: > Arnaud Delobelle writes: > > Also, it would be odd to have a language feature that generates a > > warning whenever it is used :) > > Not really. Deprecation warnings behave that way. AFAICS, a > deprecation warning is just the extreme case of a syntax warning. I don't see it that way at all. A deprecation warning makes sense, it's the primary use case for the warnings module. It says "Here is a feature which is changing, or being removed -- you MUST take steps to change your code, or be stuck with this version forever, because the language is changing." A syntax warning, as being suggested, is nothing like that. It says "Here is a legitimate feature, part of the language. It's not going away. You may have a good reason for doing what you want, but just in case you're ignorant or naive or stupid, I'm going to warn you that your perfectly legal code might be doing what something other than what you expect." This is, to my mind, particularly egregious because every single language feature might be doing something other than what you expect, if you are sufficiently ignorant, naive or stupid. Such warnings not only try to guess what users want, but they make an arbitrary, and inconsistent, decision as to how much ignorance they "protect" the user from. We don't, for example, protect the user from thinking that list.sort() returns the sorted list, or that arithmetic on floats is simple, or that locals() can be meaningfully modified, or that mutable defaults are recreated every time the function is called -- all errors FAR more common than misunderstanding for...else. If your aim is to protect users from their own mistakes, then there are far more serious mistakes that we just ignore. And if our attitude is "we're all adults here", then why on earth single out such an uncommon gotcha for special treatment? -- Steven D'Aprano From steve at pearwood.info Sun Oct 11 13:46:24 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 11 Oct 2009 22:46:24 +1100 Subject: [Python-ideas] Summary of for...else threads Message-ID: <200910112246.25168.steve@pearwood.info> Folks, I have written a summary of the multiple issues raised by the two threads on for...else (attached). Unfortunately it's rather long, but there was a lot of ground to cover. I have tried to be as fair to all positions as I am able. If I have missed anything major, please feel free to comment, but please don't start debating the issues in this thread as well. If anyone feels I have misrepresented their position, please let me know. Unless there are serious objections, I intend to post this summary to python-dev in a couple of days and ask for a ruling on the various suggestions (e.g. Yes, No, Write A PEP And It Will Be Considered). For the record, is there anyone here willing to provide patches if one or more of the proposals are accepted on python-dev? Thank you for all who have contributed. -- Steven D'Aprano -------------- next part -------------- Summary of the python-ideas threads: for/except/else syntax http://mail.python.org/pipermail/python-ideas/2009-October/005924.html SyntaxWarning for for/while/else without break or return? http://mail.python.org/pipermail/python-ideas/2009-October/006066.html Executive summary ================= The for...else and while...else syntax is one of the most under-used and misunderstood syntactic features in Python. We discuss potential changes to the behaviour and/or spelling of this feature: - remove the syntax - generate a warning if used without break - change the spelling - add additional functionality Discussion ========== Both `for` and `while` loops take an optional `else` suite, which executes if the loop completes normally, including the case of the loop not executing at all (e.g. if the `for` sequence is empty). If the loop is exited with a `break` statement, the `else` suite is skipped. (Naturally if the loop is exited with a return or by raising an exception, the `else` suite never gets a chance to execute.) In the following document, for brevity I will generally say "for...else", but everything said applies equally to while...else. The primary use-case of for...else is to implement searching: for item in sequence: if item == target: print "found" break else: print "not found" But even for this use-case, it is notable that the bisect module does not use this idiom in its implementation of binary search. According to some straw polls, even otherwise experienced and competent Python programmers are unaware of the existence, or confused about the functionality, of the `else` clause. On being presented with an example of the syntax, such programmers typically have one of three erroneous reactions: (a) The `else` clause has been outdented by mistake, or there is a missing `if` statement. (b) The `else` clause is executed if the `for` loop is not executed at all. (c) The `else` clause is executed if there is a `break`, or if there isn't, but I don't remember which. Predicting the correct behaviour seems to be extremely rare among programmers who haven't learned it. (b) and especially (c) seems to be fairly common even among programmers who say they understand the `for...else` construct. The `else` clause has another perceived problem: if there is no `break` in the loop, the `else` clause is functionally redundant. (However, they are not semantically equivalent -- see (3c) below.) That is, the following two code snippets behave identically: for item in sequence: process(item) else: other() and: for item in sequence: process(item) other() Proposals have been made that: (1) The `else` clause should be left exactly as it is. (2) The `else` keyword is misleading or confusing or both, and should be replaced. (3) The `else` clause itself is useful, but `else` without a preceding `break` is not, and so should generate an error or a warning. (4) The `else` clause should be changed to execute if the for-loop is not. (5) The `else` clause is of limited use, and can be removed altogether. (6) There should be an additional clause which is executed when the for-loop does not. Taking these proposals individually: (1) The `else` clause should be left exactly as it is. This is, naturally, the default position. Why Python tries to be easy to comprehend, the language is under no obligation to be "intuitive" to every programmer. The Zen even says "Although that way may not be obvious at first unless you're Dutch." While improved documentation is always valued, there is no consensus that `for/while...else` needs changing: even if it is a "gotcha", there's not necessarily any reason to change it. (2) The `else` keyword is misleading or confusing or both, and should be replaced. The problem many people seem to have is that the semantic connection between the keyword `else` and the behaviour of the `else` clause is tenuous at best. Perhaps the original concept was that the loop construct should be read as: execute the for-loop (or while-loop) if you reach a `break`, jump to the end of the `for...else` block else execute the `else` suite but this seems to be unintuitive to many people. One suggestion is to deprecate `else` and replace it with a key-phrase (rather than a single word) which explicitly states what is happening: for item in sequence: process(item) else no break: suite or similar variants. The advantage of this is that it is explicit about when the suite is executed. But there are disadvantages: * it requires three words rather than one; * it isn't easy to remember: it could be "else no break", "else not break", "if no break", "elif no breaks" or any of a number of other variants; * it may mislead users into imagining that `break` is a global variable, and that they can test it (or set it) outside of the loop construct; * and despite appearances, it does not describe what the Python VM actually does: there is no test. The entire for...else block is a single chunk of byte-code, and `break` jumps to the end of it: >>> code = compile("""for x in seq: ... break ... else: ... print "no break" ... """, '', 'exec') >>> dis.dis(code) 1 0 SETUP_LOOP 20 (to 23) 3 LOAD_NAME 0 (seq) 6 GET_ITER >> 7 FOR_ITER 7 (to 17) 10 STORE_NAME 1 (x) 2 13 BREAK_LOOP 14 JUMP_ABSOLUTE 7 >> 17 POP_BLOCK 4 18 LOAD_CONST 0 ('no break') 21 PRINT_ITEM 22 PRINT_NEWLINE >> 23 LOAD_CONST 1 (None) 26 RETURN_VALUE Note the line SETUP_LOOP, which jumps to 23 (beyond the `else` clause) when the BREAK_LOOP op-code is reached. For completeness, I should mention that with a slight change in syntax, programmers who want this syntax can have it right now: for item in sequence: process(item) else: # no break suite Another suggestion is to deprecate `else` and replace it with `then`: for item in sequence: process(item) then: suite This matches the way we might describe the flow control: execute the for-loop (or while-loop) then execute the `else` suite (unless you reach a break) with the understanding that a `break` jumps to the end of the entire `for...else` (for...then) block. This has two advantages: it is only one word, and it accurately describes the flow control. But the obvious disadvantage is that it requires a new keyword. Fortunately, "then" is not a verb or a noun, and consequently is unlikely to be used by much real-world code: I was not able to find it being used as an identifier in either my own code or the standard library. A third suggestion was to use the keyword "finally", but this will likely lead people to expect it to behave like try...finally, that is, to be executed even if the loop is exited by an exception or a return. (3) The `else` clause itself is useful, but `else` without a preceding `break` is not, and so should generate an error or a warning. This proposal arguably generated the most heat on the python-ideas list, with people vehemently disagreeing on whether any such warnings should be provided by the Python compiler, or left to third-party tools like PyChecker or PyLint. We can distinguish at least four positions: (3a) for...else without a break is simply wrong and should raise a SyntaxError PROS: - This would allow the Python compiler to enforce a convention which many people think is sensible. CONS: - The Python compiler enforces very few conventions. - This is a very drastic step that risks breaking existing code, and so would need to go through a deprecation period first. - It's hard (impossible?) to justify calling something an error when it in fact compiles correctly and executes as designed. - It is possible for the optimiser to remove the break in the loop (e.g. given `if __debug__: break`). This means that while the parser enforces the presence of a break, such a break isn't actually needed (and may not exist) in successfully compiled and working byte-code. - It may lead to programmers inserting meaningless breaks that can never be taken, simply to satisfy the compiler. - There's little or no evidence that for...else with no break is a significant source of bugs in real-world code. (3b) for...else without a break is pointless and programmers who use it are probably confused and the compiler should raise a warning. PROS: - The `else` clause is error-prone (e.g. accidentally dedenting the `else` from an `if...else` inside the loop). Such errors are arguably more common than deliberate use of `else` without `break`. - Some programmers are confused about when the `else` block is executed, and a warning may help them learn the feature. - There is much precedence from other languages' compilers, some of which give many warnings for things. - There is a little precedence from Python, e.g. warning when you assign or reference a global or nonlocal before defining it. CONS: - As a rule, Python's compiler is very lightweight, and it rarely gives warnings, particularly not warnings about coding standards. E.g. it does not warn if you declare a global at the module level, or if you write to locals(), even though the first is pointless and the second doesn't do as expected. - Such a warning punishes those who do understand the for...else construct and choose to use it without a `break` (see below for two possible justifications for doing so). - The compiler shouldn't try to guess what the programmer may have wanted, but should just do as instructed. - Such warnings increase the complexity of the compiler. For rare features like for...else and while...else, it isn't obvious that the benefit is worth the cost. - Excessive warnings null naive users into a false sense of security, and annoy non-naive users for no good reason. (3c) for...else without a `break` is unusual but neither wrong nor even suspect, it's a reasonable style. PROS: - A loop may go through multiple iterations of edit-compile-run. The loop may gain an `else` clause in an earlier iteration than it gains a `break`. - A for...else block is a single conceptual unit in a way that code following a loop is not. It represents a single semantic unit smaller than a function but larger than a line, and makes it obvious that the `else` suite belongs with, and follows, the `for` suite. That is, given the following code: for item in sequence: block suite without reading the code in detail it isn't obvious whether suite belongs with the loop or merely happens to follow it. But with the equivalent for...else block: for item in sequence: block else: suite there can be no doubt that they belong together as a unit. - Even if for...else without a break is silly, we're all adults here and we should allow programmers to use whatever style they see fit without penalty. (3d) for...else without a `break` is unusual enough to include in PyChecker or PyLint, but it is inappropriate for the Python compiler to warn about it. PROS: - It is a reasonable compromise position. Those who want to check for it can use a tool to do so, those who don't don't have to. - Misuse of `else` rarely produces subtle errors. It is unlikely to require heroic debugging effort to solve such mistakes. CONS: - PyChecker and PyLint are third-party software. Neither are available in the standard library, and both have a non-trivial learning curve. (4) The `else` clause should be changed to execute if the for-loop is not. When programmers unfamiliar with the `else` clause are shown the syntax and asked to predict what it will do, some expect that it will execute if the loop does not. Changing `else` to match their intuition may make the feature easier to learn. However, such a change is a major change in behaviour, and will break existing code. Consequently, even if desired by the community, it is unlikely to be implemented until Python4000, and would need to go through a long period of deprecating the existing behaviour first. (5) The `else` clause is of limited use, and can be removed altogether. The `else` clause does not have many use-cases, the primary one being searching. Removing it would simplify the syntax of loops. This too is unlikely to be implemented before Python 4000 even if desired. (6) There should be an additional clause which is executed when the for-loop does not. Such a clause is easy to do today. For lists and other sequences, this is easy with one additional test: if sequence: for item in sequence: process(item) else: print "sequence is empty" The for loop can also be moved outside of the test, if preferred. However, for iterators, the test needs a little more work: sentinel = object() item = sentinel for item in sequence: process(item) if item is sentinel: print "sequence is empty" For this reason, some people would like to see syntactic support for a clause that executes whenever the for-loop is not. This might be written as follows: for item in sequence: process(item) otherwise: print "sequence is empty" Although not strictly necessary, this might be seen as a "nice to have" syntactic sugar. The disadvantage are the introduction of a new keyword, and the increased complexity of the compiler. There is also the issue of deciding how this proposed `otherwise` block would interact with `else`: * Can a loop have both an `else` and an `otherwise` suite? * If so, do they both execute if the loop does not? * And if so, can the programmer choose in which order they execute? Conclusion ========== The default, and simplest, position to take is that nothing should change, except possibly documentation (e.g. an entry in the FAQ explaining the behaviour). Otherwise, some of the proposals could be implemented together: e.g. we could deprecate the keyword `else` in favour of an alternate spelling as well as raise a warning if there is no `break` in the loop. In any case, any change will require consensus and possibly a PEP. -- [This document is placed in the public domain.] From stephen at xemacs.org Sun Oct 11 17:33:23 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 12 Oct 2009 00:33:23 +0900 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <200910112101.05991.steve@pearwood.info> References: <4ACDA982.3070604@strank.info> <200910111356.40784.steve@pearwood.info> <871vlalf6q.fsf@uwakimon.sk.tsukuba.ac.jp> <200910112101.05991.steve@pearwood.info> Message-ID: <87tyy6j67w.fsf@uwakimon.sk.tsukuba.ac.jp> Steven D'Aprano writes: > You're kidding, right? No. > Or is anything but acquiescence to the idea of giving a warning > "off-topic"? That's not the alternative to "kidding". > Just because somebody proposes a change, doesn't mean that > opposition to that change is off-topic. "There you go again." I have no problem with opposition, and if you go back and read my posts, that's clear. It's the bogus attribution of motives and wildly exaggerated claims that need to stop, on both sides. On your side, the "proliferation of warnings" and the "warn on anything that might trip up a naive programmer who hasn't read the docs" nonsense. Neither is in question. There's *one* warning in question, the construct is legal and unambiguous as implemented. Good luck contesting those facts. So the topic is "how useful and common is the construct, correctly used?" vs. "how much code in actual use is likely to be written under the seductive spell of the meaning of 'else' in English?" (I'm open to other suggestions, within the framework of "only one warning" and "construct is unambiguous when understood.") At the moment, my assessment is "almost useless" and "it is likely that a large majority of the code using 'breakless for-else' is written with the wrong semantics expected, and a substantial fraction of those will actually result in buggy behavior." On that basis, I'm about +0.5 on issuing a warning. I don't know if my opinion matters to anybody but me but it *is* up for grabs; my current stance is not set in concrete. I can tell you that the *style* of your last five posts or so have tended to tilt my opinion against you, while Nick and Georg (except that last ad hominem, of course) have been steadily scoring a point or two per post on content. From guido at python.org Sun Oct 11 20:00:52 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 11 Oct 2009 11:00:52 -0700 Subject: [Python-ideas] SyntaxWarning for for/while/else without break or return? In-Reply-To: <871vlalf6q.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4ACDA982.3070604@strank.info> <200910102336.13786.steve@pearwood.info> <4AD0A8AB.5060800@gmail.com> <200910111356.40784.steve@pearwood.info> <871vlalf6q.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Sat, Oct 10, 2009 at 9:36 PM, Stephen J. Turnbull wrote: > We're talking about a single construct that the BDFL > has deigned to deprecate as an unfortunate choice of keywords. Wrong. I would not have the feature at all if I had to do it over. I would *not* choose another keyword. But I don't see the same level of danger in it that some here see. I am also against adding a syntax warning for this. It belongs in pylint etc. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Sun Oct 11 22:13:42 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 12 Oct 2009 06:13:42 +1000 Subject: [Python-ideas] Summary of for...else threads In-Reply-To: <200910112246.25168.steve@pearwood.info> References: <200910112246.25168.steve@pearwood.info> Message-ID: <4AD23C76.80402@gmail.com> Steven D'Aprano wrote: > Unless there are serious objections, I intend to post this summary to > python-dev in a couple of days and ask for a ruling on the various > suggestions (e.g. Yes, No, Write A PEP And It Will Be Considered). 1. Excellent summary! 2. After the discussions, my personal opinion has settled down as +1 on retaining the status quo (but welcoming doc patches), +0 on warning-if-no-break and -1 on the other options. 3. This should probably go in a PEP regardless of the reaction from python-dev. Rejected PEPs provide a good record of things we have deliberately decided *not* to do. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Sun Oct 11 22:18:25 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 12 Oct 2009 06:18:25 +1000 Subject: [Python-ideas] Summary of for...else threads In-Reply-To: <4AD23C76.80402@gmail.com> References: <200910112246.25168.steve@pearwood.info> <4AD23C76.80402@gmail.com> Message-ID: <4AD23D91.6060609@gmail.com> Nick Coghlan wrote: > Steven D'Aprano wrote: >> Unless there are serious objections, I intend to post this summary to >> python-dev in a couple of days and ask for a ruling on the various >> suggestions (e.g. Yes, No, Write A PEP And It Will Be Considered). > > 1. Excellent summary! > 2. After the discussions, my personal opinion has settled down as +1 on > retaining the status quo (but welcoming doc patches), +0 on > warning-if-no-break and -1 on the other options. > 3. This should probably go in a PEP regardless of the reaction from > python-dev. Rejected PEPs provide a good record of things we have > deliberately decided *not* to do. Having just seen Guido's latest post (i.e. he would omit the feature entirely if given the chance to redo the loop syntax and he's against adding a syntax warning), I'd suggest proposing a "Rejected PEP for the record" (with a link to Guido's post as the rejection). Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From g.brandl at gmx.net Sun Oct 11 22:48:59 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 11 Oct 2009 22:48:59 +0200 Subject: [Python-ideas] Summary of for...else threads In-Reply-To: <4AD23D91.6060609@gmail.com> References: <200910112246.25168.steve@pearwood.info> <4AD23C76.80402@gmail.com> <4AD23D91.6060609@gmail.com> Message-ID: Nick Coghlan schrieb: > Nick Coghlan wrote: >> Steven D'Aprano wrote: >>> Unless there are serious objections, I intend to post this summary to >>> python-dev in a couple of days and ask for a ruling on the various >>> suggestions (e.g. Yes, No, Write A PEP And It Will Be Considered). >> >> 1. Excellent summary! >> 2. After the discussions, my personal opinion has settled down as +1 on >> retaining the status quo (but welcoming doc patches), +0 on >> warning-if-no-break and -1 on the other options. >> 3. This should probably go in a PEP regardless of the reaction from >> python-dev. Rejected PEPs provide a good record of things we have >> deliberately decided *not* to do. > > Having just seen Guido's latest post (i.e. he would omit the feature > entirely if given the chance to redo the loop syntax and he's against > adding a syntax warning), I'd suggest proposing a "Rejected PEP for the > record" (with a link to Guido's post as the rejection). Would an entry in PEP 3099 suffice? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ncoghlan at gmail.com Sun Oct 11 23:53:05 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 12 Oct 2009 07:53:05 +1000 Subject: [Python-ideas] Summary of for...else threads In-Reply-To: References: <200910112246.25168.steve@pearwood.info> <4AD23C76.80402@gmail.com> <4AD23D91.6060609@gmail.com> Message-ID: <4AD253C1.4060900@gmail.com> Georg Brandl wrote: > Nick Coghlan schrieb: >> Nick Coghlan wrote: >>> Steven D'Aprano wrote: >>>> Unless there are serious objections, I intend to post this summary to >>>> python-dev in a couple of days and ask for a ruling on the various >>>> suggestions (e.g. Yes, No, Write A PEP And It Will Be Considered). >>> 1. Excellent summary! >>> 2. After the discussions, my personal opinion has settled down as +1 on >>> retaining the status quo (but welcoming doc patches), +0 on >>> warning-if-no-break and -1 on the other options. >>> 3. This should probably go in a PEP regardless of the reaction from >>> python-dev. Rejected PEPs provide a good record of things we have >>> deliberately decided *not* to do. >> Having just seen Guido's latest post (i.e. he would omit the feature >> entirely if given the chance to redo the loop syntax and he's against >> adding a syntax warning), I'd suggest proposing a "Rejected PEP for the >> record" (with a link to Guido's post as the rejection). > > Would an entry in PEP 3099 suffice? With links to Steven's summary and Guido's post? That would make sense (and save anyone the effort of formatting a PEP). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From steve at pearwood.info Mon Oct 12 00:25:03 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 12 Oct 2009 09:25:03 +1100 Subject: [Python-ideas] Summary of for...else threads In-Reply-To: <4AD253C1.4060900@gmail.com> References: <200910112246.25168.steve@pearwood.info> <4AD253C1.4060900@gmail.com> Message-ID: <200910120925.04745.steve@pearwood.info> On Mon, 12 Oct 2009 08:53:05 am Nick Coghlan wrote: > >> Having just seen Guido's latest post (i.e. he would omit the > >> feature entirely if given the chance to redo the loop syntax and > >> he's against adding a syntax warning), I'd suggest proposing a > >> "Rejected PEP for the record" (with a link to Guido's post as the > >> rejection). > > > > Would an entry in PEP 3099 suffice? > > With links to Steven's summary and Guido's post? That would make > sense (and save anyone the effort of formatting a PEP). I would be happy with that. -- Steven D'Aprano From stephen at xemacs.org Mon Oct 12 07:59:53 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 12 Oct 2009 14:59:53 +0900 Subject: [Python-ideas] Summary of for...else threads In-Reply-To: <200910120925.04745.steve@pearwood.info> References: <200910112246.25168.steve@pearwood.info> <4AD253C1.4060900@gmail.com> <200910120925.04745.steve@pearwood.info> Message-ID: <87my3xjgo6.fsf@uwakimon.sk.tsukuba.ac.jp> Steven D'Aprano writes: > On Mon, 12 Oct 2009 08:53:05 am Nick Coghlan wrote: > > >> Having just seen Guido's latest post (i.e. he would omit the > > >> feature entirely if given the chance to redo the loop syntax and > > >> he's against adding a syntax warning), I'd suggest proposing a > > >> "Rejected PEP for the record" (with a link to Guido's post as the > > >> rejection). > > > > > > Would an entry in PEP 3099 suffice? > > > > With links to Steven's summary and Guido's post? That would make > > sense (and save anyone the effort of formatting a PEP). > > I would be happy with that. +1 Very nice summary, by the way. Steven, you did a very good job of summarizing the facts and the logic behind these proposals, while leaving the ultimate balance up to the reader (and in the end, the BDFL). From ubershmekel at gmail.com Mon Oct 12 09:38:01 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Mon, 12 Oct 2009 09:38:01 +0200 Subject: [Python-ideas] Summary of for...else threads In-Reply-To: <87my3xjgo6.fsf@uwakimon.sk.tsukuba.ac.jp> References: <200910112246.25168.steve@pearwood.info> <4AD253C1.4060900@gmail.com> <200910120925.04745.steve@pearwood.info> <87my3xjgo6.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <9d153b7c0910120038s6f928f4ct35573d8464d79c33@mail.gmail.com> It was a very complete summary indeed. Only 1 thing - I would have liked it if the buggy examples brought to this thread were noted as evidence on the "fix for..else" parts of the summary. I also think that the threads and summary should be linked to by a PEP (3099 would suffice). This isn't the first time fists were raised concerning for..else. 2008 - "for-else" - http://209.85.229.132/search?q=cache:AA6hSopNSIYJ:mail.python.org/pipermail/python-list/2008-March/652239.html+for+else+site:mail.python.org&cd=1&hl=en&ct=clnk 2003 - "[Tutor] for/else question" - http://mail.python.org/pipermail/tutor/2003-September/025274.html 2002 - "[Tutor] while else, for else" - http://mail.python.org/pipermail/tutor/2002-October/017855.html 2000 - "else clauses in while and for loops" - http://209.85.229.132/search?q=cache:7Y1eZ5uEXScJ:mail.python.org/pipermail/python-list/2000-April/031600.html+for+else+site:mail.python.org+blake&cd=4&hl=en&ct=clnk 2000 - "History question about for .. else" - http://209.85.229.132/search?q=cache:http://mail.python.org/pipermail/python-list/2000-November/059359.html So I'm guessing we might be discussing this issue again come python 4000, --yuv From sturla at molden.no Tue Oct 13 04:00:05 2009 From: sturla at molden.no (Sturla Molden) Date: Tue, 13 Oct 2009 04:00:05 +0200 Subject: [Python-ideas] adding an __exec__ method to context managers? In-Reply-To: <4AD3DBEE.4080705@molden.no> References: <4AD3DBEE.4080705@molden.no> Message-ID: <4AD3DF25.1050708@molden.no> Sturla Molden skrev: > import pymp > > def foobar(): > > @pymp.parallel_sections > def _(mt): > > mt.section(task1) > mt.section(task2) > mt.section(task3) > > _() > This would actually be even messier if we were to emulate OpenMP completely. def foobar(): @pymp.parallel_sections def _(mt): @mt.section def _1(): task1() @mt.section def _2(): task2() @mt.section def _3(): task3() _1(); _2(); _3() _() > def foobar(): > with pymp.parallel_sections as mt: > with mt.section: > task1() > > with mt.section: > task2() > > with mt.section: > task3() > Intendation got messed up by Thunderbird :-( Another attempt: def foobar(): with pymp.parallel_sections as mt: with mt.section: task1() with mt.section: task2() with mt.section: task3() S.M. From sturla at molden.no Tue Oct 13 03:46:22 2009 From: sturla at molden.no (Sturla Molden) Date: Tue, 13 Oct 2009 03:46:22 +0200 Subject: [Python-ideas] adding an __exec__ method to context managers? Message-ID: <4AD3DBEE.4080705@molden.no> I have been trying to implement an OpenMP-like threading API for Python. For those who don't know OpenMP, it is an alternative threading API for C, C++ and Fortran, that unlike pthreads and Win32 threads has an intuitive syntax. For example, a parallel loop becomes: #pragma omp parallel for private(i) for (i=0; i References: <4AD3DBEE.4080705@molden.no> <4AD3DF25.1050708@molden.no> Message-ID: <3bdda690910122009u1af3ee6ehfe9ecf8100ba5aa1@mail.gmail.com> This is definitely an idea that python-ideas has seen before. Just a couple months ago, when the syntax of "with" was being changed to allow for "with a, b, c" this was kicked around as a possible improvement to the with statement. Taking a step back, it seems like what you really want is some easy way to create callbacks, just like Ruby blocks or the new Objective-C blocks. There are a number of ways this could be done: 1. Some kind of multiline lambda. (This is generally considered to be unpythonic.) 2. Relaxing the restrictions on decorators so that, eg. this, is legal: @pymp.parallel_for(range(100)) def _(mt): for i in mt.iter: dostuff1(i) with mt.critical: dostuff2(i) Then you can have it auto-call itself and ignore the fact that _ will be the result (presumably None) and not a callable. 3. Some sort of "out of order operation" signal, as was batted around on the list a while back: result = pymp.parallel_for( ~~DEFINE_ME_NEXT~~, range(100)) def DEFINE_ME_NEXT(mt): ... There are many potential ways to spell that. 4. Some modification to the "with" statement, as you are proposing. The resistance that you will face with this idea is that it is significantly different from how "with" works now, since it does not create a block at all. Frankly I think this list is going to face proposals for some block substitute or another every couple months between now and whenever Python finally allows for some more readable way of passing functions to other functions. ? Carl Johnson From debatem1 at gmail.com Tue Oct 13 06:47:45 2009 From: debatem1 at gmail.com (geremy condra) Date: Tue, 13 Oct 2009 00:47:45 -0400 Subject: [Python-ideas] adding an __exec__ method to context managers? In-Reply-To: <3bdda690910122009u1af3ee6ehfe9ecf8100ba5aa1@mail.gmail.com> References: <4AD3DBEE.4080705@molden.no> <4AD3DF25.1050708@molden.no> <3bdda690910122009u1af3ee6ehfe9ecf8100ba5aa1@mail.gmail.com> Message-ID: On Mon, Oct 12, 2009 at 11:09 PM, Carl Johnson wrote: > This is definitely an idea that python-ideas has seen before. Just a > couple months ago, when the syntax of "with" was being changed to > allow for "with a, b, c" this was kicked around as a possible > improvement to the with statement. > > Taking a step back, it seems like what you really want is some easy > way to create callbacks, just like Ruby blocks or the new Objective-C > blocks. There are a number of ways this could be done: > > 1. Some kind of multiline lambda. (This is generally considered to be > unpythonic.) > > 2. Relaxing the restrictions on decorators so that, eg. this, is legal: > > ?@pymp.parallel_for(range(100)) > ?def _(mt): > > ? ?for i in mt.iter: > > ? ? ? dostuff1(i) > > ? ? ? with mt.critical: > ? ? ? ? ?dostuff2(i) > > Then you can have it auto-call itself and ignore the fact that _ will > be the result (presumably None) and not a callable. > > 3. Some sort of "out of order operation" signal, as was batted around > on the list a while back: > > result = pymp.parallel_for( ~~DEFINE_ME_NEXT~~, range(100)) > def DEFINE_ME_NEXT(mt): ... > > There are many potential ways to spell that. > > 4. Some modification to the "with" statement, as you are proposing. > The resistance that you will face with this idea is that it is > significantly different from how "with" works now, since it does not > create a block at all. > > > Frankly I think this list is going to face proposals for some block > substitute or another every couple months between now and whenever > Python finally allows for some more readable way of passing functions > to other functions. > > ? Carl Johnson I agree, and I'd like to hear more about what you think ought to be done about the issue. Geremy Condra From wuwei23 at gmail.com Tue Oct 13 06:53:22 2009 From: wuwei23 at gmail.com (alex23) Date: Mon, 12 Oct 2009 21:53:22 -0700 (PDT) Subject: [Python-ideas] adding an __exec__ method to context managers? In-Reply-To: <3bdda690910122009u1af3ee6ehfe9ecf8100ba5aa1@mail.gmail.com> References: <4AD3DBEE.4080705@molden.no> <4AD3DF25.1050708@molden.no> <3bdda690910122009u1af3ee6ehfe9ecf8100ba5aa1@mail.gmail.com> Message-ID: <2b55b60a-d3f7-4c14-99de-4717fa73ffd2@h14g2000pri.googlegroups.com> Carl Johnson wrote: > Frankly I think this list is going to face proposals for some block > substitute or another every couple months between now and whenever > Python finally allows for some more readable way of passing functions > to other functions. I've never found func1(func2) to be all that unreadable... From ben+python at benfinney.id.au Tue Oct 13 08:22:05 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 13 Oct 2009 17:22:05 +1100 Subject: [Python-ideas] adding an __exec__ method to context managers? References: <4AD3DBEE.4080705@molden.no> <4AD3DF25.1050708@molden.no> <3bdda690910122009u1af3ee6ehfe9ecf8100ba5aa1@mail.gmail.com> Message-ID: <87hbu3reya.fsf@benfinney.id.au> Carl Johnson writes: > Taking a step back, it seems like what you really want is some easy > way to create callbacks, just like Ruby blocks or the new Objective-C > blocks. There are a number of ways this could be done: [?] Unless I misunderstand one or more of the options you present, you've omitted the most obvious way under current Python: create a function using ?def? and use the name to refer to that function object. > Frankly I think this list is going to face proposals for some block > substitute or another every couple months between now and whenever > Python finally allows for some more readable way of passing functions > to other functions. What is insufficiently readable about:: def foo(spam): process_wibble(spam) process_wobble(spam) return process_warble(spam) bar(foo) -- \ ?I went to a garage sale. ?How much for the garage?? ?It's not | `\ for sale.?? ?Steven Wright | _o__) | Ben Finney From cmjohnson.mailinglist at gmail.com Tue Oct 13 09:00:30 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Mon, 12 Oct 2009 21:00:30 -1000 Subject: [Python-ideas] adding an __exec__ method to context managers? In-Reply-To: <87hbu3reya.fsf@benfinney.id.au> References: <4AD3DBEE.4080705@molden.no> <4AD3DF25.1050708@molden.no> <3bdda690910122009u1af3ee6ehfe9ecf8100ba5aa1@mail.gmail.com> <87hbu3reya.fsf@benfinney.id.au> Message-ID: <3bdda690910130000k424b2e0exa7654b17334ceee8@mail.gmail.com> Ben Finney: > What is insufficiently readable about:: > > ?def foo(spam): > ? ? ?process_wibble(spam) > ? ? ?process_wobble(spam) > ? ? ?return process_warble(spam) > > ?bar(foo) The problem with that is in 3 or 4 months someone will come back to Python-ideas with another theory of how to replace it. More seriously, it puts things out of order. If you had to write for-loops as def loop(item): process(item), etc. for(loop, iterator) it would be patently obvious that the for(loop, iterator) belongs at the top, not the bottom, so that you know *what* is being iterated before you find out *how* it's being iterated. For that matter, Python has changed the perfectly sensible: def method(cls): stuff() method = classmethod(method) to @classmethod def method(cls): stuff() Why? Because it's more readable to have the decorator up top, so you know what kind of function/method to expect. So, in this particular case, I think it's more readable to have the conditions at the top instead of the bottom. It's very natural when something that starts as for i in range(100): dostuff1(i) # thread safe function dostuff2(i) # not thread safe function becomes with parallelize(range(100)) as i: dostuff1(i) # thread safe function dostuff2(i) # not thread safe function (or some other way of writing the condition at the top instead of, such as a decorator etc.) instead of def f(i): dostuff1(i) # thread safe function dostuff2(i) # not thread safe function parallelize(f, range(100)) with the loop condition at the bottom. For whatever reason, people find putting the conditions out of order onerous, and Python-ideas won't be free of periodic interruptions until there someway to conditions into their mental order. ? Carl From stephen at xemacs.org Tue Oct 13 09:22:31 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 13 Oct 2009 16:22:31 +0900 Subject: [Python-ideas] adding an __exec__ method to context managers? In-Reply-To: <3bdda690910122009u1af3ee6ehfe9ecf8100ba5aa1@mail.gmail.com> References: <4AD3DBEE.4080705@molden.no> <4AD3DF25.1050708@molden.no> <3bdda690910122009u1af3ee6ehfe9ecf8100ba5aa1@mail.gmail.com> Message-ID: <87my3vhi6g.fsf@uwakimon.sk.tsukuba.ac.jp> Carl Johnson writes: > Frankly I think this list is going to face proposals for some block > substitute or another every couple months between now and whenever > Python finally allows for some more readable way of passing functions > to other functions. I suspect all the blockheads will switch to Ruby before Python gets such a facility, given that lambda itself is considered an unfortunate un-Pythonic legacy by many, and lambda-with-suite quite beyond the pale. This use of with is the most plausible I've seen, though, I have to admit that. From ben+python at benfinney.id.au Tue Oct 13 09:25:25 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 13 Oct 2009 18:25:25 +1100 Subject: [Python-ideas] adding an __exec__ method to context managers? References: <4AD3DBEE.4080705@molden.no> <4AD3DF25.1050708@molden.no> <3bdda690910122009u1af3ee6ehfe9ecf8100ba5aa1@mail.gmail.com> <87hbu3reya.fsf@benfinney.id.au> <3bdda690910130000k424b2e0exa7654b17334ceee8@mail.gmail.com> Message-ID: <87d44rrc0q.fsf@benfinney.id.au> Carl Johnson writes: > So, in this particular case, I think it's more readable to have the > conditions at the top instead of the bottom. I'm not understanding ?conditions? here. When thinking about programming languages, a ?condition? is an expression evaluated in a boolean context. You seem to mean something different. > It's very natural when something that starts as > > for i in range(100): > dostuff1(i) # thread safe function > dostuff2(i) # not thread safe function > > becomes > > with parallelize(range(100)) as i: > dostuff1(i) # thread safe function > dostuff2(i) # not thread safe function > > (or some other way of writing the condition at the top instead of, > such as a decorator etc.) instead of > > def f(i): > dostuff1(i) # thread safe function > dostuff2(i) # not thread safe function > > parallelize(f, range(100)) > > with the loop condition at the bottom. I'm not seeing how you got from the ?start as? case to the latter cases. Where is the loop condition? -- \ ?Whatever a man prays for, he prays for a miracle. Every prayer | `\ reduces itself to this: ?Great God, grant that twice two be not | _o__) four.?? ?Ivan Turgenev | Ben Finney From stefan_ml at behnel.de Tue Oct 13 10:39:11 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 13 Oct 2009 10:39:11 +0200 Subject: [Python-ideas] adding an __exec__ method to context managers? In-Reply-To: <4AD3DBEE.4080705@molden.no> References: <4AD3DBEE.4080705@molden.no> Message-ID: Sturla Molden wrote: > I have been trying to implement an OpenMP-like threading API for Python. This need has (obviously) also been discussed on the Cython list and lead to this write-up: http://wiki.cython.org/enhancements/parallel and this ticket: http://trac.cython.org/cython_trac/ticket/211 The fact that this isn't an easy thing to decide (nor a major need, it seems) is reflected by the age of the Wiki page (June 2008, still undecided) and the amount of similar discussions on c.l.py and cython-dev/cython-ideas. I actually think that OpenMP support makes a lot more sense for Cython code (which can happily free the GIL at any granularity) than for Python code. Stefan From solipsis at pitrou.net Tue Oct 13 14:45:45 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 13 Oct 2009 12:45:45 +0000 (UTC) Subject: [Python-ideas] Python threads are perfectly fine References: <4AD3DBEE.4080705@molden.no> Message-ID: Sturla Molden writes: > > P.S. Yes I know about the GIL. It can be released by C extensions (incl. > Cython/Pyrex, f2py, ctypes). Python threads are perfectly fine for > course-grained parallelism in numerical code, with scalability and > performance close to GCC's OpenMP (I have tried). That's a very refreshing thing to hear, thanks :-) From jimjjewett at gmail.com Tue Oct 13 15:03:03 2009 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 13 Oct 2009 09:03:03 -0400 Subject: [Python-ideas] adding an __exec__ method to context managers? In-Reply-To: <3bdda690910122009u1af3ee6ehfe9ecf8100ba5aa1@mail.gmail.com> References: <4AD3DBEE.4080705@molden.no> <4AD3DF25.1050708@molden.no> <3bdda690910122009u1af3ee6ehfe9ecf8100ba5aa1@mail.gmail.com> Message-ID: On Mon, Oct 12, 2009 at 11:09 PM, Carl Johnson wrote: > Taking a step back, it seems like what you really want is some easy > way to create callbacks, just like Ruby blocks or the new Objective-C > blocks. There are a number of ways this could be done: Hmm... I saw this as bigger than a block; it was creating a full execution context, not just just a suite executed in a tweak of the current context. But I may have been reading too much into it. Would the __exec__ have its own globals and locals (which might default to a reference to or copy of the current ones)? > 4. Some modification to the "with" statement, as you are proposing. > The resistance that you will face with this idea is that it is > significantly different from how "with" works now, since it does not > create a block at all. To me, "with" says: "Take the current execution context, tweak it, run the following suite, then untweak the context". An __exec__ just strengthens the possible separation between the inner and and outer contexts -- it may be slightly less efficient, but in return, it will be a better sandbox. OpenMP would be a special type of __exec__ that also happens to handle parallelization for you. -jJ From sturla at molden.no Tue Oct 13 16:36:07 2009 From: sturla at molden.no (Sturla Molden) Date: Tue, 13 Oct 2009 16:36:07 +0200 Subject: [Python-ideas] adding an __exec__ method to context managers? In-Reply-To: References: <4AD3DBEE.4080705@molden.no> Message-ID: <4AD49057.7070607@molden.no> Stefan Behnel skrev: > I actually think that OpenMP support makes a lot more sense for Cython code > (which can happily free the GIL at any granularity) than for Python code. > > I am not talking about OpenMP support, I am talking about how we use Python threads. Another use case, BTW, which someone mailed me, is an improved sandbox for restricted execution. S.M. From stefan_ml at behnel.de Tue Oct 13 16:55:38 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 13 Oct 2009 16:55:38 +0200 Subject: [Python-ideas] adding an __exec__ method to context managers? In-Reply-To: <4AD49057.7070607@molden.no> References: <4AD3DBEE.4080705@molden.no> <4AD49057.7070607@molden.no> Message-ID: Sturla Molden wrote: > Stefan Behnel skrev: >> I actually think that OpenMP support makes a lot more sense for Cython >> code >> (which can happily free the GIL at any granularity) than for Python code. >> > I am not talking about OpenMP support, I am talking about how we use > Python threads. Fine. Same Syntax proposals apply. > Another use case, BTW, which someone mailed me, is an improved sandbox > for restricted execution. I would really prefer if this was based on a function rather than an arbitrary code block. Stefan From sturla at molden.no Tue Oct 13 17:13:49 2009 From: sturla at molden.no (Sturla Molden) Date: Tue, 13 Oct 2009 17:13:49 +0200 Subject: [Python-ideas] Python threads are perfectly fine In-Reply-To: References: <4AD3DBEE.4080705@molden.no> Message-ID: <4AD4992D.7090808@molden.no> Antoine Pitrou skrev: > Sturla Molden writes: > >> P.S. Yes I know about the GIL. It can be released by C extensions (incl. >> Cython/Pyrex, f2py, ctypes). Python threads are perfectly fine for >> course-grained parallelism in numerical code, with scalability and >> performance close to GCC's OpenMP (I have tried). >> > > That's a very refreshing thing to hear, thanks :-) > > I could show you a test I did on my laptop (dual core) a while ago: http://folk.uio.no/sturlamo/kdtree/benchmark-27022009.png The black line is scipy.spatial.cKDTree (Cython). The green line is scipy.spatial.cKDTree modified to use Python threads (GIL released whenever possible). The red line is scipy.spatial.cKDTree with some parts re-written in C, and using OpenMP. This is hardly surprising, as Python threads are just native OS threads. The slightly reduced performance of Python threads probably comes from contention for the GIL in parts of the Cython code. At least for numerical code, the heavy lifiting is done in special C and Fortran libraries such as ATLAS/LAPACK, Intel MKL, FFTW, and MINPACK. Even for code we write completely ourselves, there will always be some performance critical parts in Cython, C/C++ or Fortran. We can thus release the GIL around the worst bottlenecks, and use multi-threading in Python. The GIL becomes an issue if you never release it. S.M. From ryan.freckleton at gmail.com Tue Oct 13 17:27:04 2009 From: ryan.freckleton at gmail.com (Ryan Freckleton) Date: Tue, 13 Oct 2009 09:27:04 -0600 Subject: [Python-ideas] adding an __exec__ method to context managers? In-Reply-To: <4AD3DBEE.4080705@molden.no> References: <4AD3DBEE.4080705@molden.no> Message-ID: <318072440910130827i43d693am68980dce34f2b806@mail.gmail.com> On Mon, Oct 12, 2009 at 7:46 PM, Sturla Molden wrote: > > I have been trying to implement an OpenMP-like threading API for Python. For > those who don't know OpenMP, it is an alternative threading API for C, C++ > and Fortran, that unlike pthreads and Win32 threads has an intuitive syntax. > > The virtue is that one can take sequential code, add in a few pragmas here > and there, and end up having a multi-threaded program a human can > understand. All the mess that makes multi-threaded programs error-prone and > difficult to write is taken care of by the compiler. > > Ok, so what has this to do with Python? > > First Python already has the main machinery for OpenMP-like syntax using > closures and decorators. For example, if we have a seqential function: > Now we could e.g. imagine using this syntax instead, using __exec__ to > control execution in threads: > > def foobar(): > > ? with pymp.parallel_for( range(100) ) as mt: > ? ? ? for i in mt.iter: > > ? ? ? ? ? dostuff1(i) > > ? ? ? ? ? with mt.critical: > ? ? ? ? ? ? ? dostuff2(i) > > > def foobar(): > ? ? ? with pymp.parallel_sections as mt: > ? ? ? ? ? with mt.section: > ? ? ? ? ? task1() > > ? ? ? with mt.section: > ? ? ? ? ? task2() > > ? ? ? with mt.section: > ? ? ? ? ? task3() > > Have you looked at the API for python-safethread [http://code.google.com/p/python-safethread/wiki/Branching]? I think an API combining that and your semantics would be very cool. If you use the same technique of passing functions into the context manager, your examples become: def foobar(): with pymp.parallel_for(range(100)) as mt: for i in mt.iter: mt.add(lambda : dostuff1(i)) mt.critical(lambda : dostuff2(i)) def foobar(): with pymp.parallel_sections as mt: mt.section(task1) mt.section(task2) mt.section(task3) Another option would be to execute the closure under the covers, e.g. # The function dostuff_in_parallel is called when necessary by the pymp object. @pymp.parallel_for(range(100)) def dostuff_in_parallel(mt): dostuff1(i) with mt.critical: dostuff2(i) # The function sections is called when necessary by the pymp object, e.g. by the parallel_sections decorator. @pymp.parallel_sections def sections(mt): mt.section(task1) mt.section(task2) mt.section(task3) There are a couple PEPs about adding blocks to python that were rejected in favor of the more constrained with statement, so you may want to look at those as well. Cheers, ===== --Ryan E. Freckleton From sturla at molden.no Tue Oct 13 17:31:51 2009 From: sturla at molden.no (Sturla Molden) Date: Tue, 13 Oct 2009 17:31:51 +0200 Subject: [Python-ideas] adding an __exec__ method to context managers? In-Reply-To: <4AD49057.7070607@molden.no> References: <4AD3DBEE.4080705@molden.no> <4AD49057.7070607@molden.no> Message-ID: <4AD49D67.7080405@molden.no> Sturla Molden skrev: > I am not talking about OpenMP support, I am talking about how we use > Python threads. Which is to say that Java threads, which Python's threading module mimics, is a bad concurrency abstraction. It is error prone and difficult to use (does not fit the programmer's mind). Another ting is that some of this is to some extent a compensation for lack of os.fork in Windows. On Linux, one could call os.fork in __enter__, and achieve much the same effect as an __exec__ method. That is, the parent forks, raises an exception (jumps to __exit__), and calls os.waitpid there. The child executes the "with ctxmgr:" block, and calls sys.exit (or os._exit) in __exit__. That kind of scheme would not work on Windows (well there is Cygwin and SUA...) It would also be somewhat limited by the child and parent not sharing memory space. So I still think there is justification for a __exec__ method in context managers. By the way: I tried to implement this using a bytecode hack a while ago. That is, spawn a thread and execute the code object from the calling stack frame in __enter__. It failed whenever the with block contained a loop, as there was some variable (I think it was called _1) that the interpreter could not find. And bytecode hacks are not particularly reliable and portable either. S.M. From sturla at molden.no Tue Oct 13 17:57:08 2009 From: sturla at molden.no (Sturla Molden) Date: Tue, 13 Oct 2009 17:57:08 +0200 Subject: [Python-ideas] adding an __exec__ method to context managers? In-Reply-To: <318072440910130827i43d693am68980dce34f2b806@mail.gmail.com> References: <4AD3DBEE.4080705@molden.no> <318072440910130827i43d693am68980dce34f2b806@mail.gmail.com> Message-ID: <4AD4A354.8000105@molden.no> Ryan Freckleton skrev: > # The function dostuff_in_parallel is called when necessary by the > pymp object. > @pymp.parallel_for(range(100)) > def dostuff_in_parallel(mt): > dostuff1(i) > with mt.critical: > dostuff2(i) > I am beginning to feel a little stupid. :-( Right... a decorator would always be called for a closure, not just once on module import. So I could make function "def parallel_for(iterable)" return a decorator and auto-execute the closure. That would indeed give cleaner decorator syntax instead of calling the decorated closure manually. S.M. From bruce at leapyear.org Tue Oct 13 18:26:58 2009 From: bruce at leapyear.org (Bruce Leban) Date: Tue, 13 Oct 2009 09:26:58 -0700 Subject: [Python-ideas] adding an __exec__ method to context managers? In-Reply-To: <4AD3DBEE.4080705@molden.no> References: <4AD3DBEE.4080705@molden.no> Message-ID: It seems to me that the right way to support this kind of thing is decorators on suites: def sample_for(): @parallel.for: for i in range(n): do_something(i) def sample_sections(): @parallel.sections: @parallel.section: do_one() @parallel.section: do_two() @parallel.section: do_three() Now there's a lot I don't know about python internals that may make this more/less practical. The suite decorator would of course be able to do the same kinds of things that a function decorator can so this is not just limited to the parallel example. --- Bruce On Mon, Oct 12, 2009 at 6:46 PM, Sturla Molden wrote: > > > #pragma omp parallel for private(i) > for (i=0; i /* whatever */ > > A simple pragma tells that the loop is parallel, and that the counter is > private to the thread. Calling multiple task in separate threads are equally > simple: > > #pragma omp parallel sections > { > #pragma omp section > task1(); > > #pragma omp section > task2(); > > #pragma omp section > task3(); > > } > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Tue Oct 13 22:05:53 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 13 Oct 2009 16:05:53 -0400 Subject: [Python-ideas] adding an __exec__ method to context managers? In-Reply-To: <3bdda690910130000k424b2e0exa7654b17334ceee8@mail.gmail.com> References: <4AD3DBEE.4080705@molden.no> <4AD3DF25.1050708@molden.no> <3bdda690910122009u1af3ee6ehfe9ecf8100ba5aa1@mail.gmail.com> <87hbu3reya.fsf@benfinney.id.au> <3bdda690910130000k424b2e0exa7654b17334ceee8@mail.gmail.com> Message-ID: Carl Johnson wrote: > Ben Finney: > before you find out *how* it's being iterated. For that matter, Python > has changed the perfectly sensible: > > def method(cls): > stuff() > > method = classmethod(method) > > to > > @classmethod > def method(cls): > stuff() > > Why? Because it's more readable to have the decorator up top, so you > know what kind of function/method to expect. Part of the motivation for @ you are missing was the desire to not write the function name three times, especially when the name is long, as is required in some contexts where functions must be wrapped to interface with external systems. The classmethod use case alone would not have pushed the addition. tjr From fperez.net at gmail.com Tue Oct 13 07:49:56 2009 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 13 Oct 2009 05:49:56 +0000 (UTC) Subject: [Python-ideas] adding an __exec__ method to context managers? References: <4AD3DBEE.4080705@molden.no> Message-ID: Hi Sturla! On Tue, 13 Oct 2009 03:46:22 +0200, Sturla Molden wrote: > It occurs to me that all this would be a lot cleaner if context managers > had an optional __exec__ method. It would receive the body as a code > object, together with the local and global dicts for controlled > execution. If it does not exist, something like this would be assumed: [...] I had this *precise* conversation at SciPy'08 at length with Alex Martelli, and more briefly at SciPy'09 with Peter Norvig, because this is something I've been tossing around for quite a while. I actually implemented something similar in IPython, but which is horribly brittle because it digs the parent context out by source introspection (using a custom exception to stop the execution flow at __enter__ time). It worked actually perfectly, but I know such hacks aren't robust enough to be trusted in real production code. I am somewhat skeptical that this idea could really fly in the long haul, because of all the issues regarding exactly what gets passed in (for lots of neat things you might want the real sources and not just the code object), etc. Alex raised a number of detailed points in our conversation I don't have at the top of my head right now, but I could try to think a bit harder about the details if need be. However, recently I've found 'peace' with this topic by using a different approach, that takes care of the problem you highlight in your post with having to call the _() function afterwards. In python 3 with the new non- local keyword, this approach is actually very functional, and it solves lots of problems. It's simply a matter of having a decorator consume the called function directly. Rather than repeat all this, I'll point you to a page where I summarized the whole topic at a recent talk at our Berkeley Scientific Python users group (it also contains links to a detailed discussion on this topic we had on the ipython list): https://cirl.berkeley.edu/fperez/py4science/decorators.html My current thinking on this matter is that I'll use this approach for a while, to get a better feel for the possibilities. The syntax isn't ideal, but it's not horrible either, and it is quite flexible. I think some real-world experience with this approach can teach us a lot, in order to later revisit the question with a really solid proposal for either a with extension like __exec__ or something else. I hope this is useful, thanks a lot for bringing this up here (I've discussed this *exact* idea multiple times with colleagues, but never had the energy to carry it further on-list due to being too swamped with other things). Cheers, f From cmjohnson.mailinglist at gmail.com Wed Oct 14 10:49:57 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Tue, 13 Oct 2009 22:49:57 -1000 Subject: [Python-ideas] adding an __exec__ method to context managers? In-Reply-To: References: <4AD3DBEE.4080705@molden.no> <4AD3DF25.1050708@molden.no> <3bdda690910122009u1af3ee6ehfe9ecf8100ba5aa1@mail.gmail.com> <87hbu3reya.fsf@benfinney.id.au> <3bdda690910130000k424b2e0exa7654b17334ceee8@mail.gmail.com> Message-ID: <3bdda690910140149n45076581m547d3730e31b8c54@mail.gmail.com> Terry Reedy: > Carl Johnson wrote: > >> Why? Because it's more readable to have the decorator up top, so you >> know what kind of function/method to expect. > > Part of the motivation for @ you are missing was the desire to not write > the function name three times, especially when the name is long, as is > required in some contexts where functions must be wrapped to interface with > external systems. The classmethod use case alone would not have pushed the > addition. Yes, but if we go from using def to make a function and then giving that as a callback to using a multiline lambda, that's a drop from using the function name 2 times to using it 0 times: the same magnitude of a drop: -2 ! ;-D (Of course, I'm only mentioning this in jest, since multiline lambdas are unpythonic. I just wanted to point out that it's the same reduction in typing.) -- Carl From p.f.moore at gmail.com Wed Oct 14 11:52:40 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 14 Oct 2009 10:52:40 +0100 Subject: [Python-ideas] adding an __exec__ method to context managers? In-Reply-To: References: <4AD3DBEE.4080705@molden.no> Message-ID: <79990c6b0910140252j63f125fcxa2b9f56b31f19f57@mail.gmail.com> 2009/10/13 Bruce Leban : > It seems to me that the right way to support this kind of thing is > decorators on suites: > def sample_for(): > ?? ?@parallel.for: > ?? ? ? ?for i in range(n): > ?? ? ? ? ? ?do_something(i) You don't need decorators on suites - at a minimum you can use a defined function (possibly with a throwaway name like _): def sample_for(): @parallel.for def _(): for i in range(n): do_something(i) You just need to modify parallel.for to call the newly defined function as follows (pseudocode): def parallel.for(fn): ... old body of parallel.for ... assume it returns a function inner ... instead of "return inner", do inner() return None Summary: you don't need decorators on bare blocks, just name the block. (OK, scope issues may impact this, but you didn't define how scope is handled in a "decorated block" in any case, so I choose to assume it introduces a new scope just like def does :-)) Paul. From cool-rr at cool-rr.com Fri Oct 16 18:16:19 2009 From: cool-rr at cool-rr.com (cool-RR) Date: Fri, 16 Oct 2009 18:16:19 +0200 Subject: [Python-ideas] Making assert raise a different exception type Message-ID: Hello! I would have liked to do something like this: assert some_false_condition, ValueError("Message") Meaning that if the assertion fails, the ValueError is raised instead of AssertionError. Do you think it's something that has a chance of being implemented? If so, I will write a PEP. Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Fri Oct 16 18:52:49 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 16 Oct 2009 09:52:49 -0700 Subject: [Python-ideas] Making assert raise a different exception type In-Reply-To: References: Message-ID: I don't think so. You can already write if not some_false_condition: raise ValueError("Message") On Fri, Oct 16, 2009 at 9:16 AM, cool-RR wrote: > Hello! > > I would have liked to do something like this: > > assert some_false_condition, ValueError("Message") > > Meaning that if the assertion fails, the ValueError is raised instead?of > AssertionError. Do you think it's something that has a chance of?being > implemented? If so, I will write a PEP. > > Ram. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From phd at phd.pp.ru Fri Oct 16 19:03:38 2009 From: phd at phd.pp.ru (Oleg Broytman) Date: Fri, 16 Oct 2009 21:03:38 +0400 Subject: [Python-ideas] Making assert raise a different exception type In-Reply-To: References: Message-ID: <20091016170338.GA10040@phd.pp.ru> On Fri, Oct 16, 2009 at 09:52:49AM -0700, Guido van Rossum wrote: > if not some_false_condition: if __debug__ and not some_false_condition... Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From cool-rr at cool-rr.com Fri Oct 16 19:08:55 2009 From: cool-rr at cool-rr.com (cool-RR) Date: Fri, 16 Oct 2009 19:08:55 +0200 Subject: [Python-ideas] Making assert raise a different exception type In-Reply-To: References: Message-ID: Right. The reason I want it in an `assert` is to make use of the `-o` tag. And I would prefer to raise a more informative exception than AssertionError. On Fri, Oct 16, 2009 at 6:52 PM, Guido van Rossum wrote: > I don't think so. You can already write > > if not some_false_condition: > raise ValueError("Message") > > On Fri, Oct 16, 2009 at 9:16 AM, cool-RR wrote: > > Hello! > > > > I would have liked to do something like this: > > > > assert some_false_condition, ValueError("Message") > > > > Meaning that if the assertion fails, the ValueError is raised instead of > > AssertionError. Do you think it's something that has a chance of being > > implemented? If so, I will write a PEP. > > > > Ram. > > > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas at python.org > > http://mail.python.org/mailman/listinfo/python-ideas > > > > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -- Sincerely, Ram Rachum -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Fri Oct 16 19:15:27 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 16 Oct 2009 10:15:27 -0700 Subject: [Python-ideas] Making assert raise a different exception type In-Reply-To: References: Message-ID: Sounds like a pretty odd use case to me. If the error is so common that you care about which exception is raised an assert is probably the wrong thing (and ignoring it with -O doubly so). Note that you can put an arbitrarily complex expression resulting in a string as the second argument, so you can make the message as pretty and informative as you want -- it'll just be prefixed with AssertionError: so that it's still clear this is a "should not happen" error. On Fri, Oct 16, 2009 at 10:08 AM, cool-RR wrote: > Right. The reason I want it in an `assert` is to make use of the `-o` tag. > And I would prefer to raise a more informative exception than > AssertionError. > > On Fri, Oct 16, 2009 at 6:52 PM, Guido van Rossum wrote: >> >> I don't think so. You can already write >> >> if not some_false_condition: >> ?raise ValueError("Message") >> >> On Fri, Oct 16, 2009 at 9:16 AM, cool-RR wrote: >> > Hello! >> > >> > I would have liked to do something like this: >> > >> > assert some_false_condition, ValueError("Message") >> > >> > Meaning that if the assertion fails, the ValueError is raised instead?of >> > AssertionError. Do you think it's something that has a chance of?being >> > implemented? If so, I will write a PEP. >> > >> > Ram. >> > >> > _______________________________________________ >> > Python-ideas mailing list >> > Python-ideas at python.org >> > http://mail.python.org/mailman/listinfo/python-ideas >> > >> > >> >> >> >> -- >> --Guido van Rossum (home page: http://www.python.org/~guido/) > > > > -- > Sincerely, > Ram Rachum > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steve at pearwood.info Fri Oct 16 19:59:44 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 17 Oct 2009 04:59:44 +1100 Subject: [Python-ideas] Making assert raise a different exception type In-Reply-To: <20091016170338.GA10040@phd.pp.ru> References: <20091016170338.GA10040@phd.pp.ru> Message-ID: <200910170459.45675.steve@pearwood.info> On Sat, 17 Oct 2009 04:03:38 am Oleg Broytman wrote: > On Fri, Oct 16, 2009 at 09:52:49AM -0700, Guido van Rossum wrote: > > if not some_false_condition: > > if __debug__ and not some_false_condition... That's not quite the same as assert. With assert, the test of __debug__ happens at compile time, not runtime. With the above, it happens at runtime (at least in Python 2.6). To get the same behaviour as assert, you would need something like: if __debug__: if not some_condition: raise ValueError(msg) -- Steven D'Aprano From daniel at stutzbachenterprises.com Sat Oct 17 16:34:46 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sat, 17 Oct 2009 09:34:46 -0500 Subject: [Python-ideas] breaking cycles that include __del__ Message-ID: Right now, the presence of __del__ method prevents cycles from being collected. It is possible for an application to rummage through gc.garbage and break links between objects. However, for a large application this may be inefficient or inelegant for several reasons: - It may be most natural to break the links on objects of type Foo, even though Foo is a minority of items (e.g., if Foo is parent node with many children). - It's difficult to know how frequently to sift through the garbage - Third-party package may create noncollectable garbage that we need to sift through, even though we don't know how to safely break links on their objects. - If every package sifts through the garbage independently, there's a lot of extra work going on. I propose the following simple solution. When the garbage collector is about to add an object to gc.garbage, it checks to see if the object has a __clear__ method. If so, the garbage collector calls it, giving the object an opportunity to safely break any cycles it may be in. If it breaks the cycles, great! The object can now safely be collected and __del__ will eventually be called. If not, then the object gets added to gc.garbage after all and no harm is done. It would be simple to implement, efficient, and the cognitive load for the programmer using __del__ and __clear__ is small. Thoughts? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnodel at googlemail.com Sat Oct 17 18:01:48 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 17 Oct 2009 17:01:48 +0100 Subject: [Python-ideas] breaking cycles that include __del__ In-Reply-To: References: Message-ID: <369CD9FA-8A72-4301-A938-FC55A1643677@googlemail.com> On 17 Oct 2009, at 15:34, Daniel Stutzbach wrote: > Right now, the presence of __del__ method prevents cycles from being > collected. It is possible for an application to rummage through > gc.garbage and break links between objects. However, for a large > application this may be inefficient or inelegant for several reasons: > > - It may be most natural to break the links on objects of type Foo, > even though Foo is a minority of items (e.g., if Foo is parent node > with many children). > > - It's difficult to know how frequently to sift through the garbage > > - Third-party package may create noncollectable garbage that we need > to sift through, even though we don't know how to safely break links > on their objects. > > - If every package sifts through the garbage independently, there's > a lot of extra work going on. > > I propose the following simple solution. When the garbage collector > is about to add an object to gc.garbage, it checks to see if the > object has a __clear__ method. If so, the garbage collector calls > it, giving the object an opportunity to safely break any cycles it > may be in. If it breaks the cycles, great! The object can now > safely be collected and __del__ will eventually be called. If not, > then the object gets added to gc.garbage after all and no harm is > done. > > It would be simple to implement, efficient, and the cognitive load > for the programmer using __del__ and __clear__ is small. > > Thoughts? I remember a thread about this on comp.lang.python in March: http://mail.python.org/pipermail/python-list/2009-March/174419.html -- Arnaud From ncoghlan at gmail.com Sat Oct 17 18:17:03 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 18 Oct 2009 02:17:03 +1000 Subject: [Python-ideas] breaking cycles that include __del__ In-Reply-To: References: Message-ID: <4AD9EDFF.4000703@gmail.com> Daniel Stutzbach wrote: > It would be simple to implement, efficient, and the cognitive load for > the programmer using __del__ and __clear__ is small. I believe it would be better to find ways to streamline the existing "collectible-yet-finalised" mechanism using weak references rather than adding another way to do it. Although even before that, just documenting how weakrefs can be used instead of __del__ in the weakref module docs and crosslinking to that from the __del__ method documentation would be an improvement on the status quo. The gist is that you split your object into a "core" which requires finalisation (but can never itself be caught up in a cycle) and the main object which may participate in cycles. A weak reference to the main object is placed in a global container. When the weak reference callback is invoked, the object core is still in a valid state so it can be finalised safely, even though the main object may be in the midst of cycle collection. This recipe gives a pretty good idea of how that approach works: http://code.activestate.com/recipes/519610/ Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From cool-rr at cool-rr.com Sun Oct 18 12:22:13 2009 From: cool-rr at cool-rr.com (cool-RR) Date: Sun, 18 Oct 2009 12:22:13 +0200 Subject: [Python-ideas] Minimalist `except` syntax. Message-ID: Often I do: try: some_suite() except SomeError: pass What I suggest is that instead we'll have this: try: some_suite() except SomeError What do you think? Ram. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fetchinson at googlemail.com Sun Oct 18 12:26:24 2009 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Sun, 18 Oct 2009 12:26:24 +0200 Subject: [Python-ideas] Minimalist `except` syntax. In-Reply-To: References: Message-ID: > Often I do: > > try: > some_suite() > except SomeError: > pass > > What I suggest is that instead we'll have this: > > try: > some_suite() > except SomeError > > What do you think? Since the sole purpose of your proposal is saving 6 keystrokes I don't think it's a useful one. I do also lot of times the construct you mention but it doesn't bother me that I have to type the extra 6 keystrokes. In general, if the sole purpose of a proposal is saving a couple of keystrokes and nothing else in terms of readability, maintainability, expressiveness, etc, then I think the proposal is doomed to fail and rightly so. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From sturla at molden.no Sun Oct 18 15:40:01 2009 From: sturla at molden.no (Sturla Molden) Date: Sun, 18 Oct 2009 15:40:01 +0200 Subject: [Python-ideas] Comment syntax similar to /* */ in C? Message-ID: <4ADB1AB1.8030806@molden.no> In Python a comment start from # and spans the rest of the line, similar to // in C++. In C/C++, we can also make comments that don't span the rest of the line, but has two delimiters: /* comment */ This allows us to write: foobar(a, b /* comment on b */, c, d, e, f); foobar(a, b /* comment on b that spans two lines */, c, d, e, f); /* multiline comment often seen in C/C++ code */ I have sometimes missed this possibility when writing Python code. To my knowledge there is no equivalent to /* */ in Python. I don't care much about choice of delimiter. Perhaps ### comment ### is the most intuitive, as it has resemblance to triple-quotes for text strings? It also stands out clearly in black/white text. foobar(a, b ### comment on b ###, c, d, e, f) foobar(a, b ### comment on b that spans two lines ###, c, d, e, f) ### comment that spans multiple lines ### If this breaks a lot of existing code, perhaps something else like %% could be used? foobar(a, b %% comment on b that spans two lines %%, c, d, e, f) P.S. In Python we can emulate multiline comments using triple-quoted strings, but conceptually strings and comments are very different. I.e. strings are objects, comments are auxillary text discarded at compile time. Strings are objects created at runtime, comments are not. Regards, Sturla Molden From steve at pearwood.info Sun Oct 18 16:53:33 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 19 Oct 2009 01:53:33 +1100 Subject: [Python-ideas] Comment syntax similar to /* */ in C? In-Reply-To: <4ADB1AB1.8030806@molden.no> References: <4ADB1AB1.8030806@molden.no> Message-ID: <200910190153.34123.steve@pearwood.info> On Mon, 19 Oct 2009 12:40:01 am Sturla Molden wrote: > In Python a comment start from # and spans the rest of the line, > similar to // in C++. > > In C/C++, we can also make comments that don't span the rest of the > line, but has two delimiters: > > > /* comment */ > > > This allows us to write: > > > foobar(a, b /* comment on b */, c, d, e, f); Ewwww. That's horrible. If you ask me, in-expression comments are a misfeature the world would be better off without. > I have sometimes missed this possibility when writing Python code. To > my knowledge there is no equivalent to /* */ in Python. > > I don't care much about choice of delimiter. Perhaps ### comment ### > is the most intuitive, as it has resemblance to triple-quotes for > text strings? It also stands out clearly in black/white text. > > > foobar(a, b ### comment on b ###, c, d, e, f) > > foobar(a, b ### comment on b > that spans two lines ###, c, d, e, f) Take advantage of Python's rules for handling open parentheses: foobar(a, b, # comment on b c, d, e, f) foobar(a, b, # comment on b # more and longer comment on b c, d, e, f) And don't forget that there's no need for comments to be on the same physical line they are referring to. You are allowed to include comments surrounding the line. > ### > comment that spans multiple lines > ### Use a modern editor that lets you easily comment and uncomment multiple lines at once. > P.S. In Python we can emulate multiline comments using triple-quoted > strings, but conceptually strings and comments are very different. > I.e. strings are objects, comments are auxillary text discarded at > compile time. Strings are objects created at runtime, comments are > not. Guido's time-machine strikes again. >>> import dis >>> def test(): ... x = 1 ... """ ... This is a triple-quote comment. ... """ ... return x ... >>> dis.dis(test) 2 0 LOAD_CONST 1 (1) 3 STORE_FAST 0 (x) 6 6 LOAD_FAST 0 (x) 9 RETURN_VALUE String literals -- not just triple-quoted strings, but any string literal -- that don't go anywhere are discarded by the Python compiler, precisely so they can be used as comments. -- Steven D'Aprano From grosser.meister.morti at gmx.net Sun Oct 18 16:55:31 2009 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Sun, 18 Oct 2009 16:55:31 +0200 Subject: [Python-ideas] Comment syntax similar to /* */ in C? In-Reply-To: <4ADB1AB1.8030806@molden.no> References: <4ADB1AB1.8030806@molden.no> Message-ID: <4ADB2C63.7090504@gmx.net> I don't think multiline comments are necessary, see: foobar(a, b, # comment on b c, d, e, f) # comment on b foobar(a, b, c, d, e, f) foobar( a, # comment on a b, # multiline # comment on b c, # comment on c d, e, f) However, if multiline comments are introduced they have to be a new syntax (syntax that produces a syntax error in older python versions) and I also would like them to by syntactical (nestable), e.g.: foobar(a, b {* comment on b *}, c, d, e, f) {* foobar(a, b {* comment on b *}, c, d, e, f) egg(spam) *} tomato(spam) On 10/18/2009 03:40 PM, Sturla Molden wrote: > In Python a comment start from # and spans the rest of the line, similar > to // in C++. > > In C/C++, we can also make comments that don't span the rest of the > line, but has two delimiters: > > > /* comment */ > > > This allows us to write: > > > foobar(a, b /* comment on b */, c, d, e, f); > > > foobar(a, b /* comment on b > that spans two lines */, c, d, e, f); > > > /* multiline comment > often seen in C/C++ code > > */ > > > > > I have sometimes missed this possibility when writing Python code. To my > knowledge there is no equivalent to /* */ in Python. > > I don't care much about choice of delimiter. Perhaps ### comment ### is > the most intuitive, as it has resemblance to triple-quotes for text > strings? It also stands out clearly in black/white text. > > > foobar(a, b ### comment on b ###, c, d, e, f) > > > foobar(a, b ### comment on b > that spans two lines ###, c, d, e, f) > > > ### > comment that spans multiple lines > ### > > > If this breaks a lot of existing code, perhaps something else like %% > could be used? > > > foobar(a, b %% comment on b > that spans two lines %%, c, d, e, f) > > > > P.S. In Python we can emulate multiline comments using triple-quoted > strings, but conceptually strings and comments are very different. I.e. > strings are objects, comments are auxillary text discarded at compile > time. Strings are objects created at runtime, comments are not. > > > > Regards, > Sturla Molden > From brett at python.org Sun Oct 18 18:45:58 2009 From: brett at python.org (Brett Cannon) Date: Sun, 18 Oct 2009 09:45:58 -0700 Subject: [Python-ideas] Comment syntax similar to /* */ in C? In-Reply-To: <200910190153.34123.steve@pearwood.info> References: <4ADB1AB1.8030806@molden.no> <200910190153.34123.steve@pearwood.info> Message-ID: On Sun, Oct 18, 2009 at 07:53, Steven D'Aprano wrote: > On Mon, 19 Oct 2009 12:40:01 am Sturla Molden wrote: > > > In Python a comment start from # and spans the rest of the line, > > similar to // in C++. > > > > In C/C++, we can also make comments that don't span the rest of the > > line, but has two delimiters: > > > > > > /* comment */ > > > > > > This allows us to write: > > > > > > foobar(a, b /* comment on b */, c, d, e, f); > > Ewwww. That's horrible. If you ask me, in-expression comments are a > misfeature the world would be better off without. > > > I have sometimes missed this possibility when writing Python code. To > > my knowledge there is no equivalent to /* */ in Python. > > > > I don't care much about choice of delimiter. Perhaps ### comment ### > > is the most intuitive, as it has resemblance to triple-quotes for > > text strings? It also stands out clearly in black/white text. > > > > > > foobar(a, b ### comment on b ###, c, d, e, f) > > > > foobar(a, b ### comment on b > > that spans two lines ###, c, d, e, f) > > Take advantage of Python's rules for handling open parentheses: > > foobar(a, b, # comment on b > c, d, e, f) > > foobar(a, b, # comment on b > # more and longer comment on b > c, d, e, f) > > > And don't forget that there's no need for comments to be on the same > physical line they are referring to. You are allowed to include > comments surrounding the line. > > > > ### > > comment that spans multiple lines > > ### > > Use a modern editor that lets you easily comment and uncomment multiple > lines at once. > > > > > P.S. In Python we can emulate multiline comments using triple-quoted > > strings, but conceptually strings and comments are very different. > > I.e. strings are objects, comments are auxillary text discarded at > > compile time. Strings are objects created at runtime, comments are > > not. > > Guido's time-machine strikes again. > > >>> import dis > >>> def test(): > ... x = 1 > ... """ > ... This is a triple-quote comment. > ... """ > ... return x > ... > >>> dis.dis(test) > 2 0 LOAD_CONST 1 (1) > 3 STORE_FAST 0 (x) > > 6 6 LOAD_FAST 0 (x) > 9 RETURN_VALUE > > > String literals -- not just triple-quoted strings, but any string > literal -- that don't go anywhere are discarded by the Python compiler, > precisely so they can be used as comments. > And because of this new syntax will never be added for multi-line comments. You guys can keep talking, but this is one of those "over Guido's resignation" type of proposals. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From debatem1 at gmail.com Sun Oct 18 19:22:28 2009 From: debatem1 at gmail.com (geremy condra) Date: Sun, 18 Oct 2009 13:22:28 -0400 Subject: [Python-ideas] Minimalist `except` syntax. In-Reply-To: References: Message-ID: On Sun, Oct 18, 2009 at 6:22 AM, cool-RR wrote: > Often I do: > try: > ?? ?some_suite() > except SomeError: > ?? ?pass > What I suggest is that instead we'll have this: > try: > ?? ?some_suite() > except SomeError > What do you think? > Ram. try: some_suite() except SomeError: pass works now. not really sure I see the point in going through the changes required in order to remove the word 'pass' from the end. Anything I'm missing? Geremy Condra From debatem1 at gmail.com Sun Oct 18 19:25:29 2009 From: debatem1 at gmail.com (geremy condra) Date: Sun, 18 Oct 2009 13:25:29 -0400 Subject: [Python-ideas] Comment syntax similar to /* */ in C? In-Reply-To: <200910190153.34123.steve@pearwood.info> References: <4ADB1AB1.8030806@molden.no> <200910190153.34123.steve@pearwood.info> Message-ID: On Sun, Oct 18, 2009 at 10:53 AM, Steven D'Aprano wrote: > On Mon, 19 Oct 2009 12:40:01 am Sturla Molden wrote: >> P.S. In Python we can emulate multiline comments using triple-quoted >> strings, but conceptually strings and comments are very different. >> I.e. strings are objects, comments are auxillary text discarded at >> compile time. Strings are objects created at runtime, comments are >> not. > > Guido's time-machine strikes again. > >>>> import dis >>>> def test(): > ... ? ? x = 1 > ... ? ? """ > ... ? ? This is a triple-quote comment. > ... ? ? """ > ... ? ? return x > ... >>>> dis.dis(test) > ?2 ? ? ? ? ? 0 LOAD_CONST ? ? ? ? ? ? ? 1 (1) > ? ? ? ? ? ? ?3 STORE_FAST ? ? ? ? ? ? ? 0 (x) > > ?6 ? ? ? ? ? 6 LOAD_FAST ? ? ? ? ? ? ? ?0 (x) > ? ? ? ? ? ? ?9 RETURN_VALUE > > > String literals -- not just triple-quoted strings, but any string > literal -- that don't go anywhere are discarded by the Python compiler, > precisely so they can be used as comments. > Did not know that, interesting. Geremy Condra From cmjohnson.mailinglist at gmail.com Mon Oct 19 10:34:29 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Sun, 18 Oct 2009 22:34:29 -1000 Subject: [Python-ideas] Minimalist `except` syntax. In-Reply-To: References: Message-ID: <3bdda690910190134u61bd7237wb8856b7ad1f0644@mail.gmail.com> Why would we want to encourage people to ignore errors? From stefan_ml at behnel.de Mon Oct 19 11:59:03 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 19 Oct 2009 11:59:03 +0200 Subject: [Python-ideas] Minimalist `except` syntax. In-Reply-To: <3bdda690910190134u61bd7237wb8856b7ad1f0644@mail.gmail.com> References: <3bdda690910190134u61bd7237wb8856b7ad1f0644@mail.gmail.com> Message-ID: Carl Johnson wrote: > Why would we want to encourage people to ignore errors? Because errors should never pass silently, unless explicitly silenced. Stefan From rylesny at gmail.com Mon Oct 19 12:07:38 2009 From: rylesny at gmail.com (ryles) Date: Mon, 19 Oct 2009 03:07:38 -0700 (PDT) Subject: [Python-ideas] Minimalist `except` syntax. In-Reply-To: References: Message-ID: <712250c3-27f9-4ba0-878f-71de199e6e3e@w19g2000yqk.googlegroups.com> On Oct 18, 6:22?am, cool-RR wrote: > Often I do: > > try: > ? ? some_suite() > except SomeError: > ? ? pass > > What I suggest is that instead we'll have this: > > try: > ? ? some_suite() > except SomeError > > What do you think? > The explicit 'pass' improves readability by showing that the author made a conscious decision to ignore the error. In the suggested form, it looks more like the author took a coffee break in the middle of coding. From weasley_wx at qq.com Mon Oct 19 15:33:06 2009 From: weasley_wx at qq.com (starwing) Date: Mon, 19 Oct 2009 21:33:06 +0800 Subject: [Python-ideas] a new lambda syntax Message-ID: <4ADC6A92.3060206@qq.com> In ruby, we can add a lambda function into a function call. but it's a little ugly: [1,2,3,4].each { |x| print x } but, it really convenient. current lambda only support single line syntax. it's not very good. maybe we can add a new syntax: call: primary "(" argument-list ")" [ "lambda" parameter-list ":" suite ] the postfix lambda will translate to a single "function" object, and pass to the &arg parameter of function. so the function call will become func(*arg1, **arg2, &arg3) for example, we can write this: def print_list(*lists, &func = lambda x: true): for item in lists: map(a) lambda x: if func(x): print x call with: print_list([1,2,3], [1,2]) lambda x: if x % 2: return True else return False From masklinn at masklinn.net Mon Oct 19 15:43:59 2009 From: masklinn at masklinn.net (Masklinn) Date: Mon, 19 Oct 2009 15:43:59 +0200 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <4ADC6A92.3060206@qq.com> References: <4ADC6A92.3060206@qq.com> Message-ID: <08953A22-3C91-457C-A2DC-25926B0C128C@masklinn.net> On 19 Oct 2009, at 15:33 , starwing wrote: > In ruby, we can add a lambda function into a function call. but it's a > little ugly: > > [1,2,3,4].each { |x| > print x > } > > > but, it really convenient. current lambda only support single line > syntax. it's not very good. maybe we can add a new syntax: > call: primary "(" argument-list ")" [ "lambda" parameter-list ":" > suite ] > > the postfix lambda will translate to a single "function" object, and > pass to the &arg parameter of function. so the function call will > become > func(*arg1, **arg2, &arg3) > > for example, we can write this: > > def print_list(*lists, &func = lambda x: true): > for item in lists: > map(a) lambda x: > if func(x): > print x > > > call with: > print_list([1,2,3], [1,2]) lambda x: > if x % 2: return True > else return False > This is not going to end well. Also, http://mail.python.org/pipermail/python-ideas/2007-December/001279.html http://mail.python.org/pipermail/python-ideas/2008-April/001522.html http://mail.python.org/pipermail/python-ideas/2008-November/002340.html The BDFL has disliked every proposal so far, and due to the Ruby- inspired weirdness and magic of this one I really don't see him liking it any better than the ones that came before. -m From phd at phd.pp.ru Mon Oct 19 16:04:47 2009 From: phd at phd.pp.ru (Oleg Broytman) Date: Mon, 19 Oct 2009 18:04:47 +0400 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <4ADC6A92.3060206@qq.com> References: <4ADC6A92.3060206@qq.com> Message-ID: <20091019140447.GA8510@phd.pp.ru> On Mon, Oct 19, 2009 at 09:33:06PM +0800, starwing wrote: > current lambda only support single line > syntax. it's not very good. This is where you've made an error. Single-line lambdas are good enough, and if you need more - just create a named function. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From daniel at stutzbachenterprises.com Mon Oct 19 16:25:08 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 19 Oct 2009 09:25:08 -0500 Subject: [Python-ideas] breaking cycles that include __del__ In-Reply-To: <4AD9EDFF.4000703@gmail.com> References: <4AD9EDFF.4000703@gmail.com> Message-ID: On Sat, Oct 17, 2009 at 11:17 AM, Nick Coghlan wrote: > The gist is that you split your object into a "core" which requires > finalisation (but can never itself be caught up in a cycle) and the main > object which may participate in cycles. A weak reference to the main > object is placed in a global container. When the weak reference callback > is invoked, the object core is still in a valid state so it can be > finalised safely, even though the main object may be in the midst of > cycle collection. > > This recipe gives a pretty good idea of how that approach works: > http://code.activestate.com/recipes/519610/ > That idiom (and that recipe in particular! yuck!) is pretty convoluted, IMO. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From weasley_wx at qq.com Mon Oct 19 16:28:58 2009 From: weasley_wx at qq.com (starwing) Date: Mon, 19 Oct 2009 22:28:58 +0800 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <20091019140447.GA8510@phd.pp.ru> References: <4ADC6A92.3060206@qq.com> <20091019140447.GA8510@phd.pp.ru> Message-ID: <4ADC77AA.2090807@qq.com> Oleg Broytman ??: > On Mon, Oct 19, 2009 at 09:33:06PM +0800, starwing wrote: > >> current lambda only support single line >> syntax. it's not very good. >> > > This is where you've made an error. Single-line lambdas are good enough, > and if you need more - just create a named function. > > Oleg. > BUT, why we need a name? just to process something or make decide, sometimes code itself is enough. and, is there performance problems when you define a inner function in another function? (that's, that function will define every time you call the function or not?) From phd at phd.pp.ru Mon Oct 19 16:39:42 2009 From: phd at phd.pp.ru (Oleg Broytman) Date: Mon, 19 Oct 2009 18:39:42 +0400 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <4ADC77AA.2090807@qq.com> References: <4ADC6A92.3060206@qq.com> <20091019140447.GA8510@phd.pp.ru> <4ADC77AA.2090807@qq.com> Message-ID: <20091019143942.GB8510@phd.pp.ru> On Mon, Oct 19, 2009 at 10:28:58PM +0800, starwing wrote: > Oleg Broytman ??????: >> Single-line lambdas are good enough, >> and if you need more - just create a named function. >> > BUT, why we need a name? just to process something or make decide, > sometimes code itself is enough. A multiline function is certainly not a simple piece of code; it requires documentation - docstring, comments - and the name is a part of the documentation. > and, is there performance problems when you define a inner function in > another function? (that's, that function will define every time you call > the function or not?) A lambda, like an inner function, is recreated every time, so it's certainly no better than a named function. You should profile your program to prove there is really a performance degradation. And if there is - create a global function instead. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From masklinn at masklinn.net Mon Oct 19 17:13:14 2009 From: masklinn at masklinn.net (Masklinn) Date: Mon, 19 Oct 2009 17:13:14 +0200 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <20091019143942.GB8510@phd.pp.ru> References: <4ADC6A92.3060206@qq.com> <20091019140447.GA8510@phd.pp.ru> <4ADC77AA.2090807@qq.com> <20091019143942.GB8510@phd.pp.ru> Message-ID: <5E391173-A51F-4945-BB66-54A54773CB1F@masklinn.net> On 19 Oct 2009, at 16:39 , Oleg Broytman wrote: > On Mon, Oct 19, 2009 at 10:28:58PM +0800, starwing wrote: >> Oleg Broytman ??????: >>> Single-line lambdas are good enough, >>> and if you need more - just create a named function. >>> >> BUT, why we need a name? just to process something or make decide, >> sometimes code itself is enough. > > A multiline function is certainly not a simple piece of code; it > requires documentation - docstring, comments - and the name is a > part of > the documentation. > Why would grouping two statements require such an overhead? Do you write a comment for each line of Python code you produce? A function, named or not, is nothing more than a bloc of code (in fact that's exactly how they're called in Smalltalk), and not all blocs of code require a name, a docstring and comments. From rhamph at gmail.com Mon Oct 19 18:34:38 2009 From: rhamph at gmail.com (Adam Olsen) Date: Mon, 19 Oct 2009 10:34:38 -0600 Subject: [Python-ideas] breaking cycles that include __del__ In-Reply-To: References: <4AD9EDFF.4000703@gmail.com> Message-ID: On Mon, Oct 19, 2009 at 08:25, Daniel Stutzbach wrote: > On Sat, Oct 17, 2009 at 11:17 AM, Nick Coghlan wrote: >> >> The gist is that you split your object into a "core" which requires >> finalisation (but can never itself be caught up in a cycle) and the main >> object which may participate in cycles. A weak reference to the main >> object is placed in a global container. When the weak reference callback >> is invoked, the object core is still in a valid state so it can be >> finalised safely, even though the main object may be in the midst of >> cycle collection. >> >> This recipe gives a pretty good idea of how that approach works: >> http://code.activestate.com/recipes/519610/ > > That idiom (and that recipe in particular! yuck!) is pretty convoluted, IMO. The critical idea is to list attributes that cannot be cleared, leaving everything else to be cleared by default. This is MUCH less precarious than calling arbitrary code from within the GC, and accomplishes nearly the same thing. The use of a core further benefits the GC implementation, but not much the user. Obviously a recipe like that cannot show real benefit from it, and I admit the recipe is pretty ugly. -- Adam Olsen, aka Rhamphoryncus From daniel at stutzbachenterprises.com Mon Oct 19 18:56:00 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 19 Oct 2009 11:56:00 -0500 Subject: [Python-ideas] breaking cycles that include __del__ In-Reply-To: References: Message-ID: On Sat, Oct 17, 2009 at 9:34 AM, Daniel Stutzbach < daniel at stutzbachenterprises.com> wrote: > I propose the following simple solution. When the garbage collector is > about to add an object to gc.garbage, it checks to see if the object has a > __clear__ method. If so, the garbage collector calls it, giving the object > an opportunity to safely break any cycles it may be in. If it breaks the > cycles, great! The object can now safely be collected and __del__ will > eventually be called. If not, then the object gets added to gc.garbage > after all and no harm is done. > I should add that my personal use-case runs something like this. Obviously, most uses of __del__ revolve around object with with special resources that need to be freed "by hand" (such as a resource allocated via ctypes). To guarantee the timely freeing of resources, it's always better to close the object by hand with a .close() method or using a context manager. That works great, except there's often no easy way to test a program to verifying that .close is always called when appropriate. There's a simple recipe to make sure that .close() was called on an object, by using __del__ to make sure the object was closed "by hand": class Widget: closed = False def close(self): if self.closed: return self.closed = True # Free various resources from ctypes, etc. def __del__(self): log_an_error('%s not closed properly' % repr(self)) self.close() Unfortunately, if the object ends up in a cycle, then the recipe can't do its job because __del__ will never be called. I think I'll experiment by having my program periodically check gc.garbage and call any __clear__ methods it finds. For that matter, I may just call any __del__ methods it finds and see what happens. If the order of calls to __del__ matters, I'll try it as a bug in my __del__ methods. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Oct 19 21:35:59 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 19 Oct 2009 12:35:59 -0700 Subject: [Python-ideas] [Python-Dev] Proposal : Python Trusted Computing API In-Reply-To: References: Message-ID: On Sun, Oct 18, 2009 at 11:29 PM, Abhiram Kasina wrote: > Trusted Computing (TC) is a technology developed and promoted by the Trusted > Computing Group (TCG)[3]. So, basically the group came up with these chips > called TPM chips which are present on most motherboards nowadays. The main > purpose of it is to enhance security so that infected executables don't run. > It also provides memory curtaining such that cryptographic keys won't be > accessible and many other features. There was a criticism on this from the > FOSS community as well that it enables DRM. No wonder, it is being pushed by > Intel, Microsoft, AMD, etc.. But personally I think its a good idea from > security point of view. Hm... Given that most infections these days are JavaScript based and run in the browser, how does this provide any protection? I'm presuming you're going to say that it doesn't but that there are other use cases where it *does* provide protection; but most likely those use cases are only relevant for Windows (since that's what most attackers attack anyway). > So, currently there is an TSS (TCG Software Stack)[1] API written in C. And > TrustedJava[2] is a project which ported it to Java and is going to be > included in the standard API of Java soon. They have 2 versions of it. One > is a simple wrapper on top of the API and the other is a whole > implementation of the stack in Java. Since this intefaces with the hardware, doesn't it require some kind of cooperation from the Linux kernel? And wouldn't it be better if Python was never allowed access to any of the protected resources in the first place? > My proposal is we create an API for it in python. > Reason: I am a developer in Umit Where/what is Umit? (Google gives several meanings but it's unclear which you might mean.) > and I think Python is a very good platform > for developing applications. So, why not create an API which helps in > developing secure applications? You'd first have to tell us more about the security model. What is a "secure application" and what does it protect against? And how? > I would love to learn more and provide you with any more information. Please > let me know what you guys think of it? This is better directed at python-ideas, so I've redirected this reply there and Bcc'ed the python-dev list. > Thanks in advance > > Cheers > Abhiram > > [1] > http://www.trustedcomputinggroup.org/resources/tcg_software_stack_tss_specification > [2] http://trustedjava.sourceforge.net/index.php?item=jtss/about > [3] http://www.trustedcomputinggroup.org/ > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ubershmekel at gmail.com Mon Oct 19 22:28:02 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Mon, 19 Oct 2009 22:28:02 +0200 Subject: [Python-ideas] Minimalist `except` syntax. In-Reply-To: <712250c3-27f9-4ba0-878f-71de199e6e3e@w19g2000yqk.googlegroups.com> References: <712250c3-27f9-4ba0-878f-71de199e6e3e@w19g2000yqk.googlegroups.com> Message-ID: <9d153b7c0910191328g6da5e574x2923db301ba0a57@mail.gmail.com> On Mon, Oct 19, 2009 at 12:07 PM, ryles wrote: > The explicit 'pass' improves readability by showing that the author > made a conscious decision to ignore the error. In the suggested form, > it looks more like the author took a coffee break in the middle of > coding. -1 on allowing code that looks like a coffee break. From ncoghlan at gmail.com Mon Oct 19 22:30:05 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 20 Oct 2009 06:30:05 +1000 Subject: [Python-ideas] breaking cycles that include __del__ In-Reply-To: References: <4AD9EDFF.4000703@gmail.com> Message-ID: <4ADCCC4D.8010201@gmail.com> Daniel Stutzbach wrote: > On Sat, Oct 17, 2009 at 11:17 AM, Nick Coghlan > wrote: > > The gist is that you split your object into a "core" which requires > finalisation (but can never itself be caught up in a cycle) and the main > object which may participate in cycles. A weak reference to the main > object is placed in a global container. When the weak reference callback > is invoked, the object core is still in a valid state so it can be > finalised safely, even though the main object may be in the midst of > cycle collection. > > This recipe gives a pretty good idea of how that approach works: > http://code.activestate.com/recipes/519610/ > > > That idiom (and that recipe in particular! yuck!) is pretty convoluted, IMO. True, but as Adam pointed out, something along those lines is necessary to avoid accessing already destroyed objects in __del__ methods. My point was mainly that the idiom works, we know it works, we just don't have a particularly nice way of spelling it. (Even I thought that recipe was fairly ugly - I just linked it because it does a good job of explaining the requirements of the technique) For the diagnostic task you're talking about, wouldn't it be better to create your own collection of weakrefs to instances of the class you're interested in, checking that the object has been closed in the callback and then iterate over that collection in an atexit handler? You wouldn't have any __del__ methods of your own interfering with garbage collection then and the atexit handler would be a final diagnostic to check if the objects were getting caught up in cycles with *other* objects with __del__ methods. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ben+python at benfinney.id.au Mon Oct 19 22:50:11 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 20 Oct 2009 07:50:11 +1100 Subject: [Python-ideas] a new lambda syntax References: <4ADC6A92.3060206@qq.com> <20091019140447.GA8510@phd.pp.ru> <4ADC77AA.2090807@qq.com> Message-ID: <873a5fds70.fsf@benfinney.id.au> starwing writes: > Oleg Broytman ??: > > Single-line lambdas are good enough, and if you need more - just > > create a named function. > BUT, why we need a name? just to process something or make decide, > sometimes code itself is enough. When the code is more complex than a single expression, it's already complex enough that defining it first and then referring to it makes for clearer code. > and, is there performance problems when you define a inner function in > another function? (that's, that function will define every time you > call the function or not?) Measure the performance difference in your specific code base and decide on that basis. If there are such problems, then the lambda form won't help you: it defines a function of exactly the same type as what ?def? defines. -- \ ?I don't like country music, but I don't mean to denigrate | `\ those who do. And for the people who like country music, | _o__) denigrate means ?put down?.? ?Bob Newhart | Ben Finney From masklinn at masklinn.net Mon Oct 19 23:04:46 2009 From: masklinn at masklinn.net (Masklinn) Date: Mon, 19 Oct 2009 23:04:46 +0200 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <873a5fds70.fsf@benfinney.id.au> References: <4ADC6A92.3060206@qq.com> <20091019140447.GA8510@phd.pp.ru> <4ADC77AA.2090807@qq.com> <873a5fds70.fsf@benfinney.id.au> Message-ID: On 19 Oct 2009, at 22:50 , Ben Finney wrote: > starwing writes: >> Oleg Broytman ??: >>> Single-line lambdas are good enough, and if you need more - just >>> create a named function. >> BUT, why we need a name? just to process something or make decide, >> sometimes code itself is enough. > When the code is more complex than a single expression, it's already > complex enough that defining it first and then referring to it makes > for > clearer code. > That sounds quite recursive. Is it a hard rule or are you allowed to write three consecutive statements or expressions in some cases? From scott+python-ideas at scottdial.com Mon Oct 19 22:46:43 2009 From: scott+python-ideas at scottdial.com (Scott Dial) Date: Mon, 19 Oct 2009 16:46:43 -0400 Subject: [Python-ideas] breaking cycles that include __del__ In-Reply-To: References: Message-ID: <4ADCD033.2080204@scottdial.com> Daniel Stutzbach wrote: > Unfortunately, if the object ends up in a cycle, then the recipe can't > do its job because __del__ will never be called. The way I have always dealt with these problems is to not have overly-generalized objects. If I have a resource to acquire that needs to have a __del__ to clean-up after itself (as you suggest, in case a close() was not called), then that is *all* that object does. The only way you can get trapped into a cycle is if you design a resource-managing object that holds references to other objects, which was bad design. I don't see the need for a recipe like the one Nick linked to. Everyone can agree it is ugly, I think. But, the real goal was to split out the core of the object required for clean-up. So, I am saying, actually split it out! Classes are cheap. class _CoreWidget: def __init__(self, *args, **kwds): self.closed = False # Acquire resources def close(self): if self.closed: return self.closed = True # Free resources def __del__(self): if not self.closed: print('%s not closed properly' % repr(self)) self.close() class Widget: def __init__(self, *args, **kwds): self._core_widget = _CoreWidget(*args, **kwds) x, y = Widget(), Widget() x.y, y.x = y, x del x; del y # <__main__._CoreWidget instance at 0x00B3F418> not closed properly # <__main__._CoreWidget instance at 0x00B3E328> not closed properly -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From ncoghlan at gmail.com Tue Oct 20 00:00:14 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 20 Oct 2009 08:00:14 +1000 Subject: [Python-ideas] a new lambda syntax In-Reply-To: References: <4ADC6A92.3060206@qq.com> <20091019140447.GA8510@phd.pp.ru> <4ADC77AA.2090807@qq.com> <873a5fds70.fsf@benfinney.id.au> Message-ID: <4ADCE16E.2000208@gmail.com> Masklinn wrote: > On 19 Oct 2009, at 22:50 , Ben Finney wrote: >> starwing writes: >>> Oleg Broytman ??: >>>> Single-line lambdas are good enough, and if you need more - just >>>> create a named function. >>> BUT, why we need a name? just to process something or make decide, >>> sometimes code itself is enough. >> When the code is more complex than a single expression, it's already >> complex enough that defining it first and then referring to it makes for >> clearer code. >> > That sounds quite recursive. Is it a hard rule or are you allowed to > write three consecutive statements or expressions in some cases? The difference lies largely in whether or not you are bundling up that code and giving it to another piece of code to use. For myself, I don't actually agree it's a valid design rule - I think anonymous blocks have legitimate use cases (see Ars Technica's writeup of the Apple's new Grand Central Dispatch and C-level anonymous block system in OS X 10.6). However, for Python, the idea of having anonymous blocks runs headlong into the use of indentation for block delineation. You have to either come up with a non-indentation based syntax for the statements inside the anonymous blocks, essentially inventing a whole new language, or you have to embed significant whitespace inside an expression rather than only having it between statements. Nobody has come up with a solution to that mismatch which is superior to just naming the block and using a normal def statement to create it. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ben+python at benfinney.id.au Tue Oct 20 00:05:18 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 20 Oct 2009 09:05:18 +1100 Subject: [Python-ideas] a new lambda syntax References: <4ADC6A92.3060206@qq.com> <20091019140447.GA8510@phd.pp.ru> <4ADC77AA.2090807@qq.com> <873a5fds70.fsf@benfinney.id.au> Message-ID: <87tyxvca5d.fsf@benfinney.id.au> Masklinn writes: > On 19 Oct 2009, at 22:50 , Ben Finney wrote: > > When the code is more complex than a single expression, it's already > > complex enough that defining it first and then referring to it makes > > for clearer code. > > > That sounds quite recursive. Where's the recursion? I'm not defining complexity in terms of complexity, if that's what you mean; I'm noting that a threshold on one continuum is located around the same place as a threshold on another. > Is it a hard rule or are you allowed to write three consecutive > statements or expressions in some cases? Of course it's not a hard rule; it's a rule of thumb, if you like. -- \ ?Either he's dead or my watch has stopped.? ?Groucho Marx | `\ | _o__) | Ben Finney From fuzzyman at gmail.com Tue Oct 20 00:33:48 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 19 Oct 2009 23:33:48 +0100 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <20091019143942.GB8510@phd.pp.ru> References: <4ADC6A92.3060206@qq.com> <20091019140447.GA8510@phd.pp.ru> <4ADC77AA.2090807@qq.com> <20091019143942.GB8510@phd.pp.ru> Message-ID: <6f4025010910191533gf7d47b1gf0e7465bf96e0c84@mail.gmail.com> 2009/10/19 Oleg Broytman > On Mon, Oct 19, 2009 at 10:28:58PM +0800, starwing wrote: > > Oleg Broytman ??????: > >> Single-line lambdas are good enough, > >> and if you need more - just create a named function. > >> > > BUT, why we need a name? just to process something or make decide, > > sometimes code itself is enough. > > A multiline function is certainly not a simple piece of code; it > requires documentation - docstring, comments - and the name is a part of > the documentation. > > Having used languages that allow multiline anonymous functions (C#, Javascript) I have to say that they *can* improve the readability of code. Being forced to define functions ahead of where they belong in the logical flow of the program is not a feature of Python but a restriction. To dogmatically assert that multiline anonymous functions are *necessarily* less clear simply does the reputation of Python damage to those who use them already in other languages and know that this isn't true. I accept that it is a necessary restriction in Python (basically impossible to find a clean syntax that fits in with indentation for block structure) but to pretend that it is anything other than a restriction insults the intelligence of your audience. Michael > > and, is there performance problems when you define a inner function in > > another function? (that's, that function will define every time you call > > the function or not?) > > A lambda, like an inner function, is recreated every time, so it's > certainly no better than a named function. > You should profile your program to prove there is really a performance > degradation. And if there is - create a global function instead. > > Oleg. > -- > Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru > Programmers don't die, they just GOSUB without RETURN. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Oct 20 01:06:29 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 20 Oct 2009 10:06:29 +1100 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <20091019140447.GA8510@phd.pp.ru> References: <4ADC6A92.3060206@qq.com> <20091019140447.GA8510@phd.pp.ru> Message-ID: <200910201006.30622.steve@pearwood.info> On Tue, 20 Oct 2009 01:04:47 am Oleg Broytman wrote: > On Mon, Oct 19, 2009 at 09:33:06PM +0800, starwing wrote: > > current lambda only support single line > > syntax. it's not very good. > > This is where you've made an error. Single-line lambdas are good > enough, and if you need more - just create a named function. > > Oleg. Actually, lambda isn't limited to a single line, but to a single expression, and expressions can go over multiple lines: >>> data = [1, 2, 3, 4] >>> map(lambda n: ( ... n**3 - # comment goes here ... n**2 + ... n + 1), data) [2, 7, 22, 53] http://docs.python.org/reference/expressions.html#lambda The only limitation is that the lambda syntax is limited to a single expression, which means you can't call statements. If your "anonymous code block" is more complicated than a single short expression, it's too complicated to be "obviously correct" just from looking at it, and so it should be documented and tested. The restriction on lambda forces people to move the code block into a named function, which has the happy consequence of encouraging documentation and testing. -- Steven D'Aprano From grosser.meister.morti at gmx.net Tue Oct 20 01:11:48 2009 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Tue, 20 Oct 2009 01:11:48 +0200 Subject: [Python-ideas] Minimalist `except` syntax. In-Reply-To: <9d153b7c0910191328g6da5e574x2923db301ba0a57@mail.gmail.com> References: <712250c3-27f9-4ba0-878f-71de199e6e3e@w19g2000yqk.googlegroups.com> <9d153b7c0910191328g6da5e574x2923db301ba0a57@mail.gmail.com> Message-ID: <4ADCF234.7070207@gmx.net> On 10/19/2009 10:28 PM, Yuvgoog Greenle wrote: > On Mon, Oct 19, 2009 at 12:07 PM, ryles wrote: >> The explicit 'pass' improves readability by showing that the author >> made a conscious decision to ignore the error. In the suggested form, >> it looks more like the author took a coffee break in the middle of >> coding. > > -1 on allowing code that looks like a coffee break. Exactly what I thought, just I would not have been able to phrase it that concise. :) -1 for this syntax -0 for the idea behind it From steve at pearwood.info Tue Oct 20 01:30:27 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 20 Oct 2009 10:30:27 +1100 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <6f4025010910191533gf7d47b1gf0e7465bf96e0c84@mail.gmail.com> References: <4ADC6A92.3060206@qq.com> <20091019143942.GB8510@phd.pp.ru> <6f4025010910191533gf7d47b1gf0e7465bf96e0c84@mail.gmail.com> Message-ID: <200910201030.27448.steve@pearwood.info> On Tue, 20 Oct 2009 09:33:48 am Michael Foord wrote: > 2009/10/19 Oleg Broytman > > > On Mon, Oct 19, 2009 at 10:28:58PM +0800, starwing wrote: > > > Oleg Broytman ??????: > > >> Single-line lambdas are good enough, > > >> and if you need more - just create a named function. > > > > > > BUT, why we need a name? just to process something or make > > > decide, sometimes code itself is enough. > > > > A multiline function is certainly not a simple piece of code; it > > requires documentation - docstring, comments - and the name is a > > part of the documentation. > > Having used languages that allow multiline anonymous functions (C#, > Javascript) I have to say that they *can* improve the readability of > code. Given the difficulty in testing such anonymous functions, what do they do to the correctness of programs? > Being forced to define functions ahead of where they belong in > the logical flow of the program is not a feature of Python but a > restriction. I don't see that this restriction is worth the hours and hours people have spent on it. If I have code like this: function(1, None, True, { # define an anonymous dict 'a': set([1, 2, 4, 8, ...], 'b': set([2, 4, 6, 9, 12, 16, 20, ...], 'c': set([1234, 1342, 1432, ...], ... 'z': set([12, 99, 151, 153, 745, ...]) }, flag, count=4, parrot='red') with a complicated data argument, then I can write it as an anonymous variable, but it actually hurts readability to do so. There's no problem with creating an anonymous dict of any complexity I like, but for readability and correctness I'm better off pulling it out of the function call and giving it a name: d = { 'a': set([1, 2, 4, 16, 128, ...], 'b': set([2, 4, 6, 9, 12, 16, 20, ...], 'c': set([1234, 1342, 1432, ...], ... 'z': set([12, 99, 151, 153, 745, ...]) } # print d function(1, None, True, d, flag, count=4, parrot='red') In fact, if it's that complicated, I'm better off creating a function to generate it, so I can test it: function(1, None, True, d_factory(), flag, count=4, parrot='red') My point is that when it comes to data, we have anonymous data that is easy and straightforward to define, and yet we *still* prefer to give data names and define it ahead of where we use it. So why is code different? Why does defining code ahead of where you use it such a bad thing that the lambda restriction generates so much attention? -- Steven D'Aprano From stephen at xemacs.org Tue Oct 20 07:14:57 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 20 Oct 2009 14:14:57 +0900 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <6f4025010910191533gf7d47b1gf0e7465bf96e0c84@mail.gmail.com> References: <4ADC6A92.3060206@qq.com> <20091019140447.GA8510@phd.pp.ru> <4ADC77AA.2090807@qq.com> <20091019143942.GB8510@phd.pp.ru> <6f4025010910191533gf7d47b1gf0e7465bf96e0c84@mail.gmail.com> Message-ID: <8763aaeje6.fsf@uwakimon.sk.tsukuba.ac.jp> Michael Foord writes: > I accept that it is a necessary restriction in Python I agree with your point that it is a restriction. But the connotation of "necessary" that it would be desirable to relax it if there were a good syntax, I don't know about that. Up to about ten years ago, I used lambda in Lisp code a lot. Since then I've found that use cases are being pried from my grasp one by one. I find nowadays that even one-liners that are simply currying a multiargument function are more readable when defined as named functions. From stephen at xemacs.org Tue Oct 20 07:37:15 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 20 Oct 2009 14:37:15 +0900 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <4ADCE16E.2000208@gmail.com> References: <4ADC6A92.3060206@qq.com> <20091019140447.GA8510@phd.pp.ru> <4ADC77AA.2090807@qq.com> <873a5fds70.fsf@benfinney.id.au> <4ADCE16E.2000208@gmail.com> Message-ID: <874opueid0.fsf@uwakimon.sk.tsukuba.ac.jp> Nick Coghlan writes: > For myself, I don't actually agree it's a valid design rule - I think > anonymous blocks have legitimate use cases (see Ars Technica's writeup > of the Apple's new Grand Central Dispatch and C-level anonymous block > system in OS X 10.6). That doesn't look like what "anonymous block" means to me. It looks like a lambda. The difference is that an block resolves all its non-argument references in the calling context, eg, as a C macro without arguments would. But I don't see how you can assign a C macro to a variable and call it at runtime.... Now, the cases that Ruby programmers I know always propose to me as use cases for anonymous blocks rely on conventions for naming certain objects used by their blocks (typically iteration variables), thus avoiding the need to specify arguments for them. So a block seems to be a conventional way of currying a more general function to the context of a specific suite. From debatem1 at gmail.com Tue Oct 20 08:46:14 2009 From: debatem1 at gmail.com (geremy condra) Date: Tue, 20 Oct 2009 02:46:14 -0400 Subject: [Python-ideas] [Python-Dev] Proposal : Python Trusted Computing API In-Reply-To: References: Message-ID: On Mon, Oct 19, 2009 at 2:29 AM, Abhiram Kasina wrote: > Hi > > Trusted Computing (TC) is a technology developed and promoted by the Trusted > Computing Group (TCG)[3]. So, basically the group came up with these chips > called TPM chips which are present on most motherboards nowadays. The main > purpose of it is to enhance security so that infected executables don't run. > It also provides memory curtaining such that cryptographic keys won't be > accessible and many other features. There was a criticism on this from the > FOSS community as well that it enables DRM. No wonder, it is being pushed by > Intel, Microsoft, AMD, etc.. I personally have grave concerns about this technology, independent of who is advancing it- just reading the links you provide makes it clear that its primary purpose is to restrict the degree of control a user can exercise over their own data, programs, and machine. Is it any wonder that the same people who find DRM abhorrent find the technologies used to advance it equally distasteful? > But personally I think its a good idea from security point of view. Besides sealed storage and memory curtaining- which I admit have seductive security implications- all of the supposed advantages of a TPM can be replicated using non-interactive zero knowledge proofs. I'd rather we put that capability into the standard library- but I strongly doubt that's going to happen any time soon. > So, currently there is an TSS (TCG Software Stack)[1] API written in C. And > TrustedJava[2] is a project which ported it to Java and is going to be > included in the standard API of Java soon. They have 2 versions of it. One > is a simple wrapper on top of the API and the other is a whole > implementation of the stack in Java. > > My proposal is we create an API for it in python. I also suspect that if this were a simple undertaking, there wouldn't be a 'we' in this sentence. > Reason: I am a developer in Umit and I think Python is a very good platform > for developing applications. So, why not create an API which helps in > developing secure applications? If you were suggesting adding a crypto API to python, I'd be all for it- but you're suggesting adding the ability to have Python software vendors remotely cripple the code on your machine. I just can't get behind that, and while you're sure to hear wildly divergent opinions on this board, I suspect that mine will not be an uncommon sentiment. Geremy Condra From ben+python at benfinney.id.au Tue Oct 20 09:31:32 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 20 Oct 2009 18:31:32 +1100 Subject: [Python-ideas] [Python-Dev] Proposal : Python Trusted Computing API References: Message-ID: <87pr8icyi3.fsf@benfinney.id.au> geremy condra writes: > If you were suggesting adding a crypto API to python, I'd be all > for it- but you're suggesting adding the ability to have Python > software vendors remotely cripple the code on your machine. > I just can't get behind that, and while you're sure to hear wildly > divergent opinions on this board, I suspect that mine will not > be an uncommon sentiment. Agreed. Though I don't draw much water here, being a Python user rather than a core developer, I totally agree that technologies to subvert an owner's control over the behaviour of their own machine are anathema to a free culture. Such technologies are to be resisted, and they shall not get a jot of my support in any form if I can help it. -- \ ?As the most participatory form of mass speech yet developed, | `\ the Internet deserves the highest protection from governmental | _o__) intrusion.? ?U.S. District Court Judge Dalzell | Ben Finney From masklinn at masklinn.net Tue Oct 20 10:36:40 2009 From: masklinn at masklinn.net (Masklinn) Date: Tue, 20 Oct 2009 10:36:40 +0200 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <4ADCE16E.2000208@gmail.com> References: <4ADC6A92.3060206@qq.com> <20091019140447.GA8510@phd.pp.ru> <4ADC77AA.2090807@qq.com> <873a5fds70.fsf@benfinney.id.au> <4ADCE16E.2000208@gmail.com> Message-ID: <8DFB62D9-740A-4655-A420-675EFB951AB5@masklinn.net> On 20 Oct 2009, at 00:00 , Nick Coghlan wrote: > > However, for Python, the idea of having anonymous blocks runs headlong > into the use of indentation for block delineation. I'm not sure that's the case, Haskell uses indentation for block delineation too and doesn't (as far as I know, I might be wrong) impose further restrictions on anonymous functions than on named ones. Though this might have to do with? > You have to [?] embed significant whitespace inside an expression > rather than > only having it between statements. > haskell only dealing with expressions and not with statements, thus "blocks" are expressions and it doesn't have to deal with a mismatch between statements and expressions. From masklinn at masklinn.net Tue Oct 20 10:39:44 2009 From: masklinn at masklinn.net (Masklinn) Date: Tue, 20 Oct 2009 10:39:44 +0200 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <200910201006.30622.steve@pearwood.info> References: <4ADC6A92.3060206@qq.com> <20091019140447.GA8510@phd.pp.ru> <200910201006.30622.steve@pearwood.info> Message-ID: On 20 Oct 2009, at 01:06 , Steven D'Aprano wrote: > If your "anonymous code block" is more complicated than a single short > expression, it's too complicated to be "obviously correct" just from > looking at it, and so it should be documented and tested. I have the same issue with this as with Oleg's pronouncement (which runs along the same lines): does that mean you never write for or if statements with more than a single line in them? Yet these are also "anonymous code blocks" which you say here are "too complicated to be obviously correct just from looking at [them]". If that's the case, why should compound statements even be allowed? They're breeding grounds for multiline anonymous blocks of code? From masklinn at masklinn.net Tue Oct 20 10:42:46 2009 From: masklinn at masklinn.net (Masklinn) Date: Tue, 20 Oct 2009 10:42:46 +0200 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <200910201030.27448.steve@pearwood.info> References: <4ADC6A92.3060206@qq.com> <20091019143942.GB8510@phd.pp.ru> <6f4025010910191533gf7d47b1gf0e7465bf96e0c84@mail.gmail.com> <200910201030.27448.steve@pearwood.info> Message-ID: <8D156E33-F749-4246-A2D3-6032A4F1553F@masklinn.net> On 20 Oct 2009, at 01:30 , Steven D'Aprano wrote: > > Given the difficulty in testing such anonymous functions, what do they > do to the correctness of programs? Nothing different than what `with` or `while` statements do. > My point is that when it comes to data, we have anonymous data that is > easy and straightforward to define, and yet we *still* prefer to give > data names and define it ahead of where we use it. Not in every case. > So why is code different? Why indeed? As you said above, we have anonymous data that is easy and straightforward to define, why treat code differently and prevent this even when that code is just as easy and straightforward to define? From masklinn at masklinn.net Tue Oct 20 10:46:44 2009 From: masklinn at masklinn.net (Masklinn) Date: Tue, 20 Oct 2009 10:46:44 +0200 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <874opueid0.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4ADC6A92.3060206@qq.com> <20091019140447.GA8510@phd.pp.ru> <4ADC77AA.2090807@qq.com> <873a5fds70.fsf@benfinney.id.au> <4ADCE16E.2000208@gmail.com> <874opueid0.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On 20 Oct 2009, at 07:37 , Stephen J. Turnbull wrote: > Nick Coghlan writes: > >> For myself, I don't actually agree it's a valid design rule - I think >> anonymous blocks have legitimate use cases (see Ars Technica's >> writeup >> of the Apple's new Grand Central Dispatch and C-level anonymous block >> system in OS X 10.6). > > That doesn't look like what "anonymous block" means to me. It looks > like a lambda. > An anonymous block and a lambda are the exact same thing. > The difference is that an block resolves all its non-argument > references in the calling context Are you talking about forming closures here? > Now, the cases that Ruby programmers I know always propose to me as > use cases for anonymous blocks rely on conventions for naming certain > objects used by their blocks (typically iteration variables) Please forgive my disbelief but "what?" Ruby's blocks take arguments (they're fundamentally nothing more than anonymous functions), I've never encountered a case where "anonymous blocks [relied] on conventions for naming certain objects used within their blocks". > thus avoiding the need to specify arguments for them. See above. Some languages (Scala for instance) have "magical placeholders" to avoid having to name arguments (`_*3` is equivalent to `i => i*3`) but that's pretty much the extent of it. > So a block seems to > be a conventional way of currying a more general function to the > context of a specific suite. And I don't know what to make of that one either. Blocks don't have any more to do with currying than functions in general do. From ncoghlan at gmail.com Tue Oct 20 12:56:55 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 20 Oct 2009 20:56:55 +1000 Subject: [Python-ideas] [Python-Dev] Proposal : Python Trusted Computing API In-Reply-To: <87pr8icyi3.fsf@benfinney.id.au> References: <87pr8icyi3.fsf@benfinney.id.au> Message-ID: <4ADD9777.6010004@gmail.com> Ben Finney wrote: > geremy condra writes: > >> If you were suggesting adding a crypto API to python, I'd be all >> for it- but you're suggesting adding the ability to have Python >> software vendors remotely cripple the code on your machine. >> I just can't get behind that, and while you're sure to hear wildly >> divergent opinions on this board, I suspect that mine will not >> be an uncommon sentiment. > > Agreed. Though I don't draw much water here, being a Python user rather > than a core developer, I totally agree that technologies to subvert an > owner's control over the behaviour of their own machine are anathema to > a free culture. Such technologies are to be resisted, and they shall not > get a jot of my support in any form if I can help it. Such techologies are actually fairly pointless, as giving both a secret and a key to that secret to the same party and expecting the secret to stay that way is completely missing the point of how crypto works. You can make it difficult for the user to acquire the secret, but there are just too many attack vectors to keep *everyone* out, and once one person breaks it, the wonders of the internet mean that pretty much everyone else can break it as well. The ease with which even Python bytecode can be monkey-patched at run time to null out the trusted API calls is likely to make such an undertaking in Python even more pointless than usual. Regardless, the answer I gave on python-dev stands: this has nothing to do with Python as a language or the standard library. The OP is free to develop whatever Python extension libraries they like and see if they can attract much outside interest. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From weasley_wx at qq.com Tue Oct 20 12:43:43 2009 From: weasley_wx at qq.com (=?gbk?B?0KHS7Q==?=) Date: Tue, 20 Oct 2009 18:43:43 +0800 Subject: [Python-ideas] a new lambda syntax Message-ID: Okay...but, Two--no, Three question: - is it necessary to implement it? - is it diffcult to implement it? - is it helps to import Python statement? if so, I will try to implement it (I'm trying to read the source of Python, but it sames too complicated :-( -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Oct 20 13:04:13 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 20 Oct 2009 21:04:13 +1000 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <874opueid0.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4ADC6A92.3060206@qq.com> <20091019140447.GA8510@phd.pp.ru> <4ADC77AA.2090807@qq.com> <873a5fds70.fsf@benfinney.id.au> <4ADCE16E.2000208@gmail.com> <874opueid0.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4ADD992D.90106@gmail.com> Stephen J. Turnbull wrote: > Nick Coghlan writes: > > > For myself, I don't actually agree it's a valid design rule - I think > > anonymous blocks have legitimate use cases (see Ars Technica's writeup > > of the Apple's new Grand Central Dispatch and C-level anonymous block > > system in OS X 10.6). > > That doesn't look like what "anonymous block" means to me. It looks > like a lambda. The difference lies in the fact that in C, object references are non-local by default - you have to declare them explicitly in the current scope to make them local. Hence the code you stick inline in Apple's new C extensions can manipulate locals as if it were just a normal part of the current function. Accordingly, I find the idea of a new function-like construct where all non-argument variable references are nonlocal by default to be a potentially interesting one. It has nothing to do with Python's inherent syntax problems with nesting statements inside expressions though, which is the reason I snipped that digression from my previous message. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From fuzzyman at voidspace.org.uk Tue Oct 20 13:16:01 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 20 Oct 2009 12:16:01 +0100 Subject: [Python-ideas] What about feature X? Message-ID: <6f4025010910200416n1c1dc459r92fccfbaa1a755d@mail.gmail.com> An interesting list of what it takes to add a new feature from Eric Lippert, one of the C# team: http://blogs.msdn.com/ericlippert/archive/2009/10/05/why-no-extension-properties.aspx As I'm fond of pointing out, the answer to every question of the form "why doesn't product X have feature Y?" is the same. It's because in order for a product to have a feature, that feature must be: - thought of in the first place - desired - designed - specified - implemented - tested - documented - shipped to customers You've got to hit every single one of those things, otherwise, no feature. Other than perhaps the last entry in the list all of them apply to new features in Python. This is without covering the burden that adding new syntax or features adds to those learning the language or reading code. All the best, Michael -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at gmail.com Tue Oct 20 13:17:03 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Tue, 20 Oct 2009 12:17:03 +0100 Subject: [Python-ideas] What about feature X? Message-ID: <6f4025010910200417x3b0dc549t9b49b000987cf1ed@mail.gmail.com> An interesting list of what it takes to add a new feature from Eric Lippert, one of the C# team: http://blogs.msdn.com/ericlippert/archive/2009/10/05/why-no-extension-properties.aspx As I'm fond of pointing out, the answer to every question of the form "why doesn't product X have feature Y?" is the same. It's because in order for a product to have a feature, that feature must be: - thought of in the first place - desired - designed - specified - implemented - tested - documented - shipped to customers You've got to hit every single one of those things, otherwise, no feature. Other than perhaps the last entry in the list all of them apply to new features in Python. This is without covering the burden that adding new syntax or features adds to those learning the language or reading code. All the best, Michael -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Oct 20 13:21:02 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 20 Oct 2009 22:21:02 +1100 Subject: [Python-ideas] a new lambda syntax In-Reply-To: References: <4ADC6A92.3060206@qq.com> <200910201006.30622.steve@pearwood.info> Message-ID: <200910202221.03233.steve@pearwood.info> On Tue, 20 Oct 2009 07:39:44 pm Masklinn wrote: > On 20 Oct 2009, at 01:06 , Steven D'Aprano wrote: > > If your "anonymous code block" is more complicated than a single > > short expression, it's too complicated to be "obviously correct" > > just from looking at it, and so it should be documented and tested. > > I have the same issue with this as with Oleg's pronouncement (which > runs along the same lines): does that mean you never write for or if > statements with more than a single line in them? No, you are correct, my statement is overly strong. Please insert "very likely" between "it's" and "too complicated". > Yet these are also "anonymous code blocks" which you say here are > "too complicated to be obviously correct just from looking at > [them]". > > If that's the case, why should compound statements even be allowed? > They're breeding grounds for multiline anonymous blocks of code? There is at least one difference between compound statements and multi-statement anonymous code blocks: You don't pass compound statements around as input to other functions, so you know the environment that they're being called in, and can more easily reason about their correctness. But even then, compound statements are good candidates for being converted into named functions too. I usually start with something like this: if cond: do this do that do something else fe fi fo fum ... else: one fish two fish red fish blue fish ... and by the time I'm finished I've got something like this: if cond: call_one() else: call_two() The multi-line blocks are split into functions, each with their own documentation and tests. Of course I don't do this literally every single time. There's no hard rule about how many lines are allowed, just like there's no hard rule about how complex a data structure can be before you give it a name. But my point is that the restriction on lambda is not an odious one, or at least not so odious that it deserves the amount of heat that it generates. There are uses for multi-statement anonymous functions, but there is nothing that an anonymous function can do that a named function can't.[1] In my opinion, it simply isn't enough of a burden to define a function before you use it, using a "don't care" name, to deserve the amount of words written about potential multi-statement lambda syntax. If Python had different syntax, then perhaps we'd have anonymous functions like Ruby. But we don't, and frankly I don't think it is much of a restriction. [1] Some way to write an anonymous code block, which isn't a function and simply runs in the local environment where it is called as if it were written in-line, would be more interesting to me. -- Steven D'Aprano From masklinn at masklinn.net Tue Oct 20 13:34:11 2009 From: masklinn at masklinn.net (Masklinn) Date: Tue, 20 Oct 2009 13:34:11 +0200 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <200910202221.03233.steve@pearwood.info> References: <4ADC6A92.3060206@qq.com> <200910201006.30622.steve@pearwood.info> <200910202221.03233.steve@pearwood.info> Message-ID: <2DEB8009-3607-49FC-9C3B-ED08C9B3A245@masklinn.net> On 20 Oct 2009, at 13:21 , Steven D'Aprano wrote: > > You don't pass compound statements around as input to other functions, > so you know the environment that they're being called in, and can more > easily reason about their correctness. > But unless the language is dynamically scoped (which is getting pretty rare these days, and which Python definitely isn't) the environment in which the block is executed is not quite relevant, since the free variables are bound based on its lexical scope (the environment in which the block is created) and the rest is made of argument (as in every other function). > In my opinion, it simply isn't enough of a burden to define a function > before you use it, using a "don't care" name, to deserve the amount of > words written about potential multi-statement lambda syntax. If Python > had different syntax, then perhaps we'd have anonymous functions like > Ruby. But we don't, and frankly I don't think it is much of a > restriction. > The biggest (by far) advantages I see to good anonymous functions (note: Ruby's aren't, as far as I'm concerned, because due to their nature they don't easily scale from 1/call to 2+/call) are in flexibility, freedom of experimentation and possibility to keep the core language itself small: had Python had "full-blown" anonymous functions, it wouldn't have been necessary to add the `with` statement to the language. It could just as well have been implemented through a protocol or the stdlib, and people would have been free to toy with it in their projects long before it was added to the core language. It also gives the possibility of making OO much less imperative (right now if one wants customized code paths, one usually has to rely on extracting data from objects, performing imperative operations and then feeding the result back to some other object, but objects could just as well provide "hooks" thought callables), and of creating custom control structures (see `with` above, but it applies to far more stuff). In fact, Smalltalk quite clearly demonstrated that with messages (method calls) and blocks (anonymous functions) you basically didn't need "hard-coded" control structures anymore. Now Smalltalk probably went a bit too far for most people, even 30 years later, but it does show how powerful this combination is, and how it allows the language's users to evolve the language itself, without having to change its core syntax. From phd at phd.pp.ru Tue Oct 20 13:49:58 2009 From: phd at phd.pp.ru (Oleg Broytman) Date: Tue, 20 Oct 2009 15:49:58 +0400 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <2DEB8009-3607-49FC-9C3B-ED08C9B3A245@masklinn.net> References: <4ADC6A92.3060206@qq.com> <200910201006.30622.steve@pearwood.info> <200910202221.03233.steve@pearwood.info> <2DEB8009-3607-49FC-9C3B-ED08C9B3A245@masklinn.net> Message-ID: <20091020114958.GA21044@phd.pp.ru> On Tue, Oct 20, 2009 at 01:34:11PM +0200, Masklinn wrote: > Now Smalltalk probably went a > bit too far for most people, even 30 years later This is The Reason Number Two - there have to be a balance between what features are accepted in the language and what are rejected. Python developers decided that anonymous code blocks are allowed in a few special places and are forbidden generally. IWBN to have a PEP listing all pro et contra arguments so we don't need to repeat the same arguments over and over. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From masklinn at masklinn.net Tue Oct 20 13:54:59 2009 From: masklinn at masklinn.net (Masklinn) Date: Tue, 20 Oct 2009 13:54:59 +0200 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <20091020114958.GA21044@phd.pp.ru> References: <4ADC6A92.3060206@qq.com> <200910201006.30622.steve@pearwood.info> <200910202221.03233.steve@pearwood.info> <2DEB8009-3607-49FC-9C3B-ED08C9B3A245@masklinn.net> <20091020114958.GA21044@phd.pp.ru> Message-ID: <60AA7691-926E-42A4-B6BB-8D7D692BDED8@masklinn.net> On 20 Oct 2009, at 13:49 , Oleg Broytman wrote: > On Tue, Oct 20, 2009 at 01:34:11PM +0200, Masklinn wrote: >> Now Smalltalk probably went a >> bit too far for most people, even 30 years later > > This is The Reason Number Two Erm? what was reason number one? > - there have to be a balance between what > features are accepted in the language and what are rejected. Python > developers decided that anonymous code blocks are allowed in a few > special > places and are forbidden generally. Have they? I've seen the decision that anonymous functions generally aren't allowed to be useful, but there are anonymous code blocks everywhere. You just can't pass them around. From phd at phd.pp.ru Tue Oct 20 14:13:22 2009 From: phd at phd.pp.ru (Oleg Broytman) Date: Tue, 20 Oct 2009 16:13:22 +0400 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <60AA7691-926E-42A4-B6BB-8D7D692BDED8@masklinn.net> References: <4ADC6A92.3060206@qq.com> <200910201006.30622.steve@pearwood.info> <200910202221.03233.steve@pearwood.info> <2DEB8009-3607-49FC-9C3B-ED08C9B3A245@masklinn.net> <20091020114958.GA21044@phd.pp.ru> <60AA7691-926E-42A4-B6BB-8D7D692BDED8@masklinn.net> Message-ID: <20091020121322.GB21044@phd.pp.ru> On Tue, Oct 20, 2009 at 01:54:59PM +0200, Masklinn wrote: > On 20 Oct 2009, at 13:49 , Oleg Broytman wrote: >> On Tue, Oct 20, 2009 at 01:34:11PM +0200, Masklinn wrote: >>> Now Smalltalk probably went a >>> bit too far for most people, even 30 years later >> >> This is The Reason Number Two > Erm? what was reason number one? http://www.python.org/dev/peps/pep-3099/#core-language "multi-line arguments to function calls, for instance. That is just plain ugly." My argument about anonymous/named functions is a part of the explanation for the ugliness. >> - there have to be a balance between what >> features are accepted in the language and what are rejected. Python >> developers decided that anonymous code blocks are allowed in a few >> special >> places and are forbidden generally. > Have they? I've seen the decision that anonymous functions generally > aren't allowed to be useful, but there are anonymous code blocks > everywhere. You just can't pass them around. The number of compound statements in Python is pretty low. def, for, if, try, with, while. A PEP, please? Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From solipsis at pitrou.net Tue Oct 20 15:01:31 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 20 Oct 2009 13:01:31 +0000 (UTC) Subject: [Python-ideas] =?utf-8?q?breaking_cycles_that_include_=5F=5Fdel?= =?utf-8?b?X18=?= References: Message-ID: Daniel Stutzbach writes: > > Unfortunately, if the object ends up in a cycle, then the recipe can't do > its job because __del__ will never be called. You don't need __del__, a weakref is good enough since you only need to remember an adequate string representation of the object: class Widget: closed = False def __init__(self): # ... def error_not_closed(_, r=repr(self)): log_an_error('%s not closed properly' % r) self._wr_not_closed = weakref.ref(self, error_not_closed) def close(self): if self.closed: return self.closed = True # Destroy weakref self._wr_not_closed = None # Free various resources from ctypes, etc. Regards Antoine. From daniel at stutzbachenterprises.com Tue Oct 20 15:27:14 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 20 Oct 2009 08:27:14 -0500 Subject: [Python-ideas] breaking cycles that include __del__ In-Reply-To: References: Message-ID: On Tue, Oct 20, 2009 at 8:01 AM, Antoine Pitrou wrote: > You don't need __del__, a weakref is good enough since you only need to > remember > an adequate string representation of the object: > > def error_not_closed(_, r=repr(self)): > log_an_error('%s not closed properly' % r) > self._wr_not_closed = weakref.ref(self, error_not_closed) > Thank you! This is a much simpler recipe than the others I have seen. It should probably be placed right in the documentation for __del__, as a suggested alternative. Having direct support from the garbage collector would still be more memory efficient (instead of requiring a weakref and a bound function object), but it will do for many purposes in a pinch. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Oct 20 15:34:09 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 20 Oct 2009 23:34:09 +1000 Subject: [Python-ideas] What about feature X? In-Reply-To: <6f4025010910200417x3b0dc549t9b49b000987cf1ed@mail.gmail.com> References: <6f4025010910200417x3b0dc549t9b49b000987cf1ed@mail.gmail.com> Message-ID: <4ADDBC51.1070606@gmail.com> Michael Foord wrote: > An interesting list of what it takes to add a new feature from Eric > Lippert, one of the C# team: > > http://blogs.msdn.com/ericlippert/archive/2009/10/05/why-no-extension-properties.aspx > > As I'm fond of pointing out, the answer to every question of the form > "why doesn't product X have feature Y?" is the same. It's because in > order for a product to have a feature, that feature must be: > > * thought of in the first place > * desired > * designed > * specified > * implemented > * tested > * documented > * shipped to customers > > You've got to hit every single one of those things, otherwise, no feature. > > > Other than perhaps the last entry in the list all of them apply to new > features in Python. This is without covering the burden that adding new > syntax or features adds to those learning the language or reading code. We have the last one as well - features don't hit really widespread use until the first Python version they are in has been baked into a few Linux distros (and these days, probably a Mac OS X release as well). I don't believe there is any particularly strong cause and effect there, just a matter of comparable time frames until developers feel they aren't restricting their deployment options too much by using the new features instead of remaining compatible with the previous version. It does mean that flawed designs (e.g. contextlib.nested) can take a while to be picked up because people initially aren't using them heavily enough to notice the limitations. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From bruce at leapyear.org Tue Oct 20 17:28:16 2009 From: bruce at leapyear.org (Bruce Leban) Date: Tue, 20 Oct 2009 08:28:16 -0700 Subject: [Python-ideas] nonlocal functions Message-ID: On Tue, Oct 20, 2009 at 4:04 AM, Nick Coghlan wrote: > The difference lies in the fact that in C, object references are > non-local by default - you have to declare them explicitly in the > current scope to make them local. > > Accordingly, I find the idea of a new function-like construct where all > non-argument variable references are nonlocal by default to be a > potentially interesting one. > Hmm. That's an interesting idea. I don't know enough about internals. It might be possible to do this with a decorator def foo(): a = 1 @allnonlocal def bar(): a = 2 bar() return a foo() => 2 However, even if possible, that would make everything nonlocal with no way to pick which ones. Here's another idea: def foo(): a = 1 b = 3 nonlocal def bar(): local b a = 2 b = 4 return (a,b) foo() => (2, 3) Prepending nonlocal to a function definition is equivalent to applying nonlocal to every variable referenced in that function unless the variable is declared local. For backward compatibility, the local statement would only be only recognized inside nonlocal functions. (Thus if you have a function that uses 'local' as a variable it would not need to be changed unless you decided to stick nonlocal in front of it.) Alternative syntax: def bar() nonlocal: --- Bruce http://www.vroospeak.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Tue Oct 20 18:00:51 2009 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 20 Oct 2009 17:00:51 +0100 Subject: [Python-ideas] nonlocal functions In-Reply-To: References: Message-ID: <4ADDDEB3.3010204@mrabarnett.plus.com> Bruce Leban wrote: > On Tue, Oct 20, 2009 at 4:04 AM, Nick Coghlan > wrote: > > The difference lies in the fact that in C, object references are > non-local by default - you have to declare them explicitly in the > current scope to make them local. > > Accordingly, I find the idea of a new function-like construct where all > non-argument variable references are nonlocal by default to be a > potentially interesting one. > > > Hmm. That's an interesting idea. > > I don't know enough about internals. It might be possible to do this > with a decorator > > def foo(): > a = 1 > @allnonlocal > def bar(): > a = 2 > bar() > return a > > foo() => 2 > > However, even if possible, that would make everything nonlocal with no > way to pick which ones. Here's another idea: > > def foo(): > a = 1 > b = 3 > nonlocal def bar(): > local b > a = 2 > b = 4 > return (a,b) > > foo() => (2, 3) > > Prepending nonlocal to a function definition is equivalent to applying > nonlocal to every variable referenced in that function unless the > variable is declared local. For backward compatibility, the local > statement would only be only recognized inside nonlocal functions. (Thus > if you have a function that uses 'local' as a variable it would not need > to be changed unless you decided to stick nonlocal in front of it.) > > Alternative syntax: > def bar() nonlocal: > I'd prefer: def foo(): a = 1 b = 3 def bar(): nonlocal * local b a = 2 b = 4 return (a,b) if a later 'local' can override an earlier 'nonlocal', or: def foo(): a = 1 b = 3 def bar(): local b nonlocal * a = 2 b = 4 return (a,b) if a 'nonlocal' can act as a catch-all for any names not previously mentioned. From masklinn at masklinn.net Tue Oct 20 18:43:15 2009 From: masklinn at masklinn.net (Masklinn) Date: Tue, 20 Oct 2009 18:43:15 +0200 Subject: [Python-ideas] nonlocal functions In-Reply-To: <4ADDDEB3.3010204@mrabarnett.plus.com> References: <4ADDDEB3.3010204@mrabarnett.plus.com> Message-ID: <494DB4DC-CA0E-4F3D-BF32-8CE0E19DE5E5@masklinn.net> On 20 Oct 2009, at 18:00 , MRAB wrote: > Bruce Leban wrote: > I'd prefer: > > def foo(): > a = 1 > b = 3 > def bar(): > nonlocal * > local b > a = 2 > b = 4 > return (a,b) > > if a later 'local' can override an earlier 'nonlocal', or: > > def foo(): > a = 1 > b = 3 > def bar(): > local b > nonlocal * > a = 2 > b = 4 > return (a,b) > > if a 'nonlocal' can act as a catch-all for any names not previously > mentioned. If you go that route, I'd suggest removing all the legacy stuff and just using "let" to create a new binding in the current scope: def foo(): let a = 1 let b = 3 def bar(): a = 2 let b = 4 bar() return (a, b) # returns (2, 3) Plus it allows the compiler to statically catch typos: def foo(): let some_very_complex_type = 42 def bar(): some_very_complex_typo = "oh noes" # not a new binding, no existing binding for that name => warning(*) bar() return some_very_complex_type You can even get "two in one" and use the keyword to create explicit scopes: def foo(): let foo="whatever", bar=42: do_something_with(foo, bar) return foo # warning, no foo in scope which would probably be useful mostly for code clarity purposes. *: warning, not error, because the programmer could be manipulating locals() directly, or something like that. From debatem1 at gmail.com Tue Oct 20 18:45:55 2009 From: debatem1 at gmail.com (geremy condra) Date: Tue, 20 Oct 2009 12:45:55 -0400 Subject: [Python-ideas] [Python-Dev] Proposal : Python Trusted Computing API In-Reply-To: <4ADD9777.6010004@gmail.com> References: <87pr8icyi3.fsf@benfinney.id.au> <4ADD9777.6010004@gmail.com> Message-ID: On Tue, Oct 20, 2009 at 6:56 AM, Nick Coghlan wrote: > Ben Finney wrote: >> geremy condra writes: >> >>> If you were suggesting adding a crypto API to python, I'd be all >>> for it- but you're suggesting adding the ability to have Python >>> software vendors remotely cripple the code on your machine. >>> I just can't get behind that, and while you're sure to hear wildly >>> divergent opinions on this board, I suspect that mine will not >>> be an uncommon sentiment. >> >> Agreed. Though I don't draw much water here, being a Python user rather >> than a core developer, I totally agree that technologies to subvert an >> owner's control over the behaviour of their own machine are anathema to >> a free culture. Such technologies are to be resisted, and they shall not >> get a jot of my support in any form if I can help it. > > Such techologies are actually fairly pointless, as giving both a secret > and a key to that secret to the same party and expecting the secret to > stay that way is completely missing the point of how crypto works. You > can make it difficult for the user to acquire the secret, but there are > just too many attack vectors to keep *everyone* out, and once one person > breaks it, the wonders of the internet mean that pretty much everyone > else can break it as well. It's sometimes called the "third law of cryptography": that there is no cryptographic solution to the problem in which the attacker and intended recipient are the same person. Unfortunately, TPMs get to bend the rules a little bit using the same technique I mentioned above- zero knowledge proofs. Ultimately, it means that it is possible to build repudiation schemes which do not leak keying data even in a system that is 100% observable, although it does so at the expense of significant overhead and a loss of flexibility. > The ease with which even Python bytecode can be monkey-patched at run > time to null out the trusted API calls is likely to make such an > undertaking in Python even more pointless than usual. This is one of the reasons I suspect there was a 'we' in the 'we should build an API': it would require vast and neverending support in python core in order to enforce the demands of the TPM. > Regardless, the answer I gave on python-dev stands: this has nothing to > do with Python as a language or the standard library. The OP is free to > develop whatever Python extension libraries they like and see if they > can attract much outside interest. Unfortunately, as I said above, it does- in order to make it work, the OP would need support from core. I very much hope that support won't develop. Geremy Condra From daniel at stutzbachenterprises.com Tue Oct 20 18:58:36 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 20 Oct 2009 11:58:36 -0500 Subject: [Python-ideas] breaking cycles that include __del__ In-Reply-To: References: Message-ID: I'm playing around with this, and in following variant the weakref callback is never called (in Python 2.6 and 3.1). I do not see why. I know I've created a cycle between the callback and the object itself, but the garbage collector should detect and free the cycle. import weakref, gc class Foo: def __init__(self): self._weakref = weakref.ref(self, self.__free__) def __free__(self): print("I'm free!") x = Foo() del x gc.collect() print('test') print(gc.garbage) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhamph at gmail.com Tue Oct 20 19:31:03 2009 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 20 Oct 2009 11:31:03 -0600 Subject: [Python-ideas] breaking cycles that include __del__ In-Reply-To: References: Message-ID: On Tue, Oct 20, 2009 at 10:58, Daniel Stutzbach wrote: > I'm playing around with this, and in following variant the weakref callback > is never called (in Python 2.6 and 3.1).? I do not see why.? I know I've > created a cycle between the callback and the object itself, but the garbage > collector should detect and free the cycle. > > import weakref, gc > class Foo: > ??? def __init__(self): > ??????? self._weakref = weakref.ref(self, self.__free__) > ??? def __free__(self): > ??????? print("I'm free!") > x = Foo() > del x > gc.collect() > print('test') > print(gc.garbage) Your weakref callback is a method of your object. The callback requires that method still be alive, but isn't triggered until self is deleted. If the GC does anything it'll have to include clearing the weakref, deleting the callback. Note that, for robustness, it's preferable to make the weakref globally reachable until it's called, such as in a set. Also note that Antoine's example saved a snapshot of the object's id. Storing state is harder, and requires you to encapsulate it in some other object. A list for Antoine's 'r' argument might be a decently simple bodge. -- Adam Olsen, aka Rhamphoryncus From stefan_ml at behnel.de Tue Oct 20 19:55:22 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 20 Oct 2009 19:55:22 +0200 Subject: [Python-ideas] nonlocal functions In-Reply-To: <494DB4DC-CA0E-4F3D-BF32-8CE0E19DE5E5@masklinn.net> References: <4ADDDEB3.3010204@mrabarnett.plus.com> <494DB4DC-CA0E-4F3D-BF32-8CE0E19DE5E5@masklinn.net> Message-ID: Masklinn wrote: > If you go that route, I'd suggest removing all the legacy stuff and just > using "let" to create a new binding in the current scope: > > def foo(): > let a = 1 > let b = 3 No, please. Why would you want to require so much typing for the most common case? And how often would you forget to declare a variable that way? Stefan From scott+python-ideas at scottdial.com Tue Oct 20 19:59:05 2009 From: scott+python-ideas at scottdial.com (Scott Dial) Date: Tue, 20 Oct 2009 13:59:05 -0400 Subject: [Python-ideas] breaking cycles that include __del__ In-Reply-To: References: Message-ID: <4ADDFA69.5000707@scottdial.com> Adam Olsen wrote: > Your weakref callback is a method of your object. The callback > requires that method still be alive, but isn't triggered until self is > deleted. This is made clear in the gcmodule.c comments, but is severely lacking from the actual documentation in the weakref module. But also, Antoine's is flawed in the same manner. Antoine's example works merely because the gc module is left out of the issue (there are no cycles in it). If you introduce a cycle, then it falls apart, just like Daniel's: import weakref, gc class Foo: def __init__(self): def call_free(_, r=repr(self)): print(r) self._weakref = weakref.ref(self, call_free) x,y = Foo(), Foo() x.y, y.x = y, x del x del y gc.collect() print(gc.garbage) The whole source of the confusion is documented in the handle_weakrefs() in Modules/gcmodule.c at line 600: /* Headache time. `op` is going away, and is weakly referenced by * `wr`, which has a callback. Should the callback be invoked? If wr * is also trash, no: * * 1. There's no need to call it. The object and the weakref are * both going away, so it's legitimate to pretend the weakref is * going away first. The user has to ensure a weakref outlives its * referent if they want a guarantee that the wr callback will get * invoked. * * 2. It may be catastrophic to call it. If the callback is also in * cyclic trash (CT), then although the CT is unreachable from * outside the current generation, CT may be reachable from the * callback. Then the callback could resurrect insane objects. * * Since the callback is never needed and may be unsafe in this case, * wr is simply left in the unreachable set. Note that because we * already called _PyWeakref_ClearRef(wr), its callback will never * trigger. * * OTOH, if wr isn't part of CT, we should invoke the callback: the * weakref outlived the trash. Note that since wr isn't CT in this * case, its callback can't be CT either -- wr acted as an external * root to this generation, and therefore its callback did too. So * nothing in CT is reachable from the callback either, so it's hard * to imagine how calling it later could create a problem for us. wr * is moved to wrcb_to_call in this case. */ if (IS_TENTATIVELY_UNREACHABLE(wr)) continue; The only way to guarantee the callback occurs is if you attach it to some other object that will outlive your object, *but* must also not get pulled into the same gc generation, otherwise it will *still not be called*. I believe this is the source of your advice to store the weakref in some globally reachable set. I believe given the way modules are currently deallocated, this is guaranteed to work. Should modules ever be included in the gc, then perhaps this would have to be revisited. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From stephen at xemacs.org Tue Oct 20 20:23:57 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 21 Oct 2009 03:23:57 +0900 Subject: [Python-ideas] a new lambda syntax In-Reply-To: References: <4ADC6A92.3060206@qq.com> <20091019140447.GA8510@phd.pp.ru> <4ADC77AA.2090807@qq.com> <873a5fds70.fsf@benfinney.id.au> <4ADCE16E.2000208@gmail.com> <874opueid0.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87ws2pdiv6.fsf@uwakimon.sk.tsukuba.ac.jp> Masklinn writes: > On 20 Oct 2009, at 07:37 , Stephen J. Turnbull wrote: > > That doesn't look like what "anonymous block" means to me. It looks > > like a lambda. > > > An anonymous block and a lambda are the exact same thing. Why use different names for them, then? At least when discussing Python which calls the concept "lambda"? > > The difference is that an block resolves all its non-argument > > references in the calling context > Are you talking about forming closures here? No, I'm talking about the exact opposite, I think. Let's forget Ruby, since we're talking about anonymous functions after all, and what I "know" about Ruby is all hearsay (and evidently not correctly understood at that). In Lisp, you can do something like this: (defvar i) (defvar f (lambda () (print i))) (do ((i 10 (- i 1))) ((< i 1)) (funcall f)) outputting 10 9 8 7 6 5 4 3 2 1 and returning nil. Ie, it was my understanding of "block" that free variables in the block have what in Lisp is called "dynamic scope". > > So a block seems to be a conventional way of currying a more > > general function to the context of a specific suite. That was a brain bubble, sorry. From masklinn at masklinn.net Tue Oct 20 20:18:07 2009 From: masklinn at masklinn.net (Masklinn) Date: Tue, 20 Oct 2009 20:18:07 +0200 Subject: [Python-ideas] nonlocal functions In-Reply-To: References: <4ADDDEB3.3010204@mrabarnett.plus.com> <494DB4DC-CA0E-4F3D-BF32-8CE0E19DE5E5@masklinn.net> Message-ID: On 20 Oct 2009, at 19:55 , Stefan Behnel wrote: > Masklinn wrote: >> If you go that route, I'd suggest removing all the legacy stuff and >> just >> using "let" to create a new binding in the current scope: >> >> def foo(): >> let a = 1 >> let b = 3 > No, please. Why would you want to require so much typing for the most > common case? Simpler, clearer, non ambiguous. As opposed to the current scheme of scope inference which is ambiguous and error prone. Also, nothing would prevent combining variable creations if need be. Not like this will ever happen anyway, but a guy can dream. > And how often would you forget to declare a variable that way? > Never. From stefan_ml at behnel.de Tue Oct 20 20:23:55 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 20 Oct 2009 20:23:55 +0200 Subject: [Python-ideas] nonlocal functions In-Reply-To: References: <4ADDDEB3.3010204@mrabarnett.plus.com> <494DB4DC-CA0E-4F3D-BF32-8CE0E19DE5E5@masklinn.net> Message-ID: Masklinn wrote: > On 20 Oct 2009, at 19:55 , Stefan Behnel wrote: >> And how often would you forget to declare a variable that way? >> > Never. :) Had a good laugh on that one, thanks! Stefan From masklinn at masklinn.net Tue Oct 20 20:45:34 2009 From: masklinn at masklinn.net (Masklinn) Date: Tue, 20 Oct 2009 20:45:34 +0200 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <87ws2pdiv6.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4ADC6A92.3060206@qq.com> <20091019140447.GA8510@phd.pp.ru> <4ADC77AA.2090807@qq.com> <873a5fds70.fsf@benfinney.id.au> <4ADCE16E.2000208@gmail.com> <874opueid0.fsf@uwakimon.sk.tsukuba.ac.jp> <87ws2pdiv6.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On 20 Oct 2009, at 20:23 , Stephen J. Turnbull wrote: > Masklinn writes: >> On 20 Oct 2009, at 07:37 , Stephen J. Turnbull wrote: > >>> That doesn't look like what "anonymous block" means to me. It looks >>> like a lambda. >>> >> An anonymous block and a lambda are the exact same thing. > > Why use different names for them, then? At least when discussing > Python which calls the concept "lambda"? > > One reason is that the creators of Smalltalk liked renaming stuff. A more serious one is that as far as they were concerned, they were manipulating code blocks. There are only two places in Smalltalk where you can have code (apart from the Workspace, which is equivalent to a Python shell): methods (method bodies in Python) and blocks (lambdas). Since there are no statements and if/for/while/try/whatever are implemented through messages (method calls) and anonymous functions, it made sense to use the term blocks to designate, well, blocks. Of code. > No, I'm talking about the exact opposite, I think. Let's forget Ruby, > since we're talking about anonymous functions after all, and what I > "know" about Ruby is all hearsay (and evidently not correctly > understood at that). In Lisp, you can do something like this: > > (defvar i) > (defvar f (lambda () (print i))) > (do ((i 10 (- i 1))) > ((< i 1)) > (funcall f)) > > outputting 10 9 8 7 6 5 4 3 2 1 and returning nil. Ie, it was my > understanding of "block" that free variables in the block have what in > Lisp is called "dynamic scope". In most languages (that I know of), free variables come from the lexical scope not the dynamic one (apart from a pair of lisps, I know of no language using a dynamic scope). However as I'm not sure what precisely happens in your code `i` could be the same in the lexical and the dynamic scope so? Using Python's syntax: def foo(): i = 5 return lambda: i def bar(fn): i = "d" print fn() fn = foo() bar(fn) here, if the language is lexically scoped the last line will print "5" whereas if it's dynamically scoped it will print "d". In pretty much every language I know (and that includes Ruby and Smalltalk) it will ? as in Python ? print "5". From masklinn at masklinn.net Tue Oct 20 20:47:37 2009 From: masklinn at masklinn.net (Masklinn) Date: Tue, 20 Oct 2009 20:47:37 +0200 Subject: [Python-ideas] nonlocal functions In-Reply-To: References: <4ADDDEB3.3010204@mrabarnett.plus.com> <494DB4DC-CA0E-4F3D-BF32-8CE0E19DE5E5@masklinn.net> Message-ID: On 20 Oct 2009, at 20:23 , Stefan Behnel wrote: > Masklinn wrote: >> On 20 Oct 2009, at 19:55 , Stefan Behnel wrote: >>> And how often would you forget to declare a variable that way? >>> >> Never. > :) Had a good laugh on that one, thanks! Why? It's a very simple process: "I'm creating a binding" matches to a let. Plus the runtime could easily warn you in the rare cases where you'd forgotten a let. I can assure you it's not a very difficult habit to pick. From daniel at stutzbachenterprises.com Tue Oct 20 22:12:03 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 20 Oct 2009 15:12:03 -0500 Subject: [Python-ideas] breaking cycles that include __del__ In-Reply-To: <4ADDFA69.5000707@scottdial.com> References: <4ADDFA69.5000707@scottdial.com> Message-ID: On Tue, Oct 20, 2009 at 12:59 PM, Scott Dial < scott+python-ideas at scottdial.com >wrote: > Should modules > ever be included in the gc, then perhaps this would have to be revisited. > You mean like this? http://bugs.python.org/issue812369 -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From sturla at molden.no Tue Oct 20 23:24:57 2009 From: sturla at molden.no (Sturla Molden) Date: Tue, 20 Oct 2009 23:24:57 +0200 Subject: [Python-ideas] Remove GIL with CAS instructions? Message-ID: <4ADE2AA9.4030604@molden.no> The GIL is particularly evil on multiprocessor systems. Not just because it prevents parallel excution of Python code, but thread switches are defunct. This is because a thread that periodically releases and re-acquires the GIL (at checkintervals), more or less always wins. On a multiprocessor system, the thread it will continue to run its processor and grab the GIL back before a sleeping thread wakes up. The cooperative multithreading intended by using checkintervals only works properly on single-processor systems. Python threads only work properly on single-processor computers. On computers with multiple CPUs, one must set the affinity of the Python process is set to one CPU only for proper cooperative thread scheduling. This behaviour of the GIL is a PITA, but often overlooked - all debates seems to be about parallel scalability, though far less important. Today, most people have multicore desktop computers (e.g. mine have four cores). The Linux kernel got rid of the BKL a long time ago. It's time Python does the same. The GIL also has its virtues. When calling a C extension, it is serialized by default unless the GIL is released. An attempt was made (many years ago) to replace the GIL with fine-grained locking. It slowed down serial code. That should come as no surprise, as OS mutexes and semaphores are expensive. There are lock-free data structures of any conceivable sort. There are multithreaded garbage collectors in Java and .NET, that don't need a global lock. I've heard claims that removal of the GIL would require Python to use a garbage collector instead of reference counts. That is not true. Lock-free data structures and garbage collectors have all one thing in common: they use a compare-and-exchange (CAS) instruction present in most modern processors, e.g. CMPXCHG on x86. In fact, CAS is used to implement OS mutexes (sleeplocks) and the less expensive spinlocks. Without a CAS instruction for the platform, there would be no GIL. All platforms that has a working GIL has a working CAS. Python could use the CAS instruction for lock-free management of reference counts. All built-in types (int, float, str, list, tuple, dict, set, etc) could be made lock-free using CAS. That would allow the interpreter to execute bytecode without holding the GIL. Only calls to C extensions would be serialized with the GIL. Schematically, this is how refcounts could be managed without needing a lock: /* * Returns 1 if an exhange was made, and (*addr == old) before * the exchange, otherwise returns 0. * * uses opcode CMPXCHG on x86 * */ extern int Py_refcnt_CAS(Py_ssize_t *addr, Py_ssize_t old, Py_ssize_t new); inline void Py_DECREF(PyObject *obj) { register Py_ssize_t refcnt; do { refcnt = obj->refcnt; } while (!Py_refcnt_CAS(&obj->ob_refcnt, refcnt, refcnt-1)); refcnt = obj->refcnt; if (refcnt == 0) { if(Py_refcnt_CAS(&obj->ob_refcnt, 0, -1)) { /* deallocate the object */ } } } inline void Py_INCREF(PyObject *obj) { register Py_ssize_t refcnt; refcnt = obj->refcnt; if (refcnt >= 0) { do { if((refcnt = obj->refcnt)<0) break; } while (!Py_refcnt_CAS(&obj->ob_refcnt, refcnt, refcnt+1)); } } Regards, Sturla Molden From benjamin at python.org Tue Oct 20 23:45:39 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 20 Oct 2009 21:45:39 +0000 (UTC) Subject: [Python-ideas] Remove GIL with CAS instructions? References: <4ADE2AA9.4030604@molden.no> Message-ID: Sturla Molden writes: > Python could use the CAS instruction for lock-free management of > reference counts. All built-in types (int, float, str, list, tuple, > dict, set, etc) could be made lock-free using CAS. I eagerly await your patch! From rhamph at gmail.com Tue Oct 20 23:58:05 2009 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 20 Oct 2009 15:58:05 -0600 Subject: [Python-ideas] breaking cycles that include __del__ In-Reply-To: <4ADDFA69.5000707@scottdial.com> References: <4ADDFA69.5000707@scottdial.com> Message-ID: On Tue, Oct 20, 2009 at 11:59, Scott Dial wrote: > The only way to guarantee the callback occurs is if you attach it to > some other object that will outlive your object, *but* must also not get > pulled into the same gc generation, otherwise it will *still not be > called*. I believe this is the source of your advice to store the > weakref in some globally reachable set. I believe given the way modules > are currently deallocated, this is guaranteed to work. Should modules > ever be included in the gc, then perhaps this would have to be revisited. Right, but if you want to guarantee your method will be called on shutdown it's better to fall back on an atexit handler. Once we start tearing down modules we can't promise any sane state, so it'd be better to disable weakref callbacks entirely. -- Adam Olsen, aka Rhamphoryncus From rhamph at gmail.com Wed Oct 21 00:08:01 2009 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 20 Oct 2009 16:08:01 -0600 Subject: [Python-ideas] Remove GIL with CAS instructions? In-Reply-To: <4ADE2AA9.4030604@molden.no> References: <4ADE2AA9.4030604@molden.no> Message-ID: On Tue, Oct 20, 2009 at 15:24, Sturla Molden wrote: > There are lock-free data structures of any conceivable sort. There are > multithreaded garbage collectors in Java and .NET, that don't need a global > lock. I've heard claims that removal of the GIL would require Python to use > a garbage collector instead of reference counts. That is not true. Lock-free > data structures and garbage collectors have all one thing in common: they > use a compare-and-exchange (CAS) instruction present in most modern > processors, e.g. CMPXCHG on x86. In fact, CAS is used to implement OS > mutexes (sleeplocks) and the less expensive spinlocks. Without a CAS > instruction for the platform, there would be no GIL. All platforms that has > a working GIL has a working CAS. CPython uses *extensive* refcounting operations on objects shared between threads (builtins, globals, literals, etc). The cache contention over those refcounts means even 2 threads are significantly slower than 1 thread, nevermind the increased contention of 3+ threads. That is why it is said a true tracing GC is needed to replace the refcounting GC. -- Adam Olsen, aka Rhamphoryncus From sturla at molden.no Wed Oct 21 00:13:04 2009 From: sturla at molden.no (Sturla Molden) Date: Wed, 21 Oct 2009 00:13:04 +0200 Subject: [Python-ideas] Remove GIL with CAS instructions? In-Reply-To: References: <4ADE2AA9.4030604@molden.no> Message-ID: <4ADE35F0.10803@molden.no> Benjamin Peterson skrev: > I eagerly await your patch! > I'd gladly contribute to the effort, but I did not say it is an easy task :-) I just wanted to bring this up: - The GIL has consequences on multicore CPUs that are overlooked: thread switches are usually missed at check intervals. This could be fixed without removing the GIL: For example, there could be a wait-queue for the GIL; a thread that request the GIL puts itself in the back. - Also, fine grained locking is not the only alternative to a global lock. No locks at all is even better. Sturla Molden From solipsis at pitrou.net Wed Oct 21 00:34:15 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 20 Oct 2009 22:34:15 +0000 (UTC) Subject: [Python-ideas] =?utf-8?q?breaking_cycles_that_include_=5F=5Fdel?= =?utf-8?b?X18=?= References: <4ADDFA69.5000707@scottdial.com> Message-ID: Scott Dial writes: > > This is made clear in the gcmodule.c comments, but is severely lacking > from the actual documentation in the weakref module. But also, Antoine's > is flawed in the same manner. You're right, I've been too quick in posting this. The following works, however: import weakref, gc class Foo: _w = {} def __init__(self): k = id(self) def not_closed(_, d=Foo._w, k=k, r=repr(self)): del d[k] print ("%s not closed!" % r) Foo._w[k] = weakref.ref(self, not_closed) def close(self): # Close... # and then destroy weakref Foo._w.pop(id(self), None) x,y = Foo(), Foo() x.y, y.x = y, x y.close() del x del y gc.collect() print(gc.garbage) Cheers Antoine. From casey at pandora.com Wed Oct 21 00:57:15 2009 From: casey at pandora.com (Casey Duncan) Date: Tue, 20 Oct 2009 16:57:15 -0600 Subject: [Python-ideas] Remove GIL with CAS instructions? In-Reply-To: <4ADE35F0.10803@molden.no> References: <4ADE2AA9.4030604@molden.no> <4ADE35F0.10803@molden.no> Message-ID: <613A6649-9D4E-4410-8A07-F5E4CAB9F399@pandora.com> On Oct 20, 2009, at 4:13 PM, Sturla Molden wrote: > Benjamin Peterson skrev: >> I eagerly await your patch! > I'd gladly contribute to the effort, but I did not say it is an easy > task :-) > > I just wanted to bring this up: [..] > > - Also, fine grained locking is not the only alternative to a global > lock. No locks at all is even better. I doubt anyone here would disagree with that, but to be taken seriously, you'll need to be *much* more specific. Enlighten us. -Casey From solipsis at pitrou.net Wed Oct 21 00:55:04 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 20 Oct 2009 22:55:04 +0000 (UTC) Subject: [Python-ideas] Remove GIL with CAS instructions? References: <4ADE2AA9.4030604@molden.no> Message-ID: Hello, > The cooperative > multithreading intended by using checkintervals only works properly on > single-processor systems. Python threads only work properly on > single-processor computers. I'm trying to work on that. I have an experimental rewrite of the GIL for POSIX systems which improves latencies, while decreasing the overhead of the GIL itself (because if checked every 100 opcodes, it can be dropped and re-taken tens of thousands of times per second...). If people have interesting workloads for py3k, the Mercurial branch can be found here: http://hg.pitrou.net/public/py3k/newgil/shortlog (my measurements, by the way, are based on a small concurrency benchmark for Python I recently wrote: http://svn.python.org/view/sandbox/trunk/ccbench/ ) > There are lock-free data structures of any conceivable sort. There are > multithreaded garbage collectors in Java and .NET, that don't need a > global lock. I've heard claims that removal of the GIL would require > Python to use a garbage collector instead of reference counts. That is > not true. Lock-free data structures and garbage collectors have all one > thing in common: they use a compare-and-exchange (CAS) instruction > present in most modern processors, e.g. CMPXCHG on x86. In fact, CAS is > used to implement OS mutexes (sleeplocks) and the less expensive > spinlocks. Without a CAS instruction for the platform, there would be no > GIL. All platforms that has a working GIL has a working CAS. That does not mean that CAS (or atomic increment/decrement, for that matter) is as fast as a regular (non-atomic) increment / decrement. Besides, reference counting is not the only thing to worry about. You have to: - protect (somehow) the GC when collecting, so that things don't get modified under its feet - protect INCREFs and DECREFs - protect all mutable types (dicts being probably the most performance-critical, since they are used for namespaces and class/object attributes and methods) - protect all static/global data in C modules which have them - and probably other things I'm not thinking about An interesting experiment would be to add all those protections without even attempting to remove the GIL, and measure the overhead compared to a regular Python. IMO, if the overhead is less than 10%, then it would probably be acceptable to go ahead and try removing the GIL. But someone needs to tackle this concretely for it to happen, because I guess these ideas have already been suggested a couple of times... Regards Antoine. From sturla at molden.no Wed Oct 21 01:31:52 2009 From: sturla at molden.no (Sturla Molden) Date: Wed, 21 Oct 2009 01:31:52 +0200 Subject: [Python-ideas] Remove GIL with CAS instructions? In-Reply-To: <613A6649-9D4E-4410-8A07-F5E4CAB9F399@pandora.com> References: <4ADE2AA9.4030604@molden.no> <4ADE35F0.10803@molden.no> <613A6649-9D4E-4410-8A07-F5E4CAB9F399@pandora.com> Message-ID: <4ADE4868.2070300@molden.no> Casey Duncan skrev: > > I doubt anyone here would disagree with that, but to be taken > seriously, you'll need to be *much* more specific. Enlighten us. I just showed you thread-safe incref and decref that don't require a lock. How specific should I be? S.M. From casey at pandora.com Wed Oct 21 01:38:34 2009 From: casey at pandora.com (Casey Duncan) Date: Tue, 20 Oct 2009 17:38:34 -0600 Subject: [Python-ideas] Remove GIL with CAS instructions? In-Reply-To: <4ADE4868.2070300@molden.no> References: <4ADE2AA9.4030604@molden.no> <4ADE35F0.10803@molden.no> <613A6649-9D4E-4410-8A07-F5E4CAB9F399@pandora.com> <4ADE4868.2070300@molden.no> Message-ID: On Oct 20, 2009, at 5:31 PM, Sturla Molden wrote: > Casey Duncan skrev: >> >> I doubt anyone here would disagree with that, but to be taken >> seriously, you'll need to be *much* more specific. Enlighten us. > I just showed you thread-safe incref and decref that don't require a > lock. They looked like spinlocks to me, I was assuming you were talking about something else even more magical. I was still considering them locks. -Casey From debatem1 at gmail.com Wed Oct 21 01:39:47 2009 From: debatem1 at gmail.com (geremy condra) Date: Tue, 20 Oct 2009 19:39:47 -0400 Subject: [Python-ideas] Remove GIL with CAS instructions? In-Reply-To: References: <4ADE2AA9.4030604@molden.no> <4ADE35F0.10803@molden.no> <613A6649-9D4E-4410-8A07-F5E4CAB9F399@pandora.com> <4ADE4868.2070300@molden.no> Message-ID: On Tue, Oct 20, 2009 at 7:31 PM, Sturla Molden wrote: > Casey Duncan skrev: >> >> I doubt anyone here would disagree with that, but to be taken seriously, >> you'll need to be *much* more specific. Enlighten us. > > I just showed you thread-safe incref and decref that don't require a lock. > > How specific should I be? > I suspect it's going to take working code- after all, if it isn't worth your time to implement, I don't think very many others are going to leap up and do it, however much benefit your perceive it having. Geremy Condra From grosser.meister.morti at gmx.net Wed Oct 21 01:42:30 2009 From: grosser.meister.morti at gmx.net (=?UTF-8?B?TWF0aGlhcyBQYW56ZW5iw7Zjaw==?=) Date: Wed, 21 Oct 2009 01:42:30 +0200 Subject: [Python-ideas] [Python-Dev] Proposal : Python Trusted Computing API In-Reply-To: <87pr8icyi3.fsf@benfinney.id.au> References: <87pr8icyi3.fsf@benfinney.id.au> Message-ID: <4ADE4AE6.2090602@gmx.net> On 10/20/2009 09:31 AM, Ben Finney wrote: > geremy condra writes: > >> If you were suggesting adding a crypto API to python, I'd be all >> for it- but you're suggesting adding the ability to have Python >> software vendors remotely cripple the code on your machine. >> I just can't get behind that, and while you're sure to hear wildly >> divergent opinions on this board, I suspect that mine will not >> be an uncommon sentiment. > > Agreed. Though I don't draw much water here, being a Python user rather > than a core developer, I totally agree that technologies to subvert an > owner's control over the behaviour of their own machine are anathema to > a free culture. Such technologies are to be resisted, and they shall not > get a jot of my support in any form if I can help it. > Exactly what I wanted to say. -panzi From sturla at molden.no Wed Oct 21 01:46:15 2009 From: sturla at molden.no (Sturla Molden) Date: Wed, 21 Oct 2009 01:46:15 +0200 Subject: [Python-ideas] Remove GIL with CAS instructions? In-Reply-To: References: <4ADE2AA9.4030604@molden.no> Message-ID: <4ADE4BC7.5050205@molden.no> Antoine Pitrou skrev: > - protect (somehow) the GC when collecting, so that things don't get modified > under its feet > The GC would have to temporarily suspend all active threads registered with the interpreter. E.g. using the SuspendThread function in Windows API. > - protect INCREFs and DECREFs > Update using CAS (I just showed you C code for this). > - protect all mutable types (dicts being probably the most performance-critical, > since they are used for namespaces and class/object attributes and methods) > CAS. Lock-free lists and hash tables exist. We don't need locks to protect mutable types and class objects. Just use a lock-free hash-table for __dict__ or whatever. > - protect all static/global data in C modules which have them > Reacquire the GIL before calling functions in C extensions functions. The GIL is nice there, but not anywhere else. > An interesting experiment would be to add all those protections without even > attempting to remove the GIL, and measure the overhead compared to a regular > Python. Yes, one could add a nonsence CAS to existing incref/decref, and measure overhead. From solipsis at pitrou.net Wed Oct 21 01:50:29 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 20 Oct 2009 23:50:29 +0000 (UTC) Subject: [Python-ideas] Remove GIL with CAS instructions? References: <4ADE2AA9.4030604@molden.no> <4ADE4BC7.5050205@molden.no> Message-ID: Sturla Molden writes: > > Yes, one could add a nonsence CAS to existing incref/decref, and measure > overhead. That would be the first thing to do IMO. If it doesn't seem to decrease performance a lot, you could continue by adding locking to dict objects, and measure again. Then add lists and sets (although I doubt sets and even lists are performance-critical for general interpreter operation). If after that you have less than a 10% slowdown (personal opinion only, other people's mileage may vary), it's probably worth going on and trying to do the whole GIL removal (which doesn't mean it would necessarily be accepted, by the way, but at least would avoid it being automatically rejected ;-)). Regards Antoine. From sturla at molden.no Wed Oct 21 01:52:39 2009 From: sturla at molden.no (Sturla Molden) Date: Wed, 21 Oct 2009 01:52:39 +0200 Subject: [Python-ideas] Remove GIL with CAS instructions? In-Reply-To: References: <4ADE2AA9.4030604@molden.no> <4ADE35F0.10803@molden.no> <613A6649-9D4E-4410-8A07-F5E4CAB9F399@pandora.com> <4ADE4868.2070300@molden.no> Message-ID: <4ADE4D47.4060209@molden.no> Casey Duncan skrev: > > They looked like spinlocks to me, I was assuming you were talking > about something else even more magical. I was still considering them > locks. The code is almost like spinlocks, except the refcount acts as its own "spinlock", thus the overhead is halved. (A separate spinlock would have to be released as well.) The failed attempt to remove the GIL used OS mutexes, which are much more expensive sleeplocks. From mwm-keyword-python.b4bdba at mired.org Wed Oct 21 01:57:42 2009 From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer) Date: Tue, 20 Oct 2009 19:57:42 -0400 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <2DEB8009-3607-49FC-9C3B-ED08C9B3A245@masklinn.net> References: <4ADC6A92.3060206@qq.com> <200910201006.30622.steve@pearwood.info> <200910202221.03233.steve@pearwood.info> <2DEB8009-3607-49FC-9C3B-ED08C9B3A245@masklinn.net> Message-ID: <20091020195742.45a43d33@bhuda.mired.org> On Tue, 20 Oct 2009 13:34:11 +0200 Masklinn wrote: > The biggest (by far) advantages I see to good anonymous functions > (note: Ruby's aren't, as far as I'm concerned, because due to their > nature they don't easily scale from 1/call to 2+/call) are in > flexibility, freedom of experimentation and possibility to keep the > core language itself small: had Python had "full-blown" anonymous > functions, it wouldn't have been necessary to add the `with` statement > to the language. It could just as well have been implemented through a > protocol or the stdlib, and people would have been free to toy with it > in their projects long before it was added to the core language. Note that this is a two edged sword, in that all those people "toying" with "with"-like constructs might also be publishing code using them, meaning that instead of having one clean construct from the core developers, we'd have to deal with an unknown number of variant idioms from most anybody. A small core language doesn't help a lot when you have to know four or five different ways to build some basic construct because it's not in the core language. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From rhamph at gmail.com Wed Oct 21 02:00:29 2009 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 20 Oct 2009 18:00:29 -0600 Subject: [Python-ideas] Remove GIL with CAS instructions? In-Reply-To: References: <4ADE2AA9.4030604@molden.no> <4ADE4BC7.5050205@molden.no> Message-ID: On Tue, Oct 20, 2009 at 17:50, Antoine Pitrou wrote: > Sturla Molden writes: >> >> Yes, one could add a nonsence CAS to existing incref/decref, and measure >> overhead. > > That would be the first thing to do IMO. > If it doesn't seem to decrease performance a lot, you could continue by adding > locking to dict objects, and measure again. > Then add lists and sets (although I doubt sets and even lists are > performance-critical for general interpreter operation). > If after that you have less than a 10% slowdown (personal opinion only, other > people's mileage may vary), it's probably worth going on and trying to do the > whole GIL removal (which doesn't mean it would necessarily be accepted, by the > way, but at least would avoid it being automatically rejected ;-)). This'd show you the minimum amount of overhead, but only for a trivial case: single threaded. You're not invoking cache contention, which in my experience is much larger than the atomic op cost, and prevents the *real* gains you hope to make. Python-safethread makes the datastructures safe (lockless fast-paths) so that it can be compared properly. I tried atomic refcounting and it sucked. My buffered bodge sucked less, but still sucked. The only option left is a real tracing GC. -- Adam Olsen, aka Rhamphoryncus From mwm-keyword-python.b4bdba at mired.org Wed Oct 21 02:02:11 2009 From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer) Date: Tue, 20 Oct 2009 20:02:11 -0400 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <20091020114958.GA21044@phd.pp.ru> References: <4ADC6A92.3060206@qq.com> <200910201006.30622.steve@pearwood.info> <200910202221.03233.steve@pearwood.info> <2DEB8009-3607-49FC-9C3B-ED08C9B3A245@masklinn.net> <20091020114958.GA21044@phd.pp.ru> Message-ID: <20091020200211.756a40fb@bhuda.mired.org> On Tue, 20 Oct 2009 15:49:58 +0400 Oleg Broytman wrote: > This is The Reason Number Two - there have to be a balance between what > features are accepted in the language and what are rejected. Python > developers decided that anonymous code blocks are allowed in a few special > places and are forbidden generally. They did? Where? The closest I've seen is that multi-line expressions - and in particular function invocations - are ugly, so reject any feature that allows them - which generally includes anonymous code blocks - will probably be rejected. This does leave open the possibility of anonymous code blocks being acceptable providing the syntax avoids that problem area. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From sturla at molden.no Wed Oct 21 02:22:22 2009 From: sturla at molden.no (Sturla Molden) Date: Wed, 21 Oct 2009 02:22:22 +0200 Subject: [Python-ideas] Remove GIL with CAS instructions? In-Reply-To: References: <4ADE2AA9.4030604@molden.no> <4ADE4BC7.5050205@molden.no> Message-ID: <4ADE543E.1040302@molden.no> Adam Olsen skrev: > This'd show you the minimum amount of overhead, but only for a trivial > case: single threaded. As far as I know, the single-threaded case is the reason for keeping the GIL. From rhamph at gmail.com Wed Oct 21 06:09:01 2009 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 20 Oct 2009 22:09:01 -0600 Subject: [Python-ideas] Remove GIL with CAS instructions? In-Reply-To: <4ADE543E.1040302@molden.no> References: <4ADE2AA9.4030604@molden.no> <4ADE4BC7.5050205@molden.no> <4ADE543E.1040302@molden.no> Message-ID: On Tue, Oct 20, 2009 at 18:22, Sturla Molden wrote: > Adam Olsen skrev: >> >> This'd show you the minimum amount of overhead, but only for a trivial >> case: single threaded. > > As far as I know, the single-threaded case is the reason for keeping the > GIL. No, single-threaded is the reason for keeping the GIL *as an option*. The complete lack of any other viable option (in CPython) is the reason it's the only option. -- Adam Olsen, aka Rhamphoryncus From eero.nevalainen at indagon.com Wed Oct 21 11:13:22 2009 From: eero.nevalainen at indagon.com (Eero Nevalainen) Date: Wed, 21 Oct 2009 12:13:22 +0300 Subject: [Python-ideas] Syntax for making stuct / record / namedtuples Message-ID: <4ADED0B2.3050804@indagon.com> From the (creative) laziness department: Whenever my module A needs to pass a lot of data to module B, I find myself making a factory function in module B for creating the datastructures def my_factory_function(a, b, c): # input checking res.a = a res.b = b res.c = c return res I believe this is fairly common. Since the fields are already defined in the function signature, I'd prefer to not repeat myself and write something like this: def my_factory_function(a, b, c): args = locals() # input checking return namedtuple('MyTypeName', args) This would perceivably be possible, if locals() returned an OrderedDict and an appropriate namedtuple factory function was added. related discussion is in: http://kbyanc.blogspot.com/2007/07/python-aggregating-function-arguments.html http://code.activestate.com/recipes/500261/ Does this seem familiar or useful to anyone besides me? -- Eero Nevalainen From cmjohnson.mailinglist at gmail.com Wed Oct 21 12:03:39 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Wed, 21 Oct 2009 00:03:39 -1000 Subject: [Python-ideas] nonlocal functions In-Reply-To: <494DB4DC-CA0E-4F3D-BF32-8CE0E19DE5E5@masklinn.net> References: <4ADDDEB3.3010204@mrabarnett.plus.com> <494DB4DC-CA0E-4F3D-BF32-8CE0E19DE5E5@masklinn.net> Message-ID: <3bdda690910210303r31553cafk393c6b621c1def63@mail.gmail.com> 2009/10/20 Masklinn: > def foo(): > ? ?let a = 1 > ? ?let b = 3 > ? ?def bar(): > ? ? ? ?a = 2 > ? ? ? ?let b = 4 > ? ?bar() > ? ?return ?(a, b) # returns (2, 3) This is all fantasy, but what is your proposal when this happens: def foo(): let a = 1 stuff stuff let a = 2 #Oops, forgot about already declaring it! I also imagine that you won't allow a bare "let a"? Or does it just set itself to None? ? Carl From kristjan at ccpgames.com Wed Oct 21 12:15:52 2009 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Wed, 21 Oct 2009 10:15:52 +0000 Subject: [Python-ideas] Remove GIL with CAS instructions? In-Reply-To: <4ADE35F0.10803@molden.no> References: <4ADE2AA9.4030604@molden.no> <4ADE35F0.10803@molden.no> Message-ID: <930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local> > -----Original Message----- > From: python-ideas-bounces+kristjan=ccpgames.com at python.org > [mailto:python-ideas-bounces+kristjan=ccpgames.com at python.org] On > Behalf Of Sturla Molden > Sent: 20. okt?ber 2009 22:13 > To: python-ideas at python.org > Subject: Re: [Python-ideas] Remove GIL with CAS instructions? > > > - The GIL has consequences on multicore CPUs that are overlooked: > thread > switches are usually missed at check intervals. This could be fixed > without removing the GIL: For example, there could be a wait-queue for > the GIL; a thread that request the GIL puts itself in the back. > This depends entirely on the platform and primitives used to implement the GIL. I'm interested in windows. There, I found this article: http://fonp.blogspot.com/2007/10/fairness-in-win32-lock-objects.html So, you may be on to something. Perhaps a simple C test is in order then? I did that. I found, on my dual-core vista machine, that running "release", that both Mutexes and CriticalSections behaved as you describe, with no "fairness". Using a "semaphore" seems to retain fairness, however. "fairness" was retained in debug builds too, strangely enough. Now, Python uses none of these. On windows, it uses an "Event" object coupled with an atomically updated counter. This also behaves fairly. The test application is attached. I think that you ought to sustantiate your claims better, maybe with a specific platform and using some test like the above. On the other hand, it shows that we must be careful what we use. There has been some talk of using CriticalSections for the GIL on windows. This test ought to show the danger of that. The GIL is different than a regular lock. It is a reverse-lock, really, and therefore may need to be implemented in its own special way, if we want very fast mutexes for the rest of the system (cc to python-dev) Cheers, Kristj?n -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: giltest.cpp URL: From weasley_wx at qq.com Wed Oct 21 12:16:03 2009 From: weasley_wx at qq.com (=?ISO-8859-1?B?d3h5YXJ2?=) Date: Wed, 21 Oct 2009 18:16:03 +0800 Subject: [Python-ideas] Syntax for making stuct / record / namedtuples Message-ID: >>Whenever my module A needs to pass a lot of data to module B, I find >>myself making a factory function in module B for creating the datastructures >> >>def my_factory_function(a, b, c): >> # input checking >> res.a = a >> res.b = b >> res.c = c >> return res >> >>I believe this is fairly common. >> >>Since the fields are already defined in the function signature, I'd >>prefer to not repeat myself and write something like this: >> >>def my_factory_function(a, b, c): >> args = locals() >> # input checking >> return namedtuple('MyTypeName', args) >> >> >>This would perceivably be possible, if locals() returned an OrderedDict >>and an appropriate namedtuple factory function was added. >> >>related discussion is in: >>http://kbyanc.blogspot.com/2007/07/python-aggregating-function-arguments.html >>http://code.activestate.com/recipes/500261/ >> >>Does this seem familiar or useful to anyone besides me? in current syntax, res.a = a is not allowed, if res is a dict, the right syntax is res['a'] = a. in some languages, res.a equals res['a'], but not in python. (and isn't possible in Python, because there are many disadvantage to make this.). so, if you need put arguments into a dict, you can do this: def my_factory_function(**arg): # input check return arg arg is a dict. and, if res is a class, you are setting the attribute of res. you can also: def my_factory_function(**arg): # input check res.__dict__.update(arg) return res a named tuple is just a dict (I think :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmjohnson.mailinglist at gmail.com Wed Oct 21 12:16:29 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Wed, 21 Oct 2009 00:16:29 -1000 Subject: [Python-ideas] Syntax for making stuct / record / namedtuples In-Reply-To: <4ADED0B2.3050804@indagon.com> References: <4ADED0B2.3050804@indagon.com> Message-ID: <3bdda690910210316s1bd9630exa83b9fd147644cc3@mail.gmail.com> I would also appreciate a faster way to do the boilerplate in an __init__ of saying self.arg1 = arg1, self.arg2 = arg2, etc. But what would be the global implications of changing locals() to an odict? Is that backwards compatible? If not, perhaps we could add a new builtin, say "olocals()" instead? ?Carl From g.brandl at gmx.net Wed Oct 21 12:34:23 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 21 Oct 2009 12:34:23 +0200 Subject: [Python-ideas] Syntax for making stuct / record / namedtuples In-Reply-To: <4ADED0B2.3050804@indagon.com> References: <4ADED0B2.3050804@indagon.com> Message-ID: Eero Nevalainen schrieb: > From the (creative) laziness department: > > Whenever my module A needs to pass a lot of data to module B, I find > myself making a factory function in module B for creating the datastructures > > def my_factory_function(a, b, c): > # input checking > res.a = a > res.b = b > res.c = c > return res > > I believe this is fairly common. > > Since the fields are already defined in the function signature, I'd > prefer to not repeat myself and write something like this: > > def my_factory_function(a, b, c): > args = locals() > # input checking > return namedtuple('MyTypeName', args) > > > This would perceivably be possible, if locals() returned an OrderedDict > and an appropriate namedtuple factory function was added. Sorry, I don't understand *at all* what you are proposing here. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From masklinn at masklinn.net Wed Oct 21 12:45:14 2009 From: masklinn at masklinn.net (Masklinn) Date: Wed, 21 Oct 2009 12:45:14 +0200 Subject: [Python-ideas] nonlocal functions In-Reply-To: <3bdda690910210303r31553cafk393c6b621c1def63@mail.gmail.com> References: <4ADDDEB3.3010204@mrabarnett.plus.com> <494DB4DC-CA0E-4F3D-BF32-8CE0E19DE5E5@masklinn.net> <3bdda690910210303r31553cafk393c6b621c1def63@mail.gmail.com> Message-ID: <1CA66C4C-68AD-4ED6-8E03-AADED5D61E2E@masklinn.net> On 21 Oct 2009, at 12:03 , Carl Johnson wrote: > > This is all fantasy, but what is your proposal when this happens: > > def foo(): > let a = 1 > stuff > stuff > let a = 2 #Oops, forgot about already declaring it! > It could be a noop, but I think a warning (or an error if scopes are statically checked) would be a smarter idea, make the programmer aware that something weird's happening. > I also imagine that you won't allow a bare "let a"? Or does it just > set itself to None? There's not much sense in a bare `let a` imo, if you want it to be nothing, just write `let a = None`. From steve at pearwood.info Wed Oct 21 13:05:34 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 21 Oct 2009 22:05:34 +1100 Subject: [Python-ideas] Remove GIL with CAS instructions? In-Reply-To: References: <4ADE2AA9.4030604@molden.no> <4ADE543E.1040302@molden.no> Message-ID: <200910212205.35732.steve@pearwood.info> On Wed, 21 Oct 2009 03:09:01 pm Adam Olsen wrote: > On Tue, Oct 20, 2009 at 18:22, Sturla Molden wrote: > > Adam Olsen skrev: > >> This'd show you the minimum amount of overhead, but only for a > >> trivial case: single threaded. > > > > As far as I know, the single-threaded case is the reason for > > keeping the GIL. > > No, single-threaded is the reason for keeping the GIL *as an option*. > The complete lack of any other viable option (in CPython) is the > reason it's the only option. I'm not entirely sure what you're trying to say there, but I'd like link to what Guido said over two years ago: http://www.artima.com/weblogs/viewpost.jsp?thread=214235 [quote] ... I'd welcome a set of patches into Py3k only if the performance for a single-threaded program (and for a multi-threaded but I/O-bound program) does not decrease. I would also be happy if someone volunteered to maintain a GIL-free fork of Python... However, I want to warn that there are many downsides to removing the GIL. ... I want to point out one more time that the language doesn't require the GIL -- it's only the CPython virtual machine that has historically been unable to shed it. [end quote] -- Steven D'Aprano From eero.nevalainen at indagon.com Wed Oct 21 13:15:10 2009 From: eero.nevalainen at indagon.com (Eero Nevalainen) Date: Wed, 21 Oct 2009 14:15:10 +0300 Subject: [Python-ideas] Syntax for making stuct / record / namedtuples In-Reply-To: References: Message-ID: <4ADEED3E.3050803@indagon.com> wxyarv wrote: >>>Whenever my module A needs to pass a lot of data to module B, I find >>>myself making a factory function in module B for creating the > datastructures >>> >>>def my_factory_function(a, b, c): >>> # input checking >>> res.a = a >>> res.b = b >>> res.c = c >>> return res >>> >>>I believe this is fairly common. >>> >>>Since the fields are already defined in the function signature, I'd >>>prefer to not repeat myself and write something like this: >>> >>>def my_factory_function(a, b, c): >>> args = locals() >>> # input checking >>> return namedtuple('MyTypeName', args) >>> >>> >>>This would perceivably be possible, if locals() returned an OrderedDict >>>and an appropriate namedtuple factory function was added. >>> >>>related discussion is in: >>>http://kbyanc.blogspot.com/2007/07/python-aggregating-function-arguments.html >>>http://code.activestate.com/recipes/500261/ >>> >>>Does this seem familiar or useful to anyone besides me? > in current syntax, res.a = a is not allowed, if res is a dict, the > right syntax is res['a'] = a. in some languages, res.a equals > res['a'], but not in python. (and isn't possible in Python, because > there are many disadvantage to make this.). > so, if you need put arguments into a dict, you can do this: > def my_factory_function(**arg): > # input check > return arg > arg is a dict. > and, if res is a class, you are setting the attribute of res. you can > also: > def my_factory_function(**arg): > # input check > res.__dict__.update(arg) > return res > a named tuple is just a dict (I think :-) Right. I probably didn't express myself too well. The current recipe for 'structs' is here http://docs.python.org/tutorial/classes.html#odds-and-ends make an empty class, and assign fields as necessary. I prefer using factory functions, since I then I can define in a single place, what the 'struct' is supposed to contain, and the function signature will conveniently raise errors if I've made changes in module B, but not A. The same boilerplate code is also in all normal class init methods, you list the stuff once in the __init__ method arguments, then you list them again when you assign them, and I'm just wondering why do I have to repeat myself. As for named tuples, I quote http://docs.python.org/library/collections.html#namedtuple-factory-function-for-tuples-with-named-fields "Named tuple instances do not have per-instance dictionaries, so they are lightweight and require no more memory than regular tuples." -- Eero Nevalainen From ncoghlan at gmail.com Wed Oct 21 13:38:44 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 21 Oct 2009 21:38:44 +1000 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <20091020195742.45a43d33@bhuda.mired.org> References: <4ADC6A92.3060206@qq.com> <200910201006.30622.steve@pearwood.info> <200910202221.03233.steve@pearwood.info> <2DEB8009-3607-49FC-9C3B-ED08C9B3A245@masklinn.net> <20091020195742.45a43d33@bhuda.mired.org> Message-ID: <4ADEF2C4.9050702@gmail.com> Mike Meyer wrote: > On Tue, 20 Oct 2009 13:34:11 +0200 > Masklinn wrote: >> The biggest (by far) advantages I see to good anonymous functions >> (note: Ruby's aren't, as far as I'm concerned, because due to their >> nature they don't easily scale from 1/call to 2+/call) are in >> flexibility, freedom of experimentation and possibility to keep the >> core language itself small: had Python had "full-blown" anonymous >> functions, it wouldn't have been necessary to add the `with` statement >> to the language. It could just as well have been implemented through a >> protocol or the stdlib, and people would have been free to toy with it >> in their projects long before it was added to the core language. > > Note that this is a two edged sword, in that all those people "toying" > with "with"-like constructs might also be publishing code using them, > meaning that instead of having one clean construct from the core > developers, we'd have to deal with an unknown number of variant idioms > from most anybody. > > A small core language doesn't help a lot when you have to know four or > five different ways to build some basic construct because it's not in > the core language. Yup, this is certainly one of the concerns with making it too easy for developers to invent their own syntax (and also why Guido has flatly ruled out a macro system for Python). The bar for anonymous blocks is a high one, because: 1. Once an anonymous block syntax is accepted into the language we're pretty much stuck with it 2. While it does exist, the use case gap between anonymous functional expressions (i.e. the current lambda) and full named functions is pretty narrow (especially since the introduction of true conditional expressions on the lambda side). 3. Code is written once and read often. Anonymous blocks run a high risk of getting this back to front by making code easier to write but harder to read. The main point in favour of anonymous blocks of some kind is that having to define the subfunction before using it often *will* get the order of the code as written out of whack with respect to the order that makes the most logical sense. Both decorators and the with statement involved large elements of "don't leave important information trailing at the end where a reader may easily miss it" so it is an argument with some merit. Since it is a valid point that this can happen with arbitrary complex data objects rather than solely with named functions, one past proposal (PEP 359, the 'make' statement) involved attaching a suite to a function call, then providing the contents of that suite as a dictionary argument to the function being invoked. (Using the namespace to provide keyword arguments instead would allow a new construct to be used with existing functions, but would significantly constrain the use of temporary variables that weren't intended for use a arguments to the function (then again, that already happens with normal class definitions, and the del statement seems to handle that just fine). I should note that Guido ended up killing that PEP because he didn't like it, but I still believe some kind of "do this, but only after running this other suite first" out of order execution construct is more likely to gain acceptance than trying to come up with a full anonymous block syntax. However, nobody has suggested a particular nice syntax for that idea either, so, again it is unlikely to go anywhere anytime soon. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From fuzzyman at gmail.com Wed Oct 21 13:50:16 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Wed, 21 Oct 2009 12:50:16 +0100 Subject: [Python-ideas] Syntax for making stuct / record / namedtuples In-Reply-To: <4ADED0B2.3050804@indagon.com> References: <4ADED0B2.3050804@indagon.com> Message-ID: <6f4025010910210450g1e9dc7e5x303bb7f7da493d87@mail.gmail.com> 2009/10/21 Eero Nevalainen > From the (creative) laziness department: > > Whenever my module A needs to pass a lot of data to module B, I find myself > making a factory function in module B for creating the datastructures > > def my_factory_function(a, b, c): > # input checking > res.a = a > res.b = b > res.c = c > return res > > I believe this is fairly common. > > The C# syntax for this (anonymous types new in C# 3.0) is: var obj = new { prod.Color, prod.Price } C# creates what is effectively a named tuple parsing the field names from the way you construct it. The closest Python syntax would be something like: obj = {Color=foo, Price=bar} Or as we seem to be overloading curly braces a great deal now, how about: obj = (Color=foo, Price=bar) All the best, Michael > Since the fields are already defined in the function signature, I'd prefer > to not repeat myself and write something like this: > > def my_factory_function(a, b, c): > args = locals() > # input checking > return namedtuple('MyTypeName', args) > > > This would perceivably be possible, if locals() returned an OrderedDict and > an appropriate namedtuple factory function was added. > > related discussion is in: > > http://kbyanc.blogspot.com/2007/07/python-aggregating-function-arguments.html > http://code.activestate.com/recipes/500261/ > > Does this seem familiar or useful to anyone besides me? > > -- > Eero Nevalainen > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Wed Oct 21 13:50:13 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 21 Oct 2009 21:50:13 +1000 Subject: [Python-ideas] nonlocal functions In-Reply-To: References: <4ADDDEB3.3010204@mrabarnett.plus.com> <494DB4DC-CA0E-4F3D-BF32-8CE0E19DE5E5@masklinn.net> Message-ID: <4ADEF575.4000302@gmail.com> Masklinn wrote: > On 20 Oct 2009, at 20:23 , Stefan Behnel wrote: >> Masklinn wrote: >>> On 20 Oct 2009, at 19:55 , Stefan Behnel wrote: >>>> And how often would you forget to declare a variable that way? >>>> >>> Never. >> :) Had a good laugh on that one, thanks! > Why? It's a very simple process: "I'm creating a binding" matches to a > let. Plus the runtime could easily warn you in the rare cases where > you'd forgotten a let. > > I can assure you it's not a very difficult habit to pick. I'm with Stefan in laughing at the idea of someone never forgetting a variable declaration, and I've done a *lot* of programming in statically typed languages (and "let" really isn't that different from a type declaration). You're also omitting consideration of the ways *other* than assignment to bind a name in Python (i.e. def, with, except, for). Would those require an explicit let as well? If not, why single out assignment as a special case that requires an explicit declaration of "Yes, I really do want to create a new local here"? If yes... then whatever language you're dreaming of there, it is getting a really long way away from Python and giving up one of the major benefits of dynamic typing (i.e. no need for variable declarations). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Wed Oct 21 13:55:50 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 21 Oct 2009 21:55:50 +1000 Subject: [Python-ideas] Syntax for making stuct / record / namedtuples In-Reply-To: <4ADED0B2.3050804@indagon.com> References: <4ADED0B2.3050804@indagon.com> Message-ID: <4ADEF6C6.9050408@gmail.com> Eero Nevalainen wrote: > Does this seem familiar or useful to anyone besides me? Python 3, metaclasses and __prepare__ (aka PEP 3115). The performance of the local function namespace is critical to Python's speed characteristics, so slowing it down would not be appreciated (and odicts definitely *are* slower than regular dicts due to the extra checks involved). The performance of class namespaces during definition is much less critical however, and OrderedDict was added in part so that people could easily return it from __prepare__ methods without having to write one themselves. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From george.sakkis at gmail.com Wed Oct 21 14:02:18 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Wed, 21 Oct 2009 15:02:18 +0300 Subject: [Python-ideas] Syntax for making stuct / record / namedtuples In-Reply-To: <4ADED0B2.3050804@indagon.com> References: <4ADED0B2.3050804@indagon.com> Message-ID: <91ad5bf80910210502h68004809r4c19d7afa9b4a47a@mail.gmail.com> On Wed, Oct 21, 2009 at 12:13 PM, Eero Nevalainen wrote: > From the (creative) laziness department: > > Whenever my module A needs to pass a lot of data to module B, I find myself > making a factory function in module B for creating the datastructures > > def my_factory_function(a, b, c): > ? ?# input checking > ? ?res.a = a > ? ?res.b = b > ? ?res.c = c > ? ?return res > > I believe this is fairly common. > > Since the fields are already defined in the function signature, I'd prefer > to not repeat myself and write something like this: > > def my_factory_function(a, b, c): > ? ?args = locals() > ? ?# input checking > ? ?return namedtuple('MyTypeName', args) > > > This would perceivably be possible, if locals() returned an OrderedDict and > an appropriate namedtuple factory function was added. > > related discussion is in: > http://kbyanc.blogspot.com/2007/07/python-aggregating-function-arguments.html > http://code.activestate.com/recipes/500261/ > > Does this seem familiar or useful to anyone besides me? You can override the __new__ method in your type: class Foo(namedtuple('Foo', 'a b c')): def __new__(cls, a, b, c): # input validation and/or adaptation a = int(a) b = float(b) c = list(c) return super(Foo,cls).__new__(cls,a,b,c) >>> Foo(1, 2, 'xyz') Foo(a=1, b=2.0, c=['x', 'y', 'z']) It's not perfect (the super call is ugly, in 2.x at least) but it works. George From ncoghlan at gmail.com Wed Oct 21 14:24:58 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 21 Oct 2009 22:24:58 +1000 Subject: [Python-ideas] Syntax for making stuct / record / namedtuples In-Reply-To: <4ADED0B2.3050804@indagon.com> References: <4ADED0B2.3050804@indagon.com> Message-ID: <4ADEFD9A.8060308@gmail.com> Eero Nevalainen wrote: > Since the fields are already defined in the function signature, I'd > prefer to not repeat myself and write something like this: > > def my_factory_function(a, b, c): > args = locals() > # input checking > return namedtuple('MyTypeName', args) > This would perceivably be possible, if locals() returned an OrderedDict > and an appropriate namedtuple factory function was added. Not really necessary though. (2.x example - replace func_code with __code__ for 3.x. Alternatively, just use the inspect module instead of coding it directly) >>> from collections import namedtuple >>> def argnames(f): ... return f.func_code.co_varnames[:f.func_code.co_argcount] ... >>> def factory(a, b, c): ... return namedtuple('MyClass', argnames(factory))(a, b, c) ... >>> argnames(factory) ('a', 'b', 'c') >>> x = factory(1, 2, 3) >>> x MyClass(a=1, b=2, c=3) That's pretty wasteful though, since you're creating a new type every time through the function. Subclassing gives you an alternative way of checking the argument validity without ever repeating the list of field names: >>> def check_args(a, b, c): ... # Check args, raise exceptions, etc ... pass ... >>> class MyClass(namedtuple('MyClass', argnames(check_args))): ... def __new__(*args): ... check_args(*args[1:]) ... return super(args[0], MyClass).__new__(*args) ... >>> y = MyClass(1, 2, 3) >>> y MyClass(a=1, b=2, c=3) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From arnodel at googlemail.com Wed Oct 21 14:30:03 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Wed, 21 Oct 2009 13:30:03 +0100 Subject: [Python-ideas] Syntax for making stuct / record / namedtuples In-Reply-To: <3bdda690910210316s1bd9630exa83b9fd147644cc3@mail.gmail.com> References: <4ADED0B2.3050804@indagon.com> <3bdda690910210316s1bd9630exa83b9fd147644cc3@mail.gmail.com> Message-ID: <9bfc700a0910210530q7d9de651j839f34bca5bf62fc@mail.gmail.com> 2009/10/21 Carl Johnson : > I would also appreciate a faster way to do the boilerplate in an > __init__ of saying self.arg1 = arg1, self.arg2 = arg2, etc. There was a discussion on c.l.p and I wrote a recipe for that: http://code.activestate.com/recipes/551763/ -- Arnaud From ncoghlan at gmail.com Wed Oct 21 14:31:12 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 21 Oct 2009 22:31:12 +1000 Subject: [Python-ideas] Remove GIL with CAS instructions? In-Reply-To: <4ADE35F0.10803@molden.no> References: <4ADE2AA9.4030604@molden.no> <4ADE35F0.10803@molden.no> Message-ID: <4ADEFF10.5040701@gmail.com> Sturla Molden wrote: > Benjamin Peterson skrev: >> I eagerly await your patch! > I'd gladly contribute to the effort, but I did not say it is an easy > task :-) > > I just wanted to bring this up: > > - The GIL has consequences on multicore CPUs that are overlooked: thread > switches are usually missed at check intervals. In the rare case where none of those threads are releasing the GIL for any other reason, time.sleep(0.001) works wonders. Yeah, it's a wart, but not a particularly difficult one to deal with. That tangent aside, the major problem with removing the GIL has never been the concept of doing so, but the practicality of implementing it in a cross-platform way that doesn't hurt single threaded performance. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From masklinn at masklinn.net Wed Oct 21 14:43:07 2009 From: masklinn at masklinn.net (Masklinn) Date: Wed, 21 Oct 2009 14:43:07 +0200 Subject: [Python-ideas] nonlocal functions In-Reply-To: <4ADEF575.4000302@gmail.com> References: <4ADDDEB3.3010204@mrabarnett.plus.com> <494DB4DC-CA0E-4F3D-BF32-8CE0E19DE5E5@masklinn.net> <4ADEF575.4000302@gmail.com> Message-ID: On 21 Oct 2009, at 13:50 , Nick Coghlan wrote: > > You're also omitting consideration of the ways *other* than assignment > to bind a name in Python (i.e. def, with, except, for). Would those > require an explicit let as well? No. > If not, why single out assignment as a > special case that requires an explicit declaration of "Yes, I really > do > want to create a new local here"? Because in all those other cases, the fact that you "really do want to create a new local here" is part of the statement's semantics. Why repeat it? > giving up one of the major benefits of dynamic typing (i.e. > no need for variable declarations). I don't see that as a benefit of dynamic typing (in fact I don't even see that as relevant to dynamic typing, some dynamically typed languages have implicit locals declaration, others have explicit locals ? and scope ? declaration). And even if it were a benefit of dynamic typing, it's hardly a major one, and it's one with a bunch of drawbacks as scope inference tends to be unreliable and messy (leading to the need for `global` and `nonlocal` in Python). From masklinn at masklinn.net Wed Oct 21 14:47:22 2009 From: masklinn at masklinn.net (Masklinn) Date: Wed, 21 Oct 2009 14:47:22 +0200 Subject: [Python-ideas] nonlocal functions In-Reply-To: <4ADEF575.4000302@gmail.com> References: <4ADDDEB3.3010204@mrabarnett.plus.com> <494DB4DC-CA0E-4F3D-BF32-8CE0E19DE5E5@masklinn.net> <4ADEF575.4000302@gmail.com> Message-ID: <196BB58F-F285-415D-917E-62B847EAEAB5@masklinn.net> On 21 Oct 2009, at 13:50 , Nick Coghlan wrote: > > it is getting a really long way away from Python And I probably shouldn't have sent the previous mail that soon? Anyway on this specific quote, I do think `let` fits the Zen better than nothing/`global`/`nonlocal`: it's explicit rather than implicit (as far as scope goes), it's simple rather than complex (a single basic statement instead of 3 different forms depending on the exact case), it's more uniform (less special cases, if any), ? I understand that you disagree, but I don't think the idea of `let` is "getting a long way away from Python". From kristjan at ccpgames.com Wed Oct 21 15:12:38 2009 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Wed, 21 Oct 2009 13:12:38 +0000 Subject: [Python-ideas] Remove GIL with CAS instructions? In-Reply-To: <4ADEFF10.5040701@gmail.com> References: <4ADE2AA9.4030604@molden.no> <4ADE35F0.10803@molden.no> <4ADEFF10.5040701@gmail.com> Message-ID: <930F189C8A437347B80DF2C156F7EC7F098FF41FF8@exchis.ccp.ad.local> > -----Original Message----- > From: python-ideas-bounces+kristjan=ccpgames.com at python.org > [mailto:python-ideas-bounces+kristjan=ccpgames.com at python.org] On > Behalf Of Nick Coghlan > Sent: 21. okt?ber 2009 12:31 > To: Sturla Molden > Cc: python-ideas at python.org > Subject: Re: [Python-ideas] Remove GIL with CAS instructions? > > > - The GIL has consequences on multicore CPUs that are overlooked: > thread > > switches are usually missed at check intervals. > > In the rare case where none of those threads are releasing the GIL for > any other reason, time.sleep(0.001) works wonders. Yeah, it's a wart, > but not a particularly difficult one to deal with. > I think you may be missing the point. If the GIL is implemented in a naive way on a platform, then releasing the gil and reclaiming it (such as is done during a checkinterval) can cause the same thread to get it again. Thus, the idea that this checkinterval should allow another thread waiting for the gil to run, would not work. Apparently, "fairness" of some locking primitives is being given up for efficiency reasons in some operatins systems, like Vista. I tested this by implementing a GIL like mechanism on a multicore maching using windows CriticalSections and Mutexes, both of which would starve the other threads. Semaphores work, though and the python locks on windows (using an atomic counter and an Event object) also work. So, alghough Sturla's claim isn't valid for windows, there might be systems where the syscheckinterval mechanism for thread yielding doesn't work due to the locks in question not being "fair" on multicore platforms. K From metolone+gmane at gmail.com Wed Oct 21 16:53:54 2009 From: metolone+gmane at gmail.com (Mark Tolonen) Date: Wed, 21 Oct 2009 07:53:54 -0700 Subject: [Python-ideas] Syntax for making stuct / record / namedtuples References: <4ADED0B2.3050804@indagon.com> <3bdda690910210316s1bd9630exa83b9fd147644cc3@mail.gmail.com> Message-ID: "Carl Johnson" wrote in message news:3bdda690910210316s1bd9630exa83b9fd147644cc3 at mail.gmail.com... > I would also appreciate a faster way to do the boilerplate in an > __init__ of saying self.arg1 = arg1, self.arg2 = arg2, etc. But what > would be the global implications of changing locals() to an odict? Is > that backwards compatible? If not, perhaps we could add a new builtin, > say "olocals()" instead? >>> class X: ... def __init__(self,a,b,c): ... self.__dict__.update(locals()) ... >>> x=X(1,2,3) >>> x.a 1 >>> x.b 2 >>> x.c 3 -Mark From guido at python.org Wed Oct 21 18:42:01 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Oct 2009 09:42:01 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes Message-ID: I propose a moratorium on language changes. This would be a period of several years during which no changes to Python's grammar or language semantics will be accepted. The reason is that frequent changes to the language cause pain for implementors of alternate implementations (Jython, IronPython, PyPy, and others probably already in the wings) at little or no benefit to the average user (who won't see the changes for years to come and might not be in a position to upgrade to the latest version for years after). The main goal of the Python development community at this point should be to get widespread acceptance of Python 3000. There is tons of work to be done before we can be comfortable about Python 3.x, mostly in creating solid ports of those 3rd party libraries that must be ported to Py3k before other libraries and applications can be ported. (Other work related to Py3k acceptance might be tools to help porting, tools to help maintaining multiple versions of a codebase, documentation about porting to Python 3, and so on. Also, work like that going on in the distutils-sig is very relevant.) Note, the moratorium would only cover the language itself plus built-in functions, not the standard library. Development in the standard library is valuable and much less likely to be a stumbling block for alternate language implementations. I also want to exclude details of the CPython implementation, including the C API from being completely frozen -- for example, if someone came up with (otherwise acceptable) changes to get rid of the GIL I wouldn't object. But the moratorium would clearly apply to proposals for anonymous blocks, "yield from" (PEP 380), changes to decorator syntax, and the like. (I'm sure it won't stop *discussion* of those proposals, and that's not the purpose of the moratorium; but at least it will stop worries elsewhere that such proposals might actually be *accepted* any time soon.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Oct 21 18:56:38 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Oct 2009 09:56:38 -0700 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <4ADEF2C4.9050702@gmail.com> References: <4ADC6A92.3060206@qq.com> <200910201006.30622.steve@pearwood.info> <200910202221.03233.steve@pearwood.info> <2DEB8009-3607-49FC-9C3B-ED08C9B3A245@masklinn.net> <20091020195742.45a43d33@bhuda.mired.org> <4ADEF2C4.9050702@gmail.com> Message-ID: On Wed, Oct 21, 2009 at 4:38 AM, Nick Coghlan wrote: > The main point in favour of anonymous blocks of some kind is that having > to define the subfunction before using it often *will* get the order of > the code as written out of whack with respect to the order that makes > the most logical sense. I actually don't buy that argument. I believe that the main reason people keep arguing for anonymous blocks (and this will never stop) is simply that there is a certain segment of the programmer population that is used to solving a large variety of problems with anonymous blocks and that they don't want to learn how to solve each of those cases using the existing tools in Python. See also my proposal of a moratorium on language changes. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Wed Oct 21 19:17:33 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 21 Oct 2009 19:17:33 +0200 Subject: [Python-ideas] Syntax for making stuct / record / namedtuples In-Reply-To: <3bdda690910210316s1bd9630exa83b9fd147644cc3@mail.gmail.com> References: <4ADED0B2.3050804@indagon.com> <3bdda690910210316s1bd9630exa83b9fd147644cc3@mail.gmail.com> Message-ID: Carl Johnson schrieb: > I would also appreciate a faster way to do the boilerplate in an > __init__ of saying self.arg1 = arg1, self.arg2 = arg2, etc. But what > would be the global implications of changing locals() to an odict? Is > that backwards compatible? If not, perhaps we could add a new builtin, > say "olocals()" instead? locals() does not only return function arguments. You need to specify the order of that ordered dictionary for such a function precisely. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From stefan_ml at behnel.de Wed Oct 21 19:33:31 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 21 Oct 2009 19:33:31 +0200 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: Guido van Rossum wrote: > I propose a moratorium on language changes. This would be a period of > several years during which no changes to Python's grammar or language > semantics will be accepted. The reason is that frequent changes to the > language cause pain for implementors of alternate implementations > (Jython, IronPython, PyPy, and others probably already in the wings) As one of the lead developers of the Cython project, which is following CPython at some distance but at fast pace (and high speed ;), I honestly wouldn't mind such a moratorium. As a Python language user, I must say that I cannot remember any recent(?) language syntax proposal on python-ideas that would have made the language substantially better. Plus, a lot of the code that gets written even today is still required to stay Py2.x compatible, sometimes as old as 2.3. Especially library or tooling code will often not even use Python 2.6 features for a while, let alone Python 3.x features. That doesn't mean you can't 2to3 it, it just means that it won't use the new syntax features. I guess that makes a +1 from my side. Stefan From jnoller at gmail.com Wed Oct 21 19:43:31 2009 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 21 Oct 2009 13:43:31 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: <4222a8490910211043h361eedfeka28042106abb0420@mail.gmail.com> On Wed, Oct 21, 2009 at 12:42 PM, Guido van Rossum wrote: > I propose a moratorium on language changes. This would be a period of > several years during which no changes to Python's grammar or language > semantics will be accepted. The reason is that frequent changes to the > language cause pain for implementors of alternate implementations > (Jython, IronPython, PyPy, and others probably already in the wings) > at little or no benefit to the average user (who won't see the changes > for years to come and might not be in a position to upgrade to the > latest version for years after). > > The main goal of the Python development community at this point should > be to get widespread acceptance of Python 3000. There is tons of work > to be done before we can be comfortable about Python 3.x, mostly in > creating solid ports of those 3rd party libraries that must be ported > to Py3k before other libraries and applications can be ported. (Other > work related to Py3k acceptance might be tools to help porting, tools > to help maintaining multiple versions of a codebase, documentation > about porting to Python 3, and so on. Also, work like that going on in > the distutils-sig is very relevant.) > > Note, the moratorium would only cover the language itself plus > built-in functions, not the standard library. Development in the > standard library is valuable and much less likely to be a stumbling > block for alternate language implementations. I also want to exclude > details of the CPython implementation, including the C API from being > completely frozen -- for example, if someone came up with (otherwise > acceptable) changes to get rid of the GIL I wouldn't object. > > But the moratorium would clearly apply to proposals for anonymous > blocks, "yield from" (PEP 380), changes to decorator syntax, and the > like. (I'm sure it won't stop *discussion* of those proposals, and > that's not the purpose of the moratorium; but at least it will stop > worries elsewhere that such proposals might actually be *accepted* any > time soon.) > +1; modulo getting PEP 380 in before the moratorium ;) I think a fair amount of our time and work is better spent on the stdlib, various implementation(s) and things of that nature (ecosystem). jesse From ironfroggy at gmail.com Wed Oct 21 19:53:15 2009 From: ironfroggy at gmail.com (Calvin Spealman) Date: Wed, 21 Oct 2009 13:53:15 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: <76fd5acf0910211053m347b2ffdhf9e5ad20ae00eddb@mail.gmail.com> On Wed, Oct 21, 2009 at 12:42 PM, Guido van Rossum wrote: > I propose a moratorium on language changes. This would be a period of > several years during which no changes to Python's grammar or language > semantics will be accepted. The reason is that frequent changes to the > language cause pain for implementors of alternate implementations > (Jython, IronPython, PyPy, and others probably already in the wings) > at little or no benefit to the average user (who won't see the changes > for years to come and might not be in a position to upgrade to the > latest version for years after). > > The main goal of the Python development community at this point should > be to get widespread acceptance of Python 3000. There is tons of work > to be done before we can be comfortable about Python 3.x, mostly in > creating solid ports of those 3rd party libraries that must be ported > to Py3k before other libraries and applications can be ported. (Other > work related to Py3k acceptance might be tools to help porting, tools > to help maintaining multiple versions of a codebase, documentation > about porting to Python 3, and so on. Also, work like that going on in > the distutils-sig is very relevant.) > > Note, the moratorium would only cover the language itself plus > built-in functions, not the standard library. Development in the > standard library is valuable and much less likely to be a stumbling > block for alternate language implementations. I also want to exclude > details of the CPython implementation, including the C API from being > completely frozen -- for example, if someone came up with (otherwise > acceptable) changes to get rid of the GIL I wouldn't object. > > But the moratorium would clearly apply to proposals for anonymous > blocks, "yield from" (PEP 380), changes to decorator syntax, and the > like. (I'm sure it won't stop *discussion* of those proposals, and > that's not the purpose of the moratorium; but at least it will stop > worries elsewhere that such proposals might actually be *accepted* any > time soon.) > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas +1 to speeding up the gap-closing between 2.x and 3.x maturity and making the fielding of "When shoudl I start using 3.x?" questions easier. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy From collinw at gmail.com Wed Oct 21 19:59:37 2009 From: collinw at gmail.com (Collin Winter) Date: Wed, 21 Oct 2009 10:59:37 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: <43aa6ff70910211059s5873eb46k9ab1e0c7ec637912@mail.gmail.com> On Wed, Oct 21, 2009 at 9:42 AM, Guido van Rossum wrote: > I propose a moratorium on language changes. This would be a period of > several years during which no changes to Python's grammar or language > semantics will be accepted. The reason is that frequent changes to the > language cause pain for implementors of alternate implementations > (Jython, IronPython, PyPy, and others probably already in the wings) > at little or no benefit to the average user (who won't see the changes > for years to come and might not be in a position to upgrade to the > latest version for years after). +1 From guido at python.org Wed Oct 21 20:06:02 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Oct 2009 11:06:02 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <4222a8490910211043h361eedfeka28042106abb0420@mail.gmail.com> References: <4222a8490910211043h361eedfeka28042106abb0420@mail.gmail.com> Message-ID: On Wed, Oct 21, 2009 at 10:43 AM, Jesse Noller wrote: > +1; modulo getting PEP 380 in before the moratorium ;) I see the smiley, but just in case you were half serious, I really think we should stick to the policy and not make exceptions for personal pet peeves. > I think a fair amount of our time and work is better spent on the > stdlib, various implementation(s) and things of that nature > (ecosystem). Of course, no moratorium can stop people from *proposing* changes, but it helps core developers and others who "just want to get stuff done" focus if they know we're not changing the language for quite a while. Effectively the moratorium would freeze the language at the 3.1 version at least for 3.2 and possibly for 3.3. Also 2.7 could not add language changes except possibly changes to keep up with 3.1 (but only if *very* strict backwards compatibility with 2.6 is maintained also -- I want the upgrade from 2.6 to 2.7 to be a breeze for everyone). Obviously there are many things we could change in the standard library that would still affect the ability to upgrade easily (see the recent issues with 2.6.3 and 2.6.4), and we should be exercising a lot of restraint in this are as well. But language changes affect other implementations the most (I think). They also coincidentally speak most to the imagination of the young, eager-to-add-their-pet-feature amateur language designer crowd who so often fill python-ideas with heated discussions. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From arnodel at googlemail.com Wed Oct 21 19:19:57 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Wed, 21 Oct 2009 18:19:57 +0100 Subject: [Python-ideas] Syntax for making stuct / record / namedtuples In-Reply-To: References: <4ADED0B2.3050804@indagon.com> <3bdda690910210316s1bd9630exa83b9fd147644cc3@mail.gmail.com> <9bfc700a0910210530q7d9de651j839f34bca5bf62fc@mail.gmail.com> Message-ID: <0BCF55FE-6204-4BFF-8E4D-B33FD5E94C3E@googlemail.com> On 21 Oct 2009, at 16:56, Bruce Leban wrote: > Reading your code I have no idea what these lines mean > > > autoassign(function) -> method > autoassign(*argnames) -> decorator > autoassign(exclude=argnames) -> decorator > > Reading the code, what I *think* this does > > """Decorator that assigns all function parameters to attributes of > the class. > You can specify a list of parameters to process or a list of > parameters to exclude. > Given another function, does something complicated that belongs in > another recipe. > """ > I don't understand that last sentence. > I'm only half-joking. It's not obvious what that function parameter > does and you don't give any examples of it's use. I give two examples. One in the docstring (first example), the other one in the tests below the definition of the decorator (class Test2). > Just reading the code with unrelated assignments grouped together > was confusing enough -- definitely not my coding style. Sorry I confused you :) > > More importantly not at all clear why this is better than > > self.__dict__.update(locals()) > It's not a very nice construct. And it doesn't work as is: >>> class Foo(object): ... def __init__(self, a, b, c): ... self.__dict__.update(locals()) ... >>> foo=Foo(1, 2, 3) >>> foo.self <__main__.Foo object at 0x10048c790> Probably not what was intended... So you can write something like: >>> class Foo(object): ... def __init__(self, a, b, c): ... self.__dict__.update(((n, v) for (n, v) in locals ().items() if n!='self')) ... >>> foo=Foo(1, 2, 3) >>> foo.self Traceback (most recent call last): File "", line 1, in AttributeError: 'Foo' object has no attribute 'self' But that reads even worse. Also, it's a bit more versatile. -- Arnaud From dangyogi at gmail.com Wed Oct 21 20:14:06 2009 From: dangyogi at gmail.com (Bruce Frederiksen) Date: Wed, 21 Oct 2009 14:14:06 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: <7d702edf0910211114n79194605h92affa2fc0de3739@mail.gmail.com> Nuts, I accidentally sent this just to Guido (again...) Sorry Guido! ------------------------------------- I, for one, am very glad to see this! I might suggest that changes to correct "bugs" in the language definition (and I don't just mean the documentation) should still be allowed. I'd like to also point out that the import mechanism (including the concept of the python path, packages, and module initialization) seems to still be messy. For example, getting two copies of the same module when imported with and without a package prefix, and weird import ordering dependencies. I guess these would be covered under the "bugs in the language definition" above. I've wondered too about opening up the whole import process with more hooks to allow other notions of "compiling" and "loading" (for example, being able to import pickles of complex object networks that have been created through some other "compile" process that might still need file modification times checked, but with different file suffixes). This might help to give some degree of extensibility and support for domain specific languages, for example. I agree that Python will gain much more through better implementations than through further additions to the language. (And I don't mean to put down the CPython implementation in any way, it's carried us all a very long way!) Excluding the C implementation from the moratorium makes sense. +1 -Bruce On Wed, Oct 21, 2009 at 12:42 PM, Guido van Rossum wrote: > I propose a moratorium on language changes. This would be a period of > several years during which no changes to Python's grammar or language > semantics will be accepted. The reason is that frequent changes to the > language cause pain for implementors of alternate implementations > (Jython, IronPython, PyPy, and others probably already in the wings) > at little or no benefit to the average user (who won't see the changes > for years to come and might not be in a position to upgrade to the > latest version for years after). > > The main goal of the Python development community at this point should > be to get widespread acceptance of Python 3000. There is tons of work > to be done before we can be comfortable about Python 3.x, mostly in > creating solid ports of those 3rd party libraries that must be ported > to Py3k before other libraries and applications can be ported. (Other > work related to Py3k acceptance might be tools to help porting, tools > to help maintaining multiple versions of a codebase, documentation > about porting to Python 3, and so on. Also, work like that going on in > the distutils-sig is very relevant.) > > Note, the moratorium would only cover the language itself plus > built-in functions, not the standard library. Development in the > standard library is valuable and much less likely to be a stumbling > block for alternate language implementations. I also want to exclude > details of the CPython implementation, including the C API from being > completely frozen -- for example, if someone came up with (otherwise > acceptable) changes to get rid of the GIL I wouldn't object. > > But the moratorium would clearly apply to proposals for anonymous > blocks, "yield from" (PEP 380), changes to decorator syntax, and the > like. (I'm sure it won't stop *discussion* of those proposals, and > that's not the purpose of the moratorium; but at least it will stop > worries elsewhere that such proposals might actually be *accepted* any > time soon.) > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/ > ) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jnoller at gmail.com Wed Oct 21 20:14:06 2009 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 21 Oct 2009 14:14:06 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <4222a8490910211043h361eedfeka28042106abb0420@mail.gmail.com> Message-ID: <4222a8490910211114o52e62d81q4617420c56baa915@mail.gmail.com> On Wed, Oct 21, 2009 at 2:06 PM, Guido van Rossum wrote: > I see the smiley, but just in case you were half serious, I really > think we should stick to the policy and not make exceptions for > personal pet peeves. The only reason I would argue against that would be for the fact that AFAIK, it's "nearly done and ready" - that being said, that is a specious argument. But I agree - no exceptions makes sense. > Obviously there are many things we could change in the standard > library that would still affect the ability to upgrade easily (see the > recent issues with 2.6.3 and 2.6.4), and we should be exercising a lot > of restraint in this are as well. But language changes affect other > implementations the most (I think). They also coincidentally speak > most to the imagination of the young, eager-to-add-their-pet-feature > amateur language designer crowd who so often fill python-ideas with > heated discussions. Agreed; there's been discussion on disutils-sig and other places (such as the language summit) revolving around a "single, shared" standard library for all of the implementations. So long as the syntax is unchanged, we can upgrade/improve the stdlib over time, and make it used for the various implementations. jesse From bruce at leapyear.org Wed Oct 21 20:19:34 2009 From: bruce at leapyear.org (Bruce Leban) Date: Wed, 21 Oct 2009 11:19:34 -0700 Subject: [Python-ideas] Syntax for making stuct / record / namedtuples In-Reply-To: <0BCF55FE-6204-4BFF-8E4D-B33FD5E94C3E@googlemail.com> References: <4ADED0B2.3050804@indagon.com> <3bdda690910210316s1bd9630exa83b9fd147644cc3@mail.gmail.com> <9bfc700a0910210530q7d9de651j839f34bca5bf62fc@mail.gmail.com> <0BCF55FE-6204-4BFF-8E4D-B33FD5E94C3E@googlemail.com> Message-ID: On Wed, Oct 21, 2009 at 10:19 AM, Arnaud Delobelle wrote: > > On 21 Oct 2009, at 16:56, Bruce Leban wrote: > > Reading your code I have no idea what these lines mean >> >> >> autoassign(function) -> method >> autoassign(*argnames) -> decorator >> autoassign(exclude=argnames) -> decorator >> >> Reading the code, what I *think* this does >> >> """Decorator that assigns all function parameters to attributes of the >> class. >> You can specify a list of parameters to process or a list of parameters to >> exclude. >> Given another function, does something complicated that belongs in another >> recipe. >> """ >> >> > I don't understand that last sentence. > > I'm only half-joking. It's not obvious what that function parameter does >> and you don't give any examples of it's use. >> > > I give two examples. One in the docstring (first example), the other one > in the tests below the definition of the decorator (class Test2). > I meant there are no examples of using autoassign with a function. > > Just reading the code with unrelated assignments grouped together was >> confusing enough -- definitely not my coding style. >> > > Sorry I confused you :) > Write once, read many times. :-) Especially for recipes, clear > concise. > > >> More importantly not at all clear why this is better than >> >> self.__dict__.update(locals()) >> >> > It's not a very nice construct. And it doesn't work as is: > > >>> class Foo(object): > ... def __init__(self, a, b, c): > ... self.__dict__.update(locals()) > ... > >>> foo=Foo(1, 2, 3) > >>> foo.self > <__main__.Foo object at 0x10048c790> > > Probably not what was intended... So you can write something like: > > >>> class Foo(object): > ... def __init__(self, a, b, c): > ... self.__dict__.update(((n, v) for (n, v) in locals().items() > if n!='self')) > ... > >>> foo=Foo(1, 2, 3) > >>> foo.self > Traceback (most recent call last): > File "", line 1, in > AttributeError: 'Foo' object has no attribute 'self' > > But that reads even worse. > > Also, it's a bit more versatile. > > I don't get the function case. And I have no idea what you mean by -> decorator or -> method. I do understand how the other two cases work but those first three lines of the doc don't tell me anything. > -- > Arnaud > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at rcn.com Wed Oct 21 20:26:49 2009 From: python at rcn.com (Raymond Hettinger) Date: Wed, 21 Oct 2009 14:26:49 -0400 (EDT) Subject: [Python-ideas] Proposal: Moratorium on Python language changes Message-ID: <20091021142649.BRH02340@ms19.lnh.mail.rcn.net> [Guido van Rossum] > Note, the moratorium would only cover the language itself plus > built-in functions, not the standard library. That makes sense. There may be a few areas that still have some rough edges where you may want to allow changes if needed (tweaks to the nested with-statement syntax, bytes/text interaction, star-args unpacking, or string formatting). These areas probably have not been exercised much and there may still be problems that need to be ironed-out. I don't have anything specific in mind. Am just thinking that those features aren't yet mature. Also, do you know if there any plans afoot to do something with function annotations? > I also want to exclude > details of the CPython implementation, including the C API from being > completely frozen -- for example, if someone came up with (otherwise > acceptable) changes to get rid of the GIL I wouldn't object. FWIW, I have pending updates for the set/frozenset implementation (no api change). Also, I'm hoping the recently submitted C implementation for decimal gets accepted (as performance issues seem to be slowing broader use of the module). It looks like substantial work has already been done. > But the moratorium would clearly apply to proposals for anonymous > blocks, "yield from" (PEP 380), changes to decorator syntax, and the > like. (I'm sure it won't stop *discussion* of those proposals, and > that's not the purpose of the moratorium; but at least it will stop > worries elsewhere that such proposals might actually be *accepted* any > time soon.) Are you rejecting PEP 380? Raymond From guido at python.org Wed Oct 21 20:27:23 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Oct 2009 11:27:23 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <7d702edf0910211114n79194605h92affa2fc0de3739@mail.gmail.com> References: <7d702edf0910211114n79194605h92affa2fc0de3739@mail.gmail.com> Message-ID: On Wed, Oct 21, 2009 at 11:14 AM, Bruce Frederiksen wrote: > I might suggest that changes to correct "bugs" in the language definition > (and I don't just mean the documentation) should still be allowed. Depends on your definition of "bug". I would like to see this on a case-by-case basis. > I'd like to also point out that the import mechanism (including the concept > of the python path, packages, and module initialization) seems to still be > messy.? For example, getting two copies of the same module when imported > with and without a package prefix, and weird import ordering dependencies. Well, for better or for worse, changing that is *also* going to be messy and a compatibility nightmare. To some extent the details of the import mechanism is up to the implementation anyway, so this may not realistically be affected by the moratorium. If you want to discuss these issues in detail I recommend that you start a new thread and be prepared to hear "but that's just pilot error" about many of the perceived "bugs". > I guess these would be covered under the "bugs in the language definition" > above.? I've wondered too about opening up the whole import process with > more hooks to allow other notions of "compiling" and "loading" (for example, > being able to import pickles of complex object networks that have been > created through some other "compile" process that might still need file > modification times checked, but with different file suffixes).? This might > help to give some degree of extensibility and support for domain specific > languages, for example. Actually, I think we already have enough import hooks, and this is an area where I would tread very carefully. There is nothing stopping people from using these mechanisms now, but still they are rarely used -- I don't expect that adding new types of hooks will change that. > I agree that Python will gain much more through better implementations than > through further additions to the language.? (And I don't mean to put down > the CPython implementation in any way, it's carried us all a very long > way!)? Excluding the C implementation from the moratorium makes sense. > > +1 > > -Bruce -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Wed Oct 21 20:24:07 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 21 Oct 2009 20:24:07 +0200 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <4222a8490910211043h361eedfeka28042106abb0420@mail.gmail.com> Message-ID: Guido van Rossum schrieb: > On Wed, Oct 21, 2009 at 10:43 AM, Jesse Noller wrote: >> +1; modulo getting PEP 380 in before the moratorium ;) > > I see the smiley, but just in case you were half serious, I really > think we should stick to the policy and not make exceptions for > personal pet peeves. Well, the moratorium isn't yet in place (which I also gather from you posting that on the ideas list), so would be nothing wrong implementing something before it starts... :) Anyway, I'm not sure how much impact this will have; already there are very few real language changes going on; after 2.6, whose such changes mostly were Python 3 backports. In 2.7, the multi-with syntax is the only real language change so far. (When you say "builtins", do you e.g. include the str.format() mini-language? There has been a lot of work on that lately, to fix ambiguous corner-cases.) I'm not saying it's a bad idea, but while it might stop a few threads on python-dev, it won't change a lot in terms of how core development time is spent. cheers, Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From qrczak at knm.org.pl Wed Oct 21 20:29:07 2009 From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: Wed, 21 Oct 2009 20:29:07 +0200 Subject: [Python-ideas] nonlocal functions In-Reply-To: References: <4ADDDEB3.3010204@mrabarnett.plus.com> <494DB4DC-CA0E-4F3D-BF32-8CE0E19DE5E5@masklinn.net> <4ADEF575.4000302@gmail.com> Message-ID: <3f4107910910211129p4081f80gc31de6b0341b9217@mail.gmail.com> 2009/10/21 Masklinn : >> If not, why single out assignment as a >> special case that requires an explicit declaration of "Yes, I really do >> want to create a new local here"? > > Because in all those other cases, the fact that you "really do want to > create a new local here" is part of the statement's semantics. Why repeat > it? Actually in the other cases an existing variable is assigned if it was already defined in the current scope, or it is defined if it was not - like in the assignment syntax. -- Marcin Kowalczyk qrczak at knm.org.pl From guido at python.org Wed Oct 21 20:41:05 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Oct 2009 11:41:05 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <20091021142649.BRH02340@ms19.lnh.mail.rcn.net> References: <20091021142649.BRH02340@ms19.lnh.mail.rcn.net> Message-ID: On Wed, Oct 21, 2009 at 11:26 AM, Raymond Hettinger wrote: > [Guido van Rossum] >> Note, the moratorium would only cover the language itself plus >> built-in functions, not the standard library. > > That makes sense. > > There may be a few areas that still have some rough edges where you > may want to allow changes if needed (tweaks to the nested with-statement > syntax, bytes/text interaction, star-args unpacking, or string formatting). > These areas probably have not been exercised much and there may still be > problems that need to be ironed-out. ?I don't have anything specific > in mind. ?Am just thinking that those features aren't yet mature. No, the moratorium would freeze the language at the 3.1 version, at least for 3.2 and 2.7 and possibly 3.3 (see my earlier post). Allowing for exceptions like these provides too much wiggle room. (E.g is the decorator syntax broken?) Outright bugs in the implementation should be excepted (subject to discussion) but the accepted grammar and semantics should be frozen unless they are unimplementable. > Also, do you know if there any plans afoot to do something with function annotations? I don't. Giving them more semantics would fall under the moratorium; however some implementations might have optional extensions that don't. I can also see that implementations which support interaction with APIs defined in other languages (chiefly IronPython and Jython) might define semantics for function annotation syntax in certain cases where it benefits interaction with other languages -- this would be in code that is naturally restricted to that implementation anyway. >> ?I also want to exclude >> details of the CPython implementation, including the C API from being >> completely frozen -- for example, if someone came up with (otherwise >> acceptable) changes to get rid of the GIL I wouldn't object. > > FWIW, I have pending updates for the set/frozenset implementation > (no api change). If it's just performance that's fine. If there's more I'd like to hear more details first. > Also, I'm hoping the recently submitted C implementation for decimal > gets accepted (as performance issues seem to be slowing broader > use of the module). ?It looks like substantial work has already > been done. I don't think that "work on a feature has already begun" should be an argument for granting an exception. However I expect that in this specific case this falls under the library, which is not directly covered by the moratorium. I think I saw that it could be made optional, so that if the C implementation isn't found the Python implementation still works? >> But the moratorium would clearly apply to proposals for anonymous >> blocks, "yield from" (PEP 380), changes to decorator syntax, and the >> like. (I'm sure it won't stop *discussion* of those proposals, and >> that's not the purpose of the moratorium; but at least it will stop >> worries elsewhere that such proposals might actually be *accepted* any >> time soon.) > > Are you rejecting PEP 380? No, just deferring it. It didn't make Python 3.1 so I think we'll have to live without it for a long time no matter what. It could be revisited once the moratorium is lifted (for 3.3 or 3.4). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Oct 21 20:44:16 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Oct 2009 11:44:16 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <4222a8490910211043h361eedfeka28042106abb0420@mail.gmail.com> Message-ID: On Wed, Oct 21, 2009 at 11:24 AM, Georg Brandl wrote: > Well, the moratorium isn't yet in place (which I also gather from you > posting that on the ideas list), so would be nothing wrong implementing > something before it starts... :) No, the moratorium would freeze the language at the version implemented in 3.1. If necessary we'd have to roll back core language changes (primarily syntax, or new builtins) made since 3.1 was released. > Anyway, I'm not sure how much impact this will have; already there are > very few real language changes going on; after 2.6, whose such changes > mostly were Python 3 backports. ?In 2.7, the multi-with syntax is the > only real language change so far. > > (When you say "builtins", do you e.g. include the str.format() mini-language? > There has been a lot of work on that lately, to fix ambiguous corner-cases.) I think fixing ambiguous corner cases can proceed. > I'm not saying it's a bad idea, but while it might stop a few threads on > python-dev, it won't change a lot in terms of how core development > time is spent. My main goal is to set expectations right -- people who spend effort porting to 3.1 should not be confronted with further effort caused by the upgrade to 3.2. (An alternative would be to postpone the release of 3.2 until the moratorium is over, but this would unnecessarily constrain further development of the library.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From daniel at stutzbachenterprises.com Wed Oct 21 20:45:50 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Wed, 21 Oct 2009 13:45:50 -0500 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <7d702edf0910211114n79194605h92affa2fc0de3739@mail.gmail.com> References: <7d702edf0910211114n79194605h92affa2fc0de3739@mail.gmail.com> Message-ID: On Wed, Oct 21, 2009 at 1:14 PM, Bruce Frederiksen wrote: > I might suggest that changes to correct "bugs" in the language definition > (and I don't just mean the documentation) should still be allowed. > I suggest allowing an exception for changes that make life *easier* for alternative implementations. That would allow for fixing "bugs", making small improvements to new features added in Python 3, etc. As long as it passes the test of making life easier for alternative implementations instead of more difficult. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From debatem1 at gmail.com Wed Oct 21 20:52:41 2009 From: debatem1 at gmail.com (geremy condra) Date: Wed, 21 Oct 2009 14:52:41 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: On Wed, Oct 21, 2009 at 12:42 PM, Guido van Rossum wrote: > I propose a moratorium on language changes. This would be a period of > several years during which no changes to Python's grammar or language > semantics will be accepted. The reason is that frequent changes to the > language cause pain for implementors of alternate implementations > (Jython, IronPython, PyPy, and others probably already in the wings) > at little or no benefit to the average user (who won't see the changes > for years to come and might not be in a position to upgrade to the > latest version for years after). > > The main goal of the Python development community at this point should > be to get widespread acceptance of Python 3000. There is tons of work > to be done before we can be comfortable about Python 3.x, mostly in > creating solid ports of those 3rd party libraries that must be ported > to Py3k before other libraries and applications can be ported. (Other > work related to Py3k acceptance might be tools to help porting, tools > to help maintaining multiple versions of a codebase, documentation > about porting to Python 3, and so on. Also, work like that going on in > the distutils-sig is very relevant.) > > Note, the moratorium would only cover the language itself plus > built-in functions, not the standard library. Development in the > standard library is valuable and much less likely to be a stumbling > block for alternate language implementations. I also want to exclude > details of the CPython implementation, including the C API from being > completely frozen -- for example, if someone came up with (otherwise > acceptable) changes to get rid of the GIL I wouldn't object. > > But the moratorium would clearly apply to proposals for anonymous > blocks, "yield from" (PEP 380), changes to decorator syntax, and the > like. (I'm sure it won't stop *discussion* of those proposals, and > that's not the purpose of the moratorium; but at least it will stop > worries elsewhere that such proposals might actually be *accepted* any > time soon.) > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) I can't disagree with the sentiment, and I wholeheartedly agree with the idea that shifting the focus from language development to library development is a good idea- but I would caution against making the moratorium too hard-and-fast, or too long lasting, assuming that the goal isn't to make it de facto permanent. Towards that end, I'd also like to propose a very public, very accessible 'sandbox' specifically for the development and testing of new language features while the moratorium is in effect. Its goal would be to keep interest in changes to core language design ongoing by keeping the barrier to entry low, while simultaneously separating it from core development. With any luck, it would mean that when the moratorium lifts, Python will be able to take its pick from the best of the language proposals, while still having given other implementations the opportunity to study their behavior "in the wild" for a period of months or years. Geremy Condra From guido at python.org Wed Oct 21 20:58:45 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Oct 2009 11:58:45 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <7d702edf0910211114n79194605h92affa2fc0de3739@mail.gmail.com> Message-ID: On Wed, Oct 21, 2009 at 11:45 AM, Daniel Stutzbach wrote: > On Wed, Oct 21, 2009 at 1:14 PM, Bruce Frederiksen > wrote: >> >> I might suggest that changes to correct "bugs" in the language definition >> (and I don't just mean the documentation) should still be allowed. > > I suggest allowing an exception for changes that make life *easier* for > alternative implementations.? That would allow for fixing "bugs", making > small improvements to new features added in Python 3, etc.? As long as it > passes the test of making life easier for alternative implementations > instead of more difficult. This is reasonable. We should discuss this at the Language Summit at the next PyCon. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From daniel at stutzbachenterprises.com Wed Oct 21 21:02:47 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Wed, 21 Oct 2009 14:02:47 -0500 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: On Wed, Oct 21, 2009 at 1:52 PM, geremy condra wrote: > Towards that end, I'd also like to propose a very public, very > accessible 'sandbox' specifically for the development and testing of > new language features while the moratorium is in effect. > Once the transition to Mercurial is complete, making experimental forks of CPython to play with new language features will be a breeze. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Wed Oct 21 21:02:29 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Oct 2009 12:02:29 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: On Wed, Oct 21, 2009 at 11:52 AM, geremy condra wrote: > Towards that end, I'd also like to propose a very public, very > accessible 'sandbox' specifically for the development and testing of > new language features while the moratorium is in effect. Its goal > would be to keep interest in changes to core language design > ongoing by keeping the barrier to entry low, while simultaneously > separating it from core development. With any luck, it would mean > that when the moratorium lifts, Python will be able to take its pick > from the best of the language proposals, while still having given > other implementations the opportunity to study their behavior > "in the wild" for a period of months or years. I can't stop people from forking the language to do experiments, but one of the goals I have for the moratorium is actually to *reduce* the interest in core language changes, and to *raise* the barrier to entry. Most language change proposals are just fluff, and they will be just as unneeded three years from now as they are today. Once the moratorium is lifted, users should be able expect the normal, slow, conservative evolution of the language to continue -- not to see the floodgates lifted for a barrage of new features. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From dinov at microsoft.com Wed Oct 21 21:07:11 2009 From: dinov at microsoft.com (Dino Viehland) Date: Wed, 21 Oct 2009 19:07:11 +0000 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <4222a8490910211043h361eedfeka28042106abb0420@mail.gmail.com> Message-ID: <1A472770E042064698CB5ADC83A12ACD04B114F3@TK5EX14MBXC116.redmond.corp.microsoft.com> Guido wrote: > > I think a fair amount of our time and work is better spent on the > > stdlib, various implementation(s) and things of that nature > > (ecosystem). > > Of course, no moratorium can stop people from *proposing* changes, but > it helps core developers and others who "just want to get stuff done" > focus if they know we're not changing the language for quite a while. > > Effectively the moratorium would freeze the language at the 3.1 > version at least for 3.2 and possibly for 3.3. Also 2.7 could not add > language changes except possibly changes to keep up with 3.1 (but only > if *very* strict backwards compatibility with 2.6 is maintained also > -- I want the upgrade from 2.6 to 2.7 to be a breeze for everyone). > > Obviously there are many things we could change in the standard > library that would still affect the ability to upgrade easily (see the > recent issues with 2.6.3 and 2.6.4), and we should be exercising a lot > of restraint in this are as well. But language changes affect other > implementations the most (I think). They also coincidentally speak > most to the imagination of the young, eager-to-add-their-pet-feature > amateur language designer crowd who so often fill python-ideas with > heated discussions. This generally sounds nice for us and will make it easier to catch up to 3.x as well as make it easier to continue to flush out other missing built-in modules. But I would actually be really happy with just a partial freeze though - I actually find support for new standard library features is typically the biggest hit for us and that core language features (modulo import related features which are always difficult ) have generally not been an issue. For example consider multiprocessing - which IronPython still doesn't support. That's a much bigger work item than everything that changed related to parsing in 2.6. New built-in functions can also be a huge pain - again compare float.fromhex to everything that happened to the parser in 2.6. I'm pretty sure I spent more time on float.fromhex even w/ the already existing awesome test suite. So I definitely think not adding significant new functionality to the core interpreter would be great but from an alternate implementation perspective I'd also be happy to see some wiggle room on small new features. From jnoller at gmail.com Wed Oct 21 21:13:22 2009 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 21 Oct 2009 15:13:22 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <1A472770E042064698CB5ADC83A12ACD04B114F3@TK5EX14MBXC116.redmond.corp.microsoft.com> References: <4222a8490910211043h361eedfeka28042106abb0420@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04B114F3@TK5EX14MBXC116.redmond.corp.microsoft.com> Message-ID: <4222a8490910211213y5b0adedcle3f1fe48196466a8@mail.gmail.com> On Wed, Oct 21, 2009 at 3:07 PM, Dino Viehland wrote: > For example consider multiprocessing - which IronPython still doesn't > support. ? That's a much bigger work item than everything that changed > related to parsing in 2.6. ?New built-in functions can also be a huge > pain - again compare float.fromhex to everything that happened to the > parser in 2.6. ?I'm pretty sure I spent more time on float.fromhex even > w/ the already existing awesome test suite. I don't think IronPython should support multiprocessing, personally. While there are a few things within it that probably make sense (managers and the like) does it really make a ton of sense to run multiple iron python VMs from a single parent? The same applies (in the case of multiprocessing) to Jython - a single JVM forking multiple JVMs doesn't necessarily make sense when you have perfectly functional threads. Then again, this depends on your perspective. When splitting out the stdlib was discussed at the language summit last year, this was discussed, and at the time a few people suggested allowing some stdlib modules to be marked at "cpython-only" - this means other implementations could omit/skip those modules (and not ship them). jesse From solipsis at pitrou.net Wed Oct 21 21:15:02 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 21 Oct 2009 19:15:02 +0000 (UTC) Subject: [Python-ideas] Proposal: Moratorium on Python language changes References: Message-ID: Daniel Stutzbach writes: > > Once the transition to Mercurial is complete, making experimental forks of > CPython to play with new language features will be a breeze. There are already Mercurial branches, they are read-only but it's sufficient if you want to clone them and create your own branches while still tracking upstream changes. Regards Antoine. From python at mrabarnett.plus.com Wed Oct 21 21:29:58 2009 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 21 Oct 2009 20:29:58 +0100 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <20091021142649.BRH02340@ms19.lnh.mail.rcn.net> Message-ID: <4ADF6136.1060904@mrabarnett.plus.com> Guido van Rossum wrote: > On Wed, Oct 21, 2009 at 11:26 AM, Raymond Hettinger wrote: >> [Guido van Rossum] >>> Note, the moratorium would only cover the language itself plus >>> built-in functions, not the standard library. >> That makes sense. >> >> There may be a few areas that still have some rough edges where you >> may want to allow changes if needed (tweaks to the nested with-statement >> syntax, bytes/text interaction, star-args unpacking, or string formatting). >> These areas probably have not been exercised much and there may still be >> problems that need to be ironed-out. I don't have anything specific >> in mind. Am just thinking that those features aren't yet mature. > > No, the moratorium would freeze the language at the 3.1 version, at > least for 3.2 and 2.7 and possibly 3.3 (see my earlier post). Allowing > for exceptions like these provides too much wiggle room. (E.g is the > decorator syntax broken?) > > Outright bugs in the implementation should be excepted (subject to > discussion) but the accepted grammar and semantics should be frozen > unless they are unimplementable. > I'd like there to be the possibility of a change if we were to discover a case (a corner case, perhaps) where everyone agrees that what Python is actually doing is unPythonic due to some unforeseen combination of factors (and I'm not talking about mutable default parameters). Who knows, it could happen! :-) From fdrake at acm.org Wed Oct 21 21:39:43 2009 From: fdrake at acm.org (Fred Drake) Date: Wed, 21 Oct 2009 15:39:43 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: <9cee7ab80910211239l452eb8f6sec7c0929829bedeb@mail.gmail.com> On Wed, Oct 21, 2009 at 12:42 PM, Guido van Rossum wrote: > I propose a moratorium on language changes. This would be a period of > several years during which no changes to Python's grammar or language > semantics will be accepted. +1 While there are changes that I'd like to see, a moratorium for a few years would benefit all of us in the long run: - It's easier for companies to justify training. - It's easier to gain adoption for long-term projects. - It's easier to hold the attention of potential contributors for bug fixes. (And for many other reasons, but those are big ones in my book these days.) -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller From eric at trueblade.com Wed Oct 21 21:16:13 2009 From: eric at trueblade.com (Eric Smith) Date: Wed, 21 Oct 2009 15:16:13 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes Message-ID: Sorry for top posting and replying to a random message. I'm on a mobile device, and I fear a decision will be reached before I'm back at a real computer. It would be a shame to prevent any enhancements that would enable us to move forward on the distutils front, now that we finally have some momentum there. I'm thinking primarily of namespace packages (can't easily search for Martin's PEP just now). Most of the work will be in libraries, but adding a few core features might make things easier or better. But even if that didn't make the cut, I'm at least +0 on the idea. -- Eric. "Guido van Rossum" wrote: >On Wed, Oct 21, 2009 at 11:26 AM, Raymond Hettinger wrote: >> [Guido van Rossum] >>> Note, the moratorium would only cover the language itself plus >>> built-in functions, not the standard library. >> >> That makes sense. >> >> There may be a few areas that still have some rough edges where you >> may want to allow changes if needed (tweaks to the nested with-statement >> syntax, bytes/text interaction, star-args unpacking, or string formatting). >> These areas probably have not been exercised much and there may still be >> problems that need to be ironed-out. ?I don't have anything specific >> in mind. ?Am just thinking that those features aren't yet mature. > >No, the moratorium would freeze the language at the 3.1 version, at >least for 3.2 and 2.7 and possibly 3.3 (see my earlier post). Allowing >for exceptions like these provides too much wiggle room. (E.g is the >decorator syntax broken?) > >Outright bugs in the implementation should be excepted (subject to >discussion) but the accepted grammar and semantics should be frozen >unless they are unimplementable. > >> Also, do you know if there any plans afoot to do something with function annotations? > >I don't. Giving them more semantics would fall under the moratorium; >however some implementations might have optional extensions that >don't. I can also see that implementations which support interaction >with APIs defined in other languages (chiefly IronPython and Jython) >might define semantics for function annotation syntax in certain cases >where it benefits interaction with other languages -- this would be in >code that is naturally restricted to that implementation anyway. > >>> ?I also want to exclude >>> details of the CPython implementation, including the C API from being >>> completely frozen -- for example, if someone came up with (otherwise >>> acceptable) changes to get rid of the GIL I wouldn't object. >> >> FWIW, I have pending updates for the set/frozenset implementation >> (no api change). > >If it's just performance that's fine. If there's more I'd like to hear >more details first. > >> Also, I'm hoping the recently submitted C implementation for decimal >> gets accepted (as performance issues seem to be slowing broader >> use of the module). ?It looks like substantial work has already >> been done. > >I don't think that "work on a feature has already begun" should be an >argument for granting an exception. However I expect that in this >specific case this falls under the library, which is not directly >covered by the moratorium. I think I saw that it could be made >optional, so that if the C implementation isn't found the Python >implementation still works? > >>> But the moratorium would clearly apply to proposals for anonymous >>> blocks, "yield from" (PEP 380), changes to decorator syntax, and the >>> like. (I'm sure it won't stop *discussion* of those proposals, and >>> that's not the purpose of the moratorium; but at least it will stop >>> worries elsewhere that such proposals might actually be *accepted* any >>> time soon.) >> >> Are you rejecting PEP 380? > >No, just deferring it. It didn't make Python 3.1 so I think we'll have >to live without it for a long time no matter what. It could be >revisited once the moratorium is lifted (for 3.3 or 3.4). > >-- >--Guido van Rossum (home page: http://www.python.org/~guido/) >_______________________________________________ >Python-ideas mailing list >Python-ideas at python.org >http://mail.python.org/mailman/listinfo/python-ideas > From jnoller at gmail.com Wed Oct 21 21:49:27 2009 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 21 Oct 2009 15:49:27 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: <4222a8490910211249r266c1aa2x5cd20498413ce39f@mail.gmail.com> On Wed, Oct 21, 2009 at 3:16 PM, Eric Smith wrote: > Sorry for top posting and replying to a random message. I'm on a mobile device, and I fear a decision will be reached before I'm back at a real computer. > > It would be a shame to prevent any enhancements that would enable us to move forward on the distutils front, now that we finally have some momentum there. I'm thinking primarily of namespace packages (can't easily search for Martin's PEP just now). Most of the work will be in libraries, but adding a few core features might make things easier or better. > > But even if that didn't make the cut, I'm at least +0 on the idea. > -- > Eric. Distutils != core language syntax. Distutils was already mentioned in guido's original email, and given it's a standard library module, it can continue to evolve. jesse From dinov at microsoft.com Wed Oct 21 21:57:39 2009 From: dinov at microsoft.com (Dino Viehland) Date: Wed, 21 Oct 2009 19:57:39 +0000 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <4222a8490910211213y5b0adedcle3f1fe48196466a8@mail.gmail.com> References: <4222a8490910211043h361eedfeka28042106abb0420@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04B114F3@TK5EX14MBXC116.redmond.corp.microsoft.com> <4222a8490910211213y5b0adedcle3f1fe48196466a8@mail.gmail.com> Message-ID: <1A472770E042064698CB5ADC83A12ACD04B11A73@TK5EX14MBXC116.redmond.corp.microsoft.com> Jesse wrote: > I don't think IronPython should support multiprocessing, personally. > While there are a few things within it that probably make sense > (managers and the like) does it really make a ton of sense to run > multiple iron python VMs from a single parent? The problem w/ this is that people are going to use multiprocessing and then someone is going to try running code which uses it on IronPython/Jython and it isn't going to work. So maybe we could have a version of multiprocessing that transparently works in a completely different fashion under the hood but it seems like the module should still exist and be usable. From debatem1 at gmail.com Wed Oct 21 22:00:54 2009 From: debatem1 at gmail.com (geremy condra) Date: Wed, 21 Oct 2009 16:00:54 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: On Wed, Oct 21, 2009 at 3:02 PM, Guido van Rossum wrote: > On Wed, Oct 21, 2009 at 11:52 AM, geremy condra wrote: >> Towards that end, I'd also like to propose a very public, very >> accessible 'sandbox' specifically for the development and testing of >> new language features while the moratorium is in effect. Its goal >> would be to keep interest in changes to core language design >> ongoing by keeping the barrier to entry low, while simultaneously >> separating it from core development. With any luck, it would mean >> that when the moratorium lifts, Python will be able to take its pick >> from the best of the language proposals, while still having given >> other implementations the opportunity to study their behavior >> "in the wild" for a period of months or years. > > I can't stop people from forking the language to do experiments, but > one of the goals I have for the moratorium is actually to *reduce* the > interest in core language changes, and to *raise* the barrier to > entry. Most language change proposals are just fluff, and they will be > just as unneeded three years from now as they are today. Once the > moratorium is lifted, users should be able expect the normal, slow, > conservative evolution of the language to continue -- not to see the > floodgates lifted for a barrage of new features. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > Granted that most language change proposals could only be described as "terrible" by a person of unusual charity, but if the goal here isn't to permanently forbid all changes, then it makes sense (to me) to ensure that we're in a better position when we leave the moratorium than when we go into it. Right now, the barrier for proposing a language change is pretty much nonexistent- you dreamed it up in the shower, read about it in a trade magazine, your dog told you it was a good idea, doesn't matter- put it on python-ideas. The problem (as I see it) is that after proposing the idea, wasting a lot of time, and causing a lot of drama, the fact remains that even if the OP finally gets their way, the odds are good that it will take a substantial amount of work from people who have better things to do- the core developers- to make it work. And it only gets worse for the other implementors, who probably didn't have as much input in the process as the CPython devs did. Changing the way that core language changes are proposed and evaluated changes that a lot, though. In the scenario I've proposed, we are in effect saying "write the code and if it gets a lot of support we'll talk about including it in a few years". It shifts the responsibility for development from the core dev team to the one proposing the change. I'm confident that that alone would cut out most of the proposals on python-ideas, while ensuring that whatever was up for consideration at the point where the moratorium lifts was high-quality, well thought out code that had probably been looked at pretty closely by other implementors for several years. Personally, I think that's a much better place to be than either where we are now or where we'll be in a few years without some sort of brainstorm bin. Geremy Condra From g.brandl at gmx.net Wed Oct 21 21:59:24 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 21 Oct 2009 21:59:24 +0200 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <4ADF6136.1060904@mrabarnett.plus.com> References: <20091021142649.BRH02340@ms19.lnh.mail.rcn.net> <4ADF6136.1060904@mrabarnett.plus.com> Message-ID: MRAB schrieb: > Guido van Rossum wrote: >> On Wed, Oct 21, 2009 at 11:26 AM, Raymond Hettinger wrote: >>> [Guido van Rossum] >>>> Note, the moratorium would only cover the language itself plus >>>> built-in functions, not the standard library. >>> That makes sense. >>> >>> There may be a few areas that still have some rough edges where you >>> may want to allow changes if needed (tweaks to the nested with-statement >>> syntax, bytes/text interaction, star-args unpacking, or string formatting). >>> These areas probably have not been exercised much and there may still be >>> problems that need to be ironed-out. I don't have anything specific >>> in mind. Am just thinking that those features aren't yet mature. >> >> No, the moratorium would freeze the language at the 3.1 version, at >> least for 3.2 and 2.7 and possibly 3.3 (see my earlier post). Allowing >> for exceptions like these provides too much wiggle room. (E.g is the >> decorator syntax broken?) >> >> Outright bugs in the implementation should be excepted (subject to >> discussion) but the accepted grammar and semantics should be frozen >> unless they are unimplementable. >> > I'd like there to be the possibility of a change if we were to discover > a case (a corner case, perhaps) where everyone agrees that what Python > is actually doing is unPythonic due to some unforeseen combination of > factors (and I'm not talking about mutable default parameters). Who > knows, it could happen! :-) Please relax. We're not going to pass a law, or institute a commit hook that will reject each and every change to the relevant files. Whatever eventual form the moratorium is going to take, it does not stand above reasonable discussion. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From jnoller at gmail.com Wed Oct 21 22:07:55 2009 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 21 Oct 2009 16:07:55 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <1A472770E042064698CB5ADC83A12ACD04B11A73@TK5EX14MBXC116.redmond.corp.microsoft.com> References: <4222a8490910211043h361eedfeka28042106abb0420@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04B114F3@TK5EX14MBXC116.redmond.corp.microsoft.com> <4222a8490910211213y5b0adedcle3f1fe48196466a8@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04B11A73@TK5EX14MBXC116.redmond.corp.microsoft.com> Message-ID: <4222a8490910211307ke04c44ds984831c3c87b8cc0@mail.gmail.com> On Wed, Oct 21, 2009 at 3:57 PM, Dino Viehland wrote: > Jesse wrote: >> I don't think IronPython should support multiprocessing, personally. >> While there are a few things within it that probably make sense >> (managers and the like) does it really make a ton of sense to run >> multiple iron python VMs from a single parent? > > The problem w/ this is that people are going to use multiprocessing > and then someone is going to try running code which uses it on > IronPython/Jython and it isn't going to work. ?So maybe we could have > a version of multiprocessing that transparently works in a completely > different fashion under the hood but it seems like the module should > still exist and be usable. > I'm game, I just have reservations about doing this - something like multiprocessing that isn't multiprocessing means certain things/expectations applications might have could break in a dummy context. Since we're derailing, I'm game talking about this privately so we can pass ideas back and forth. jesse From grosser.meister.morti at gmx.net Wed Oct 21 22:20:44 2009 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Wed, 21 Oct 2009 22:20:44 +0200 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: <4ADF6D1C.7070709@gmx.net> I think if such a moratorium is instated it should be announced way before the feature freeze of the version that will represent the feature set of the moratorium. So this should have been announced before 3.1 was released. -panzi From brett at python.org Wed Oct 21 22:24:52 2009 From: brett at python.org (Brett Cannon) Date: Wed, 21 Oct 2009 13:24:52 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: On Wed, Oct 21, 2009 at 09:42, Guido van Rossum wrote: > I propose a moratorium on language changes. This would be a period of > several years during which no changes to Python's grammar or language > semantics will be accepted. The reason is that frequent changes to the > language cause pain for implementors of alternate implementations > (Jython, IronPython, PyPy, and others probably already in the wings) > at little or no benefit to the average user (who won't see the changes > for years to come and might not be in a position to upgrade to the > latest version for years after). > > +1 from me. In this rather long thread someone proposed allowing changes that make implementation of the language easier, which I am also fine with, but that could almost be classified a bug fix and taken on a case-by-case basis. But completely new exposed syntax or semantics for the language and built-ins should be off limits. And for the "several years", I say make through 3.4, letting in new features for Python 3.5 which should be open for development in mid 2013. > The main goal of the Python development community at this point should > be to get widespread acceptance of Python 3000. There is tons of work > to be done before we can be comfortable about Python 3.x, mostly in > creating solid ports of those 3rd party libraries that must be ported > to Py3k before other libraries and applications can be ported. (Other > work related to Py3k acceptance might be tools to help porting, tools > to help maintaining multiple versions of a codebase, documentation > about porting to Python 3, and so on. Also, work like that going on in > the distutils-sig is very relevant.) > > The moratorium should also give us time and space to focus on bug fixes for the language which is good. > Note, the moratorium would only cover the language itself plus > built-in functions, not the standard library. Development in the > standard library is valuable and much less likely to be a stumbling > block for alternate language implementations. I also want to exclude > details of the CPython implementation, including the C API from being > completely frozen -- for example, if someone came up with (otherwise > acceptable) changes to get rid of the GIL I wouldn't object. > > Sounds reasonable to me. > But the moratorium would clearly apply to proposals for anonymous > blocks, "yield from" (PEP 380), changes to decorator syntax, and the > like. (I'm sure it won't stop *discussion* of those proposals, and > that's not the purpose of the moratorium; but at least it will stop > worries elsewhere that such proposals might actually be *accepted* any > time soon.) Sounds good to me. While I heard some people on Twitter want PEP 380, if we are going to freeze it should be across the board. I can see someone arguing that at least PEP 380 is in a PEP format, but even then that doesn't mean its fully thought out (and I am not explicitly talking about PEP 380 as I have not read it in ages). A side benefit to this is it will give us time to come up with a PEP that clearly defines exactly what is required to propose and get accepted changes to the language (I also want to do this for the standard library, but I view that as a different PEP). Hopefully that will cut down on the wild proposal on python-ideas and give people a much clearer idea of what is required of them. As I view this as a developer doc thing I am willing to write these PEPs (eventually =). One thing I would like to see happen for this moratorium is us following a more rigid release schedule in terms of feature freeze. So I would propose that starting with 3.2 we feature freeze 18 months after 3.2's feature freeze. Now I am not suggesting we lock down the final release date to 18 months as who knows how long it would take to shake out the bugs, but we can easily say that every 18 months we feature freeze for the next minor release (major releases don't apply to this schedule and I am not worrying about micro releases as that is not under discussion here). This would give us and the community a much clearer indication of when the moratorium will be lifted. So we could say that 3.2 is feature frozen on June 15, 2010, Python 3.3 on January 15, 2012, and Python 3.4 on June 15, 2013. We can obviously choose some other set of dates (went with this to avoid releasing on New Years, although it might be fun to shift to Oct 5 as that's the date Flying Circus was first broadcast), but it would be nice to be able to say that "after Python 3.4, which is feature frozen on XXX, we will start looking at new features again for Python 3.5". -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Wed Oct 21 22:28:13 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 21 Oct 2009 20:28:13 +0000 (UTC) Subject: [Python-ideas] Proposal: Moratorium on Python language changes References: Message-ID: Guido van Rossum writes: > > Note, the moratorium would only cover the language itself plus > built-in functions, not the standard library. Out of curiousity, would it preclude adding e.g. a method to a built-in type? (I'm not saying this with anything specific in mind) Regards Antoine. From brett at python.org Wed Oct 21 22:32:47 2009 From: brett at python.org (Brett Cannon) Date: Wed, 21 Oct 2009 13:32:47 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <4ADF6D1C.7070709@gmx.net> References: <4ADF6D1C.7070709@gmx.net> Message-ID: On Wed, Oct 21, 2009 at 13:20, Mathias Panzenb?ck < grosser.meister.morti at gmx.net> wrote: > I think if such a moratorium is instated it should be announced way before > the > feature freeze of the version that will represent the feature set of the > moratorium. So this should have been announced before 3.1 was released. > That's not necessary. If you look at the PEPs that are even remotely going to fall under this moratorium are PEPs 3142, 382, 380, 369, 335, and 286. The newest one, PEP 382, was checked in back in April which is five months ago. This is not about to cut someone off from getting their suggestion into Python that was on the verge of acceptance. If anything this is a good time to do this as the hazy "does this go into 2.7 or not" question has held back new additions to the language. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From sturla at molden.no Wed Oct 21 22:37:38 2009 From: sturla at molden.no (Sturla Molden) Date: Wed, 21 Oct 2009 22:37:38 +0200 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: <4ADF7112.30905@molden.no> Guido van Rossum skrev: > I propose a moratorium on language changes. This would be a period of > several years during which no changes to Python's grammar or language > semantics will be accepted. The reason is that frequent changes to the > language cause pain for implementors of alternate implementations > (Jython, IronPython, PyPy, and others probably already in the wings) > at little or no benefit to the average user Isn't this the same as saying it is time to produce an industry standard (as in ISO, ECMA, ANSI, IEEE) for the Python language? ISO standardization of C++ has not prevented GNU, Borland, Microsoft, and others to implement proprietary language extensions in their C++ compilers. But if we stick to writing standard C++, it will compile on standard compliant C++ compilers from any vendor. As I see it, the problem is not "language changes", but one implementation (CPython) being the "de facto" Python language standard. Maybe there should be a real Python standard? A syntax change in CPython would be a CPython specific language extension (like GNU extensions to C), not a language change. Today, a syntax change in CPython is a redefinition of the Python language. That will be avoided with a standard. Cython (and Pyrex before) adds many extensions to the Python language, while aiming at being compliant with "pure Python". But what is "pure Python" anyway? It would be much easier for everyone if there was a piece of paper defining the Python language -- and possibly certain critical parts of the standard library. Also, an industry standard for the Python language would help adoption of Python in many organizations. Except for Java, only standardized languages tend to be considered for large projects. Many has strict policies of only using standardized languages to ensure availability of compilers/interpreters in the future. Personally I think Python 3000 is mature enough for a standard. Sturla Molden From solipsis at pitrou.net Wed Oct 21 23:17:42 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 21 Oct 2009 21:17:42 +0000 (UTC) Subject: [Python-ideas] Proposal: Moratorium on Python language changes References: <4ADF7112.30905@molden.no> Message-ID: Sturla Molden writes: > > Isn't this the same as saying it is time to produce an industry standard > (as in ISO, ECMA, ANSI, IEEE) for the Python language? If we had plenty of workforce available it might be an idea. But we have not, and fixing bugs and making improvements is a much better use of volunteer time than language lawyering, IMO. > Except for Java, only standardized > languages tend to be considered for large projects. The "except for Java" statement makes me think that standardization isn't a real criterion here. Stability may be, however. Antoine. From guido at python.org Wed Oct 21 23:40:45 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Oct 2009 14:40:45 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: On Wed, Oct 21, 2009 at 1:28 PM, Antoine Pitrou wrote: > Guido van Rossum writes: >> >> Note, the moratorium would only cover the language itself plus >> built-in functions, not the standard library. > > Out of curiousity, would it preclude adding e.g. a method to a built-in type? > (I'm not saying this with anything specific in mind) And I can't answer without anything specific in front of me. I think it's time for someone to draft a PEP for me summarizing some of the points of this discussion. Some specific quick responses before I run off to do real work: - no, I don't think it should have been announced before 3.1 was frozen - no, I don't think we need an ISO standard - no, it's not decided yet - no, it's not an absolute law (actually typical laws have lots of gray areas too) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Wed Oct 21 23:38:08 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 21 Oct 2009 23:38:08 +0200 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: Brett Cannon schrieb: > I propose a moratorium on language changes. This would be a period of > several years during which no changes to Python's grammar or language > semantics will be accepted. The reason is that frequent changes to the > language cause pain for implementors of alternate implementations > (Jython, IronPython, PyPy, and others probably already in the wings) > at little or no benefit to the average user (who won't see the changes > for years to come and might not be in a position to upgrade to the > latest version for years after). > > > +1 from me. In this rather long thread someone proposed allowing changes > that make implementation of the language easier, which I am also fine > with, but that could almost be classified a bug fix and taken on a > case-by-case basis. But completely new exposed syntax or semantics for > the language and built-ins should be off limits. > > And for the "several years", I say make through 3.4, letting in new > features for Python 3.5 which should be open for development in mid 2013. That's a looong time (for me at least), but still an acceptable timeframe. > The main goal of the Python development community at this point should > be to get widespread acceptance of Python 3000. There is tons of work > to be done before we can be comfortable about Python 3.x, mostly in > creating solid ports of those 3rd party libraries that must be ported > to Py3k before other libraries and applications can be ported. (Other > work related to Py3k acceptance might be tools to help porting, tools > to help maintaining multiple versions of a codebase, documentation > about porting to Python 3, and so on. Also, work like that going on in > the distutils-sig is very relevant.) > > > The moratorium should also give us time and space to focus on bug fixes > for the language which is good. Do you really think so, or do you say it because you wish it was true? The occasional syntax change doesn't block many developer resources. IMO it's the tedious interminable discussions on library and infrastructure issues, e.g. distribution/packaging or the future of WSGI, and none of these is going to be cut off or settled. > But the moratorium would clearly apply to proposals for anonymous > blocks, "yield from" (PEP 380), changes to decorator syntax, and the > like. (I'm sure it won't stop *discussion* of those proposals, and > that's not the purpose of the moratorium; but at least it will stop > worries elsewhere that such proposals might actually be *accepted* any > time soon.) > > > Sounds good to me. While I heard some people on Twitter want PEP 380, if > we are going to freeze it should be across the board. I can see someone > arguing that at least PEP 380 is in a PEP format, but even then that > doesn't mean its fully thought out (and I am not explicitly talking > about PEP 380 as I have not read it in ages). > > A side benefit to this is it will give us time to come up with a PEP > that clearly defines exactly what is required to propose and get > accepted changes to the language (I also want to do this for the > standard library, but I view that as a different PEP). Hopefully that > will cut down on the wild proposal on python-ideas and give people a > much clearer idea of what is required of them. As I view this as a > developer doc thing I am willing to write these PEPs (eventually =). > > One thing I would like to see happen for this moratorium is us following > a more rigid release schedule in terms of feature freeze. So I would > propose that starting with 3.2 we feature freeze 18 months after 3.2's > feature freeze. Now I am not suggesting we lock down the final release > date to 18 months as who knows how long it would take to shake out the > bugs, but we can easily say that every 18 months we feature freeze for > the next minor release (major releases don't apply to this schedule and > I am not worrying about micro releases as that is not under discussion > here). +1 for predictable release schedules. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From lists at cheimes.de Wed Oct 21 23:56:47 2009 From: lists at cheimes.de (Christian Heimes) Date: Wed, 21 Oct 2009 23:56:47 +0200 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: <4ADF839F.6050601@cheimes.de> Stefan Behnel wrote: > As a Python language user, I must say that I cannot remember any recent(?) > language syntax proposal on python-ideas that would have made the language > substantially better. Plus, a lot of the code that gets written even today > is still required to stay Py2.x compatible, sometimes as old as 2.3. > Especially library or tooling code will often not even use Python 2.6 > features for a while, let alone Python 3.x features. That doesn't mean you > can't 2to3 it, it just means that it won't use the new syntax features. Dito! The last useful (and today widely used) syntax change in the 2.x series was the 'with' statement. I don't see urgent need for any syntax or language chances in the next couple of year. The only new feature I can think of, that might require new syntax, is parallel processing and executing multithreaded, GIL-less code. Such a feature is going to take a long time before it's implemented. I'm +1 on your proposal, Guido. Do you propose a fixed time span for the moratorium or a status quo? Can we continue changing the language once Jython and IronPython have fully implemented Python 2.7 / 3.2? Christian From eric at trueblade.com Thu Oct 22 00:18:55 2009 From: eric at trueblade.com (Eric Smith) Date: Wed, 21 Oct 2009 18:18:55 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <4222a8490910211249r266c1aa2x5cd20498413ce39f@mail.gmail.com> References: <4222a8490910211249r266c1aa2x5cd20498413ce39f@mail.gmail.com> Message-ID: <4ADF88CF.3040507@trueblade.com> Jesse Noller wrote: > Distutils != core language syntax. This, I know :) > Distutils was already mentioned in > guido's original email, and given it's a standard library module, it > can continue to evolve. I was specifically referring to PEP 382, which I'm hoping can be used to solve some of the heartburn that setuptools causes some people. I believe we'll need to modify the import machinery for it, since the PEP itself says "The import statement is extended so that it directly considers *.pth files during import". But I'll admit that I've not studied the PEP enough to know to implement it or how much it helps the situation. Eric. From fdrake at acm.org Thu Oct 22 00:33:25 2009 From: fdrake at acm.org (Fred Drake) Date: Wed, 21 Oct 2009 18:33:25 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <4ADF88CF.3040507@trueblade.com> References: <4222a8490910211249r266c1aa2x5cd20498413ce39f@mail.gmail.com> <4ADF88CF.3040507@trueblade.com> Message-ID: <9cee7ab80910211533w52691686k177a4eaf8a51af8e@mail.gmail.com> On Wed, Oct 21, 2009 at 6:18 PM, Eric Smith wrote: > "The import statement is extended so that it directly considers *.pth files > during import". This sounds like the implementation of the import statement, not the language definition. -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller From brett at python.org Thu Oct 22 00:49:13 2009 From: brett at python.org (Brett Cannon) Date: Wed, 21 Oct 2009 15:49:13 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <9cee7ab80910211533w52691686k177a4eaf8a51af8e@mail.gmail.com> References: <4222a8490910211249r266c1aa2x5cd20498413ce39f@mail.gmail.com> <4ADF88CF.3040507@trueblade.com> <9cee7ab80910211533w52691686k177a4eaf8a51af8e@mail.gmail.com> Message-ID: On Wed, Oct 21, 2009 at 15:33, Fred Drake wrote: > On Wed, Oct 21, 2009 at 6:18 PM, Eric Smith wrote: > > "The import statement is extended so that it directly considers *.pth > files > > during import". > > This sounds like the implementation of the import statement, not the > language definition. But that is a change in semantics for __import__ which is a built-in. It's gray area. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Thu Oct 22 01:18:41 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 21 Oct 2009 23:18:41 +0000 (UTC) Subject: [Python-ideas] Proposal: Moratorium on Python language changes References: Message-ID: Brett Cannon writes: > > One thing I would like to see happen for this moratorium is us following > a more rigid release schedule in terms of feature freeze. So I would propose > that starting with 3.2 we feature freeze 18 months after 3.2's feature > freeze. Time-based releases can be good, but 18 months until the feature freeze (which, realistically, gives us a new release every 2 years) sounds much too long to me. Regards Antoine. From jnoller at gmail.com Thu Oct 22 01:45:03 2009 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 21 Oct 2009 19:45:03 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: <4222a8490910211645n3cb4a61al8167d6acda9a315c@mail.gmail.com> On Wed, Oct 21, 2009 at 7:18 PM, Antoine Pitrou wrote: > Brett Cannon writes: >> >> One thing I would like to see happen for this moratorium is us following >> a more rigid release schedule in terms of feature freeze. So I would propose >> that starting with 3.2 we feature freeze 18 months after 3.2's feature >> freeze. > > Time-based releases can be good, but 18 months until the feature freeze (which, > realistically, gives us a new release every 2 years) sounds much too long to me. > > Regards > > Antoine. +1 18 months seems too long. From contact at xavierho.com Thu Oct 22 01:47:09 2009 From: contact at xavierho.com (Xavier Ho) Date: Thu, 22 Oct 2009 09:47:09 +1000 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: <2d56febf0910211647g458af068l2caacb0a35a375b7@mail.gmail.com> On Thu, Oct 22, 2009 at 9:18 AM, Antoine Pitrou wrote: > Time-based releases can be good, but 18 months until the feature freeze > (which, > realistically, gives us a new release every 2 years) sounds much too long > to me. > > Agreed. We'll need to first list out the goals of what we can accomplish within the frozen syntax timeframe, and then look at how long it will take to reach those goals. Only then can we decide how long it would be ideal. Also, even if when this is in effect (+1 fro me by the way), we also need to agree on some kind of exceptions so we're not shooting down at our own feet. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Thu Oct 22 01:47:16 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 22 Oct 2009 10:47:16 +1100 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: <200910221047.17051.steve@pearwood.info> On Thu, 22 Oct 2009 06:02:29 am Guido van Rossum wrote: > On Wed, Oct 21, 2009 at 11:52 AM, geremy condra wrote: > > Towards that end, I'd also like to propose a very public, very > > accessible 'sandbox' specifically for the development and testing > > of new language features while the moratorium is in effect. ... > > I can't stop people from forking the language to do experiments, but > one of the goals I have for the moratorium is actually to *reduce* > the interest in core language changes, and to *raise* the barrier to > entry. Most language change proposals are just fluff, and they will > be just as unneeded three years from now as they are today. Once the > moratorium is lifted, users should be able expect the normal, slow, > conservative evolution of the language to continue -- not to see the > floodgates lifted for a barrage of new features. I was going to make a similar suggestion to Geremy, an official experimental branch in perpetual alpha, specifically to allow people to experiment with language changes. This doesn't mean that when the moratorium is lifted there will be a sudden flood of new features, radically changing the language: features will still need to go through precisely the same procedure to be accepted into the language as they do now. But it will be a place to point people and tell them "don't just tell us how you imagine this feature will work, show us". (For the record, I'm not suggesting that this experimental branch should be open to commits from anyone. You still have to convince those with privileges that your patch is worth looking at.) As far as the moratorium goes, I'm going to be the negative-nellie who says that I don't think this will actually make much practical difference. Python is, and always has been, very conservative about adding new language features. And so far, we've only heard from one developer of another Python implementation, Dino Viehland of IronPython, who said that language features are generally not a problem for him. As far as I can see, the only practical difference it will make is that it will cut short some (but not all) threads on python-ideas and comp.lang.python: anyone who suggests a language change will be told "Moratorium. Come back in 2013." This risks giving newbies the impression that Python is moribund and closed to new ideas -- not just the amateur language designers who have a wild and wacky ideas, but also those with clue who have good, thought-out ideas, as well as the quiet ones who don't have ideas but like those who do. Programming language popularity is not quite a zero-sum game, but languages do live or die by the ability to attract and keep new users. Particularly in Open Source languages, those users tend to be attracted more by "fresh and lively" than "stable and boring". So this is a cost that needs to be considered against the dubious (to me at least) proposition that a feature-freeze in the language will speed up uptake of 3.1. If library authors are going to support 3.x, they need to pick a minimum version and write for that. A new-feature freeze for 3.2 and 3.3 isn't going to make their life easier if they're writing for 3.1, for the obvious reason that they need to use only features available in 3.1. The only language changes that hurt them are non-backward compatible changes like the removal of features, but even there the benefit is minimal because Python already is very cautious about backwards compatibility. Features aren't removed or functionality changed without a long deprecation period, possibly even longer than the proposed moratorium. So I'm not convinced that this moratorium will actually give any advantage. -- Steven D'Aprano From jnoller at gmail.com Thu Oct 22 02:23:52 2009 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 21 Oct 2009 20:23:52 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: <4222a8490910211723p46d94edbge6a309440074c41f@mail.gmail.com> On Wed, Oct 21, 2009 at 5:40 PM, Guido van Rossum wrote: > On Wed, Oct 21, 2009 at 1:28 PM, Antoine Pitrou wrote: >> Guido van Rossum writes: >>> >>> Note, the moratorium would only cover the language itself plus >>> built-in functions, not the standard library. >> >> Out of curiousity, would it preclude adding e.g. a method to a built-in type? >> (I'm not saying this with anything specific in mind) > > And I can't answer without anything specific in front of me. > > I think it's time for someone to draft a PEP for me summarizing some > of the points of this discussion. > > Some specific quick responses before I run off to do real work: > > - no, I don't think it should have been announced before 3.1 was frozen > > - no, I don't think we need an ISO standard > > - no, it's not decided yet > > - no, it's not an absolute law (actually typical laws have lots of > gray areas too) > I'll volunteer to write it up; I like writing. I'll send it to python-dev when it's ready. jesse From jimjjewett at gmail.com Thu Oct 22 02:28:13 2009 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 21 Oct 2009 20:28:13 -0400 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <200910201030.27448.steve@pearwood.info> References: <4ADC6A92.3060206@qq.com> <20091019143942.GB8510@phd.pp.ru> <6f4025010910191533gf7d47b1gf0e7465bf96e0c84@mail.gmail.com> <200910201030.27448.steve@pearwood.info> Message-ID: On Mon, Oct 19, 2009 at 7:30 PM, Steven D'Aprano wrote: > On Tue, 20 Oct 2009 09:33:48 am Michael Foord wrote: >> Having used languages that allow multiline anonymous functions (C#, >> Javascript) I have to say that they *can* improve the readability of >> code. Could you provide some examples? I've seen plenty of usage in Javascript, but none that I would personally judge as improving readability -- and many that hurt badly. Even if this turns out to be a personal preference thing (some people think it is better, others just don't happen to agree), the advantage is likely small enough that it is outweighed by the advantages of standardization. (I've seen many people claim that the advantages are huge, but the people making such claims often turn out not to be familiar with nested functions.) > Given the difficulty in testing such anonymous functions, what do they > do to the correctness of programs? Anonymity of functions is generally not the biggest source of problems in Javascript. With C compilers, there is pressure to conform to the standard; with Javascript interpreters there is pressure to support existing broken content without any manual intervention (so no compiler switches). The major goal of HTML5 is to at least document and standardize the idiotic workarounds that are required to do that when parsing a web page labeled as HTML; there are similar, less advanced, plans for Javascript and the Browser Object Model, but ... I suspect "untested environment" will continue to be a much bigger issue for at least half a decade. >> Being forced to define functions ahead of where they belong in >> the logical flow of the program is not a feature of Python but a >> restriction. Agreed. But I have never felt the urge to read (as opposed to write) a full definition in the middle of call. (And if you aren't in the middle of a call, then the previous line isn't really that far ahead.) I have wished to just pass a name, and define the function later; the fact that I can't always do that is a quirk of python's execution model, but adding anonymous functions wouldn't help. And any syntax heavier than a single keyword would probably add so much noise that it would be worse than the out-of-order effect. In other words, I could understand the value of f(a, 5, delay g, "asdf") def g(x): return x But the cost is already high enough to make me unsure that it would be a worthwhile feature. > My point is that when it comes to data, we have anonymous data that is > easy and straightforward to define, and yet we *still* prefer to give > data names and define it ahead of where we use it. So why is code > different? Why does defining code ahead of where you use it such a bad > thing that the lambda restriction generates so much attention? Data is static -- it just sits there. A name makes it shrink (in mental cost) but has no other effect -- it is like putting physical objects in a suitcase and zipping the thing shut. A function is part of the instructions; defining it somewhere else is like setting a bookmark to follow a footnote -- and then having to come back. That said, I suspect that much of the actual attention is from misunderstandings. -jJ From jimjjewett at gmail.com Thu Oct 22 02:41:04 2009 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 21 Oct 2009 20:41:04 -0400 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <2DEB8009-3607-49FC-9C3B-ED08C9B3A245@masklinn.net> References: <4ADC6A92.3060206@qq.com> <200910201006.30622.steve@pearwood.info> <200910202221.03233.steve@pearwood.info> <2DEB8009-3607-49FC-9C3B-ED08C9B3A245@masklinn.net> Message-ID: On Tue, Oct 20, 2009 at 7:34 AM, Masklinn wrote: > But unless the language is dynamically scoped (which is getting pretty rare > these days, and which Python definitely isn't) err ... current python. I believe that changed around python 2.2. I've wanted a way to reverse that more often than I've wanted a way to avoid naming a function. (I originally wrote "far more", but ... the urge doesn't actually come up most years, so maybe it was a solution finding a problem.) > The biggest (by far) advantages I see to good anonymous functions (note: > Ruby's aren't, as far as I'm concerned, because due to their nature they > don't easily scale from 1/call to 2+/call) are in flexibility, freedom of > experimentation and possibility to keep the core language itself small: had > Python had "full-blown" anonymous functions, it wouldn't have been necessary > to add the `with` statement to the language. I'm not sure how the lack of a name buys you any of that. Callables as first-class objects that can be passed around, yes -- but naming them is fine. And to be honest, that flexibility and freedom seem like they would be amplified by dynamic scope, so that you could "tune" library functions by passing in arbitrary callables that could affect the "local" variables. The advantages of lexical scoping seem to be mostly about closures, rather than about extending an existing external function. Could you explain how (even named) multiline functions (as opposed to dynamic scope) would have served in place of "with:"? -jJ From sturla at molden.no Thu Oct 22 04:56:40 2009 From: sturla at molden.no (Sturla Molden) Date: Thu, 22 Oct 2009 04:56:40 +0200 Subject: [Python-ideas] [Python-Dev] GIL behaviour under Windows In-Reply-To: References: <4ADE2AA9.4030604@molden.no> <4ADE35F0.10803@molden.no> <930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local> <930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local> <1256132087.5069.39.camel@localhost> <4ADF247E.8040401@molden.no> Message-ID: <4ADFC9E8.1010400@molden.no> Antoine Pitrou skrev: > Kristj?n sent me a patch which I applied and is supposed to fix this. > Anyway, thanks for the numbers. The GIL does seem to fare a bit better (zero > latency with the Pi calculation in the background) than under Linux, although it > may be caused by the limited resolution of time.time() under Windows. > > My critisism of the GIL on python-ideas was partly motivated by this: http://blip.tv/file/2232410 However, David Beazley is not talking about Windows. Since the GIL is apparently not a mutex on Windows, it could behave differently. So I wrote a small script that contructs a GIL battle, and record how often a check-interval results in a thread-switch or not. For monitoring check intervals, I used a small C extension to read _Py_Ticker from ceval.c. It is not declared static so I could easily hack into it. With two threads and a check interval og 100, only 61 of 100000 check intervals failed to produce a thread-switch in the interpreter. I'd call that rather fair. :-) And in case someone asks, the nthreads=1 case is just for verification. S.M. D:\>test.py check interval = 1 nthreads=1, swiched=0, missed=100000 nthreads=2, swiched=57809, missed=42191 nthreads=3, swiched=91535, missed=8465 nthreads=4, swiched=99751, missed=249 nthreads=5, swiched=95839, missed=4161 nthreads=6, swiched=100000, missed=0 D:\>test.py check interval = 10 nthreads=1, swiched=0, missed=100000 nthreads=2, swiched=99858, missed=142 nthreads=3, swiched=99992, missed=8 nthreads=4, swiched=100000, missed=0 nthreads=5, swiched=100000, missed=0 nthreads=6, swiched=100000, missed=0 D:\>test.py check interval = 100 nthreads=1, swiched=0, missed=100000 nthreads=2, swiched=99939, missed=61 nthreads=3, swiched=100000, missed=0 nthreads=4, swiched=100000, missed=0 nthreads=5, swiched=100000, missed=0 nthreads=6, swiched=100000, missed=0 D:\>test.py check interval = 1000 nthreads=1, swiched=0, missed=100000 nthreads=2, swiched=99999, missed=1 nthreads=3, swiched=100000, missed=0 nthreads=4, swiched=100000, missed=0 nthreads=5, swiched=100000, missed=0 nthreads=6, swiched=100000, missed=0 From sturla at molden.no Thu Oct 22 05:09:44 2009 From: sturla at molden.no (Sturla Molden) Date: Thu, 22 Oct 2009 05:09:44 +0200 Subject: [Python-ideas] [Python-Dev] GIL behaviour under Windows In-Reply-To: <4ADFC9E8.1010400@molden.no> References: <4ADE2AA9.4030604@molden.no> <4ADE35F0.10803@molden.no> <930F189C8A437347B80DF2C156F7EC7F098FF41F74@exchis.ccp.ad.local> <930F189C8A437347B80DF2C156F7EC7F098FF42001@exchis.ccp.ad.local> <1256132087.5069.39.camel@localhost> <4ADF247E.8040401@molden.no> <4ADFC9E8.1010400@molden.no> Message-ID: <4ADFCCF8.5010807@molden.no> Sturla Molden skrev: > > However, David Beazley is not talking about Windows. Since the GIL is > apparently not a mutex on Windows, it could behave differently. So I > wrote a small script that contructs a GIL battle, and record how often > a check-interval results in a thread-switch or not. For monitoring > check intervals, I used a small C extension to read _Py_Ticker from > ceval.c. It is not declared static so I could easily hack into it. Anyway, if anyone wants to run a GIL battle, here is the code I used. If it turns out the GIL is far worse with pthreads, as it is implemented with a mutex, it might be a good idea to reimplement it with an event object as it is on Windows. Sturla Molden ---------------- In python: from giltest import * from time import clock import threading import sys def thread(rank, battle, start): while not start.isSet(): if rank == 0: start.set() try: while 1: battle.record(rank) except: pass if __name__ == '__main__': sys.setcheckinterval(1000) print "check interval = %d" % sys.getcheckinterval() for nthreads in range(1,7): start = threading.Event() battle = GIL_Battle(100000) threads = [threading.Thread(target=thread, args=(i,battle,start)) for i in range(1,nthreads)] for t in threads: t.setDaemon(True) t.start() thread(0, battle, start) for t in threads: t.join() s,m = battle.report() print "nthreads=%d, swiched=%d, missed=%d" % (nthreads, s, m) In Cython or Pyrex: from exceptions import Exception cdef extern from *: ctypedef int vint "volatile int" vint _Py_Ticker class StopBattle(Exception): pass cdef class GIL_Battle: """ tests the fairness of the GIL """ cdef vint prev_tick, prev_rank, switched, missed cdef int trials def __cinit__(GIL_Battle self, int trials=100000): self.prev_tick = _Py_Ticker self.prev_rank = -1 self.missed = 0 self.switched = 0 self.trials = trials def record(GIL_Battle self, int rank): if self.trials == self.switched + self.missed: raise StopBattle if self.prev_rank == -1: self.prev_tick = _Py_Ticker self.prev_rank = rank else: if _Py_Ticker > self.prev_tick: if self.prev_rank == rank: self.missed += 1 else: self.switched += 1 self.prev_tick = _Py_Ticker self.prev_rank = rank else: self.prev_tick = _Py_Ticker def report(GIL_Battle self): return int(self.switched), int(self.missed) From rrr at ronadam.com Thu Oct 22 07:06:35 2009 From: rrr at ronadam.com (Ron Adam) Date: Thu, 22 Oct 2009 00:06:35 -0500 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: <4ADFE85B.3030504@ronadam.com> Guido van Rossum wrote: > I propose a moratorium on language changes. This would be a period of > several years during which no changes to Python's grammar or language > semantics will be accepted. The reason is that frequent changes to the > language cause pain for implementors of alternate implementations > (Jython, IronPython, PyPy, and others probably already in the wings) > at little or no benefit to the average user (who won't see the changes > for years to come and might not be in a position to upgrade to the > latest version for years after). > > The main goal of the Python development community at this point should > be to get widespread acceptance of Python 3000. There is tons of work > to be done before we can be comfortable about Python 3.x, mostly in > creating solid ports of those 3rd party libraries that must be ported > to Py3k before other libraries and applications can be ported. (Other > work related to Py3k acceptance might be tools to help porting, tools > to help maintaining multiple versions of a codebase, documentation > about porting to Python 3, and so on. Also, work like that going on in > the distutils-sig is very relevant.) > > Note, the moratorium would only cover the language itself plus > built-in functions, not the standard library. Development in the > standard library is valuable and much less likely to be a stumbling > block for alternate language implementations. I also want to exclude > details of the CPython implementation, including the C API from being > completely frozen -- for example, if someone came up with (otherwise > acceptable) changes to get rid of the GIL I wouldn't object. > > But the moratorium would clearly apply to proposals for anonymous > blocks, "yield from" (PEP 380), changes to decorator syntax, and the > like. (I'm sure it won't stop *discussion* of those proposals, and > that's not the purpose of the moratorium; but at least it will stop > worries elsewhere that such proposals might actually be *accepted* any > time soon.) ++1 Yay! I really like the idea of keeping the core small, simple, and efficient, and having most of the real interfacing and data processing *magic* done through libraries. So yes, please refocus development efforts to improving the library. I think if a library could benifit substantually from some core improvement, then that can be discussed at that time. Such improvement to the core will be for a real concrete need to enable functionality in an extension or library, rather than improving the core with expectations of having some unknown abstract or perceived benefit later. Ron Adam From larry.bugbee at gmail.com Thu Oct 22 08:24:14 2009 From: larry.bugbee at gmail.com (Larry Bugbee) Date: Wed, 21 Oct 2009 23:24:14 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes Message-ID: On Wed, Oct 21, 2009 at 12:42 PM, Guido van Rossum wrote: > I propose a moratorium on language changes. This would be a > period of several years during which no changes to Python's > grammar or language semantics will be accepted. I agree with a moratorium. I cannot imagine it doing any harm. I'd even go so far as to say new language syntax and semantics should only come in only at top level version numbers like 2.0, 3.0, 4.0, etc. ...not at 3.2, 3.3, 3.4, etc. And at this later date, I submit language changes should not come in unless there is a compelling reason. That said, two areas where language support would be welcome: - the optional declaration of data types - parallel processing FWIW, I do a lot of crypto and for now have chosen to remain at 2.5.x. For me, byte strings are all important, and a few quick forays into Python 3 and its default Unicode suggested I should proceed very carefully. My experiments with Python 2.6 left me uncomfortable as well. There I ran into a couple of standard libraries that had the same API but their implementation/behavior was different. (No, that was many months ago and I do not remember which. Sorry.) In time I will move to Py3. From larry.bugbee at gmail.com Thu Oct 22 06:20:20 2009 From: larry.bugbee at gmail.com (Larry Bugbee) Date: Wed, 21 Oct 2009 21:20:20 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes Message-ID: <84296706-04D5-4F63-99D3-6966766125A0@gmail.com> On Wed, Oct 21, 2009 at 12:42 PM, Guido van Rossum wrote: > I propose a moratorium on language changes. This would be a > period of several years during which no changes to Python's > grammar or language semantics will be accepted. I think a moratorium would be Goodness. I cannot imagine it doing any harm. I'd even go so far as to say new language syntax and semantics should only come in only at top level version numbers like 2.0, 3.0, 4.0, etc. ...not at 3.2, 3.3, 3.4, etc. And at this later date, I submit language changes should not come in unless there is a compelling reason. Two areas where language support would be welcome: - the optional declaration of data types - parallel processing FWIW, I do a lot of crypto and for now have chosen to remain at 2.5.x. For me, byte strings are all important, and a few quick forays into Python 3 and its default Unicode suggested I should proceed very carefully. My experiments with Python 2.6 left me uncomfortable as well. There I ran into a couple of standard libraries that had the same API but their implementation/behavior was different. (No, that was many months ago and I do not remember which. Sorry.) In time I will move to Py3. From stephen at xemacs.org Thu Oct 22 09:00:22 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 22 Oct 2009 16:00:22 +0900 Subject: [Python-ideas] a new lambda syntax In-Reply-To: References: <4ADC6A92.3060206@qq.com> <200910201006.30622.steve@pearwood.info> <200910202221.03233.steve@pearwood.info> <2DEB8009-3607-49FC-9C3B-ED08C9B3A245@masklinn.net> Message-ID: <87k4yndibd.fsf@uwakimon.sk.tsukuba.ac.jp> Jim Jewett writes: > On Tue, Oct 20, 2009 at 7:34 AM, Masklinn wrote: > > But unless the language is dynamically scoped (which is getting pretty rare > > these days, and which Python definitely isn't) > > err ... current python. I believe that changed around python 2.2. > > I've wanted a way to reverse that more often than I've wanted a way to > avoid naming a function. It's funny ... the language I use most is Emacs Lisp and that has taught me a severe aversion to dynamic scoping by default. Sometimes it's useful, but when I am using say Common Lisp an occasional defvar just isn't that painful. I don't think I've ever missed dynamic scope in Python since my first few days with it. From stephen at xemacs.org Thu Oct 22 09:14:58 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 22 Oct 2009 16:14:58 +0900 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <20091021142649.BRH02340@ms19.lnh.mail.rcn.net> Message-ID: <87iqe7dhn1.fsf@uwakimon.sk.tsukuba.ac.jp> Guido van Rossum writes: > On Wed, Oct 21, 2009 at 11:26 AM, Raymond Hettinger wrote: > > Are you rejecting PEP 380? > > No, just deferring it. It didn't make Python 3.1 so I think we'll have > to live without it for a long time no matter what. It could be > revisited once the moratorium is lifted (for 3.3 or 3.4). Would it be useful to allow work on it, maybe including a deferred approval (ie, PEP is approved for implementation in version 3.3 [or "when the moratorium is lifted"]), to proceed on a branch? Branches will be very cheap in hg. I realize this may "bring work forward" for you and other interested parties, on the other hand I think there is a "strike while the iron's hot" aspect for implementators of the PEP. In particular, having a working and approved feature on a branch would allow those interested in the deferred PEP to merge into a (private) fresh branch for compatibility testing of the work being done on the mainline. From stephen at xemacs.org Thu Oct 22 09:30:50 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 22 Oct 2009 16:30:50 +0900 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <4ADF7112.30905@molden.no> References: <4ADF7112.30905@molden.no> Message-ID: <87hbtrdgwl.fsf@uwakimon.sk.tsukuba.ac.jp> Sturla Molden writes: > Guido van Rossum skrev: > > I propose a moratorium on language changes. > Isn't this the same as saying it is time to produce an industry standard > (as in ISO, ECMA, ANSI, IEEE) for the Python language? No. The language reference is quite precise and accurate AFAICS. There may be an argument for an industry standard, but this isn't it. > As I see it, the problem is not "language changes", but one > implementation (CPython) being the "de facto" Python language > standard. Maybe there should be a real Python standard? Would it make a difference? Surely the CPython implementation would still be the de facto "reference interpretation" for resolving ambiguities. Note that many argue that the success of the Internet is due to deliberate adherence to a PEP-like process that involves actually implementing and demonstrating usefulness of new features. OTOH, the ISO and other standards bodies are famous for standards produced by language lawyers. These standards are sometimes unusable, and too often ignored in favor of well-known best practices. In the end, I think it would be hard to convince the developer community to abandon the PEP process. So any ISO effort would be layered on top of it, and CPython would still have distinguished status. From ubershmekel at gmail.com Thu Oct 22 09:38:42 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Thu, 22 Oct 2009 09:38:42 +0200 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <200910221047.17051.steve@pearwood.info> References: <200910221047.17051.steve@pearwood.info> Message-ID: <9d153b7c0910220038h641375ccl55ec3aad0885c8e5@mail.gmail.com> On Thu, Oct 22, 2009 at 1:47 AM, Steven D'Aprano wrote: > On Thu, 22 Oct 2009 06:02:29 am Guido van Rossum wrote: >> On Wed, Oct 21, 2009 at 11:52 AM, geremy condra > wrote: >> > Towards that end, I'd also like to propose a very public, very >> > accessible 'sandbox' specifically for the development and testing >> > of new language features while the moratorium is in effect. > ... > I was going to make a similar suggestion to Geremy, an official > experimental branch in perpetual alpha, > ... +1 for the sandbox idea. If crazy suggestions that come to python-ideas are plainly rejected because of the moratorium then I'm -1 on inventing discussion killers. If the moratorium PEP shows how a new language feature is to be introduced in an official sandbox etc, then I'm +1 for the moratorium. --yuv From sturla at molden.no Thu Oct 22 10:49:14 2009 From: sturla at molden.no (Sturla Molden) Date: Thu, 22 Oct 2009 10:49:14 +0200 Subject: [Python-ideas] a new lambda syntax In-Reply-To: References: <4ADC6A92.3060206@qq.com> <200910201006.30622.steve@pearwood.info> <200910202221.03233.steve@pearwood.info> <2DEB8009-3607-49FC-9C3B-ED08C9B3A245@masklinn.net> <20091020195742.45a43d33@bhuda.mired.org> <4ADEF2C4.9050702@gmail.com> Message-ID: <4AE01C8A.5030404@molden.no> Guido van Rossum skrev: > I actually don't buy that argument. I believe that the main reason > people keep arguing for anonymous blocks (and this will never stop) is > simply that there is a certain segment of the programmer population > that is used to solving a large variety of problems with anonymous > blocks and that they don't want to learn how to solve each of those > cases using the existing tools in Python. > > I think anonymous blocks have proven their usefulness in concurrency programming. Just take a look at OpenMP or Apple's GCD. But I'll make an argument against anonymous blocks anyway: We can create "anonymous" blocks by throw-away names like "def _()", but why clutter the scope with names that are never used? So assume we would allow write "def():" to designate an anonymous block. I think 99% would abuse it like this: name = def(): This could just as well be written: def name(): Thus, certainly "name = def():" should not be allowed. If that is not allowed then, an anynymous block cannot be called or referenced, as there are no name associated with it. So to be useful, it would always have to be associated with a decorator. That would be the only way to get access to it: @decorator def(): Now we can see that the use for this anonymous block "def()" simply goes away, as it too is redundant syntax. Then it all comes down to whether or not allow decorators on code blocks: @decorator Since blocks do not create new scopes, I think the answer should be "no". S.M. From ziade.tarek at gmail.com Thu Oct 22 11:30:36 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 22 Oct 2009 11:30:36 +0200 Subject: [Python-ideas] A standard location for Python configuration files. Message-ID: <94bdd2610910220230p11017058y8f1f8c4db508dc99@mail.gmail.com> Hello, As discussed in issue #7175 and on irc, I would like to propose the addition of a new function called 'getuserconfig' in site.py. This function will return the path to a directory that will contain configuration files for Python. The name I am suggesting for the path of the folder is: ~/.python with an alternative location for win32 : ~/python This win32 alternative is proposed to avoid the fact that it's not possible in some flavors of windows to manually create a folder that starts with a dot when using file graphical browser. That's disturbing for end users. This will be used by Distutils to gather the configuration files it uses right now ([py]distutils.cfg and pypirc) and gently deprecate the old location (~) I am bringing this discussion to python-ideas to make sure those are the best choices, and to propose ~/[.]python as the new standard location for any user configuration file Python currently uses or might use in the future. Regards, Tarek From fetchinson at googlemail.com Thu Oct 22 11:47:15 2009 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Thu, 22 Oct 2009 11:47:15 +0200 Subject: [Python-ideas] A standard location for Python configuration files. In-Reply-To: <94bdd2610910220230p11017058y8f1f8c4db508dc99@mail.gmail.com> References: <94bdd2610910220230p11017058y8f1f8c4db508dc99@mail.gmail.com> Message-ID: > Hello, > > As discussed in issue #7175 and on irc, I would like to propose the > addition of a new function called 'getuserconfig' in site.py. > > This function will return the path to a directory that will contain > configuration files for Python. > > The name I am suggesting for the path of the folder is: ~/.python > > with an alternative location for win32 : ~/python > > This win32 alternative is proposed to avoid the fact that it's not > possible in some flavors of windows to manually create a > folder that starts with a dot when using file graphical browser. > That's disturbing for end users. > > This will be used by Distutils to gather the configuration files it > uses right now ([py]distutils.cfg and pypirc) and gently deprecate the > old location (~) > > I am bringing this discussion to python-ideas to make sure those are > the best choices, and to propose ~/[.]python as the new standard > location for any user configuration file Python currently uses or > might use in the future. What configuration files exactly do you think will be stored in this location in the future (other than the ones for distutils)? In let's say today's python 2.5 or 2.6 what config files would you put there if this capability would be there currently? Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From lists at cheimes.de Thu Oct 22 11:53:46 2009 From: lists at cheimes.de (Christian Heimes) Date: Thu, 22 Oct 2009 11:53:46 +0200 Subject: [Python-ideas] A standard location for Python configuration files. In-Reply-To: <94bdd2610910220230p11017058y8f1f8c4db508dc99@mail.gmail.com> References: <94bdd2610910220230p11017058y8f1f8c4db508dc99@mail.gmail.com> Message-ID: <4AE02BAA.3000309@cheimes.de> Tarek Ziad? wrote: > As discussed in issue #7175 and on irc, I would like to propose the > addition of a new function called 'getuserconfig' in site.py. > > This function will return the path to a directory that will contain > configuration files for Python. > > The name I am suggesting for the path of the folder is: ~/.python > > with an alternative location for win32 : ~/python PEP 370 already addresses this issue of version specific and general config files. Even if you prefer to use a different naming schema than I've suggested in PEP 370 you shouldn't use your suggested paths. For Unix ~/.python might already be used. At least it's used on my box as a script that is loaded through PYTHONSTARTUP to active tab completion in interactive Python shells. The path ~/python on Windows violates Windows naming schema. Application must not use the root directory of a user's home folder to store data. Instead you should use the application data folder. Christian From ziade.tarek at gmail.com Thu Oct 22 11:56:18 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 22 Oct 2009 11:56:18 +0200 Subject: [Python-ideas] A standard location for Python configuration files. In-Reply-To: References: <94bdd2610910220230p11017058y8f1f8c4db508dc99@mail.gmail.com> Message-ID: <94bdd2610910220256y5be116c3od77d3233a272c365@mail.gmail.com> On Thu, Oct 22, 2009 at 11:47 AM, Daniel Fetchinson wrote: >> Hello, >> >> As discussed in issue #7175 and on irc, I would like to propose the >> addition of a new function called 'getuserconfig' in site.py. >> >> This function will return the path to a directory that will contain >> configuration files for Python. >> >> The name I am suggesting for the path of the folder is: ? ~/.python >> >> with an alternative location for win32 : ~/python >> >> This win32 alternative is proposed to avoid the fact that it's not >> possible in some flavors of windows to manually create a >> folder that starts with a dot when using file graphical browser. >> That's disturbing for end users. >> >> This will be used by Distutils to gather the configuration files it >> uses right now ([py]distutils.cfg and pypirc) and gently deprecate the >> old location (~) >> >> I am bringing this discussion to python-ideas to make sure those are >> the best choices, and to propose ~/[.]python as the new standard >> location for any user configuration file Python currently uses or >> might use in the future. > > What configuration files exactly do you think will be stored in this > location in the future (other than the ones for distutils)? In let's > say today's python 2.5 or 2.6 what config files would you put there if > this capability would be there currently? .pythonrc for example From fetchinson at googlemail.com Thu Oct 22 11:56:31 2009 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Thu, 22 Oct 2009 11:56:31 +0200 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: > I propose a moratorium on language changes. This would be a period of > several years during which no changes to Python's grammar or language > semantics will be accepted. +1 I only wish that the leads of many other software projects I use would be equally wise to declare a moratorium once in a while. It's a sign of maturity and sadly many developers don't realize this and keep adding features all the time for the sake of adding features. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From phd at phd.pp.ru Thu Oct 22 11:58:09 2009 From: phd at phd.pp.ru (Oleg Broytman) Date: Thu, 22 Oct 2009 13:58:09 +0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: <20091022095809.GD13617@phd.pp.ru> On Wed, Oct 21, 2009 at 09:42:01AM -0700, Guido van Rossum wrote: > I propose a moratorium on language changes. This would be a period of > several years during which no changes to Python's grammar or language > semantics will be accepted. The reason is that frequent changes to the > language cause pain for implementors of alternate implementations +1 for the moratorium during the life of 3.2. +0.5 for 3.3. -1 for the reason. In my no so humble opinion the moratorium is a benefit for the language, the implementation (CPython), the developers and the users (software developers that use CPython). If alternative implementations see their benefits - well, good for them. Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From ziade.tarek at gmail.com Thu Oct 22 11:59:21 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 22 Oct 2009 11:59:21 +0200 Subject: [Python-ideas] A standard location for Python configuration files. In-Reply-To: <4AE02BAA.3000309@cheimes.de> References: <94bdd2610910220230p11017058y8f1f8c4db508dc99@mail.gmail.com> <4AE02BAA.3000309@cheimes.de> Message-ID: <94bdd2610910220259x2f2421d2geb7c4b7dfe4bd3f7@mail.gmail.com> On Thu, Oct 22, 2009 at 11:53 AM, Christian Heimes wrote: > Tarek Ziad? wrote: >> As discussed in issue #7175 and on irc, I would like to propose the >> addition of a new function called 'getuserconfig' in site.py. >> >> This function will return the path to a directory that will contain >> configuration files for Python. >> >> The name I am suggesting for the path of the folder is: ? ~/.python >> >> with an alternative location for win32 : ~/python > > PEP 370 already addresses this issue of version specific and general > config files. > > Even if you prefer to use a different naming schema than I've suggested > in PEP 370 you shouldn't use your suggested paths. For Unix ~/.python > might already be used. At least it's used on my box as a script that is > loaded through PYTHONSTARTUP to active tab completion in interactive > Python shells. The path ~/python on Windows violates Windows naming > schema. Application must not use the root directory of a user's home > folder to store data. Instead you should use the application data folder. Yes, but it doesn't address the configuration files issue. If I make the same analogy (./local <-> /usr/local) where in .local would I place those files ? Would it be ".local/etc" ? Tarek From solipsis at pitrou.net Thu Oct 22 12:10:16 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 22 Oct 2009 10:10:16 +0000 (UTC) Subject: [Python-ideas] =?utf-8?q?A_standard_location_for_Python_configura?= =?utf-8?q?tion=09files=2E?= References: <94bdd2610910220230p11017058y8f1f8c4db508dc99@mail.gmail.com> <4AE02BAA.3000309@cheimes.de> Message-ID: Christian Heimes writes: > > PEP 370 already addresses this issue of version specific and general > config files. Please, no. As I said in #7175, ~/.local is the user-specific equivalent of /usr or /usr/local. Do you put your configuration files in /usr/local ? Why put them in .local ? An Unix user will look for Python configuration files in his home directory, and then perhaps in a subdirectory named .python or .py. *Not* in .local, which is a place for locally installed stuff, not user-editable config files. > Even if you prefer to use a different naming schema than I've suggested > in PEP 370 you shouldn't use your suggested paths. For Unix ~/.python > might already be used. At least it's used on my box as a script that is > loaded through PYTHONSTARTUP to active tab completion in interactive > Python shells. I don't find this argument very serious. The Python interpreter reclaiming the ~/.python directory only seems natural and reasonable to me. If we follow your argument we can't introduce *any* new location just because someone might already use it. This is quite unacceptable. From lists at cheimes.de Thu Oct 22 13:13:36 2009 From: lists at cheimes.de (Christian Heimes) Date: Thu, 22 Oct 2009 13:13:36 +0200 Subject: [Python-ideas] A standard location for Python configuration files. In-Reply-To: References: <94bdd2610910220230p11017058y8f1f8c4db508dc99@mail.gmail.com> <4AE02BAA.3000309@cheimes.de> Message-ID: <4AE03E60.6030402@cheimes.de> Antoine Pitrou wrote: > Please, no. As I said in #7175, ~/.local is the user-specific equivalent of /usr > or /usr/local. Do you put your configuration files in /usr/local ? Why put them > in .local ? > > An Unix user will look for Python configuration files in his home directory, and > then perhaps in a subdirectory named .python or .py. *Not* in .local, > which is a place for locally installed stuff, not user-editable config files. First of all we should discuss if we are talking about automatically generated config files or human editable config files. If the configuration files are frequently edited by a user than ~/.local or ~/.config aren't the best place. > I don't find this argument very serious. The Python interpreter reclaiming the > ~/.python directory only seems natural and reasonable to me. If we follow your > argument we can't introduce *any* new location just because someone might > already use it. This is quite unacceptable. The path ~/.python is a so obvious choice for a file or directory that contains Python specific stuff that it probably used by lots of people. If we are going to introduce a directory in the user's home directory and not in some subfolder like ~/.local or ~/.config, then I like to stick to another Unix tradition. Configuration directories are usually suffixed with ".d". How about ~/.python.d/? Christian From phd at phd.pp.ru Thu Oct 22 13:18:45 2009 From: phd at phd.pp.ru (Oleg Broytman) Date: Thu, 22 Oct 2009 15:18:45 +0400 Subject: [Python-ideas] A standard location for Python configuration files. In-Reply-To: <4AE03E60.6030402@cheimes.de> References: <94bdd2610910220230p11017058y8f1f8c4db508dc99@mail.gmail.com> <4AE02BAA.3000309@cheimes.de> <4AE03E60.6030402@cheimes.de> Message-ID: <20091022111845.GB16548@phd.pp.ru> On Thu, Oct 22, 2009 at 01:13:36PM +0200, Christian Heimes wrote: > Configuration directories are usually > suffixed with ".d". How about ~/.python.d/? I have 40 config directories in my $HOME, and none of them ends with ".d". -1. ".d" are usually directories in /etc that are referenced from main config file; for example /etc/logrotate.conf does "include /etc/logrotate.d/". Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From ziade.tarek at gmail.com Thu Oct 22 13:36:08 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 22 Oct 2009 13:36:08 +0200 Subject: [Python-ideas] A standard location for Python configuration files. In-Reply-To: <4AE03E60.6030402@cheimes.de> References: <94bdd2610910220230p11017058y8f1f8c4db508dc99@mail.gmail.com> <4AE02BAA.3000309@cheimes.de> <4AE03E60.6030402@cheimes.de> Message-ID: <94bdd2610910220436l325c0a53j9b21f32b6e1838ee@mail.gmail.com> On Thu, Oct 22, 2009 at 1:13 PM, Christian Heimes wrote: > First of all we should discuss if we are talking about automatically > generated config files or human editable config files. If the > configuration files are frequently edited by a user than ~/.local or > ~/.config aren't the best place. In Distutils, those are human-readable files. > The path ~/.python is a so obvious choice for a file or directory that > contains Python specific stuff that it probably used by lots of people. > If we are going to introduce a directory in the user's home directory > and not in some subfolder like ~/.local or ~/.config, then I like to > stick to another Unix tradition. Configuration directories are usually > suffixed with ".d". How about ~/.python.d/? I can understand the concerns about ~/.python but I find ~/.python.d/ rather cryptic. What about a more explicit name like ~/.pythonconfig then ? Tarek From ziade.tarek at gmail.com Thu Oct 22 13:36:52 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 22 Oct 2009 13:36:52 +0200 Subject: [Python-ideas] A standard location for Python configuration files. In-Reply-To: <94bdd2610910220436l325c0a53j9b21f32b6e1838ee@mail.gmail.com> References: <94bdd2610910220230p11017058y8f1f8c4db508dc99@mail.gmail.com> <4AE02BAA.3000309@cheimes.de> <4AE03E60.6030402@cheimes.de> <94bdd2610910220436l325c0a53j9b21f32b6e1838ee@mail.gmail.com> Message-ID: <94bdd2610910220436m703b36f1k300bd98041c3b27@mail.gmail.com> On Thu, Oct 22, 2009 at 1:36 PM, Tarek Ziad? wrote: > In Distutils, those are human-readable files. (and editable) From solipsis at pitrou.net Thu Oct 22 13:41:52 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 22 Oct 2009 11:41:52 +0000 (UTC) Subject: [Python-ideas] =?utf-8?q?A_standard_location_for_Python_configura?= =?utf-8?q?tion=09files=2E?= References: <94bdd2610910220230p11017058y8f1f8c4db508dc99@mail.gmail.com> <4AE02BAA.3000309@cheimes.de> <4AE03E60.6030402@cheimes.de> Message-ID: > First of all we should discuss if we are talking about automatically > generated config files or human editable config files. If the > configuration files are frequently edited by a user than ~/.local or > ~/.config aren't the best place. Well, I'm not sure what a non-human editable config file would be. Of course, stuff which is not meant to be edited by a human can (should?) go into .local. But for a human-editable config file, to place it in .local is quite unintuitive. > The path ~/.python is a so obvious choice for a file or directory that > contains Python specific stuff that it probably used by lots of people. > If we are going to introduce a directory in the user's home directory > and not in some subfolder like ~/.local or ~/.config, then I like to > stick to another Unix tradition. Configuration directories are usually > suffixed with ".d". How about ~/.python.d/? Well, I have only two of them here: ".pylint.d" and ".emacs.d"; the other configuration directories don't have a ".d" suffix. But the point is really that it's natural and legitimate for Python to use a ".python" directory. It's not like we are planning to call it ".config" or ".settings". Regards Antoine. From ben+python at benfinney.id.au Thu Oct 22 14:09:52 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 22 Oct 2009 23:09:52 +1100 Subject: [Python-ideas] A standard location for Python configuration files. References: <94bdd2610910220230p11017058y8f1f8c4db508dc99@mail.gmail.com> <4AE02BAA.3000309@cheimes.de> <4AE03E60.6030402@cheimes.de> Message-ID: <87iqe77hpr.fsf@benfinney.id.au> Christian Heimes writes: > Configuration directories are usually suffixed with ".d". How about > ~/.python.d/? No, configuration directories are not usually named with a ?.d? suffix. A configuration directory named with a ?.d? suffix is a usage that has the specific connotation of ?when reading the configuration, read *all files* in this directory as a unified set of configuration settings?. Examples from a Debian system I have access to include ?/etc/cron.d/?, ?/etc/logrotate.d/?, ?/etc/apache/conf.d/?, ?/etc/fonts.d/?, ?/etc/rc2.d/?. (There are exceptions that don't follow that semantic as well, like ?/etc/init.d/?, ?$HOME/.emacs.d/?, and ?/etc/pam.d/?. It's not a universally-honoured convention.) Whereas ?directory containing discrete configuration files, each of which will be read only under specific circumstances? are generally *not* named with a ?.d? suffix. If you're going to have a set of discretely-interpreted configuration files, it's best to avoid naming it with the ?.d? suffix. -- \ ?There was a point to this story, but it has temporarily | `\ escaped the chronicler's mind.? ?Douglas Adams | _o__) | Ben Finney From jimjjewett at gmail.com Thu Oct 22 16:06:06 2009 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 22 Oct 2009 10:06:06 -0400 Subject: [Python-ideas] nonlocal functions In-Reply-To: <196BB58F-F285-415D-917E-62B847EAEAB5@masklinn.net> References: <4ADDDEB3.3010204@mrabarnett.plus.com> <494DB4DC-CA0E-4F3D-BF32-8CE0E19DE5E5@masklinn.net> <4ADEF575.4000302@gmail.com> <196BB58F-F285-415D-917E-62B847EAEAB5@masklinn.net> Message-ID: On Wed, Oct 21, 2009 at 8:47 AM, Masklinn wrote: >... I do think `let` fits the Zen better than > nothing/`global`/`nonlocal`: ... > instead of 3 different forms depending on the exact Sure. But that isn't the real choice. The need to write to outer variables is so rare that the vast majority of programs (let alone individual functions) never use global or nonlocal. So the real choice is whether to type out the redundant `let` or to just let the compiler infer it. (Well, unless you wanted to change the rules for let variables, to make deeper scoping and nesting more common.) -jJ From bruce at leapyear.org Thu Oct 22 17:57:15 2009 From: bruce at leapyear.org (Bruce Leban) Date: Thu, 22 Oct 2009 08:57:15 -0700 Subject: [Python-ideas] nonlocal functions In-Reply-To: References: <4ADDDEB3.3010204@mrabarnett.plus.com> <494DB4DC-CA0E-4F3D-BF32-8CE0E19DE5E5@masklinn.net> <4ADEF575.4000302@gmail.com> <196BB58F-F285-415D-917E-62B847EAEAB5@masklinn.net> Message-ID: On Wed, Oct 21, 2009 at 8:47 AM, Masklinn wrote: >... I do think `let` fits the Zen better than > nothing/`global`/`nonlocal`: ... > instead of 3 different forms depending on the exact First, I hardly think introducing a 'let' keyword as you suggest is pythonic. It's adding new syntax of a different flavor than the current syntax and breaking existing code. Why is it a different flavor? Because apparently you didn't notice that global and nonlocal are statements in their own right, not modifications of an assignment statement. So you're suggesting: global one nonlocal two let three = value It's also a different flavor because global says what it means, nonlocal says what it means and let says nothing about what it means. And breaking existing code without a damn good reason? I don't see that happening. It seems a bit inconsistent to me that global/nonlocal act differently with respect to reading and writing variables. That's a far worse problem than you're trying to solve. (See below.) But changing that would break a lot of code. So we live with it. My proposal (whether or not it's a good idea) wouldn't break any existing code. The new syntax proposed is currently invalid so can't appear in existing programs. It would add a new local statement that is only recognized in contexts that don't exist right now (and therefore can't break existing programs). The truth is that global and nonlocal add cognitive load to the person reading your code because they can't just read the one line to see what it's doing. This increases the burden on the developer to be clear in what they're writing. If my proposal allows us to decrease cognitive load on the reader, then it might be worth considering. As an aside, if instead of using the global/nonlocal statement we wrote: global.one = nonlocal.two Then the reader knows exactly what's global at the time it gets read. But I don't see that happening anytime soon given the preference for TOOWTDI and taking out the global/nonlocal statements would break too much code. And :-) we'd have to figure out what this statement means: foo = bar + global.bar + nonlocal.bar --- Bruce http://www.vroospeak.com >>> def f(): def g(): print(i) i = 5 g() >>> f() 5 >>> def f2(): def g(): i = 5 g() print(i) >>> f2() Traceback (most recent call last): File "", line 1, in f2() File "", line 5, in f2 print(i) NameError: global name 'i' is not defined >>> def f3(): i = None def g(): nonlocal i i = 5 g() print(i) >>> f3() 5 -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Thu Oct 22 18:26:12 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 23 Oct 2009 01:26:12 +0900 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <9d153b7c0910220038h641375ccl55ec3aad0885c8e5@mail.gmail.com> References: <200910221047.17051.steve@pearwood.info> <9d153b7c0910220038h641375ccl55ec3aad0885c8e5@mail.gmail.com> Message-ID: <87aazjcs4b.fsf@uwakimon.sk.tsukuba.ac.jp> Yuvgoog Greenle writes: > official sandbox That's an oxymoron, IMHO. From stephen at xemacs.org Thu Oct 22 18:36:24 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 23 Oct 2009 01:36:24 +0900 Subject: [Python-ideas] A standard location for Python configuration files. In-Reply-To: <94bdd2610910220436l325c0a53j9b21f32b6e1838ee@mail.gmail.com> References: <94bdd2610910220230p11017058y8f1f8c4db508dc99@mail.gmail.com> <4AE02BAA.3000309@cheimes.de> <4AE03E60.6030402@cheimes.de> <94bdd2610910220436l325c0a53j9b21f32b6e1838ee@mail.gmail.com> Message-ID: <878wf3crnb.fsf@uwakimon.sk.tsukuba.ac.jp> Tarek Ziad? writes: > I can understand the concerns about ~/.python but I find ~/.python.d/ > rather cryptic. True, but it's an existing convention quite widely used. Mac OS X itself has at least nine. That increases by more than an order of magnitude on my system if I don't filter "/opt/local" (MacPorts). I'm not even going to bother trying to count on Gentoo Linux. > What about a more explicit name like ~/.pythonconfig then ? -0.5 vs /.python.d. From g.brandl at gmx.net Thu Oct 22 18:30:07 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 22 Oct 2009 18:30:07 +0200 Subject: [Python-ideas] A standard location for Python configuration files. In-Reply-To: <4AE03E60.6030402@cheimes.de> References: <94bdd2610910220230p11017058y8f1f8c4db508dc99@mail.gmail.com> <4AE02BAA.3000309@cheimes.de> <4AE03E60.6030402@cheimes.de> Message-ID: Christian Heimes schrieb: > Antoine Pitrou wrote: >> I don't find this argument very serious. The Python interpreter reclaiming the >> ~/.python directory only seems natural and reasonable to me. If we follow your >> argument we can't introduce *any* new location just because someone might >> already use it. This is quite unacceptable. +1 > The path ~/.python is a so obvious choice for a file or directory that > contains Python specific stuff that it probably used by lots of people. Such as? What program would use .python with a good reason? I'd like to see some proof that anything is using .python. I've looked at Google codesearch, and the only thing the string constant ".python" seems to be used for is an alternate file name extension. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From guido at python.org Thu Oct 22 19:02:39 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 22 Oct 2009 10:02:39 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <87iqe7dhn1.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20091021142649.BRH02340@ms19.lnh.mail.rcn.net> <87iqe7dhn1.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Thu, Oct 22, 2009 at 12:14 AM, Stephen J. Turnbull wrote: > Guido van Rossum writes: > ?> On Wed, Oct 21, 2009 at 11:26 AM, Raymond Hettinger wrote: > > ?> > Are you rejecting PEP 380? > ?> > ?> No, just deferring it. It didn't make Python 3.1 so I think we'll have > ?> to live without it for a long time no matter what. It could be > ?> revisited once the moratorium is lifted (for 3.3 or 3.4). > > Would it be useful to allow work on it, maybe including a deferred > approval (ie, PEP is approved for implementation in version 3.3 [or > "when the moratorium is lifted"]), to proceed on a branch? ?Branches > will be very cheap in hg. ?I realize this may "bring work forward" for > you and other interested parties, on the other hand I think there is a > "strike while the iron's hot" aspect for implementators of the PEP. > > In particular, having a working and approved feature on a branch would > allow those interested in the deferred PEP to merge into a (private) > fresh branch for compatibility testing of the work being done on the > mainline. I'd be okay with this, for the specific case of PEP 380. The work was all done but there was remaining discussion about precise semantics, which dragged the approval out enough to miss the 2.6 release. Looking back I still don't think the arguments put forward by a Danish gentleman (whose name I've forgotten, sorry) are worth the complications in semantics and implementation; but if there was a branch and some people tried to use it for real code it would be a lot easier to tell (especially if there were two branches implementing the competing proposals). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Oct 22 19:05:14 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 22 Oct 2009 10:05:14 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: On Wed, Oct 21, 2009 at 11:24 PM, Larry Bugbee wrote: > I agree with a moratorium. ?I cannot imagine it doing any harm. [...] > That said, two areas where language support would be welcome: This is the typical response I expected. It is however internally inconsistent. Either you agree with a moratorium or you don't. Agreeing with a moratorium except for your personal pet peeve equals no moratorium, since there are too many pet peeves. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From laban.patrick at gmail.com Thu Oct 22 20:29:54 2009 From: laban.patrick at gmail.com (Patrick Laban) Date: Thu, 22 Oct 2009 11:29:54 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: <5ee958ff0910221129j554b5b78x3a978262b4fe3251@mail.gmail.com> Please forgive me if anything I say has already been mentioned. I am certainly in support of a moratorium on Python core and switch the focus to Python libraries. I'm sure others have noticed that as of late many proposals have been of lower quality or are the same old ideas that keep getting requested. My only concern would be that the technology field is always changing and some great new technologies may come up during the moratorium. To alleviate this concern I propose that the primary Python core developers either meet once a year, or at the halfway point of the moratorium. They would then discuss whether or not any changes within the field are major enough to warrant a change within Python core. With these meetings we would be able to ensure that the Python core is only worked on if absolutely necessary, leaving us free to concentrate on library development. --Patrick Laban -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.peters at gmail.com Thu Oct 22 20:40:16 2009 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 22 Oct 2009 14:40:16 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: <1f7befae0910221140i4a83ed63jb6ae46b53fe3ecee@mail.gmail.com> [Guido] > I propose a moratorium on language changes. This would be a period of > several years during which no changes to Python's grammar or language > semantics will be accepted. Eh. I'll be a solid +1 on this /if/ you use your time machine to begin the moratorium right after the "with" statement was introduced. The rationale you gave applied as much then as it does now -- and if you do this, then we won't need to discuss it now, since it will already have been done. If you won't use your time machine, fine, +0.9. I'm taking off a tenth so you realize there's a steep cost for refusing it interfere with history. From brett at python.org Thu Oct 22 20:54:02 2009 From: brett at python.org (Brett Cannon) Date: Thu, 22 Oct 2009 11:54:02 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <5ee958ff0910221129j554b5b78x3a978262b4fe3251@mail.gmail.com> References: <5ee958ff0910221129j554b5b78x3a978262b4fe3251@mail.gmail.com> Message-ID: On Thu, Oct 22, 2009 at 11:29, Patrick Laban wrote: > Please forgive me if anything I say has already been mentioned. I am > certainly in support of a moratorium on Python core and switch the focus to > Python libraries. I'm sure others have noticed that as of late many > proposals have been of lower quality or are the same old ideas that keep > getting requested. > > My only concern would be that the technology field is always changing and > some great new technologies may come up during the moratorium. To alleviate > this concern I propose that the primary Python core developers either meet > once a year, or at the halfway point of the moratorium. They would then > discuss whether or not any changes within the field are major enough to > warrant a change within Python core. > > We already meet once a year at PyCon US and since last year instigated a language summit so we can just discuss things. But a moratorium is a moratorium. This is an all-or-nothing approach. As Guido has said, add gray area to it and suddenly everyone is trying to get their feature in. I really don't think going a release or two w/o changes to the language is going to make Python obsolete. We already have features in 3.x that people just don't know about because they have not started using it yet. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Thu Oct 22 21:08:12 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 22 Oct 2009 12:08:12 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <5ee958ff0910221129j554b5b78x3a978262b4fe3251@mail.gmail.com> References: <5ee958ff0910221129j554b5b78x3a978262b4fe3251@mail.gmail.com> Message-ID: On Thu, Oct 22, 2009 at 11:29 AM, Patrick Laban wrote: > Please forgive me if anything I say has already been mentioned.? I am > certainly in support of a moratorium on Python core and switch the focus to > Python libraries.? I'm sure others have noticed that as of late many > proposals have been of lower quality or are the same old ideas that keep > getting requested. > > My only concern would be that the technology field is always changing and > some great new technologies may come up during the moratorium.? To alleviate > this concern I propose that the primary Python core developers either meet > once a year, or at the halfway point of the moratorium.? They would then > discuss whether or not any changes within the field are major enough to > warrant a change within Python core. > > With these meetings we would be able to ensure that the Python core is only > worked on if absolutely necessary, leaving us free to concentrate on library > development. That's the moratorium-with-exceptions again, which cannot work because everyone has their favorite "can't wait" exception. New technology should be dealt with in a library first anyway. Anyway, the "Python core" includes a lot of stuff that isn't covered by the moratorium (which only prohibits changes to syntax and associated semantics, not implementation issues). I expect that any issues with the moratorium will be discussed on python-dev first; but the yearly language summit will be a good point to review it too. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Oct 22 21:10:35 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 22 Oct 2009 12:10:35 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <1f7befae0910221140i4a83ed63jb6ae46b53fe3ecee@mail.gmail.com> References: <1f7befae0910221140i4a83ed63jb6ae46b53fe3ecee@mail.gmail.com> Message-ID: On Thu, Oct 22, 2009 at 11:40 AM, Tim Peters wrote: > [Guido] >> I propose a moratorium on language changes. This would be a period of >> several years during which no changes to Python's grammar or language >> semantics will be accepted. > > Eh. ?I'll be a solid +1 on this /if/ you use your time machine to > begin the moratorium right after the "with" statement was introduced. > The rationale you gave applied as much then as it does now -- and if > you do this, then we won't need to discuss it now, since it will > already have been done. Well, my intention is for it to begin right after 3.1 was released. While the time machine can do stuff with SVN or Hg branches, it's not powerful enough to mess with code already released. Though honestly I don't recall exactly what was added between the with statement and the 3.1 release apart from 'nonlocal'. > If you won't use your time machine, fine, +0.9. ?I'm taking off a > tenth so you realize there's a steep cost for refusing it interfere > with history. Ah, you've moved from fractional winks to fractional votes. A nice upgrade. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From laban.patrick at gmail.com Thu Oct 22 21:15:55 2009 From: laban.patrick at gmail.com (Patrick Laban) Date: Thu, 22 Oct 2009 12:15:55 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <5ee958ff0910221129j554b5b78x3a978262b4fe3251@mail.gmail.com> Message-ID: <5ee958ff0910221215y30b869efx88bdbd4bf9c096@mail.gmail.com> On Thu, Oct 22, 2009 at 12:08 PM, Guido van Rossum wrote: > > That's the moratorium-with-exceptions again, which cannot work because > everyone has their favorite "can't wait" exception. > > New technology should be dealt with in a library first anyway. > > Anyway, the "Python core" includes a lot of stuff that isn't covered > by the moratorium (which only prohibits changes to syntax and > associated semantics, not implementation issues). > > I expect that any issues with the moratorium will be discussed on > python-dev first; but the yearly language summit will be a good point > to review it too. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/ > ) > With apologies, in this discussion I lost track of the fact that the moratorium is specific to the syntax and semantics issue. In this case I am in full support of the moratorium. It gets tiring seeing the same ideas proposed over and over again. Beyond python-dev the language summit should have a limited discussion about the moratorium, but beyond that it should stand. --Patrick Laban -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziade.tarek at gmail.com Thu Oct 22 21:24:45 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 22 Oct 2009 21:24:45 +0200 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <5ee958ff0910221129j554b5b78x3a978262b4fe3251@mail.gmail.com> Message-ID: <94bdd2610910221224w247e74d7pe446257c777fbfd@mail.gmail.com> On Thu, Oct 22, 2009 at 9:08 PM, Guido van Rossum wrote: [..] > That's the moratorium-with-exceptions again, which cannot work because > everyone has their favorite "can't wait" exception. > > New technology should be dealt with in a library first anyway. So.. (just a thought) could that possibly mean we could have a different release cycle for the stdlib ? (this has been suggested lately) Or its just a "freeze" of what's not located in /Lib, with the usual release cycle ? Regards Tarek From larry.bugbee at gmail.com Thu Oct 22 21:35:30 2009 From: larry.bugbee at gmail.com (Larry Bugbee) Date: Thu, 22 Oct 2009 12:35:30 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: > > That said, two areas where language support would be welcome: > > This is the typical response I expected. ... I was very careful to not say when. ;-)) From robert.kern at gmail.com Thu Oct 22 21:48:39 2009 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 22 Oct 2009 14:48:39 -0500 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <1f7befae0910221140i4a83ed63jb6ae46b53fe3ecee@mail.gmail.com> Message-ID: On 2009-10-22 14:10 PM, Guido van Rossum wrote: > On Thu, Oct 22, 2009 at 11:40 AM, Tim Peters wrote: >> [Guido] >>> I propose a moratorium on language changes. This would be a period of >>> several years during which no changes to Python's grammar or language >>> semantics will be accepted. >> >> Eh. I'll be a solid +1 on this /if/ you use your time machine to >> begin the moratorium right after the "with" statement was introduced. >> The rationale you gave applied as much then as it does now -- and if >> you do this, then we won't need to discuss it now, since it will >> already have been done. > > Well, my intention is for it to begin right after 3.1 was released. > While the time machine can do stuff with SVN or Hg branches, it's not > powerful enough to mess with code already released. > > Though honestly I don't recall exactly what was added between the with > statement and the 3.1 release apart from 'nonlocal'. Can you recall, off the top of your head, what language changes would need to be rolled back between the trunk and 3.1? I'm basically +1 from the peanut gallery, but I'm not really sure what changes have already been committed to the trunk. >> If you won't use your time machine, fine, +0.9. I'm taking off a >> tenth so you realize there's a steep cost for refusing it interfere >> with history. > > Ah, you've moved from fractional winks to fractional votes. A nice upgrade. :-) It's one of the things I miss about the good old days of c.l.py, actually. :-( -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From debatem1 at gmail.com Thu Oct 22 22:18:13 2009 From: debatem1 at gmail.com (geremy condra) Date: Thu, 22 Oct 2009 16:18:13 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <87aazjcs4b.fsf@uwakimon.sk.tsukuba.ac.jp> References: <200910221047.17051.steve@pearwood.info> <9d153b7c0910220038h641375ccl55ec3aad0885c8e5@mail.gmail.com> <87aazjcs4b.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Thu, Oct 22, 2009 at 12:26 PM, Stephen J. Turnbull wrote: > Yuvgoog Greenle writes: > > ?> official sandbox > > That's an oxymoron, IMHO. We can call it "the officially sanctioned area for the discussion and development of unsanctioned extensions to Python", then. Or TOSAFTDADOUETP, for short. I kind of like that last one. Geremy Condra From brett at python.org Thu Oct 22 22:18:49 2009 From: brett at python.org (Brett Cannon) Date: Thu, 22 Oct 2009 13:18:49 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <94bdd2610910221224w247e74d7pe446257c777fbfd@mail.gmail.com> References: <5ee958ff0910221129j554b5b78x3a978262b4fe3251@mail.gmail.com> <94bdd2610910221224w247e74d7pe446257c777fbfd@mail.gmail.com> Message-ID: On Thu, Oct 22, 2009 at 12:24, Tarek Ziad? wrote: > On Thu, Oct 22, 2009 at 9:08 PM, Guido van Rossum > wrote: > [..] > > That's the moratorium-with-exceptions again, which cannot work because > > everyone has their favorite "can't wait" exception. > > > > New technology should be dealt with in a library first anyway. > > So.. (just a thought) could that possibly mean we could have a > different release cycle for the stdlib ? > (this has been suggested lately) > > It wouldn't preclude it, but that's a different discussion. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Thu Oct 22 23:02:27 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 23 Oct 2009 07:02:27 +1000 Subject: [Python-ideas] Remove GIL with CAS instructions? In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F098FF41FF8@exchis.ccp.ad.local> References: <4ADE2AA9.4030604@molden.no> <4ADE35F0.10803@molden.no> <4ADEFF10.5040701@gmail.com> <930F189C8A437347B80DF2C156F7EC7F098FF41FF8@exchis.ccp.ad.local> Message-ID: <4AE0C863.10709@gmail.com> Kristj?n Valur J?nsson wrote: > I think you may be missing the point. If the GIL is implemented in a > naive way on a platform, then releasing the gil and reclaiming it > (such as is done during a checkinterval) can cause the same thread to > get it again. Thus, the idea that this checkinterval should allow > another thread waiting for the gil to run, would not work. > Apparently, "fairness" of some locking primitives is being given up > for efficiency reasons in some operatins systems, like Vista. I > tested this by implementing a GIL like mechanism on a multicore > maching using windows CriticalSections and Mutexes, both of which > would starve the other threads. Semaphores work, though and the > python locks on windows (using an atomic counter and an Event object) > also work. > > So, alghough Sturla's claim isn't valid for windows, there might be > systems where the syscheckinterval mechanism for thread yielding > doesn't work due to the locks in question not being "fair" on > multicore platforms. No, I understand the fairness problem - releasing the GIL for a whole millisecond is a workaround for exactly that issue. (Python must have suffered from this back on Windows NT 4, as that is where I first discovered this trick to fix a program that wasn't switching threads properly - before I added the time.sleep call to explicitly release the GIL and give other threads a chance to run before attempting to reacquire it, the program was running each thread to completion before starting the next one). It's a sledgehammer approach to the problem though and incurs a speed penalty even on platforms where the GIL thread scheduling is fairer. So it's better if we can find a way to ensure proper scheduling fairness across all platforms. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From guido at python.org Thu Oct 22 23:09:17 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 22 Oct 2009 14:09:17 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <1f7befae0910221140i4a83ed63jb6ae46b53fe3ecee@mail.gmail.com> Message-ID: On Thu, Oct 22, 2009 at 12:48 PM, Robert Kern wrote: > Can you recall, off the top of your head, what language changes would need > to be rolled back between the trunk and 3.1? I'm basically +1 from the > peanut gallery, but I'm not really sure what changes have already been > committed to the trunk. Not off the top of my head, but a diff between the Grammar file tagged with r31 and the py3k branch's head suggests that the only grammatical change was a refactoring of star-expressions which just fixed an ambiguity. The change of the 'with' statement to allow multiple handlers made it into 3.1 so it's safe. I haven't researched recent changes to the builtins, but svn.python.org is your friend. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ziade.tarek at gmail.com Thu Oct 22 23:10:57 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 22 Oct 2009 23:10:57 +0200 Subject: [Python-ideas] A standard location for Python configuration files. In-Reply-To: <878wf3crnb.fsf@uwakimon.sk.tsukuba.ac.jp> References: <94bdd2610910220230p11017058y8f1f8c4db508dc99@mail.gmail.com> <4AE02BAA.3000309@cheimes.de> <4AE03E60.6030402@cheimes.de> <94bdd2610910220436l325c0a53j9b21f32b6e1838ee@mail.gmail.com> <878wf3crnb.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <94bdd2610910221410r7e62eae1tbd1165df3677e507@mail.gmail.com> On Thu, Oct 22, 2009 at 6:36 PM, Stephen J. Turnbull wrote: > Tarek Ziad? writes: > > ?> I can understand the concerns about ~/.python but I find ~/.python.d/ > ?> rather cryptic. > > True, but it's an existing convention quite widely used. > ?Mac OS X > itself has at least nine. That increases by more than an order of > magnitude on my system if I don't filter "/opt/local" (MacPorts). ?I'm > not even going to bother trying to count on Gentoo Linux. If we stick to per-user configuration files located in ~, the only one I have found on my Mac OS X and Debian was from Pylint (.pylint.d) (I use MacPorts on Mac) But others known softwares I have don't use .d for their config directories: Mac: - .dropbox/ - .macports/ - .cups/ Debian: - .aptitude/ - .ipython/ - .subversion/ and so on.. (I couldn't find any documented convention yet on those directories) At the end, I also think ~/.python is the best choice, If a Python software use this name, it's not a problem since the old places will still be used, but with a deprecation warning, meaning that the incriminated software will have until Python 2.8 and 3.3 to change its configuration filename. Tarek From ncoghlan at gmail.com Thu Oct 22 23:20:23 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 23 Oct 2009 07:20:23 +1000 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <4ADF7112.30905@molden.no> Message-ID: <4AE0CC97.50707@gmail.com> Antoine Pitrou wrote: > Sturla Molden writes: >> Isn't this the same as saying it is time to produce an industry standard >> (as in ISO, ECMA, ANSI, IEEE) for the Python language? > > If we had plenty of workforce available it might be an idea. > But we have not, and fixing bugs and making improvements is a much better use of > volunteer time than language lawyering, IMO. > >> Except for Java, only standardized >> languages tend to be considered for large projects. > > The "except for Java" statement makes me think that standardization isn't a real > criterion here. Stability may be, however. I know of a number of large projects that are quite happy to make extensive use of Python. They tend to standardise on older versions (more precisely, versions that were current when the project started), but they use it. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ben+python at benfinney.id.au Fri Oct 23 00:05:13 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 23 Oct 2009 09:05:13 +1100 Subject: [Python-ideas] Proposal: Moratorium on Python language changes References: <200910221047.17051.steve@pearwood.info> <9d153b7c0910220038h641375ccl55ec3aad0885c8e5@mail.gmail.com> <87aazjcs4b.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <8763a76q5i.fsf@benfinney.id.au> "Stephen J. Turnbull" writes: > Yuvgoog Greenle writes: > > > official sandbox > > That's an oxymoron, IMHO. Perhaps you're interpreting it as ?sandbox of official things?? That would appear to be an oxymoron. My interpretation of it was more like ?place with official status as a sandbox?, which isn't an oxymoron IMO. -- \ ?I still have my Christmas Tree. I looked at it today. Sure | `\ enough, I couldn't see any forests.? ?Steven Wright | _o__) | Ben Finney From mwm-keyword-python.b4bdba at mired.org Fri Oct 23 00:56:28 2009 From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer) Date: Thu, 22 Oct 2009 18:56:28 -0400 Subject: [Python-ideas] nonlocal functions In-Reply-To: References: <4ADDDEB3.3010204@mrabarnett.plus.com> <494DB4DC-CA0E-4F3D-BF32-8CE0E19DE5E5@masklinn.net> <4ADEF575.4000302@gmail.com> <196BB58F-F285-415D-917E-62B847EAEAB5@masklinn.net> Message-ID: <20091022185628.172c8996@bhuda.mired.org> On Thu, 22 Oct 2009 08:57:15 -0700 Bruce Leban wrote: > taking out the global/nonlocal statements would break too much code. And :-) > we'd have to figure out what this statement means: > > foo = bar + global.bar + nonlocal.bar I know there's a smiley, but it seems obvious. Further, it seems *useful*. If I have a local bar, a nonlocal bar and a global bar right now, I'm pretty much SOL for getting the global and local bars values (ok, I can get the global bar with some chicanery, but I can't see how to get the nonlocal bar). Making those namespaces explicit would seem to be as much of a win as making "self" explicit. Of course, you might argue that anyone who created three such variables named bar deserves to be barred, but... http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From tim.peters at gmail.com Fri Oct 23 01:11:57 2009 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 22 Oct 2009 19:11:57 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <1f7befae0910221140i4a83ed63jb6ae46b53fe3ecee@mail.gmail.com> Message-ID: <1f7befae0910221611u15b544cex9806787b52f6bc95@mail.gmail.com> [Guido] >>> I propose a moratorium on language changes. This would be a period of >>> several years during which no changes to Python's grammar or language >>> semantics will be accepted. [Tim] >> Eh. ?I'll be a solid +1 on this /if/ you use your time machine to >> begin the moratorium right after the "with" statement was introduced. >> The rationale you gave applied as much then as it does now -- and if >> you do this, then we won't need to discuss it now, since it will >> already have been done. [Guido] > Well, my intention is for it to begin right after 3.1 was released. Well, I believe that /was/ your intention. > While the time machine can do stuff with SVN or Hg branches, it's not > powerful enough to mess with code already released. LOL -- say that often enough, and the newbies might even believe it <0.6 wink>. > Though honestly I don't recall exactly what was added between the with > statement and the 3.1 release apart from 'nonlocal'. Excellent! My plan worked, then. I looked, and, e.g., both the switch statement and anonymous inline meta-decorators no longer exist in current Pythons. I can live with nonlocal -- its continued existence was probably just due to a local leak in the time machine's rear Tesla coils. They've always been a bit flaky. >> If you won't use your time machine, fine, +0. > Ah, you've moved from fractional winks to fractional votes. A nice upgrade. :-) I have no idea what you're talking about. the-more-things-change-the-more-i-don't-ly y'rs - tim From greg at krypto.org Fri Oct 23 01:34:31 2009 From: greg at krypto.org (Gregory P. Smith) Date: Thu, 22 Oct 2009 16:34:31 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <1f7befae0910221140i4a83ed63jb6ae46b53fe3ecee@mail.gmail.com> References: <1f7befae0910221140i4a83ed63jb6ae46b53fe3ecee@mail.gmail.com> Message-ID: <52dc1c820910221634t3a75bf48raecb6918db71e1ae@mail.gmail.com> On Thu, Oct 22, 2009 at 11:40 AM, Tim Peters wrote: > [Guido] >> I propose a moratorium on language changes. This would be a period of >> several years during which no changes to Python's grammar or language >> semantics will be accepted. > > Eh. ?I'll be a solid +1 on this /if/ you use your time machine to > begin the moratorium right after the "with" statement was introduced. > The rationale you gave applied as much then as it does now -- and if > you do this, then we won't need to discuss it now, since it will > already have been done. > > If you won't use your time machine, fine, +0.9. ?I'm taking off a > tenth so you realize there's a steep cost for refusing it interfere > with history. +1.1 from me. Someone's gotta pick up Tim's 0.1 slack. ;) From jared.grubb at gmail.com Fri Oct 23 02:15:09 2009 From: jared.grubb at gmail.com (Jared Grubb) Date: Thu, 22 Oct 2009 17:15:09 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <200910221047.17051.steve@pearwood.info> References: <200910221047.17051.steve@pearwood.info> Message-ID: <9514797F-A69F-4F0A-A108-800A6D9AA285@gmail.com> On 21 Oct 2009, at 16:47, Steven D'Aprano wrote: > As far as I can see, the only practical difference it will make is > that > it will cut short some (but not all) threads on python-ideas and > comp.lang.python: anyone who suggests a language change will be > told "Moratorium. Come back in 2013." It would be a shame to discourage new ideas just because we are not willing to implement them "now". I hope that any moratorium would not cut discussions of new ideas short, would not change the tone of python-ideas, and would not discourage writing PEP's for new language features (with the understanding that it will be a while before they actually get implemented). Jared From guido at python.org Fri Oct 23 02:31:06 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 22 Oct 2009 17:31:06 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <9514797F-A69F-4F0A-A108-800A6D9AA285@gmail.com> References: <200910221047.17051.steve@pearwood.info> <9514797F-A69F-4F0A-A108-800A6D9AA285@gmail.com> Message-ID: On Thu, Oct 22, 2009 at 5:15 PM, Jared Grubb wrote: > It would be a shame to discourage new ideas just because we are not willing > to implement them "now". > > I hope that any moratorium would not cut discussions of new ideas short, > would not change the tone of python-ideas, and would not discourage writing > PEP's for new language features (with the understanding that it will be a > while before they actually get implemented). Actually one of my goals with the moratorium is to discourage discussion of certain ideas that keep coming up forever and draining the energy of the list. Also, I certainly don't hope that when the moratorium is lifted there are 20 language PEPs waiting for approval. Python's evolution needs to slow down as the user community grows. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From debatem1 at gmail.com Fri Oct 23 02:57:53 2009 From: debatem1 at gmail.com (geremy condra) Date: Thu, 22 Oct 2009 20:57:53 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <200910221047.17051.steve@pearwood.info> <9514797F-A69F-4F0A-A108-800A6D9AA285@gmail.com> Message-ID: On Thu, Oct 22, 2009 at 8:31 PM, Guido van Rossum wrote: > On Thu, Oct 22, 2009 at 5:15 PM, Jared Grubb wrote: >> It would be a shame to discourage new ideas just because we are not willing >> to implement them "now". >> >> I hope that any moratorium would not cut discussions of new ideas short, >> would not change the tone of python-ideas, and would not discourage writing >> PEP's for new language features (with the understanding that it will be a >> while before they actually get implemented). > > Actually one of my goals with the moratorium is to discourage > discussion of certain ideas that keep coming up forever and draining > the energy of the list. Personally, I think mandating that you bring working code to the table when proposing a language change would take the number of requests for, say, removing the GIL to pretty much nil. > Also, I certainly don't hope that when the > moratorium is lifted there are 20 language PEPs waiting for approval. > Python's evolution needs to slow down as the user community grows. Again, I doubt that very many of the people proposing some of these changes have either the technical skills to pull them off or the patience to maintain them for a year and a half while waiting for the moratorium to lift. My guess is that you'll have about 300 half-baked or just-started projects and only one or two good ones ready for PEP consideration. Over the period of time you're talking about, that doesn't seem -IMO- to be too much, too fast. Your mileage may certainly vary. Geremy Condra From guido at python.org Fri Oct 23 03:07:29 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 22 Oct 2009 18:07:29 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <200910221047.17051.steve@pearwood.info> <9514797F-A69F-4F0A-A108-800A6D9AA285@gmail.com> Message-ID: On Thu, Oct 22, 2009 at 5:57 PM, geremy condra wrote: > On Thu, Oct 22, 2009 at 8:31 PM, Guido van Rossum wrote: >> On Thu, Oct 22, 2009 at 5:15 PM, Jared Grubb wrote: >>> It would be a shame to discourage new ideas just because we are not willing >>> to implement them "now". >>> >>> I hope that any moratorium would not cut discussions of new ideas short, >>> would not change the tone of python-ideas, and would not discourage writing >>> PEP's for new language features (with the understanding that it will be a >>> while before they actually get implemented). >> >> Actually one of my goals with the moratorium is to discourage >> discussion of certain ideas that keep coming up forever and draining >> the energy of the list. > > Personally, I think mandating that you bring working code to the table > when proposing a language change would take the number of requests > for, say, removing the GIL to pretty much nil. Actually removing the GIL is not subject to the moratorium, and it seems that some people *are* working on code. >> Also, I certainly don't hope that when the >> moratorium is lifted there are 20 language PEPs waiting for approval. >> Python's evolution needs to slow down as the user community grows. > > Again, I doubt that very many of the people proposing some of these > changes have either the technical skills to pull them off or the > patience to maintain them for a year and a half while waiting for the > moratorium to lift. My guess is that you'll have about 300 half-baked > or just-started projects and only one or two good ones ready for PEP > consideration. Over the period of time you're talking about, that > doesn't seem -IMO- to be too much, too fast. Your mileage may > certainly vary. I hope to discourage those 300 people to the point where there won't be any half-baked or unbaked projects. Let them contribute to Perl 6. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jimjjewett at gmail.com Fri Oct 23 03:22:16 2009 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 22 Oct 2009 21:22:16 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <5ee958ff0910221129j554b5b78x3a978262b4fe3251@mail.gmail.com> Message-ID: On Thu, Oct 22, 2009 at 3:08 PM, Guido van Rossum wrote: > Anyway, the "Python core" includes a lot of stuff that isn't covered > by the moratorium (which only prohibits changes to syntax and > associated semantics, not implementation issues). Changes to semantics are rare enough that I'm not sure a moratorium (but allowing bug fixes) would matter. (Particularly if you stick to the claim that details of import resolution are implementation-specific.) I have no opinion on changes to syntax; they're already uncommon, but saying they'll wait until 2013 doesn't bother me. The limit on builtins may be more of a problem. For example, I liked Brett's work on signatures. If he gets motivated to work on it again, I don't want him to say "Drat; I really need access to this one extra function attribute, which isn't currently exposed at the python level. I guess I'll have to wait a few years." It would be reasonable to say that changes to builtins will be rare, and will typically be strictly additions to functionality, as though they were replaced by a subclass which overrode nothing except that certain Exceptions were now handled internally. (In other words, *if* the name previously existed, it either keeps the same meaning, or gets wrapped in something that just calls the old meaning as a try suite, with all new functionality inside the except suite.) -jJ From brett at python.org Fri Oct 23 04:06:24 2009 From: brett at python.org (Brett Cannon) Date: Thu, 22 Oct 2009 19:06:24 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <5ee958ff0910221129j554b5b78x3a978262b4fe3251@mail.gmail.com> Message-ID: On Thu, Oct 22, 2009 at 18:22, Jim Jewett wrote: > On Thu, Oct 22, 2009 at 3:08 PM, Guido van Rossum > wrote: > > Anyway, the "Python core" includes a lot of stuff that isn't covered > > by the moratorium (which only prohibits changes to syntax and > > associated semantics, not implementation issues). > > Changes to semantics are rare enough that I'm not sure a moratorium > (but allowing bug fixes) would matter. (Particularly if you stick to > the claim that details of import resolution are > implementation-specific.) > > I have no opinion on changes to syntax; they're already uncommon, but > saying they'll wait until 2013 doesn't bother me. > > The limit on builtins may be more of a problem. For example, I liked > Brett's work on signatures. If he gets motivated to work on it again, > I don't want him to say "Drat; I really need access to this one extra > function attribute, which isn't currently exposed at the python level. > I guess I'll have to wait a few years." > > Since my PEP 362 keeps (surprisingly) coming up), I will use it as an example. If the code actually gets checked in then chances are it will be put into inspect first to work out any details. If it happens to receive uptake, it can get pushed up to a built-in or something when the moratorium is lifted. But assigning to __signature__ or something on functions is not really covered by this as anyone could do that on their own w/o changing Python. What would be gray area is if help() was changed to recognize the attribute. But that could potentially be negotiated as implementation detail. > It would be reasonable to say that changes to builtins will be rare, > and will typically be strictly additions to functionality, as though > they were replaced by a subclass which overrode nothing except that > certain Exceptions were now handled internally. (In other words, *if* > the name previously existed, it either keeps the same meaning, or gets > wrapped in something that just calls the old meaning as a try suite, > with all new functionality inside the except suite.) I still prefer Guido's hard-line on changes and we don't even do additions. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Fri Oct 23 04:30:46 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 23 Oct 2009 13:30:46 +1100 Subject: [Python-ideas] a new lambda syntax In-Reply-To: <4AE01C8A.5030404@molden.no> References: <4ADC6A92.3060206@qq.com> <4AE01C8A.5030404@molden.no> Message-ID: <200910231330.46259.steve@pearwood.info> On Thu, 22 Oct 2009 07:49:14 pm Sturla Molden wrote: > If that is not allowed then, an anynymous block cannot be called or > referenced, as there are no name associated with it. That is clearly wrong. You don't need a name to call an object. You just need the object. >>> (lambda x: x+1)(42) 43 >>> [None, lambda s: s.upper(), None][1]('abc') 'ABC' > So to be useful, > it would always have to be associated with a decorator. That would > be the only way to get access to it: > > @decorator > def(): > That doesn't even make sense to me. I don't understand what the decorator is doing there, or how it helps you access the anonymous function. Under today's semantics (modified to allow anonymous def), that would create an anonymous function containing , then pass that function to decorator(), and treat the return result as anonymous. Since you're not keeping a reference to the anonymous function, it will then be garbage collected. Passing it into a decorator first won't change that (unless the decorator stores the function somewhere as a side-effect). In other words, the above is just like this: decorator(lambda: ) which is legal but pointless unless you do something with the result. > Now we can see that the use for this anonymous block "def()" simply > goes away, as it too is redundant syntax. Why? Functions aren't the only thing you can apply decorators to. You can also call: @decorator class K: pass in Python 2.6 and better. -- Steven D'Aprano From debatem1 at gmail.com Fri Oct 23 05:04:58 2009 From: debatem1 at gmail.com (geremy condra) Date: Thu, 22 Oct 2009 23:04:58 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <200910221047.17051.steve@pearwood.info> <9514797F-A69F-4F0A-A108-800A6D9AA285@gmail.com> Message-ID: On Thu, Oct 22, 2009 at 9:07 PM, Guido van Rossum wrote: > On Thu, Oct 22, 2009 at 5:57 PM, geremy condra wrote: >> On Thu, Oct 22, 2009 at 8:31 PM, Guido van Rossum wrote: >>> On Thu, Oct 22, 2009 at 5:15 PM, Jared Grubb wrote: >>>> It would be a shame to discourage new ideas just because we are not willing >>>> to implement them "now". >>>> >>>> I hope that any moratorium would not cut discussions of new ideas short, >>>> would not change the tone of python-ideas, and would not discourage writing >>>> PEP's for new language features (with the understanding that it will be a >>>> while before they actually get implemented). >>> >>> Actually one of my goals with the moratorium is to discourage >>> discussion of certain ideas that keep coming up forever and draining >>> the energy of the list. >> >> Personally, I think mandating that you bring working code to the table >> when proposing a language change would take the number of requests >> for, say, removing the GIL to pretty much nil. > > Actually removing the GIL is not subject to the moratorium, and it > seems that some people *are* working on code. > Alright then, adding default arguments that evaluate when the function is called. >>> Also, I certainly don't hope that when the >>> moratorium is lifted there are 20 language PEPs waiting for approval. >>> Python's evolution needs to slow down as the user community grows. >> >> Again, I doubt that very many of the people proposing some of these >> changes have either the technical skills to pull them off or the >> patience to maintain them for a year and a half while waiting for the >> moratorium to lift. My guess is that you'll have about 300 half-baked >> or just-started projects and only one or two good ones ready for PEP >> consideration. Over the period of time you're talking about, that >> doesn't seem -IMO- to be too much, too fast. Your mileage may >> certainly vary. > > I hope to discourage those 300 people to the point where there won't > be any half-baked or unbaked projects. Let them contribute to Perl 6. > :-) The trick is figuring out in advance which is the 300 and which is the 1 or 2. My suggestion is that we allow them to effectively self-select; that we allow those who are skilled enough and patient enough- and who happen to have ideas of unusual merit- to prove that to the Python community. If the results are great, great! Two years down the line Python has a great new feature. If not, oh well- it's not like you are under any obligation to merge it. And there's probably merit in keeping the really wretched ones around too, if only to say "no way, tried it, had to kill it with fire". Either way, Python wins. Geremy Condra From guido at python.org Fri Oct 23 05:38:51 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 22 Oct 2009 20:38:51 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <200910221047.17051.steve@pearwood.info> <9514797F-A69F-4F0A-A108-800A6D9AA285@gmail.com> Message-ID: On Thu, Oct 22, 2009 at 8:04 PM, geremy condra wrote: > The trick is figuring out in advance which is the 300 and which is the > 1 or 2. My suggestion is that we allow them to effectively self-select; > that we allow those who are skilled enough and patient enough- and > who happen to have ideas of unusual merit- to prove that to the > Python community. If the results are great, great! Two years down the > line Python has a great new feature. If not, oh well- it's not like you > are under any obligation to merge it. And there's probably merit in > keeping the really wretched ones around too, if only to say "no way, > tried it, had to kill it with fire". Either way, Python wins. I hope so. But no feature is an island. They must work together. Two or three features that are each individually well thought-out and implemented in separate branches might still be a disaster when combined. Also, ten new features, even if they all fit together, may just be too much new stuff for the user community to digest at once. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From debatem1 at gmail.com Fri Oct 23 05:45:13 2009 From: debatem1 at gmail.com (geremy condra) Date: Thu, 22 Oct 2009 23:45:13 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <200910221047.17051.steve@pearwood.info> <9514797F-A69F-4F0A-A108-800A6D9AA285@gmail.com> Message-ID: On Thu, Oct 22, 2009 at 11:38 PM, Guido van Rossum wrote: > On Thu, Oct 22, 2009 at 8:04 PM, geremy condra wrote: >> The trick is figuring out in advance which is the 300 and which is the >> 1 or 2. My suggestion is that we allow them to effectively self-select; >> that we allow those who are skilled enough and patient enough- and >> who happen to have ideas of unusual merit- to prove that to the >> Python community. If the results are great, great! Two years down the >> line Python has a great new feature. If not, oh well- it's not like you >> are under any obligation to merge it. And there's probably merit in >> keeping the really wretched ones around too, if only to say "no way, >> tried it, had to kill it with fire". Either way, Python wins. > > I hope so. > > But no feature is an island. They must work together. Two or three > features that are each individually well thought-out and implemented > in separate branches might still be a disaster when combined. > > Also, ten new features, even if they all fit together, may just be too > much new stuff for the user community to digest at once. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > Exactly- with a sandbox in place, the community could look at real code and evaluate features not just as abstract ideas, but as real working implementations- a first chance to see how well they might work in practice, how much code they break, and how well those changes work together, or, as you point out, don't. Geremy Condra From guido at python.org Fri Oct 23 05:50:54 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 22 Oct 2009 20:50:54 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <200910221047.17051.steve@pearwood.info> <9514797F-A69F-4F0A-A108-800A6D9AA285@gmail.com> Message-ID: On Thu, Oct 22, 2009 at 8:45 PM, geremy condra wrote: > Exactly- with a sandbox in place, the community could look at > real code and evaluate features not just as abstract ideas, but > as real working implementations- a first chance to see how > well they might work in practice, how much code they break, > and how well those changes work together, or, as you point > out, don't. You keep stressing the opportunities for new features, but I want to crush that hope -- new features should be rare, and should become rarer as the user community grows. The time to experiment was approximately 10 years ago. Go invent a new language (and hope for it to become popular :-) if you want to experiment. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From debatem1 at gmail.com Fri Oct 23 06:11:35 2009 From: debatem1 at gmail.com (geremy condra) Date: Fri, 23 Oct 2009 00:11:35 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <200910221047.17051.steve@pearwood.info> <9514797F-A69F-4F0A-A108-800A6D9AA285@gmail.com> Message-ID: On Thu, Oct 22, 2009 at 11:50 PM, Guido van Rossum wrote: > On Thu, Oct 22, 2009 at 8:45 PM, geremy condra wrote: >> Exactly- with a sandbox in place, the community could look at >> real code and evaluate features not just as abstract ideas, but >> as real working implementations- a first chance to see how >> well they might work in practice, how much code they break, >> and how well those changes work together, or, as you point >> out, don't. > > You keep stressing the opportunities for new features, but I want to > crush that hope -- new features should be rare, and should become > rarer as the user community grows. The time to experiment was > approximately 10 years ago. Go invent a new language (and hope for it > to become popular :-) if you want to experiment. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > I don't know how many more language changes we'll see in Python- I don't think anybody does- but if the intent is that the moratorium lift at some point, then I would suggest that we should start preparing for that time now. Geremy Condra From debatem1 at gmail.com Fri Oct 23 06:15:50 2009 From: debatem1 at gmail.com (geremy condra) Date: Fri, 23 Oct 2009 00:15:50 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <200910221047.17051.steve@pearwood.info> <9514797F-A69F-4F0A-A108-800A6D9AA285@gmail.com> Message-ID: On Thu, Oct 22, 2009 at 11:50 PM, Guido van Rossum wrote: > On Thu, Oct 22, 2009 at 8:45 PM, geremy condra wrote: >> Exactly- with a sandbox in place, the community could look at >> real code and evaluate features not just as abstract ideas, but >> as real working implementations- a first chance to see how >> well they might work in practice, how much code they break, >> and how well those changes work together, or, as you point >> out, don't. > > You keep stressing the opportunities for new features, but I want to > crush that hope -- new features should be rare, and should become > rarer as the user community grows. The time to experiment was > approximately 10 years ago. Go invent a new language (and hope for it > to become popular :-) if you want to experiment. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > Let me add an addendum to my previous comment, because as I reread this it just strikes me as sounding so ominous. Is it really your intent that Python's development slow to the same crawl as Fortran or C? Put another way, is it your intent to eventually raise the moratorium? Geremy Condra From debatem1 at gmail.com Fri Oct 23 06:52:11 2009 From: debatem1 at gmail.com (geremy condra) Date: Fri, 23 Oct 2009 00:52:11 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: On Wed, Oct 21, 2009 at 4:24 PM, Brett Cannon wrote: > > > On Wed, Oct 21, 2009 at 09:42, Guido van Rossum wrote: >> >> I propose a moratorium on language changes. This would be a period of >> several years during which no changes to Python's grammar or language >> semantics will be accepted. The reason is that frequent changes to the >> language cause pain for implementors of alternate implementations >> (Jython, IronPython, PyPy, and others probably already in the wings) >> at little or no benefit to the average user (who won't see the changes >> for years to come and might not be in a position to upgrade to the >> latest version for years after). >> > > +1 from me. In this rather long thread someone proposed allowing changes > that make implementation of the language easier, which I am also fine with, > but that could almost be classified a bug fix and taken on a case-by-case > basis. But completely new exposed syntax or semantics for the language and > built-ins should be off limits. > And for the "several years", I say make through 3.4, letting in new features > for Python 3.5 which should be open for development in mid 2013. > >> >> The main goal of the Python development community at this point should >> be to get widespread acceptance of Python 3000. There is tons of work >> to be done before we can be comfortable about Python 3.x, mostly in >> creating solid ports of those 3rd party libraries that must be ported >> to Py3k before other libraries and applications can be ported. (Other >> work related to Py3k acceptance might be tools to help porting, tools >> to help maintaining multiple versions of a codebase, documentation >> about porting to Python 3, and so on. Also, work like that going on in >> the distutils-sig is very relevant.) >> > > The moratorium should also give us time and space to focus on bug fixes for > the language which is good. > >> >> Note, the moratorium would only cover the language itself plus >> built-in functions, not the standard library. Development in the >> standard library is valuable and much less likely to be a stumbling >> block for alternate language implementations. I also want to exclude >> details of the CPython implementation, including the C API from being >> completely frozen -- for example, if someone came up with (otherwise >> acceptable) changes to get rid of the GIL I wouldn't object. >> > > Sounds reasonable to me. > >> >> But the moratorium would clearly apply to proposals for anonymous >> blocks, "yield from" (PEP 380), changes to decorator syntax, and the >> like. (I'm sure it won't stop *discussion* of those proposals, and >> that's not the purpose of the moratorium; but at least it will stop >> worries elsewhere that such proposals might actually be *accepted* any >> time soon.) > > Sounds good to me. While I heard some people on Twitter want PEP 380, if we > are going to freeze it should be across the board. I can see someone arguing > that at least PEP 380 is in a PEP format, but even then that doesn't mean > its fully thought out (and I am not explicitly talking about PEP 380 as I > have not read it in ages). > A side benefit to this is it will give us time to come up with a PEP that > clearly defines exactly what is required to propose and get accepted changes > to the language (I also want to do this for the standard library, but I view > that as a different PEP). Hopefully that will cut down on the wild proposal > on python-ideas and give people a much clearer idea of what is required of > them. As I view this as a developer doc thing I am willing to write these > PEPs (eventually =). I second this- especially if it includes a code-first requirement. > One thing I would like to see happen for this moratorium is us following a > more rigid release schedule in terms of feature freeze. So I would propose > that starting with 3.2 we feature freeze 18 months after 3.2's feature > freeze. Now I am not suggesting we lock down the final release date to 18 > months as who knows how long it would take to shake out the bugs, but we can > easily say that every 18 months we feature freeze for the next minor release > (major releases don't apply to this schedule and I am not worrying about > micro releases as that is not under discussion here). Ditto. > This would give us and the community a much clearer indication of when the > moratorium will be lifted. So we could say that 3.2 is feature frozen on > June 15, 2010, Python 3.3 on January 15, 2012, and Python 3.4 on June 15, > 2013. We can obviously choose some other set of dates (went with this to > avoid releasing on New Years, although it might be fun to shift to Oct 5 as > that's the date Flying Circus was first broadcast), but it would be nice to > be able to say that "after Python 3.4, which is feature frozen on XXX, we > will start looking at new features again for Python 3.5". > -Brett Sounds good to me. Geremy Condra From ubershmekel at gmail.com Fri Oct 23 09:46:52 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Fri, 23 Oct 2009 09:46:52 +0200 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <200910221047.17051.steve@pearwood.info> <9514797F-A69F-4F0A-A108-800A6D9AA285@gmail.com> Message-ID: <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> On Fri, Oct 23, 2009 at 5:50 AM, Guido van Rossum wrote: > You keep stressing the opportunities for new features, but I want to > crush that hope > [ snip ] > Go invent a new language (and hope for it > to become popular :-) if you want to experiment. Crushing hope and pointing people to the door is ugly [*]. Why freeze? Instead, define how change occurs and make it as slow as you like. Define the python kilo (1000x) development cycle. For example the "moratorium" could state: --- S&B = syntax and builtins Python only introduces syntax and builtins once every X years[**] and only in major version changes. Between pyNk releases S&B are locked. New S&B can and should be tested on the next major revision's experimental branch(es). New S&B will only be considered for inclusion during the 2 years prior a pyNk major version. If a core language feature is to be accepted it needs: 1. A community tested and reviewed patch 2. A migration strategy (ie 3to4 code). 3. Python-dev acceptance (which can only be applied for during the 2 year S&B period). --- I suggest a new mailing list titled "python-experimental" for discussion and development of experimental branches. Also, commit privileges should be really common and easy to get for this branch of branches. Mercurial could help in implementing this. I realize that anyone can take the python source and do whatever they want on any free source host but having an official lab[***] for this can give another step between code that's in the wild and code that's in an official (non-experimental) branch and thus save core developer's time. --yuv [*] Beautiful is better than ugly. [**] There were about 8 years between python 2 and python 3, extrapolating from the discussion that this was too fast, lets say X=10, sound reasonable? [***] I prefer to call it a "playground" but now that python's all serious business and not a language for fun I don't know how well that would go. ps (short for pike shedding) - the "moratorium" has an evil sound to it. All python needs is a Core-Feature-Development-Cycle-Roadmap or something catchier. From abhiram.casina at gmail.com Fri Oct 23 11:13:44 2009 From: abhiram.casina at gmail.com (Abhiram Kasina) Date: Fri, 23 Oct 2009 14:43:44 +0530 Subject: [Python-ideas] Proposal : Python Trusted Computing API In-Reply-To: References: Message-ID: Hi all Thank you very much for the response. I have tried to answer all the issues as much as I can. > Hm... Given that most infections these days are JavaScript based and > run in the browser, how does this provide any protection? I'm > presuming you're going to say that it doesn't but that there are other > use cases where it *does* provide protection; but most likely those > use cases are only relevant for Windows (since that's what most > attackers attack anyway). Guido, I don't know the exact security model and I am currently going through it. But it has a mechanism in which none of the passwords or anything can be ever given out. So, phishing and pharming could be fought better. I agree that most of the cases of viruses are in Windows, but there are many cases in the past in which they have affected linux machines as well. So, if security is being made better, it should be used. > Since this intefaces with the hardware, doesn't it require some kind > of cooperation from the Linux kernel? And wouldn't it be better if > Python was never allowed access to any of the protected resources in > the first place? There would be a wrapper on top of the drivers running at the kernel level. Lets say there is a Python application which uses some sort of password authentication. If the password is present somehow in the memory, it could be stolen. But if its present inside a completely separate hardware, which doesn't allow it to be retrieved, I think that will be better. > Where/what is Umit? (Google gives several meanings but it's unclear > which you might mean.) By Umit, I meant Umit Project [1]. Its the GUI for nmap and its being built completely in Python. > You'd first have to tell us more about the security model. What is a > "secure application" and what does it protect against? And how? Yes, I will take some time to understand the model and present you the details of the model well when I get some time. > ?I personally have grave concerns about this technology, independent of > ?who is advancing it- just reading the links you provide makes it clear > ?that its primary purpose is to restrict the degree of control a user can > ?exercise over their own data, programs, and machine. Is it any wonder > ?that the same people who find DRM abhorrent find the technologies > ?used to advance it equally distasteful? Geremy, I have read the TPM specifications to some extent and the only thing enabling DRM here is the presence of a system-specific key. And it is heavily impractical to encrypt every music file with that key, when the vendor is selling it. It could be done even now,? they want to do it. > ?If you were suggesting adding a crypto API to python, I'd be all > ?for it- but you're suggesting adding the ability to have Python > ?software vendors remotely cripple the code on your machine. > ?I just can't get behind that, and while you're sure to hear wildly > ?divergent opinions on this board, I suspect that mine will not > ?be an uncommon sentiment. I appreciate the opinion and I am not suggesting adding a crypto API. Thanks Abhiram Kasina [1] http://www.umitproject.org/ From sturla at molden.no Fri Oct 23 16:22:35 2009 From: sturla at molden.no (Sturla Molden) Date: Fri, 23 Oct 2009 16:22:35 +0200 Subject: [Python-ideas] add a global tick register to the interpreter? Message-ID: <4AE1BC2B.5080307@molden.no> (Please forgive me if this belongs on python-dev.) When measuring the performance of a Python or Python program, it is often required to keep precise count of interpreter tics and periodic checks. There is no way to to this reliably, except modify ceval.c and rebuild Python. I think facilities for performance measures should be available. It should not be neccesary edit ceval.c and rebuild Python just to measure how Python itself or a Python program performs. I am therefore proposing this change to ceval.c: New global declarations: /* tick and check counters */ volatile PY_LONG_LONG _Py_Interpreter_Ticks = 0; volatile PY_LONG_LONG _Py_Interpreter_Checks = 0; The big infinite loop in PyEval_EvalFrameEx becomes for (;;Py_Interpreter_Ticks++) { #ifdef WITH_TSC if (inst1 == 0) { with periodic checks like this: if (--_Py_Ticker < 0) { _Py_Interpreter_Checks++; The reason for this change is that _Py_Ticker cannot be used for performance measures, as the variable can be set to zero during check intervals themselves or asynchronously from callbacks/signals. The variables _Py_Interpreter_Checks and Py_Interpreter_Ticks would be available to extension developers using the declaration: extern const volatile PY_LONG_LONG _Py_Interpreter_Ticks; extern const volatile PY_LONG_LONG _Py_Interpreter_Checks; #define PyEval_GetInterpreterTicks (_Py_Interpreter_Ticks) /* or an inline function */ #define PyEval_GetInterpreterChecks (_Py_Interpreter_Checks) (A 'const volatile' is a C variable that cannot be written to, but can suddenly be changed from the environment.) From the Python side we would have: sys.tickcounter # or sys.getinterpreterticks? sys.checkcounter The counters must be 64 bit because of the speed of Python. On my computer I measure close to 3 million interpreter ticks per second. That would exhaust a (signed) 32 bit counter in 11 minutes. A signed 64 bit counter could go on for 24 thousand years. (This is not covered by Guido's moratorium as it is an implementation and library issue.) Regards, Sturla Molden From solipsis at pitrou.net Fri Oct 23 16:39:14 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 23 Oct 2009 14:39:14 +0000 (UTC) Subject: [Python-ideas] add a global tick register to the interpreter? References: <4AE1BC2B.5080307@molden.no> Message-ID: Sturla Molden writes: > > When measuring the performance of a Python or Python program, it is often > required to keep precise count of interpreter tics and periodic checks. Well... interpreter ticks only measures the number of opcodes executed, not the number of seconds elapsed. It's not a reliable performance measure at all, unless you want to measure ticking itself. From dangyogi at gmail.com Fri Oct 23 16:49:37 2009 From: dangyogi at gmail.com (Bruce Frederiksen) Date: Fri, 23 Oct 2009 10:49:37 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> References: <9514797F-A69F-4F0A-A108-800A6D9AA285@gmail.com> <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> Message-ID: <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> On Fri, Oct 23, 2009 at 3:46 AM, Yuvgoog Greenle wrote: > On Fri, Oct 23, 2009 at 5:50 AM, Guido van Rossum > wrote: > > You keep stressing the opportunities for new features, but I want to > > crush that hope > > [ snip ] > > Go invent a new language (and hope for it > > to become popular :-) if you want to experiment. > > Crushing hope and pointing people to the door is ugly [*]. > > Why freeze? Instead, define how change occurs and make it as slow as > you like. OK, I'm an outsider here. But looking at this whole discussion a picture emerges. So I'll toss this out here for discussion (and I'm really going out on a limb here, but it's OK to shoot the outsider!) "I propose a moratorium on language changes. This would be a period of several years during which no changes to Python's grammar or language semantics will be accepted. The reason is that frequent changes to the language cause pain for implementors of alternate implementations..." What I see happening is a realization as we move into 2010 that the C language is no longer the future. We may not know yet what language is the future. And we might imagine that it ought to be Python. This discussion is being done within the CPython group. That has two parts: C and Python. I see the moratorium as a declaration that Python needs a new implementation. It needs a new horse to ride on. What the moratorium is saying is that the Python language will no longer evolve on the CPython implementation. The moratorium will never be lifted for CPython. When a new implementation comes along, that new implementation will necessitate it's own language changes and have it's own future. The moratorium will be lifted when this new implementation (whatever that ends up being) is chosen. And then the Python language will be riding a new horse, with a new reference implementation. The real questions (if this is even in the right ballpark), are along the lines of: 1. How will this new implementation be chosen? What qualities are we looking for? Are we opening up a contest? Are there deadlines? 2. Will there only be one winner, or might there be multiple winners (perhaps one for each platform)? 3. How will the group of developers now working on CPython transition to this new implementation? This is especially ticklish when we realize that it will be some other group that initially develops the new implementation to the point of being accepted. Does the current CPython group simply ride off into the sunset at that point? Is Guido really the BDFL (Benevolent Dictator For the Language), or only the BDFC (Benevolent Dictator For CPython)? (I, for one, would very much like to see the current group continue to hold the reins on the language. They have an outstanding track record in language design!) 4. What are the legal issues? Are there restrictions on which license the new implementation can use? How is it adopted by the PSF? What limits does that place on the other group that developed this thing? What do they have to agree to? There, I said it. Lock n Load! -------------- next part -------------- An HTML attachment was scrubbed... URL: From phd at phd.pp.ru Fri Oct 23 17:00:39 2009 From: phd at phd.pp.ru (Oleg Broytman) Date: Fri, 23 Oct 2009 19:00:39 +0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> References: <9514797F-A69F-4F0A-A108-800A6D9AA285@gmail.com> <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> Message-ID: <20091023150039.GA2730@phd.pp.ru> On Fri, Oct 23, 2009 at 10:49:37AM -0400, Bruce Frederiksen wrote: > The moratorium will never be lifted > for CPython. Why do you think so? Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From sturla at molden.no Fri Oct 23 17:02:31 2009 From: sturla at molden.no (Sturla Molden) Date: Fri, 23 Oct 2009 17:02:31 +0200 Subject: [Python-ideas] add a global tick register to the interpreter? In-Reply-To: References: <4AE1BC2B.5080307@molden.no> Message-ID: <4AE1C587.7040902@molden.no> Antoine Pitrou skrev: > Well... interpreter ticks only measures the number of opcodes executed, not the > number of seconds elapsed. It's not a reliable performance measure at all, > unless you want to measure ticking itself. > I did not mention that: In Windows we get precise timings using QueryPerformanceCounter and QueryPerformanceFrequency. On Linux it would be uclock and UCLOCKS_PER_SEC. An C extension using the proposed API would have to call these functions itself. The proposed registers are thus not for timing, but for measuring how many times the interpreter has ticked and tocked. Or we could make it easier to write cross-platform extensions and have a function that retreives these numbers together with a high-precision timing in microseconds: int PyEval_GetPerformanceStats( PY_LONG_LONG *ticks, PY_LONG_LONG *checks, double *us); Sturla Molden From guido at python.org Fri Oct 23 17:30:44 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 23 Oct 2009 08:30:44 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <20091023150039.GA2730@phd.pp.ru> References: <9514797F-A69F-4F0A-A108-800A6D9AA285@gmail.com> <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> <20091023150039.GA2730@phd.pp.ru> Message-ID: Let's wait for the PEP before derailing this discussion any more. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jnoller at gmail.com Fri Oct 23 17:44:46 2009 From: jnoller at gmail.com (Jesse Noller) Date: Fri, 23 Oct 2009 11:44:46 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <9514797F-A69F-4F0A-A108-800A6D9AA285@gmail.com> <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> <20091023150039.GA2730@phd.pp.ru> Message-ID: <4222a8490910230844k6359ea36wc6f96a363cfc3d08@mail.gmail.com> On Fri, Oct 23, 2009 at 11:30 AM, Guido van Rossum wrote: > Let's wait for the PEP before derailing this discussion any more. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) I'm crossing my fingers to crank on it this weekend. Sorry for the delay jesse From ncoghlan at gmail.com Fri Oct 23 17:45:18 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 24 Oct 2009 01:45:18 +1000 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> References: <9514797F-A69F-4F0A-A108-800A6D9AA285@gmail.com> <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> Message-ID: <4AE1CF8E.1080709@gmail.com> Bruce Frederiksen wrote: > OK, I'm an outsider here. But looking at this whole discussion a > picture emerges. So I'll toss this out here for discussion (and I'm > really going out on a limb here, but it's OK to shoot the outsider!) > > "I propose a moratorium on language changes. This would be a period of > several years during which no changes to Python's grammar or language > semantics will be accepted. The reason is that frequent changes to the > language cause pain for implementors of alternate implementations..." > > What I see happening is a realization as we move into 2010 that the C > language is no longer the future. We may not know yet what language is > the future. And we might imagine that it ought to be Python. This > discussion is being done within the CPython group. That has two parts: > C and Python. I see the moratorium as a declaration that Python needs a > new implementation. It needs a new horse to ride on. I don't believe this is an accurate characterisation of the situation (neither in regards to the state of C, nor in regards to the state of Python). C isn't going anywhere - it is still massively important as a portable language that can be made to run pretty much anywhere with minimal application overhead relative to raw assembler (and often faster than hand-coded assembler, since humans aren't smart enough to manually optimise code for many modern processors). Developers of new platforms don't need to tediously hand code lots of tools - they just need to teach a C compiler to generate code for their platform and the bootstrapping process gets off to a flying start. However, C retains that relevance without the core language changing very much - the last major update was C99, and the update before that was C89, which was itself a consolidation of assorted features introduced by various alternative implementations since K&R C was documented in 1978. Some would even say that it is the very simplicity and stability of C that has lead to its ubiquity. So that's 2 language updates in more than 30 years since a C language spec was first articulated. Python has had more major updates than that in barely half the time. Now, for a long time, that didn't really matter because only the reference interpreter saw serious use. Stackless was around, but able to pick up most changes directly from the reference implementation (since it only modified the VM and was able to retain most of the rest of the core C code), while Jython fell by the wayside for a while when new style objects were introduced in Python 2.2. The situation now is significantly different. There are at least 3 major Python implementations under active development and in production use that I know of (CPython, Jython, IronPython), other major implementations that, if not in production use yet, will be eventually (e.g. PyPy, Unladen Swallow) and various other tools based around existing Python syntax (Pyrex, Cython). Changes to the core language spec now affect a *lot* more than just CPython - there are ripple effects that spread out through the whole language ecosystem. On the other hand, standard library changes to modules that other Python implementations are able to use without modification don't generally have anywhere near the same impact. Guido's proposed moratorium is a formal recognition of the fact that tinkering with the underlying language isn't what Python really needs right now. It *is* possible for a language to become "feature complete" from a practical point of view, after which tinkering at the edges and introducing special syntax for niche use cases becomes a distraction from more productive endeavours. We changed a lot of things in Py3k - by saying "no, we aren't going to change anything else in the core language spec for at least the next few years" we would be consciously taking a step back and waiting to see how all those changes play out in the real world before we start trying to build on them. We would be giving the alternative implementations a chance to come up to speed with their own Python 3.x implementations, giving major applications and frameworks a chance to do forward ports without difficult decisions as to which version to target and giving users a chance to get used to a new feature set before throwing yet more deltas at them (e.g. I don't think anyone has even begun to scratch the surface of the power and flexibility provided by PEP 3115). Are language changes going to be completely impossible under such a moratorium? Of course they aren't - if a solid enough case is made for a change, then we aren't going to dig our heels in just because we made a formal statement that we didn't want to change certain things for the next few years. If somebody comes up with a language improvement that is as groundbreaking as the with statement was for Python, or as asynchronous blocks would be for C, then of course we would still give it serious consideration. However, even a cursory glance at Python ideas shows that most current suggestions for tweaks to the core language don't really measure up to that standard. Even PEP 380 (generator delegation), which is a decent, well thought out suggestion, has to rely fairly heavily on language consistency and coherency arguments because the real world use cases that can't already be handled a different way are fairly minimal. Thanks for posting what you did though - while I believe your perception of the situation is incorrect, it helped crystalise in my mind *why* I think Guido's suggestion is a good idea. Previously I hadn't posted anything, as I wasn't sure whether or not I agreed with the concept. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From debatem1 at gmail.com Fri Oct 23 17:57:45 2009 From: debatem1 at gmail.com (geremy condra) Date: Fri, 23 Oct 2009 11:57:45 -0400 Subject: [Python-ideas] Proposal : Python Trusted Computing API In-Reply-To: References: Message-ID: On Fri, Oct 23, 2009 at 5:13 AM, Abhiram Kasina wrote: > Hi all > > Thank you very much for the response. I have tried to answer all the > issues as much as I can. > >> Hm... Given that most infections these days are JavaScript based and >> run in the browser, how does this provide any protection? I'm >> presuming you're going to say that it doesn't but that there are other >> use cases where it *does* provide protection; but most likely those >> use cases are only relevant for Windows (since that's what most >> attackers attack anyway). > > Guido, I don't know the exact security model and I am currently going > through it. But it has a mechanism in which none of the passwords or > anything can be ever given out. So, phishing and pharming could be > fought better. I agree that most of the cases of viruses are in > Windows, but there are many cases in the past in which they have > affected linux machines as well. So, if security is being made better, > it should be used. > This is dangerously incorrect. Password storage *is* a feature of TPMs, but does no good against network-based attacks, since it cannot (obviously) replace legacy systems for which a normal transmit-and-verify system is used for authentication. >> Since this intefaces with the hardware, doesn't it require some kind >> of cooperation from the Linux kernel? And wouldn't it be better if >> Python was never allowed access to any of the protected resources in >> the first place? > > There would be a wrapper on top of the drivers running at the kernel > level. Lets say there is a Python application which uses some sort of > password authentication. If the password is present somehow in the > memory, it could be stolen. But if its present inside a completely > separate hardware, which doesn't allow it to be retrieved, I think > that will be better. > My understanding is that this would require very tight integration with each of the Python implementations as a result of the fact that Python stores data on an internal heap. I may be wrong, IANAE. >> Where/what is Umit? (Google gives several meanings but it's unclear >> which you might mean.) > > By Umit, I meant Umit Project [1]. Its the GUI for nmap and its being > built completely in Python. > >> You'd first have to tell us more about the security model. What is a >> "secure application" and what does it protect against? And how? > > > Yes, I will take some time to understand the model and present you the > details of the model well when I get some time. > >> ?I personally have grave concerns about this technology, independent of >> ?who is advancing it- just reading the links you provide makes it clear >> ?that its primary purpose is to restrict the degree of control a user can >> ?exercise over their own data, programs, and machine. Is it any wonder >> ?that the same people who find DRM abhorrent find the technologies >> ?used to advance it equally distasteful? > > Geremy, I have read the TPM specifications to some extent and the only > thing enabling DRM here is the presence of a system-specific key. And > it is heavily impractical to encrypt every music file with that key, > when the vendor is selling it. It could be done even now,? they want > to do it. > Honestly, this statement boggles me. 1) DRM doesn't work that way, and 2) if you believe that remote attestation does not imply DRM then I simply have nothing more to say here. >> ?If you were suggesting adding a crypto API to python, I'd be all >> ?for it- but you're suggesting adding the ability to have Python >> ?software vendors remotely cripple the code on your machine. >> ?I just can't get behind that, and while you're sure to hear wildly >> ?divergent opinions on this board, I suspect that mine will not >> ?be an uncommon sentiment. > > I appreciate the opinion and I am not suggesting adding a crypto API. > You seem to have missed a few pages in the specification- but no matter. You can get an overview of the cryptography API in sections 4.3.4 and 5.8.2.7 of the TSS 1.2 specification document. It describes the API calls needed to get random data, sign and verify messages, perform cryptographic hashes, and, of course, to encrypt and decrypt data. Geremy Condra From bruce at leapyear.org Fri Oct 23 17:59:37 2009 From: bruce at leapyear.org (Bruce Leban) Date: Fri, 23 Oct 2009 08:59:37 -0700 Subject: [Python-ideas] add a global tick register to the interpreter? Message-ID: <4AE1C587.7040902@molden.no> And why do you want opcode count rather than precise timing? From debatem1 at gmail.com Fri Oct 23 18:18:22 2009 From: debatem1 at gmail.com (geremy condra) Date: Fri, 23 Oct 2009 12:18:22 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <4AE1CF8E.1080709@gmail.com> References: <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> <4AE1CF8E.1080709@gmail.com> Message-ID: On Fri, Oct 23, 2009 at 11:45 AM, Nick Coghlan wrote: > Bruce Frederiksen wrote: >> OK, I'm an outsider here. ?But looking at this whole discussion a >> picture emerges. ?So I'll toss this out here for discussion (and I'm >> really going out on a limb here, but it's OK to shoot the outsider!) >> >> "I propose a moratorium on language changes. This would be a period of >> several years during which no changes to Python's grammar or language >> semantics will be accepted. The reason is that frequent changes to the >> language cause pain for implementors of alternate implementations..." >> >> What I see happening is a realization as we move into 2010 that the C >> language is no longer the future. ?We may not know yet what language is >> the future. ?And we might imagine that it ought to be Python. ?This >> discussion is being done within the CPython group. ?That has two parts: >> C and Python. ?I see the moratorium as a declaration that Python needs a >> new implementation. ?It needs a new horse to ride on. > > I don't believe this is an accurate characterisation of the situation > (neither in regards to the state of C, nor in regards to the state of > Python). > Agreed. > C isn't going anywhere - it is still massively important as a portable > language that can be made to run pretty much anywhere with minimal > application overhead relative to raw assembler (and often faster than > hand-coded assembler, since humans aren't smart enough to manually > optimise code for many modern processors). Developers of new platforms > don't need to tediously hand code lots of tools - they just need to > teach a C compiler to generate code for their platform and the > bootstrapping process gets off to a flying start. > > However, C retains that relevance without the core language changing > very much - the last major update was C99, and the update before that > was C89, which was itself a consolidation of assorted features > introduced by various alternative implementations since K&R C was > documented in 1978. Some would even say that it is the very simplicity > and stability of C that has lead to its ubiquity. > Importantly, while Python is not Java, Python is also not C. The analogy between Python's life cycle and that of a language that was designed entirely in-house and before the words "internet" or "open source" were coined is at best flawed and at worst dangerously misleading. I would point instead to the development of a language like PHP as the more accurate metaphor- and despite its syntax and core language changes, it remains more popular than Python. > So that's 2 language updates in more than 30 years since a C language > spec was first articulated. Python has had more major updates than that > in barely half the time. Now, for a long time, that didn't really matter > because only the reference interpreter saw serious use. Stackless was > around, but able to pick up most changes directly from the reference > implementation (since it only modified the VM and was able to retain > most of the rest of the core C code), while Jython fell by the wayside > for a while when new style objects were introduced in Python 2.2. > > The situation now is significantly different. There are at least 3 major > Python implementations under active development and in production use > that I know of (CPython, Jython, IronPython), other major > implementations that, if not in production use yet, will be eventually > (e.g. PyPy, Unladen Swallow) and various other tools based around > existing Python syntax (Pyrex, Cython). > > Changes to the core language spec now affect a *lot* more than just > CPython - there are ripple effects that spread out through the whole > language ecosystem. On the other hand, standard library changes to > modules that other Python implementations are able to use without > modification don't generally have anywhere near the same impact. > Granted, and this is why I support the moratorium. However, I also support eventually *lifting* the moratorium- Python competes more against today's highly dynamic languages than yesterday's quite static languages. Stand too still for too long and we risk standing nowhere at all. > Guido's proposed moratorium is a formal recognition of the fact that > tinkering with the underlying language isn't what Python really needs > right now. It *is* possible for a language to become "feature complete" > from a practical point of view, after which tinkering at the edges and > introducing special syntax for niche use cases becomes a distraction > from more productive endeavours. > And I agree that it isn't what Python needs *right now*. But that is totally different from saying that there simply aren't any good ideas left. We should give those ideas an opportunity to be heard so that in a few years when everybody is caught up we stand in a position to make good, well-informed decisions about what the language needs. > We changed a lot of things in Py3k - by saying "no, we aren't going to > change anything else in the core language spec for at least the next few > years" we would be consciously taking a step back and waiting to see how > all those changes play out in the real world before we start trying to > build on them. > Agreed. > We would be giving the alternative implementations a chance to come up > to speed with their own Python 3.x implementations, giving major > applications and frameworks a chance to do forward ports without > difficult decisions as to which version to target and giving users a > chance to get used to a new feature set before throwing yet more deltas > at them (e.g. I don't think anyone has even begun to scratch the surface > of the power and flexibility provided by PEP 3115). > Agreed. > Are language changes going to be completely impossible under such a > moratorium? Of course they aren't - if a solid enough case is made for a > change, then we aren't going to dig our heels in just because we made a > formal statement that we didn't want to change certain things for the > next few years. If somebody comes up with a language improvement that is > as groundbreaking as the with statement was for Python, or as > asynchronous blocks would be for C, then of course we would still give > it serious consideration. > Actually, that's almost exactly the opposite of what Guido said- that there will be no one-change exceptions. > However, even a cursory glance at Python ideas shows that most current > suggestions for tweaks to the core language don't really measure up to > that standard. Even PEP 380 (generator delegation), which is a decent, > well thought out suggestion, has to rely fairly heavily on language > consistency and coherency arguments because the real world use cases > that can't already be handled a different way are fairly minimal. > Arguments against past proposals are not arguments against future proposals. Unless Guido's time machine got an extra supercollider while I wasn't looking, nobody on this board knows what's coming down the pipe next, and my argument is that if the suggestion is super-awesome, popularly supported, and maintained until the end of the moratorium, then it should have a really good chance of getting in. To paraphrase a discussion I had on this topic last night, the fact that there's a long, hard road to getting a feature into the language isn't a problem. The idea that there shouldn't be a road is. Geremy Condra From guido at python.org Fri Oct 23 20:04:42 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 23 Oct 2009 11:04:42 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> <4AE1CF8E.1080709@gmail.com> Message-ID: On Fri, Oct 23, 2009 at 9:18 AM, geremy condra wrote: > Arguments against past proposals are not arguments against > future proposals. Unless Guido's time machine got an extra > supercollider while I wasn't looking, nobody on this board knows > what's coming down the pipe next, and my argument is that if > the suggestion is super-awesome, popularly supported, and > maintained until the end of the moratorium, then it should have > a really good chance of getting in. To paraphrase a discussion > I had on this topic last night, the fact that there's a long, hard > road to getting a feature into the language isn't a problem. The > idea that there shouldn't be a road is. I need to give my elbow more rest for a couple weeks, so I'm limiting myself to ultra-short emails. I take exception to "popularly supported" above. Python is still not a democracy. You can fill in the rest. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fuzzyman at gmail.com Fri Oct 23 20:06:30 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Fri, 23 Oct 2009 19:06:30 +0100 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> <4AE1CF8E.1080709@gmail.com> Message-ID: <6f4025010910231106y6b7b7194idb415caace8e672d@mail.gmail.com> 2009/10/23 Guido van Rossum > On Fri, Oct 23, 2009 at 9:18 AM, geremy condra wrote: > > Arguments against past proposals are not arguments against > > future proposals. Unless Guido's time machine got an extra > > supercollider while I wasn't looking, nobody on this board knows > > what's coming down the pipe next, and my argument is that if > > the suggestion is super-awesome, popularly supported, and > > maintained until the end of the moratorium, then it should have > > a really good chance of getting in. To paraphrase a discussion > > I had on this topic last night, the fact that there's a long, hard > > road to getting a feature into the language isn't a problem. The > > idea that there shouldn't be a road is. > > I need to give my elbow more rest for a couple weeks, so I'm limiting > myself to ultra-short emails. > > I take exception to "popularly supported" above. Python is still not a > democracy. You can fill in the rest. > > Despite the title this was never really a 'proposal' was it. ;-) All the best, Michael > -- > --Guido van Rossum (home page: http://www.python.org/~guido/ > ) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From debatem1 at gmail.com Fri Oct 23 20:23:58 2009 From: debatem1 at gmail.com (geremy condra) Date: Fri, 23 Oct 2009 14:23:58 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> <4AE1CF8E.1080709@gmail.com> Message-ID: On Fri, Oct 23, 2009 at 2:04 PM, Guido van Rossum wrote: > On Fri, Oct 23, 2009 at 9:18 AM, geremy condra wrote: >> Arguments against past proposals are not arguments against >> future proposals. Unless Guido's time machine got an extra >> supercollider while I wasn't looking, nobody on this board knows >> what's coming down the pipe next, and my argument is that if >> the suggestion is super-awesome, popularly supported, and >> maintained until the end of the moratorium, then it should have >> a really good chance of getting in. To paraphrase a discussion >> I had on this topic last night, the fact that there's a long, hard >> road to getting a feature into the language isn't a problem. The >> idea that there shouldn't be a road is. > > I need to give my elbow more rest for a couple weeks, so I'm limiting > myself to ultra-short emails. > > I take exception to "popularly supported" above. Python is still not a > democracy. You can fill in the rest. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > Didn't say it was a democracy. Assumed that it would still involve some input from its user base. Is that wrong? Geremy Condra From solipsis at pitrou.net Fri Oct 23 20:53:11 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 23 Oct 2009 18:53:11 +0000 (UTC) Subject: [Python-ideas] add a global tick register to the interpreter? References: <4AE1BC2B.5080307@molden.no> <4AE1C587.7040902@molden.no> Message-ID: Sturla Molden writes: > > In Windows we get precise timings using QueryPerformanceCounter and > QueryPerformanceFrequency. On Linux it would be uclock and > UCLOCKS_PER_SEC. An C extension using the proposed API would have to > call these functions itself. The proposed registers are thus not for > timing, but for measuring how many times the interpreter has ticked and > tocked. Well, given I plan to propose a removal of the whole "ticking" scheme (see the "newgil" branch in the sandbox if you are curious), I am personally not terribly interested in giving access to the current counters :-) However, a cross-platform API for accessing high-resolution timers could be nice. Regards Antoine. From steve at pearwood.info Sat Oct 24 05:52:47 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 24 Oct 2009 14:52:47 +1100 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <4AE1CF8E.1080709@gmail.com> References: <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> <4AE1CF8E.1080709@gmail.com> Message-ID: <200910241452.47933.steve@pearwood.info> On Sat, 24 Oct 2009 02:45:18 am Nick Coghlan wrote: > However, C retains that relevance without the core language changing > very much - the last major update was C99, and the update before that > was C89, which was itself a consolidation of assorted features > introduced by various alternative implementations since K&R C was > documented in 1978. Some would even say that it is the very > simplicity and stability of C that has lead to its ubiquity. In other words, there were various (I'd say "many") incompatible implementations of C that introduced their own core changes over the years, and eventually the C standard caught up with what those implementations had already done. And the process continues: google on "non-standard C" for examples. E.g.: "C has followed the standardization path of most languages. First, the language was created and used. As its popularity grew, it began to spawn a number of different dialects. Then, C went through a standardization process, with the main core of the language being standardized. It is common for compiler implementations to support standard C with additional, system-dependent features. This results in the creation of much non-standard C code." "C is standardized by both ANSI and ISO. However, there is no reasonable expectation that a C compiler will follow the standard without including additional features." http://archive.adaic.com/docs/reports/lawlis/k.htm That's hardly the same as saying that C was unchanging or that there was a moratorium on changes to the language. That's more like the language continued to evolve, in mutually inconsistent directions, and it was merely the standard that was out of touch with the practice of C across various incompatible implementations. There's plenty of working non-standard C code in the world. I'm not sure that it's even possible to write non-standard Python code. There are, naturally, libraries that only exist for one platform or one implementation, but that's not the same thing. > Python has had more major updates than > that in barely half the time. Now, for a long time, that didn't > really matter because only the reference interpreter saw serious use. > Stackless was around, but able to pick up most changes directly from > the reference implementation (since it only modified the VM and was > able to retain most of the rest of the core C code), while Jython > fell by the wayside for a while when new style objects were > introduced in Python 2.2. "Fell by the wayside"? Are you saying that the popularity of Jython *declined* because of the introduction of newstyle classes in CPython? I have an alternative interpretation. Jython 2.2 (equivalent to CPython 2.2, including new-style classes) was finally released in August 2007, a year after CPython had released 2.5. This was Jython's first production release in six years: http://www.jython.org/archive/22/news.html If so, that suggests to me that Jython developers abandoned Jython (probably for CPython) because they thought Jython was moribund. They voted with their feet in favour of an implementation with regular and useful changes to the core over an implementation which was stable and unchanging. [...] > Changes to the core language spec now affect a *lot* more than just > CPython - there are ripple effects that spread out through the whole > language ecosystem. On the other hand, standard library changes to > modules that other Python implementations are able to use without > modification don't generally have anywhere near the same impact. We keep being told this, but so far only one developer of an alternative implementation has spoken up. According to IronPython's Dino Viehland, language features are rarely a problem for him, but some library modules are. I would really like to hear from some other implementation developers. I'd also like to hear from the maintainers of large Python projects or libraries -- why aren't they migrating to 3.1? Is it because they fear 3.2 will include new core features? I can't imagine this is a problem, because if they're supporting 3.1, they have to use the features in 3.1 and not those in 3.2 -- whether those features are in the core or the library. Adding features to newer versions can't hurt them, although obviously changing or removing features will. But Python is already conservative about changing and removing features, so a moratorium doesn't change that. > Guido's proposed moratorium is a formal recognition of the fact that > tinkering with the underlying language isn't what Python really needs > right now. It *is* possible for a language to become "feature > complete" from a practical point of view, after which tinkering at > the edges and introducing special syntax for niche use cases becomes > a distraction from more productive endeavours. Perhaps so, but many of the changes being suggested are better described as "filing off the rough edges". Not every new feature has to be as big a change as the introduction of while, or decorators. Small changes which are easy to implement can make a big difference to usability to. -- Steven D'Aprano From stephen at xemacs.org Sat Oct 24 07:30:36 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 24 Oct 2009 14:30:36 +0900 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> <4AE1CF8E.1080709@gmail.com> Message-ID: <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> geremy condra writes: > Didn't say it was a democracy. Assumed that it would still involve > some input from its user base. Is that wrong? Your phrase "popularly supported" is ambiguous. If you mean "attracts widespread support in the form of description of varied and plausible use cases" for new syntax and built-ins, or something like that, I'm sure the answer is "yes". The question posed by the moratorium proposal is, "When?" It seems to me that what Guido is heading for here is very similar to the "punctuated equilibrium" concept (associated with the evolutionary biologist Stephen Jay Gould, the wikipedia article is pretty good, and fairly short). The basic idea is that long periods of relative stability are "punctuated" by periods of rapid evolution. In biology, the "bleeding edge" involves literal deaths, and Nature doesn't hesitate to waste large percentages of a population over the "stable" period as well as decimating it during the rapid evolution phases. In software, it may make sense to have the stable periods be *much* more stable, since artificial systems are more fragile than natural ones. C, because it occupies an "ecological niche" as a high-level assembly language, has been quite static since its original definition, even when it evolves. Python need not be, since its niche is very different. But it makes sense to propose to compress the evolution into short periods with many changes, and have very stable periods of "moratorium" between. Whether it will work well or not, we'll have to try it to see. But it's not just Guido's intuition that says it will work to the advantage of Python. From scott+python-ideas at scottdial.com Sat Oct 24 08:03:34 2009 From: scott+python-ideas at scottdial.com (Scott Dial) Date: Sat, 24 Oct 2009 02:03:34 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> References: <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> <4AE1CF8E.1080709@gmail.com> <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4AE298B6.7010708@scottdial.com> Stephen J. Turnbull wrote: > In software, it may make sense to have the stable periods be *much* > more stable, since artificial systems are more fragile than natural > C, because it occupies an "ecological niche" as a high-level > assembly language, has been quite static since its original > definition, even when it evolves. I was going to stay out of this discussion, but I find it highly annoying to see all these references to C being "static". One need only glance at the length of GCC's changes for the major version numbers to see that it is not static by any stretch of the imagination. Furthermore, the abundance of code revision (even in Python itself!) to compile without warning or error in various version of C compilers over the years makes the label "quite static" as laughable. I believe what you really mean, and what everyone really means, is that there is a core language that is unchanged over long periods of time. Changes that caused previously compiling C programs to no longer compile correctly would be considered a blatant failure of the compiler. That is not to say there are "no new features," rather only features that are backwards compatible. As is repeatedly pointed out, people who really do high-quality, professional development for Python do not develop for the latest release of Python, they standardize on a minimum version number and use only the features in that language; this is no different than writing ISO C90 code rather than ISO C99. I fail to see how this is not *already* the model that Python and Python developers are already using. At it's heart, I think the point of what Guido proposed is for everyone to stop fscking with the language and just start using it (specifically the 3.x version of it). But, I think Guido's frustration with the lack of traction of 3.x is misplaced; it's less than a year ago that it was released, and many ignored 3.0 due to the io module. Nevertheless, it is Guido's language and he can do whatever he wants. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From stephen at xemacs.org Sat Oct 24 08:52:47 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 24 Oct 2009 15:52:47 +0900 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <4AE298B6.7010708@scottdial.com> References: <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> <4AE1CF8E.1080709@gmail.com> <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <4AE298B6.7010708@scottdial.com> Message-ID: <878wf1b7wg.fsf@uwakimon.sk.tsukuba.ac.jp> Scott Dial writes: > I was going to stay out of this discussion, but I find it highly > annoying to see all these references to C being "static". One need only > glance at the length of GCC's changes for the major version numbers What does the ChangeLog for an implementation have to do with the stability of the language definition? Guido's moratorium does not attempt to regulate adaptation to new platforms, optimizations, or changes to improve conformance to the language reference, and (AIUI) not even changes to improve conformance of the implementation to programmer expectation where the language reference was ambiguous. Surely the vast majority of GCC changes fit one of those categories. Seems to me you have made a very powerful argument-by-analogy that Guido's moratorium provides plenty of room for hyperactive core development. From debatem1 at gmail.com Sat Oct 24 19:45:40 2009 From: debatem1 at gmail.com (geremy condra) Date: Sat, 24 Oct 2009 13:45:40 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> References: <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> <4AE1CF8E.1080709@gmail.com> <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Sat, Oct 24, 2009 at 1:30 AM, Stephen J. Turnbull wrote: > geremy condra writes: > > ?> Didn't say it was a democracy. Assumed that it would still involve > ?> some input from its user base. Is that wrong? > > Your phrase "popularly supported" is ambiguous. ?If you mean "attracts > widespread support in the form of description of varied and plausible > use cases" for new syntax and built-ins, or something like that, I'm > sure the answer is "yes". ?The question posed by the moratorium > proposal is, "When?" > > It seems to me that what Guido is heading for here is very similar to > the "punctuated equilibrium" concept (associated with the evolutionary > biologist Stephen Jay Gould, the wikipedia article is pretty good, and > fairly short). ?The basic idea is that long periods of relative > stability are "punctuated" by periods of rapid evolution. ?In biology, > the "bleeding edge" involves literal deaths, and Nature doesn't > hesitate to waste large percentages of a population over the "stable" > period as well as decimating it during the rapid evolution phases. > > In software, it may make sense to have the stable periods be *much* > more stable, since artificial systems are more fragile than natural > ones. ?C, because it occupies an "ecological niche" as a high-level > assembly language, has been quite static since its original > definition, even when it evolves. ?Python need not be, since its niche > is very different. ?But it makes sense to propose to compress the > evolution into short periods with many changes, and have very stable > periods of "moratorium" between. ?Whether it will work well or not, > we'll have to try it to see. ?But it's not just Guido's intuition that > says it will work to the advantage of Python. > > Let's just clear up a misconception: I am *for a moratorium*. As in, +1 from me. Having said that, I do not agree that development on changes to the language should stop, just that it should stop being integrated until we've had a long breather, allowed the other implementations to catch up to some degree, and given ourselves time to assess what the actual state of the language is post-2.x. The problem is that once we're out of the moratorium we'll be back in the same situation: every change will put the other implementors behind the one with the most man-hours to devote to it, we still won't know how any two changes to the language will interact, and we still won't be able to reasonably and empirically evaluate the worth of any particular proposed feature. We'd need another moratorium straight off. My solution to this problem is a simple one: create a sandbox where we can leverage Mercurial's ability to create cheap and easy branches to get a glimpse of possible future Pythons. We allow people to keep coming up with ideas and keep developing ideas on the off chance that one of them works so well that we want to integrate it after the moratorium lifts. There is no obligation here, no demands made on anyone's time who does not care to take a look at whats happening in the sandbox. In a few words, there is no cost. If done properly this will give both the CPython user base and the other implementors a period of not weeks but months or years to evaluate a proposed change to the language; it will force those proposing such changes to actually come up with code to match them, sparing the core devs time that could be better spent on the libraries or on improving Python 3 support in the community; it might even help thin out the requests for those features that are likely to be rejected, since seeing that an implementation has been rejected is frequently more compelling than hearing that a similar proposal has been dismissed in the past. As I say, it seems like a good idea to me, and if it turns out later not to have been it has virtually no consequences. Others will doubtless have different opinions. Geremy Condra From ncoghlan at gmail.com Sun Oct 25 06:33:09 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 25 Oct 2009 15:33:09 +1000 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> <4AE1CF8E.1080709@gmail.com> <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4AE3E315.5060500@gmail.com> geremy condra wrote: > My solution to this problem is a simple one: create > a sandbox where we can leverage Mercurial's ability > to create cheap and easy branches to get a glimpse > of possible future Pythons. We allow people to keep > coming up with ideas and keep developing ideas on > the off chance that one of them works so well that > we want to integrate it after the moratorium lifts. > There is no obligation here, no demands made on > anyone's time who does not care to take a look > at whats happening in the sandbox. In a few words, > there is no cost. Note that one of the major reasons behind moving to a DVCS at all it precisely to make it easier for people to collaborate on such experiments without having to involve python-dev. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From debatem1 at gmail.com Sun Oct 25 07:25:10 2009 From: debatem1 at gmail.com (geremy condra) Date: Sun, 25 Oct 2009 02:25:10 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <4AE3E315.5060500@gmail.com> References: <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> <4AE1CF8E.1080709@gmail.com> <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <4AE3E315.5060500@gmail.com> Message-ID: On Sun, Oct 25, 2009 at 1:33 AM, Nick Coghlan wrote: > geremy condra wrote: >> My solution to this problem is a simple one: create >> a sandbox where we can leverage Mercurial's ability >> to create cheap and easy branches to get a glimpse >> of possible future Pythons. We allow people to keep >> coming up with ideas and keep developing ideas on >> the off chance that one of them works so well that >> we want to integrate it after the moratorium lifts. >> There is no obligation here, no demands made on >> anyone's time who does not care to take a look >> at whats happening in the sandbox. In a few words, >> there is no cost. > > Note that one of the major reasons behind moving to a DVCS at all it > precisely to make it easier for people to collaborate on such > experiments without having to involve python-dev. > > Cheers, > Nick. I'm a little unclear on what you're driving at here- my goal is to keep these projects off the minds of the core devs until they're about ready to go, to make it easy for all the implementors to carefully assess both the features themselves and the code used to implement them as they near that point, and to make certain that there is a common point of reference for the debate over inclusion afterwards. In other words, not specifically to remove python-dev from the process, but rather to allow it to spend its time more productively elsewhere until it can be usefully spent there. Geremy Condra From ncoghlan at gmail.com Sun Oct 25 08:01:17 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 25 Oct 2009 17:01:17 +1000 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> <4AE1CF8E.1080709@gmail.com> <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <4AE3E315.5060500@gmail.com> Message-ID: <4AE3F7BD.3040301@gmail.com> geremy condra wrote: > On Sun, Oct 25, 2009 at 1:33 AM, Nick Coghlan wrote: > I'm a little unclear on what you're driving at here- > my goal is to keep these projects off the minds > of the core devs until they're about ready to go, to > make it easy for all the implementors to carefully > assess both the features themselves and the > code used to implement them as they near that > point, and to make certain that there is a common > point of reference for the debate over inclusion > afterwards. In other words, not specifically to > remove python-dev from the process, but rather > to allow it to spend its time more productively > elsewhere until it can be usefully spent there. Having an "official sandbox" is completely irrelevant (and probably counterproductive) if core development is happening in a DVCS. If people want to experiment, they will be able to experiment by branching into their own repository. Trying to create an "official sandbox" (or, more likely, use the existing sandbox) defeats the whole point of the exercise, since it just brings the CPython admins back into the loop and will generate a pile of irrelevant traffic on python-checkins. In short, a moratorium that said "we won't accept things into trunk, but you can still pester us to add things to our official sandbox" would be a pointless exercise. Moving to a DVCS, putting a core language change moratorium in place for a few years, then looking to see what improvements have been developed outside the core tree as the moratorium is running down (and what feedback has been received over that time) would be far more sensible. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From stephen at xemacs.org Sun Oct 25 09:34:58 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 25 Oct 2009 17:34:58 +0900 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> <4AE1CF8E.1080709@gmail.com> <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87ws2jan2l.fsf@uwakimon.sk.tsukuba.ac.jp> geremy condra writes: > Let's just clear up a misconception: I am *for a moratorium*. I don't think anybody said otherwise. So, in the meantime, you propose a "sandbox": > We allow people to keep coming up with ideas and keep developing > ideas on the off chance that one of them works so well that we want > to integrate it after the moratorium lifts. > There is no obligation here, no demands made on anyone's time who > does not care to take a look at whats happening in the sandbox. In > a few words, there is no cost. > > If done properly ... it will be work for somebody. Doing things properly always is. As Nick points out, Mercurial already allows that personal sandboxes, quite cheaply. Why should python.org and/or the PSF provide such a sandbox? I can think of one reason: providing a place to showcase new features in variants of the major implementations that are in actual use in other projects, and to collect common code of the various implementations so it can be shared. Maybe there's room for some speculative features, but I don't yet see that as a primary purpose here. > As I say, it seems like a good idea to me, and if it turns out > later not to have been it has virtually no consequences. That would be a shame if it works out poorly partly because it's been sold as cheap and risk-free, and as a consequence nobody puts in the resources needed to "do it properly". From ziade.tarek at gmail.com Sun Oct 25 11:12:25 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 25 Oct 2009 11:12:25 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? Message-ID: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> Hello, That's not my idea, it has been suggested by several people in the distutils discussions. But I'd like to bring it here because I couldn't find a clear answer or position on this subject in the lists archives (let me know if there's one). And depending on the answers, I might suggest it as a topic for the next language summit. What about having a different release cycle for the stdlib, and shipping Python in two distinct releases: - Python : core + stdlib - Python-Stdlib : stdlib The Python release would remain unchanged, but its cycle would be longer (as the moratorium seems to imply). stdlib would have a shorter release cycle (18 months?) allowing people to upgrade it without having to wait for the next full Python release. Each stdlib release would be compatible with a range of Python versions. Regards, Tarek From solipsis at pitrou.net Sun Oct 25 12:11:51 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 25 Oct 2009 11:11:51 +0000 (UTC) Subject: [Python-ideas] stdlib with its own release cycle ? References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> Message-ID: Tarek Ziad? writes: > > What about having a different release cycle for the stdlib, and > shipping Python in two distinct releases: > > - Python : core + stdlib > - Python-Stdlib : stdlib > > The Python release would remain unchanged, but its cycle would be > longer (as the moratorium seems to imply). I don't think the moratorium should imply any longer release cycle. Many improvements aren't of the kind that the moratorium aims at freezing. We could actually make the whole release cycle shorter, knowing that releases will be less disruptive. Regards Antoine. From g.brandl at gmx.net Sun Oct 25 12:56:35 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 25 Oct 2009 12:56:35 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> Message-ID: Antoine Pitrou schrieb: > Tarek Ziad? writes: >> >> What about having a different release cycle for the stdlib, and >> shipping Python in two distinct releases: >> >> - Python : core + stdlib >> - Python-Stdlib : stdlib >> >> The Python release would remain unchanged, but its cycle would be >> longer (as the moratorium seems to imply). > > I don't think the moratorium should imply any longer release cycle. Many > improvements aren't of the kind that the moratorium aims at freezing. We could > actually make the whole release cycle shorter, knowing that releases will be > less disruptive. Exactly. Since with the moratorium in effect, we are basically changing *nothing but* the stdlib, it has its own release cycle already :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ziade.tarek at gmail.com Sun Oct 25 13:39:22 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 25 Oct 2009 13:39:22 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> Message-ID: <94bdd2610910250539l4148138bvace06287e4c67867@mail.gmail.com> On Sun, Oct 25, 2009 at 12:56 PM, Georg Brandl wrote: > Antoine Pitrou schrieb: >> Tarek Ziad? writes: >>> >>> What about having a different release cycle for the stdlib, and >>> shipping Python in two distinct releases: >>> >>> - Python : core + stdlib >>> - Python-Stdlib : stdlib >>> >>> The Python release would remain unchanged, but its cycle would be >>> longer (as the moratorium seems to imply). >> >> I don't think the moratorium should imply any longer release cycle. Many >> improvements aren't of the kind that the moratorium aims at freezing. We could >> actually make the whole release cycle shorter, knowing that releases will be >> less disruptive. > > Exactly. Since with the moratorium in effect, we are basically changing > *nothing but* the stdlib, it has its own release cycle already :) Great then ! how long the release cycle should be then, in your opinion ? Tarek From g.brandl at gmx.net Sun Oct 25 14:55:25 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 25 Oct 2009 14:55:25 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: <94bdd2610910250539l4148138bvace06287e4c67867@mail.gmail.com> References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <94bdd2610910250539l4148138bvace06287e4c67867@mail.gmail.com> Message-ID: Tarek Ziad? schrieb: > On Sun, Oct 25, 2009 at 12:56 PM, Georg Brandl wrote: >> Antoine Pitrou schrieb: >>> Tarek Ziad? writes: >>>> >>>> What about having a different release cycle for the stdlib, and >>>> shipping Python in two distinct releases: >>>> >>>> - Python : core + stdlib >>>> - Python-Stdlib : stdlib >>>> >>>> The Python release would remain unchanged, but its cycle would be >>>> longer (as the moratorium seems to imply). >>> >>> I don't think the moratorium should imply any longer release cycle. Many >>> improvements aren't of the kind that the moratorium aims at freezing. We could >>> actually make the whole release cycle shorter, knowing that releases will be >>> less disruptive. >> >> Exactly. Since with the moratorium in effect, we are basically changing >> *nothing but* the stdlib, it has its own release cycle already :) > > Great then ! > > how long the release cycle should be then, in your opinion ? I'd say, less than the usual 18 months, but no shorter than 12 months. We need some time to accumulate new features, but with concentration on the stdlib, also make good changes available soon enough. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From orsenthil at gmail.com Sun Oct 25 15:31:16 2009 From: orsenthil at gmail.com (Senthil Kumaran) Date: Sun, 25 Oct 2009 20:01:16 +0530 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> Message-ID: <20091025143116.GA4934@ubuntu.ubuntu-domain> On Sun, Oct 25, 2009 at 11:12:25AM +0100, Tarek Ziad? wrote: > What about having a different release cycle for the stdlib, and > shipping Python in two distinct releases: > > - Python : core + stdlib > - Python-Stdlib : stdlib -1 on this idea. This would make things complicated, which is against the zen of python. :) I see the discussion on smaller release cycle in the thread, that should be fine.. -- Senthil "There is no distinctly American criminal class except Congress." -- Mark Twain From dreamingforward at gmail.com Sun Oct 25 17:20:44 2009 From: dreamingforward at gmail.com (average) Date: Sun, 25 Oct 2009 09:20:44 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <4AE3F7BD.3040301@gmail.com> References: <4AE1CF8E.1080709@gmail.com> <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <4AE3E315.5060500@gmail.com> <4AE3F7BD.3040301@gmail.com> Message-ID: <913f9f570910250920of841269o65656651bd53d030@mail.gmail.com> > Having an "official sandbox" is completely irrelevant (and probably > counterproductive) if core development is happening in a DVCS. If people > want to experiment, they will be able to experiment by branching into > their own repository. Trying to create an "official sandbox" (or, more > likely, use the existing sandbox) defeats the whole point of the > exercise, since it just brings the CPython admins back into the loop and > will generate a pile of irrelevant traffic on python-checkins. > > In short, a moratorium that said "we won't accept things into trunk, but > you can still pester us to add things to our official sandbox" would be > a pointless exercise. > > Moving to a DVCS, putting a core language change moratorium in place for > a few years, then looking to see what improvements have been developed > outside the core tree as the moratorium is running down (and what > feedback has been received over that time) would be far more sensible. 100% in agreement. marcos From guido at python.org Sun Oct 25 17:29:28 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 25 Oct 2009 09:29:28 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <4AE3F7BD.3040301@gmail.com> References: <4AE1CF8E.1080709@gmail.com> <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <4AE3E315.5060500@gmail.com> <4AE3F7BD.3040301@gmail.com> Message-ID: On Sun, Oct 25, 2009 at 12:01 AM, Nick Coghlan wrote: > Moving to a DVCS, putting a core language change moratorium in place for > a few years, then looking to see what improvements have been developed > outside the core tree as the moratorium is running down (and what > feedback has been received over that time) would be far more sensible. But the merges into the core will be tough. Imagine submitting a 5000-line patch and saying "I've worked on this for a year, please adopt it." Will we do enough code review to assert the code quality? -- --Guido van Rossum PS. My elbow needs a couple more weeks of rest. Limiting myself to ultra-short emails. From grosser.meister.morti at gmx.net Sun Oct 25 17:38:22 2009 From: grosser.meister.morti at gmx.net (=?UTF-8?B?TWF0aGlhcyBQYW56ZW5iw7Zjaw==?=) Date: Sun, 25 Oct 2009 17:38:22 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> Message-ID: <4AE47EFE.6020805@gmx.net> On 10/25/2009 12:56 PM, Georg Brandl wrote: > Antoine Pitrou schrieb: >> Tarek Ziad? writes: >>> >>> What about having a different release cycle for the stdlib, and >>> shipping Python in two distinct releases: >>> >>> - Python : core + stdlib >>> - Python-Stdlib : stdlib >>> >>> The Python release would remain unchanged, but its cycle would be >>> longer (as the moratorium seems to imply). >> >> I don't think the moratorium should imply any longer release cycle. Many >> improvements aren't of the kind that the moratorium aims at freezing. We could >> actually make the whole release cycle shorter, knowing that releases will be >> less disruptive. > > Exactly. Since with the moratorium in effect, we are basically changing > *nothing but* the stdlib, it has its own release cycle already :) > > Georg > As I had understood it thats not what the moratorium is about. The *language* (grammar, syntax) will not be changed. That doesn't freeze the interpreter at all. Even removing the GIL would not change the syntax (or semantics!) of the language, but it's definitely not just a stdlib change. -panzi From g.brandl at gmx.net Sun Oct 25 17:40:15 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 25 Oct 2009 17:40:15 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: <4AE47EFE.6020805@gmx.net> References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <4AE47EFE.6020805@gmx.net> Message-ID: Mathias Panzenb?ck schrieb: > On 10/25/2009 12:56 PM, Georg Brandl wrote: >> Antoine Pitrou schrieb: >>> Tarek Ziad? writes: >>>> >>>> What about having a different release cycle for the stdlib, and >>>> shipping Python in two distinct releases: >>>> >>>> - Python : core + stdlib >>>> - Python-Stdlib : stdlib >>>> >>>> The Python release would remain unchanged, but its cycle would be >>>> longer (as the moratorium seems to imply). >>> >>> I don't think the moratorium should imply any longer release cycle. Many >>> improvements aren't of the kind that the moratorium aims at freezing. We could >>> actually make the whole release cycle shorter, knowing that releases will be >>> less disruptive. >> >> Exactly. Since with the moratorium in effect, we are basically changing >> *nothing but* the stdlib, it has its own release cycle already :) > > As I had understood it thats not what the moratorium is about. The *language* > (grammar, syntax) will not be changed. That doesn't freeze the interpreter at > all. Even removing the GIL would not change the syntax (or semantics!) of the > language, but it's definitely not just a stdlib change. So, how many of those changes do you think will be done? As you say, they don't change any semantics, just maybe (hopefully) performance. Stdlib changes will therefore be most visible. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Sun Oct 25 17:42:51 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 25 Oct 2009 17:42:51 +0100 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <4AE1CF8E.1080709@gmail.com> <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <4AE3E315.5060500@gmail.com> <4AE3F7BD.3040301@gmail.com> Message-ID: Guido van Rossum schrieb: > On Sun, Oct 25, 2009 at 12:01 AM, Nick Coghlan wrote: >> Moving to a DVCS, putting a core language change moratorium in place for >> a few years, then looking to see what improvements have been developed >> outside the core tree as the moratorium is running down (and what >> feedback has been received over that time) would be far more sensible. > > But the merges into the core will be tough. Imagine submitting a > 5000-line patch and saying "I've worked on this for a year, please > adopt it." Will we do enough code review to assert the code quality? I can see two remedies: * The branch changes are structured in relatively short, well-reviewable commits. Less likely for a branch worked on for a year though. * A core developer is involved from the start and acts as a just-in-time reviewer. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From guido at python.org Sun Oct 25 18:21:06 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 25 Oct 2009 10:21:06 -0700 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> Message-ID: On Sun, Oct 25, 2009 at 4:56 AM, Georg Brandl wrote: > Exactly. Since with the moratorium in effect, we are basically changing > *nothing but* the stdlib, it has its own release cycle already :) Not true. There is much core work that can be done without changing the language definition (e.g. removing the GIL, speedups). Also, extension modules written in C presumably don't fall under the separate stdlib release but are excluded from the moratorium. That said a separate stdlib release has been asked for at the language summit and I am supportive. But it's link with the moratorium is minimal. -- --Guido van Rossum PS. My elbow needs a couple more weeks of rest. Limiting myself to ultra-short emails. From g.brandl at gmx.net Sun Oct 25 18:26:57 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 25 Oct 2009 18:26:57 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> Message-ID: Guido van Rossum schrieb: > On Sun, Oct 25, 2009 at 4:56 AM, Georg Brandl wrote: >> Exactly. Since with the moratorium in effect, we are basically changing >> *nothing but* the stdlib, it has its own release cycle already :) > > Not true. There is much core work that can be done without changing > the language definition (e.g. removing the GIL, speedups). Also, > extension modules written in C presumably don't fall under the > separate stdlib release but are excluded from the moratorium. I'd have counted them among the stdlib. You're of course right, other core work can and should be done, but it won't be as visible as new stdlib modules or improved APIs in there. Anyway, we'll probably get a better picture of what 3.2 will look like after the PEP is written and we have a rough release schedule. Before that it's moot to decide on a separate stdlib release. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From solipsis at pitrou.net Sun Oct 25 18:39:25 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 25 Oct 2009 17:39:25 +0000 (UTC) Subject: [Python-ideas] stdlib with its own release cycle ? References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <94bdd2610910250539l4148138bvace06287e4c67867@mail.gmail.com> Message-ID: Tarek Ziad? writes: > On Sun, Oct 25, 2009 at 12:56 PM, Georg Brandl wrote: > > > > Exactly. Since with the moratorium in effect, we are basically changing > > *nothing but* the stdlib, it has its own release cycle already :) > > Great then ! > > how long the release cycle should be then, in your opinion ? I think 12 months would be cool. Moreover, it's easy to compute :-) Antoine. From solipsis at pitrou.net Sun Oct 25 18:46:06 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 25 Oct 2009 17:46:06 +0000 (UTC) Subject: [Python-ideas] OT: external maintenance References: <4AE1CF8E.1080709@gmail.com> <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <4AE3E315.5060500@gmail.com> <4AE3F7BD.3040301@gmail.com> Message-ID: Guido van Rossum writes: > > But the merges into the core will be tough. Imagine submitting a > 5000-line patch and saying "I've worked on this for a year, please > adopt it." Will we do enough code review to assert the code quality? Pardon for hijacking the moratorium discussion, but this is the kind of problem we already have with externally-maintained modules (such as json). The way they are updated is that once in a while (once a year perhaps), their author comes with a huge patch that we should ideally review, except that it's very difficult to do so -- especially given that it's about a piece of code that we don't know well in the first place. > PS. My elbow needs a couple more weeks of rest. Limiting myself to > ultra-short emails. I hope you'll get better soon! Regards Antoine. From g.brandl at gmx.net Sun Oct 25 19:18:21 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 25 Oct 2009 19:18:21 +0100 Subject: [Python-ideas] OT: external maintenance In-Reply-To: References: <4AE1CF8E.1080709@gmail.com> <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <4AE3E315.5060500@gmail.com> <4AE3F7BD.3040301@gmail.com> Message-ID: Antoine Pitrou schrieb: > Guido van Rossum writes: >> >> But the merges into the core will be tough. Imagine submitting a >> 5000-line patch and saying "I've worked on this for a year, please >> adopt it." Will we do enough code review to assert the code quality? > > Pardon for hijacking the moratorium discussion, but this is the kind of problem > we already have with externally-maintained modules (such as json). The way they > are updated is that once in a while (once a year perhaps), their author comes > with a huge patch that we should ideally review, except that it's very difficult > to do so -- especially given that it's about a piece of code that we don't know > well in the first place. But we haven't really put a review policy into place for core commits, and since the authors of those packages all (?) are core devs, we trust them enough to commit to the core. It's different with DVCS branches created by "only" community members. >> PS. My elbow needs a couple more weeks of rest. Limiting myself to >> ultra-short emails. > > I hope you'll get better soon! +1. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ziade.tarek at gmail.com Sun Oct 25 19:43:18 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 25 Oct 2009 19:43:18 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> Message-ID: <94bdd2610910251143y1f8d9f47r570b5cca4b28489d@mail.gmail.com> Guido van Rossum schrieb: > That said a separate stdlib release has been asked for at the language > summit and I am supportive. But it's link with the moratorium is > minimal. On Sun, Oct 25, 2009 at 6:26 PM, Georg Brandl wrote: > > Anyway, we'll probably get a better picture of what 3.2 will look like > after the PEP is written and we have a rough release schedule. ?Before that > it's moot to decide on a separate stdlib release. I guess I'll wait and see then. :) Here's Distutils use case FYI. I don't know if it can be applied to other packages in the stdlib but.. : The work that is being done in Distutils is going to really speed up once the PEPs we are working on are over and hopefully accepted, and this package would benefit from a shorter cycle. The other idea would be to do extra standalone releases of Distutils, between two Python releases. I am currently doing nightly builds of the trunk that can be installed from 2.4 to 3.1 to make it easier for people to try out the current Distutils trunk. But it's just for testing purposes, because it's currently using a dirty .pth trick to shadow the existing distutils. Regards Tarek From debatem1 at gmail.com Sun Oct 25 21:08:38 2009 From: debatem1 at gmail.com (geremy condra) Date: Sun, 25 Oct 2009 16:08:38 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <4AE3F7BD.3040301@gmail.com> References: <4AE1CF8E.1080709@gmail.com> <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <4AE3E315.5060500@gmail.com> <4AE3F7BD.3040301@gmail.com> Message-ID: On Sun, Oct 25, 2009 at 3:01 AM, Nick Coghlan wrote: > geremy condra wrote: >> On Sun, Oct 25, 2009 at 1:33 AM, Nick Coghlan wrote: >> I'm a little unclear on what you're driving at here- >> my goal is to keep these projects off the minds >> of the core devs until they're about ready to go, to >> make it easy for all the implementors to carefully >> assess both the features themselves and the >> code used to implement them as they near that >> point, and to make certain that there is a common >> point of reference for the debate over inclusion >> afterwards. In other words, not specifically to >> remove python-dev from the process, but rather >> to allow it to spend its time more productively >> elsewhere until it can be usefully spent there. > > Having an "official sandbox" is completely irrelevant (and probably > counterproductive) if core development is happening in a DVCS. I don't see any evidence for either of these claims, and IMO the argument that somehow allowing people to "go play over there" until they've got something that could seriously be considered for merging is going to slow down core development is a little silly. >If people > want to experiment, they will be able to experiment by branching into > their own repository. Trying to create an "official sandbox" (or, more > likely, use the existing sandbox) defeats the whole point of the > exercise, since it just brings the CPython admins back into the loop and > will generate a pile of irrelevant traffic on python-checkins. > I don't see the sandbox working this way. I see it being somewhat more separated from the official core, probably to the point where it wouldn't generate checkin messages and hopefully to the point where it wouldn't require a dev to start a publicly viewable branch. This also avoids the problem of having to go through a kajillion different places to see what kinds of proposals have active support or are close to completion and which aren't, which should help to discourage people from duplicating effort, as well as giving the various implementors more advance warning on what might be coming down the pipe. > In short, a moratorium that said "we won't accept things into trunk, but > you can still pester us to add things to our official sandbox" would be > a pointless exercise. > You see the difficulty of occasionally reviewing a substantive, testable proposal for a language change as being enough to outweigh the advantages Guido has attributed to the moratorium? And here I was thinking that adopting a code-first-jaw-later approach would save the devs time. > Moving to a DVCS, putting a core language change moratorium in place for > a few years, then looking to see what improvements have been developed > outside the core tree as the moratorium is running down (and what > feedback has been received over that time) would be far more sensible. > That's the goal, I'd just also like to have all those improvements centralized for easy review and comparison. Geremy Condra From debatem1 at gmail.com Sun Oct 25 21:13:42 2009 From: debatem1 at gmail.com (geremy condra) Date: Sun, 25 Oct 2009 16:13:42 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <87ws2jan2l.fsf@uwakimon.sk.tsukuba.ac.jp> References: <9d153b7c0910230046y5cf14c26q19b4cd4c11ea8909@mail.gmail.com> <7d702edf0910230749g36856ad7wb5afe7c5fb533156@mail.gmail.com> <4AE1CF8E.1080709@gmail.com> <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <87ws2jan2l.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Sun, Oct 25, 2009 at 4:34 AM, Stephen J. Turnbull wrote: > geremy condra writes: > > ?> Let's just clear up a misconception: I am *for a moratorium*. > > I don't think anybody said otherwise. > > So, in the meantime, you propose a "sandbox": > > ?> We allow people to keep coming up with ideas and keep developing > ?> ideas on the off chance that one of them works so well that we want > ?> to integrate it after the moratorium lifts. > ?> There is no obligation here, no demands made on anyone's time who > ?> does not care to take a look at whats happening in the sandbox. In > ?> a few words, there is no cost. > ?> > ?> If done properly > > ... it will be work for somebody. Doing things properly always > is. ?As Nick points out, Mercurial already allows that personal > sandboxes, quite cheaply. ?Why should python.org and/or the PSF > provide such a sandbox? ?I can think of one reason: providing a place > to showcase new features in variants of the major implementations that > are in actual use in other projects, and to collect common code of the > various implementations so it can be shared. ?Maybe there's room for > some speculative features, but I don't yet see that as a primary > purpose here. > > ?> As I say, it seems like a good idea to me, and if it turns out > ?> later not to have been it has virtually no consequences. > > That would be a shame if it works out poorly partly because it's been > sold as cheap and risk-free, and as a consequence nobody puts in the > resources needed to "do it properly". > I wouldn't be advocating it if I weren't willing to put in the resources- but it won't actually work unless it has support from both the python-dev and python-ideas communities. Having said that, I do see it as being cheap and risk free because if it never sees wide use then it will simply fade into the night, another one in the thousands of places that people can publish a repository, which is, as Nick and others have pointed out, the status quo for the duration of the moratorium. Geremy Condra From debatem1 at gmail.com Sun Oct 25 21:16:22 2009 From: debatem1 at gmail.com (geremy condra) Date: Sun, 25 Oct 2009 16:16:22 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <4AE3E315.5060500@gmail.com> <4AE3F7BD.3040301@gmail.com> Message-ID: On Sun, Oct 25, 2009 at 12:29 PM, Guido van Rossum wrote: > On Sun, Oct 25, 2009 at 12:01 AM, Nick Coghlan wrote: >> Moving to a DVCS, putting a core language change moratorium in place for >> a few years, then looking to see what improvements have been developed >> outside the core tree as the moratorium is running down (and what >> feedback has been received over that time) would be far more sensible. > > But the merges into the core will be tough. Imagine submitting a > 5000-line patch and saying "I've worked on this for a year, please > adopt it." Will we do enough code review to assert the code quality? > > -- > --Guido van Rossum > > PS. My elbow needs a couple more weeks of rest. Limiting myself to > ultra-short emails. Frankly, if a feature isn't interesting enough to convince a core dev to take a few good hard looks at the code over the length of time you've proposed for the moratorium, it probably doesn't deserve to be in the language anyway. At least, that's my view. Geremy Condra From debatem1 at gmail.com Sun Oct 25 21:23:27 2009 From: debatem1 at gmail.com (geremy condra) Date: Sun, 25 Oct 2009 16:23:27 -0400 Subject: [Python-ideas] OT: external maintenance In-Reply-To: References: <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <4AE3E315.5060500@gmail.com> <4AE3F7BD.3040301@gmail.com> Message-ID: On Sun, Oct 25, 2009 at 2:18 PM, Georg Brandl wrote: > Antoine Pitrou schrieb: >> Guido van Rossum writes: >>> >>> But the merges into the core will be tough. Imagine submitting a >>> 5000-line patch and saying "I've worked on this for a year, please >>> adopt it." Will we do enough code review to assert the code quality? >> >> Pardon for hijacking the moratorium discussion, but this is the kind of problem >> we already have with externally-maintained modules (such as json). The way they >> are updated is that once in a while (once a year perhaps), their author comes >> with a huge patch that we should ideally review, except that it's very difficult >> to do so -- especially given that it's about a piece of code that we don't know >> well in the first place. > > But we haven't really put a review policy into place for core commits, and > since the authors of those packages all (?) are core devs, we trust them > enough to commit to the core. ?It's different with DVCS branches created > by "only" community members. > My suggestion then would be to review them. There's a couple of years lead time before that needs to be done, and there will probably only be a very small number of these projects that ever bear any fruit. As I said in another thread, if thats not enough time to attract some developer's interest, then the feature probably wasn't good enough to land anyway. Geremy Condra From guido at python.org Mon Oct 26 00:06:55 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 25 Oct 2009 16:06:55 -0700 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> Message-ID: On Sun, Oct 25, 2009 at 10:26 AM, Georg Brandl wrote: > Guido van Rossum schrieb: >> On Sun, Oct 25, 2009 at 4:56 AM, Georg Brandl wrote: >>> Exactly. Since with the moratorium in effect, we are basically changing >>> *nothing but* the stdlib, it has its own release cycle already :) >> >> Not true. There is much core work that can be done without changing >> the language definition (e.g. removing the GIL, speedups). Also, >> extension modules written in C presumably don't fall under the >> separate stdlib release but are excluded from the moratorium. > > I'd have counted them among the stdlib. Hmm, but C extensions don't work with other implementations (at least not with Jython, IronPython, PyPy). > You're of course right, other > core work can and should be done, but it won't be as visible as new stdlib > modules or improved APIs in there. Depends. Imagine Unladen Swallow merged in -- it would be huge from all kinds of perspectives (pro and con, I suppose) but wouldn't change the language at all. > Anyway, we'll probably get a better picture of what 3.2 will look like > after the PEP is written and we have a rough release schedule. Before that > it's moot to decide on a separate stdlib release. Hm, I think the separate-stdlib-release idea needs separate discussion quite independent of the moratorium. -- --Guido van Rossum PS. My elbow needs a couple more weeks of rest. Limiting myself to ultra-short emails. From steve at pearwood.info Mon Oct 26 00:22:44 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 26 Oct 2009 10:22:44 +1100 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <200910261022.44281.steve@pearwood.info> On Sat, 24 Oct 2009 04:30:36 pm Stephen J. Turnbull wrote: > It seems to me that what Guido is heading for here is very similar to > the "punctuated equilibrium" concept (associated with the > evolutionary biologist Stephen Jay Gould, the wikipedia article is > pretty good, and fairly short). I argue that you've got it backwards. Python has been relatively stable in the ways that matter for almost all of it's history. Nearly all the changes to the language in 1.x and 2.x have either been backwards-compatible, or managed carefully with a deprecation schedule or __future__. The result of this is that code written for Python 1.5, and possibly older, will still work in Python 2.6. It's easy enough to ignore new features if you need to support an older version. This does not apply to 3.x. If you want to see punctuated equilibrium in software development, the change from 2.x to 3.x is an good example: a relatively large change in a non-backwards compatible way, with no gentle migration path. Supporting 3.x is all-or-nothing: you can't support 3.1 and 2.6 with the same code base, except possibly for the most trivial code. I suggest that the causes of the slow uptake of 3.x isn't too many changes to the core, but three factors: (1) There's no gentle migration path from 2.x to 3.x in the same way that there have been gentle migration paths from every version to the next version in 1.x and 2.x. Instead you've got a discontinuous change. Library maintainers have to choose between: - support 2.x only - support 3.x only - maintain two incompatible code bases. The path of least effort is to support 2.x only, because they're already doing that. (2) For many people, 3.x doesn't offer any obvious compelling advantages over 2.x. (Of course, your mileage may vary.) Reading the What's New for 3.0, I see *many* nice features and cleanups, many Nice To Have new features, but no obvious Must Have to encourage me to migrate. http://docs.python.org/dev/3.0/whatsnew/3.0.html (3) It's the bootstrap problem: most systems that ship with CPython still ship with 2.x, and will continue to do so until people are regularly using 3.x, but people generally use whatever their system ships with. I don't see that a moratorium on new features will help with any of these issues. > In software, it may make sense to have the stable periods be *much* > more stable, The principle of "release early, release often" argues against that claim. The 2.x series has been very successful. Why change a successful procedure for one which is, at best, unproven, and at worst, condemns the language to the (unfair) label "moribund"? I don't believe that the experience of the C language is relevant. The C *standard* was stable, but actual C compilers evolved like mad, providing non-standard features. Because C is so low-level, the difference between a built-in and a library is very slight. > But it makes sense to propose to compress > the evolution into short periods with many changes, As has happened in the change to Python 3.x. > and have very stable periods of "moratorium" between. We keep coming back to this idea that the volume of change in a language is, in and of itself, harmful. I dispute that. It is *incompatible* change that is harmful. Developers have been slow to move to 3.x not because it's different from 2.x, but because it is different in inconveniently incompatible ways. There have been some major new features in the 2.x series, e.g. new-style classes, generators and decorators. As far as I can tell, those versions didn't suffer from lack of uptake as 3.x has suffered. It's not new features that frightens developers off, but incompatible change. -- Steven D'Aprano From brett at python.org Mon Oct 26 00:24:55 2009 From: brett at python.org (Brett Cannon) Date: Sun, 25 Oct 2009 16:24:55 -0700 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> Message-ID: On Sun, Oct 25, 2009 at 16:06, Guido van Rossum wrote: > On Sun, Oct 25, 2009 at 10:26 AM, Georg Brandl wrote: > > Guido van Rossum schrieb: > >> On Sun, Oct 25, 2009 at 4:56 AM, Georg Brandl wrote: > >>> Exactly. Since with the moratorium in effect, we are basically changing > >>> *nothing but* the stdlib, it has its own release cycle already :) > >> > >> Not true. There is much core work that can be done without changing > >> the language definition (e.g. removing the GIL, speedups). Also, > >> extension modules written in C presumably don't fall under the > >> separate stdlib release but are excluded from the moratorium. > > > > I'd have counted them among the stdlib. > > Hmm, but C extensions don't work with other implementations (at least > not with Jython, IronPython, PyPy). > > > You're of course right, other > > core work can and should be done, but it won't be as visible as new > stdlib > > modules or improved APIs in there. > > Depends. Imagine Unladen Swallow merged in -- it would be huge from > all kinds of perspectives (pro and con, I suppose) but wouldn't change > the language at all. > > > Anyway, we'll probably get a better picture of what 3.2 will look like > > after the PEP is written and we have a rough release schedule. Before > that > > it's moot to decide on a separate stdlib release. > > Hm, I think the separate-stdlib-release idea needs separate discussion > quite independent of the moratorium. This has been debated in the stdlib-sig (came up when we talked about argparse) if you want some viewpoints. It is definitely a separate discussion and I expect it will be brought up at the language summit. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at gmail.com Mon Oct 26 00:38:18 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 26 Oct 2009 00:38:18 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> Message-ID: <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> 2009/10/25 Guido van Rossum > On Sun, Oct 25, 2009 at 10:26 AM, Georg Brandl wrote: > > Guido van Rossum schrieb: > >> On Sun, Oct 25, 2009 at 4:56 AM, Georg Brandl wrote: > >>> Exactly. Since with the moratorium in effect, we are basically changing > >>> *nothing but* the stdlib, it has its own release cycle already :) > >> > >> Not true. There is much core work that can be done without changing > >> the language definition (e.g. removing the GIL, speedups). Also, > >> extension modules written in C presumably don't fall under the > >> separate stdlib release but are excluded from the moratorium. > > > > I'd have counted them among the stdlib. > > Hmm, but C extensions don't work with other implementations (at least > not with Jython, IronPython, PyPy). > > Well, you can use Python C extensions with Ironclad [1]. The maintainer hopes to port the core to Jython at some point as well. Michael Foord [1] http://code.google.com/p/ironclad/ > > You're of course right, other > > core work can and should be done, but it won't be as visible as new > stdlib > > modules or improved APIs in there. > > Depends. Imagine Unladen Swallow merged in -- it would be huge from > all kinds of perspectives (pro and con, I suppose) but wouldn't change > the language at all. > > > Anyway, we'll probably get a better picture of what 3.2 will look like > > after the PEP is written and we have a rough release schedule. Before > that > > it's moot to decide on a separate stdlib release. > > Hm, I think the separate-stdlib-release idea needs separate discussion > quite independent of the moratorium. > > -- > --Guido van Rossum > > PS. My elbow needs a couple more weeks of rest. Limiting myself to > ultra-short emails. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Oct 26 00:43:41 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 25 Oct 2009 16:43:41 -0700 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> Message-ID: On Sun, Oct 25, 2009 at 4:38 PM, Michael Foord wrote: > Well, you can use Python C extensions with Ironclad [1]. The maintainer > hopes to port the core to Jython at some point as well. What do you personally think of Ironclad? And (separately) of those hopes? > Michael Foord > > [1] http://code.google.com/p/ironclad/ -- --Guido van Rossum PS. My elbow needs a couple more weeks of rest. Limiting myself to ultra-short emails. From fuzzyman at gmail.com Mon Oct 26 01:12:25 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 26 Oct 2009 01:12:25 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> Message-ID: <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> 2009/10/25 Guido van Rossum > On Sun, Oct 25, 2009 at 4:38 PM, Michael Foord wrote: > > Well, you can use Python C extensions with Ironclad [1]. The maintainer > > hopes to port the core to Jython at some point as well. > > What do you personally think of Ironclad? And (separately) of those hopes? > > It seems like it ought to be an impossible task - reimplementing the Python C API for another platform. It actually runs surprisingly well (astonishingly) and I know of at least one bank in London now using it in production. The Ironclad implementation reuses the Python C source wherever possible in order to minimise the core that actually needs implementing. A *large* number of the Numpy and SciPy tests pass with it (~1000 of each last time I checked) and *generally* performance is pretty good. I'd like to see Ironclad in wider use. The hopes of the Ironclad maintainer to reimplement the core for Jython is certainly *plausible*, but it of course depends on him finding time in the future. Personally when I write IronPython code I try to avoid a dependency on C extension modules as it seems to me that the *point* of IronPython is to make use of the .NET framework (otherwise you might as well just use CPython). Where Ironclad is being used is where people want to interface existing Python systems to existing .NET systems and that makes a lot of sense (you'd rather avoid rewriting chunks of either if you can and Ironclad acts as a bridge). All the best, Michael > > Michael Foord > > > > [1] http://code.google.com/p/ironclad/ > > -- > --Guido van Rossum > > PS. My elbow needs a couple more weeks of rest. Limiting myself to > ultra-short emails. > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Oct 26 01:51:20 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 25 Oct 2009 17:51:20 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <200910261022.44281.steve@pearwood.info> References: <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <200910261022.44281.steve@pearwood.info> Message-ID: On Sun, Oct 25, 2009 at 4:22 PM, Steven D'Aprano wrote: > On Sat, 24 Oct 2009 04:30:36 pm Stephen J. Turnbull wrote: >> It seems to me that what Guido is heading for here is very similar to >> the "punctuated equilibrium" concept (associated with the >> evolutionary biologist Stephen Jay Gould, the wikipedia article is >> pretty good, and fairly short). > > I argue that you've got it backwards. [...] Much though I like the writing of Gould and others, and much though we talk about Python "evolving", I don't think that punctuated equilibrium makes sense as an analogy, no matter which way to turn it. The proposed moratorium is a *conscious decision*, an intentional policy meant to have a certain effect. This is just the opposite of evolution in nature (unless you believe in "intelligent design" :-). [...] > I suggest that the causes of the slow uptake of 3.x isn't too many > changes to the core, but three factors: While I mentioned 3.x in my original message about the moratorium, I didn't mean to imply that the moratorium would solve the slow uptake directly. The intent was to give people who would otherwise work on language change proposals more time and motivation to work on porting 3rd party packages to Py3k. Also to give other implementations (in particular PyPy, IronPython, Jython) more breathing room to catch up. Also to stop the pointless discussions about anonymous blocks (though that may have been naive :-). -- --Guido van Rossum PS. My elbow needs a couple more weeks of rest. Limiting myself to ultra-short emails. From guido at python.org Mon Oct 26 01:56:26 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 25 Oct 2009 17:56:26 -0700 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> Message-ID: On Sun, Oct 25, 2009 at 5:12 PM, Michael Foord wrote: > > > 2009/10/25 Guido van Rossum >> >> On Sun, Oct 25, 2009 at 4:38 PM, Michael Foord wrote: >> > Well, you can use Python C extensions with Ironclad [1]. The maintainer >> > hopes to port the core to Jython at some point as well. >> >> What do you personally think of Ironclad? And (separately) of those hopes? >> > > It seems like it ought to be an impossible task - reimplementing the Python > C API for another platform. It actually runs surprisingly well > (astonishingly) and I know of at least one bank in London now using it in > production. The Ironclad implementation reuses the Python C source wherever > possible in order to minimise the core that actually needs implementing. > > A *large* number of the Numpy and SciPy tests pass with it (~1000 of each > last time I checked) and *generally* performance is pretty good. I'd like to > see Ironclad in wider use. > > The hopes of the Ironclad maintainer to reimplement the core for Jython is > certainly *plausible*, but it of course depends on him finding time in the > future. > > Personally when I write IronPython code I try to avoid a dependency on C > extension modules as it seems to me that the *point* of IronPython is to > make use of the .NET framework (otherwise you might as well just use > CPython). Where Ironclad is being used is where people want to interface > existing Python systems to existing .NET systems and that makes a lot of > sense (you'd rather avoid rewriting chunks of either if you can and Ironclad > acts as a bridge). Hm. In the Java world, there are many target environments where depending on C extensions would not fly at all, since the only commonality between platforms is the JVM interface. (And sometimes not even that -- e.g. Android uses a different VM to run Java.) I'm guessing that in the .NET world this is much less of an issue, since (a) the platform is more homogeneous and under control of Microsoft (Mono notwithstanding) and (b) .NET explicitly targets other languages (though it prefers "managed C++", it supports plain C++). Have I got this right? I see the prospects for an IronClad-like thing in the Java world as pretty slim because of this. > Michael >> > [1] http://code.google.com/p/ironclad/ -- --Guido van Rossum PS. My elbow needs a couple more weeks of rest. Limiting myself to ultra-short emails. From debatem1 at gmail.com Mon Oct 26 02:14:40 2009 From: debatem1 at gmail.com (geremy condra) Date: Sun, 25 Oct 2009 21:14:40 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <200910261022.44281.steve@pearwood.info> Message-ID: On Sun, Oct 25, 2009 at 8:51 PM, Guido van Rossum wrote: > Also to stop the pointless discussions about anonymous blocks (though > that may have been naive :-). > > -- > --Guido van Rossum Tell everybody that proposes it to show you the code. 99% of them will STFU, and it's probably worth a little bit of time to see what the other 1% come up with, even if its never going to get in. Worst case scenario, you've spared yourself the headache of dealing with people who can't (or won't) help you implement what they so urgently desire. Best case scenario, maybe you even get a great feature out of it. Geremy Condra From debatem1 at gmail.com Mon Oct 26 02:21:22 2009 From: debatem1 at gmail.com (geremy condra) Date: Sun, 25 Oct 2009 21:21:22 -0400 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> Message-ID: On Sun, Oct 25, 2009 at 8:56 PM, Guido van Rossum wrote: > On Sun, Oct 25, 2009 at 5:12 PM, Michael Foord wrote: >> >> >> 2009/10/25 Guido van Rossum >>> >>> On Sun, Oct 25, 2009 at 4:38 PM, Michael Foord wrote: >>> > Well, you can use Python C extensions with Ironclad [1]. The maintainer >>> > hopes to port the core to Jython at some point as well. >>> >>> What do you personally think of Ironclad? And (separately) of those hopes? >>> >> >> It seems like it ought to be an impossible task - reimplementing the Python >> C API for another platform. It actually runs surprisingly well >> (astonishingly) and I know of at least one bank in London now using it in >> production. The Ironclad implementation reuses the Python C source wherever >> possible in order to minimise the core that actually needs implementing. >> >> A *large* number of the Numpy and SciPy tests pass with it (~1000 of each >> last time I checked) and *generally* performance is pretty good. I'd like to >> see Ironclad in wider use. >> >> The hopes of the Ironclad maintainer to reimplement the core for Jython is >> certainly *plausible*, but it of course depends on him finding time in the >> future. >> >> Personally when I write IronPython code I try to avoid a dependency on C >> extension modules as it seems to me that the *point* of IronPython is to >> make use of the .NET framework (otherwise you might as well just use >> CPython). Where Ironclad is being used is where people want to interface >> existing Python systems to existing .NET systems and that makes a lot of >> sense (you'd rather avoid rewriting chunks of either if you can and Ironclad >> acts as a bridge). > > Hm. In the Java world, there are many target environments where > depending on C extensions would not fly at all, since the only > commonality between platforms is the JVM interface. (And sometimes not > even that -- e.g. Android uses a different VM to run Java.) I'm > guessing that in the .NET world this is much less of an issue, since > (a) the platform is more homogeneous and under control of Microsoft > (Mono notwithstanding) and (b) .NET explicitly targets other languages > (though it prefers "managed C++", it supports plain C++). Have I got > this right? I see the prospects for an IronClad-like thing in the Java > world as pretty slim because of this. > >> Michael You've always got the JNI. I wrote the better part of a system to wrap Android's Java libraries for Python using it last summer, and although I stopped when I had enough working to use the Android components I needed from Python, I think it would have been comparatively easy to finish the job. Geremy Condra From stephen at xemacs.org Mon Oct 26 04:28:12 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 26 Oct 2009 12:28:12 +0900 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <4AE1CF8E.1080709@gmail.com> <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <4AE3E315.5060500@gmail.com> <4AE3F7BD.3040301@gmail.com> Message-ID: <87hbtmal6b.fsf@uwakimon.sk.tsukuba.ac.jp> geremy condra writes: > > Having an "official sandbox" is completely irrelevant (and probably > > counterproductive) if core development is happening in a DVCS. > > I don't see any evidence for either of these claims, Given the degree of decoupling you describe, there's no need for Python to host anything more than a simple CGI for registering remote branches. Probably for the purpose we have in mind the CGI should check that it actually *is* a branch of an official Python branch. > and IMO the argument that somehow allowing people to "go play over > there" until they've got something that could seriously be > considered for merging is going to slow down core development is a > little silly. It's a waste of their effort to "play over there" in hopes that their feature will ever be considered. If it's small enough to be considered and approved without a major use case, it probably can be maintained as a patch on the tracker indefinitely. > This also avoids the problem of having to go through a kajillion > different places to see what kinds of proposals have active support > or are close to completion and which aren't, This is a chunk of management effort to do properly. Take a look at git.kernel.org and tell me which of those branches matter, and to whom. Unless you're a kernel activist, I bet you cannot. > You see the difficulty of occasionally reviewing a substantive, > testable proposal for a language change as being enough to > outweigh the advantages Guido has attributed to the moratorium? > And here I was thinking that adopting a code-first-jaw-later > approach would save the devs time. It would. But consider the PEP for ipaddr. That discussion was a total waste of everybody's time because the proponent never put forward a use case in working code for the most controversial feature, making it impossible for anyone to figure out WTF he was thinking. It turns out he wasn't thinking about anything at all, the "feature" was merely an artifact of his implementation, and he didn't understand what others were talking about. When he finally did get around to thinking about what they were talking about, he immediately accepted a sensible proposal, the compromise was accepted with some grumbling, and work can proceed. I forsee lots and lots of those if the only code to review is the implementation of the feature in the officious sandbox. Much better if we can look at the big picture in the context of a real app developed wa-a-ay over there, and a sub-repo containing the relevant branch of Python core. Since it *is* a branch of Python core, you just clone the core, and do "hg merge" and you've got a reviewable implementation. > That's the goal, I'd just also like to have all those improvements > centralized for easy review and comparison. There will be way too many of them to be reviewed at all, let alone easily, if all people have to do is post a branch to an official sandbox. From stephen at xemacs.org Mon Oct 26 05:07:42 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 26 Oct 2009 13:07:42 +0900 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <200910261022.44281.steve@pearwood.info> Message-ID: <87fx96ajch.fsf@uwakimon.sk.tsukuba.ac.jp> Guido van Rossum writes: > The proposed moratorium is a *conscious decision*, an intentional > policy meant to have a certain effect. This is just the opposite of > evolution in nature (unless you believe in "intelligent design" :-). By Clarke's Law, I see no way to distinguish between the presence and the absence of intelligent design in nature. > While I mentioned 3.x in my original message about the moratorium, I > didn't mean to imply that the moratorium would solve the slow uptake > directly. The intent was to give people who would otherwise work on > language change proposals more time and motivation to work on porting > 3rd party packages to Py3k. Time, yes, but I'm not sure I see where the motivation comes from. Do you mean something like: remove all hope from "design + maybe implementation" proposals, with the intent of encouraging "design + implementation + clear application to use case" proposals. where porting to Py3k should be a prolific source of use cases? ISTM that has been the desired historical criterion for inclusion in Python anyway. So a moratorium might reduce the number of "frivolous" proposals, but is it really going to encourage work on porting? (Those are all really yes/no questions from my standpoint.) If not, Steven d'Aprano's line that Python has been quite stable anyway, up to Python 3, and so (IIU him C) there's really no need for a formal moratorium, becomes very plausible. That is, since there doesn't seem to be a strong call for a moratorium from the Jython / Cython / IronPython / PyPy end. (So far, that it. I don't suppose the final word has been spoken by those developers, yet.) > PS. My elbow needs a couple more weeks of rest. Limiting myself to > ultra-short emails. May the Intelligent Designer have mercy on your elbow! From debatem1 at gmail.com Mon Oct 26 05:02:25 2009 From: debatem1 at gmail.com (geremy condra) Date: Mon, 26 Oct 2009 00:02:25 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <87hbtmal6b.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <4AE3E315.5060500@gmail.com> <4AE3F7BD.3040301@gmail.com> <87hbtmal6b.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Sun, Oct 25, 2009 at 11:28 PM, Stephen J. Turnbull wrote: > geremy condra writes: > > ?> > Having an "official sandbox" is completely irrelevant (and probably > ?> > counterproductive) if core development is happening in a DVCS. > ?> > ?> I don't see any evidence for either of these claims, > > Given the degree of decoupling you describe, there's no need for > Python to host anything more than a simple CGI for registering remote > branches. ?Probably for the purpose we have in mind the CGI should > check that it actually *is* a branch of an official Python branch. > Hadn't thought of that. I'd prefer that there be a hosting facility involved, but it's a long way from being critical- assuming the other objections to the sandbox could be settled, this would get a +1 from me. > ?> and IMO the argument that somehow allowing people to "go play over > ?> there" until they've got something that could seriously be > ?> considered for merging is going to slow down core development is a > ?> little silly. > > It's a waste of their effort to "play over there" in hopes that their > feature will ever be considered. ?If it's small enough to be > considered and approved without a major use case, it probably can be > maintained as a patch on the tracker indefinitely. > There is an enormous risk that they are wasting their time. The goal is to save the core devs' time, not those proposing features. > ?> This also avoids the problem of having to go through a kajillion > ?> different places to see what kinds of proposals have active support > ?> or are close to completion and which aren't, > > This is a chunk of management effort to do properly. ?Take a look at > git.kernel.org and tell me which of those branches matter, and to whom. > Unless you're a kernel activist, I bet you cannot. > I don't envision this ever reaching the scale of the kernel, and if you feel there is any risk of it doing so then I suggest we implement it ASAP, since there are evidently thousands of heretofore unseen core developers with a curious aversion to centralized version control systems. > ?> You see the difficulty of occasionally reviewing a substantive, > ?> testable proposal for a language change as being enough to > ?> outweigh the advantages Guido has attributed to the moratorium? > ?> And here I was thinking that adopting a code-first-jaw-later > ?> approach would save the devs time. > > It would. ?But consider the PEP for ipaddr. ?That discussion was a > total waste of everybody's time because the proponent never put > forward a use case in working code for the most controversial feature, > making it impossible for anyone to figure out WTF he was thinking. ?It > turns out he wasn't thinking about anything at all, the "feature" was > merely an artifact of his implementation, and he didn't understand > what others were talking about. ?When he finally did get around to > thinking about what they were talking about, he immediately accepted a > sensible proposal, the compromise was accepted with some grumbling, > and work can proceed. > I honestly don't see what this has to do with this proposal. 1) ipaddr made no effort to change the language itself, and would in any event be eligible for consideration under the normal rules during the course of the moratorium. 2) the situation was complicated by recriminations about the openness of python, the role of the mailinglists vs the bugtracker, and numerous other issues that we don't need to go into here. That such a wide-ranging discussion would cause consternation should surprise no one. 3) According to people who know a great deal more about netmasks than I ever will, the things the author was unaware of would fill several large books, and even Guido repeatedly emphasized his lack of knowledge on the subject. By contrast, the group of core Python developers is uniquely qualified to talk about changes to the language's syntax and behavior. > I forsee lots and lots of those if the only code to review is the > implementation of the feature in the officious sandbox. ?Much better > if we can look at the big picture in the context of a real app > developed wa-a-ay over there, and a sub-repo containing the relevant > branch of Python core. ?Since it *is* a branch of Python core, you > just clone the core, and do "hg merge" and you've got a reviewable > implementation. We're talking about changes to the core language. I would suggest that the only "apps" possible under those conditions would be extensive test suites and real-life use. Realistically, we can only have the former without large-scale acceptance of a program similar to, but larger in scope than, the one I am proposing. > ?> That's the goal, I'd just also like to have all those improvements > ?> centralized for easy review and comparison. > > There will be way too many of them to be reviewed at all, let alone > easily, if all people have to do is post a branch to an official > sandbox. > I doubt this. Experience suggests that the number of individuals proposing changes with the technical chops to pull them off is relatively small, and my own admittedly limited familiarity with human nature leads me to conclude that the formidable obstacles piled along the road to acceptance will discourage all but the most determined proposals. Geremy Condra From ben+python at benfinney.id.au Mon Oct 26 06:20:06 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Mon, 26 Oct 2009 16:20:06 +1100 Subject: [Python-ideas] Proposal: Moratorium on Python language changes References: <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <200910261022.44281.steve@pearwood.info> <87fx96ajch.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87ocnu4tq1.fsf@benfinney.id.au> "Stephen J. Turnbull" writes: > Guido van Rossum writes: > > PS. My elbow needs a couple more weeks of rest. Limiting myself to > > ultra-short emails. > > May the Intelligent Designer have mercy on your elbow! Hold up there. I thought the intelligent designer *is* Guido? -- \ ?Intellectual property is to the 21st century what the slave | `\ trade was to the 16th.? ?David Mertz | _o__) | Ben Finney From stephen at xemacs.org Mon Oct 26 07:12:22 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 26 Oct 2009 15:12:22 +0900 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <4AE3E315.5060500@gmail.com> <4AE3F7BD.3040301@gmail.com> <87hbtmal6b.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <8763a2adkp.fsf@uwakimon.sk.tsukuba.ac.jp> geremy condra writes: > There is an enormous risk that they are wasting their time. The > goal [of a sandbox] is to save the core devs' time, not those > proposing features. The core devs' time will be saved by ignoring the sandbox (other than their own features), and if it intrudes on them (eg, in the commit notices), they will request that it forcibly be shut up. Cf. the short exchange you had with Nick on the subject of commit notices. ISTM that rather the goal is to ensure that features that would otherwise be overlooked or be blocked because the proponent is unwilling to follow through to the extent required (ie, by a new proposal at the time of ending the moratorium) be given more consideration. I don't think a sandbox will help with that, unless it is designed to give incentives and help in producing *better* proposals. Just collecting them in one place is not going to help. > > It would. ?But consider the PEP for ipaddr. > > I honestly don't see what this has to do with this proposal. It's a common pattern of discussion, for both stdlib and core syntax/builtins, and I chose it because it was close to the boundary of approval. I didn't want to choose nonstarters like multiline blocks or trivial suggestions like a warning for "breakless for-else". > 2) the situation was complicated by recriminations about > the openness of python, the role of the mailinglists vs > the bugtracker, and numerous other issues that we don't > need to go into here. Are you sure you're not confusing it with the distutils thread(s)? > 3) According to people who know a great deal more about > netmasks than I ever will, the things the author was > unaware of would fill several large books, and even Guido > repeatedly emphasized his lack of knowledge on the subject. > By contrast, the group of core Python developers is > uniquely qualified to talk about changes to the language's > syntax and behavior. But those are precisely the people whose time you plan to save. The only way to save their time is to do less reviewing. Having an official sandbox might do that by giving the dross a place to collect, so the core devs *never* look at it. > We're talking about changes to the core language. I would suggest > that the only "apps" possible under those conditions would be > extensive test suites and real-life use. Yup. Precisely the kind of stuff that happens in real projects, not in sandboxes. > > There will be way too many of them to be reviewed at all, let alone > > easily, if all people have to do is post a branch to an official > > sandbox. > > I doubt this. Experience suggests that the number of individuals > proposing changes with the technical chops to pull them off is > relatively small, Sure, but they generally also are core developers. I don't see why an "official sandbox" will be of use to them. From fuzzyman at gmail.com Mon Oct 26 10:47:43 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 26 Oct 2009 10:47:43 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> Message-ID: <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> 2009/10/26 Guido van Rossum > On Sun, Oct 25, 2009 at 5:12 PM, Michael Foord wrote: > > > > > > 2009/10/25 Guido van Rossum > >> > >> On Sun, Oct 25, 2009 at 4:38 PM, Michael Foord > wrote: > >> > Well, you can use Python C extensions with Ironclad [1]. The > maintainer > >> > hopes to port the core to Jython at some point as well. > >> > >> What do you personally think of Ironclad? And (separately) of those > hopes? > >> > > > > It seems like it ought to be an impossible task - reimplementing the > Python > > C API for another platform. It actually runs surprisingly well > > (astonishingly) and I know of at least one bank in London now using it in > > production. The Ironclad implementation reuses the Python C source > wherever > > possible in order to minimise the core that actually needs implementing. > > > > A *large* number of the Numpy and SciPy tests pass with it (~1000 of each > > last time I checked) and *generally* performance is pretty good. I'd like > to > > see Ironclad in wider use. > > > > The hopes of the Ironclad maintainer to reimplement the core for Jython > is > > certainly *plausible*, but it of course depends on him finding time in > the > > future. > > > > Personally when I write IronPython code I try to avoid a dependency on C > > extension modules as it seems to me that the *point* of IronPython is to > > make use of the .NET framework (otherwise you might as well just use > > CPython). Where Ironclad is being used is where people want to interface > > existing Python systems to existing .NET systems and that makes a lot of > > sense (you'd rather avoid rewriting chunks of either if you can and > Ironclad > > acts as a bridge). > > Hm. In the Java world, there are many target environments where > depending on C extensions would not fly at all, since the only > commonality between platforms is the JVM interface. (And sometimes not > even that -- e.g. Android uses a different VM to run Java.) I'm > guessing that in the .NET world this is much less of an issue, since > (a) the platform is more homogeneous and under control of Microsoft > (Mono notwithstanding) and (b) .NET explicitly targets other languages > (though it prefers "managed C++", it supports plain C++). Have I got > this right? I see the prospects for an IronClad-like thing in the Java > world as pretty slim because of this. > Hehe, you actually raise several points in that short email. I'll address them one by one and try not to ramble too much. Firstly, although you are correct that .NET supports a managed variant of C++ (that runs 'on .NET') and it is the same set of tools that you also use to compile native code (unmanaged C/C++) this has nothing to do with .NET. Python for Windows is compiled with the Visual C++ compiler but it doesn't run on .NET. .NET doesn't even use the MSVCRT that compiled native code links against - something that causes Ironclad 'difficulties' when managed and native code need to share file handles. Ironclad itself has binary compatibility with Python C extensions, they don't need to be recompiled. It uses the .NET FFI (P/Invoke) to work with these extensions and on the JVM would use its FFI. My understanding is that Android now allows native code, so if Dalvik has the same FFI APIs and you can compile the Python extensions for it *and* Jython runs on Dalvik (not currently the case I believe?) then it could work... The .NET framework, even leaving aside Mono, is also diverse. There are several versions of the .NET framework: .NET CE (Compact Edition) that runs on mobile devices and the XBox 360, CoreCLR that runs on Silverlight and an embedded version. There is also a Mono application called MonoTouch that compiles .NET code for the iPhone (using Ahead Of Time compilation to generate native code). The embedded version will never support IronPython. The compact edition of .NET currently lacks the System.Reflection.Emit (code generation APIs) that IronPython uses. You aren't supposed to use interpreters on the iPhone so likewise System.Reflection.Emit can't be compiled with MonoTouch. I mention them because the Microsoft dynamic languages team are looking at a cut down version of IronPython (with some of the more dynamic features restricted) that can run on .NET CE and could be compiled with MonoTouch. It is possible that Ironclad could work with this. The CoreCLR in Silverlight does run IronPython but because of the sandboxing you can't call into native code, so you can't use Ironclad in the browser. Despite the places where Ironclad doesn't work, *most* .NET code runs on desktops or servers using the standard .NET framework - and this is where Ironclad is used. Despite the variety of different JVMs this is an analogous situation and I don't see why a port wouldn't be just as useful. As for Mono; Ironclad *currently* only works on .NET and on Windows. This is because it was originally created by Resolver Systems who only develop for Windows. There was *some* work by Seo Sanghyeon to port Ironclad to Mono, and there is nothing about the techniques Ironclad uses that would prevent it working on Mono. (Seo got distracted and the work was uncompleted but we're looking for volunteers to pick up porting Ironclad to Mono.) All the best, Michael Foord > > > Michael > > >> > [1] http://code.google.com/p/ironclad/ > > -- > --Guido van Rossum > > PS. My elbow needs a couple more weeks of rest. Limiting myself to > ultra-short emails. > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjamin at python.org Mon Oct 26 15:50:57 2009 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 26 Oct 2009 14:50:57 +0000 (UTC) Subject: [Python-ideas] stdlib with its own release cycle ? References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <94bdd2610910250539l4148138bvace06287e4c67867@mail.gmail.com> Message-ID: Tarek Ziad? writes: > how long the release cycle should be then, in your opinion ? I like a year. From guido at python.org Mon Oct 26 15:59:00 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 26 Oct 2009 07:59:00 -0700 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <87fx96ajch.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <200910261022.44281.steve@pearwood.info> <87fx96ajch.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Sun, Oct 25, 2009 at 9:07 PM, Stephen J. Turnbull wrote: > Guido van Rossum writes: > > ?> The proposed moratorium is a *conscious decision*, an intentional > ?> policy meant to have a certain effect. This is just the opposite of > ?> evolution in nature (unless you believe in "intelligent design" :-). > > By Clarke's Law, I see no way to distinguish between the presence and > the absence of intelligent design in nature. So using Occam's razor we might as well assume there is none. > ?> While I mentioned 3.x in my original message about the moratorium, I > ?> didn't mean to imply that the moratorium would solve the slow uptake > ?> directly. The intent was to give people who would otherwise work on > ?> language change proposals more time and motivation to work on porting > ?> 3rd party packages to Py3k. > > Time, yes, but I'm not sure I see where the motivation comes from. ?Do > you mean something like: > > ? ?remove all hope from "design + maybe implementation" proposals, > ? ?with the intent of encouraging "design + implementation + clear > ? ?application to use case" proposals. > > where porting to Py3k should be a prolific source of use cases? ?ISTM > that has been the desired historical criterion for inclusion in Python > anyway. ?So a moratorium might reduce the number of "frivolous" > proposals, but is it really going to encourage work on porting? > (Those are all really yes/no questions from my standpoint.) But hard to answer. Have faith. :-) > If not, Steven d'Aprano's line that Python has been quite stable > anyway, up to Python 3, and so (IIU him C) there's really no need for > a formal moratorium, becomes very plausible. We're seeing an influx of new developers. They don't all remember or understand why we had to limit releases to once per 18+ months. The moratorium reminds them and their fans of the importance of *not* adding new features. > That is, since there > doesn't seem to be a strong call for a moratorium from the Jython / > Cython / IronPython / PyPy end. ?(So far, that it. ?I don't suppose > the final word has been spoken by those developers, yet.) > > ?> PS. My elbow needs a couple more weeks of rest. Limiting myself to > ?> ultra-short emails. > > May the Intelligent Designer have mercy on your elbow! I trust my doctor a little more. -- --Guido van Rossum PS. My elbow needs a couple more weeks of rest. Limiting myself to ultra-short emails. From jnoller at gmail.com Mon Oct 26 16:09:22 2009 From: jnoller at gmail.com (Jesse Noller) Date: Mon, 26 Oct 2009 11:09:22 -0400 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <94bdd2610910250539l4148138bvace06287e4c67867@mail.gmail.com> Message-ID: <4222a8490910260809g3ecdcc4eq71933b39119746fe@mail.gmail.com> On Mon, Oct 26, 2009 at 10:50 AM, Benjamin Peterson wrote: > Tarek Ziad? writes: >> how long the release cycle should be then, in your opinion ? > > I like a year. +1 From fuzzyman at gmail.com Mon Oct 26 16:10:05 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 26 Oct 2009 16:10:05 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <94bdd2610910250539l4148138bvace06287e4c67867@mail.gmail.com> Message-ID: <6f4025010910260810n4e607a31h7c139d97ba4ac429@mail.gmail.com> 2009/10/26 Benjamin Peterson > Tarek Ziad? writes: > > how long the release cycle should be then, in your opinion ? > > I like a year. > +1 > > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Oct 26 17:40:27 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 26 Oct 2009 09:40:27 -0700 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <94bdd2610910250539l4148138bvace06287e4c67867@mail.gmail.com> Message-ID: On Sun, Oct 25, 2009 at 10:39 AM, Antoine Pitrou wrote: >> how long the release cycle should be then, in your opinion ? > > I think 12 months would be cool. Moreover, it's easy to compute :-) There are very good reasons we currently have releases 18+ months apart (it wasn't always this way; others can explain). I don't think those reasons have gone stale or are void given the moratorium. -- --Guido van Rossum PS. My elbow needs a couple more weeks of rest. Limiting myself to ultra-short emails. From guido at python.org Mon Oct 26 17:50:35 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 26 Oct 2009 09:50:35 -0700 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> Message-ID: On Mon, Oct 26, 2009 at 2:47 AM, Michael Foord wrote: > Ironclad itself has binary compatibility with Python C extensions, they > don't need to be recompiled. It uses the .NET FFI (P/Invoke) to work with > these extensions and on the JVM would use its FFI. Ok, but that assumes the C extensions are already compiled for CPython on that platform (presumably .NET). This sounds limiting, and hacky (need lots of faith in the DLL loader). I also assume that calls back into the CPython runtime are somehow intercepted and somewhat limited to what is reasonable to support in the .NET world -- unless you basically link all of CPython in and then you might as well just make RPCs to CPython... > My understanding is that Android now allows native code, Theoretically, yes. I don't think it's a good use of the platform though, you get huge binaries, long startup times, and quite limited functionality. Check out "Android Scripting Environment". > so if Dalvik has > the same FFI APIs and you can compile the Python extensions for it *and* > Jython runs on Dalvik (not currently the case I believe?) Not yet, and not easy (the bytecode generator would have to be retargeted and probably a bunch of unsupported Java features would have to be worked around). Phones are slow and small... > then it could work... > The .NET framework, even leaving aside Mono, is also diverse. There are > several versions of the .NET framework: .NET CE (Compact Edition) that runs > on mobile devices and the XBox 360, CoreCLR that runs on Silverlight and an > embedded version. There is also a Mono application called MonoTouch that > compiles .NET code for the iPhone (using Ahead Of Time compilation to > generate native code). How hard was it to get IronPython to work on all? How much of the libraries work on all? Does Ironclad work on all? > The embedded version will never support IronPython. The compact edition of > .NET currently lacks the System.Reflection.Emit (code generation APIs) that > IronPython uses. You aren't supposed to use interpreters on the iPhone so > likewise System.Reflection.Emit can't be compiled with MonoTouch. I mention > them because the Microsoft dynamic languages team are looking at a cut down > version of IronPython (with some of the more dynamic features restricted) > that can run on .NET CE and could be compiled with MonoTouch. It is possible > that Ironclad could work with this. Forget I even asked. :-) > The CoreCLR in Silverlight does run IronPython but because of the sandboxing > you can't call into native code, so you can't use Ironclad in the browser. > > Despite the places where Ironclad doesn't work, *most* .NET code runs on > desktops or servers using the standard .NET framework - and this is where > Ironclad is used. Despite the variety of different JVMs this is an analogous > situation and I don't see why a port wouldn't be just as useful. Why not just use CPython? At Google we mix Python and Java using RPCs intead of mixing runtimes. It works quite well. > As for Mono; Ironclad *currently* only works on .NET and on Windows. This is > because it was originally created by Resolver Systems who only develop for > Windows. There was *some* work by Seo Sanghyeon to port Ironclad to Mono, > and there is nothing about the techniques Ironclad uses that would prevent > it working on Mono. (Seo got distracted and the work was uncompleted but > we're looking for volunteers to pick up porting Ironclad to Mono.) >> >> > [1] http://code.google.com/p/ironclad/ -- --Guido van Rossum PS. My elbow needs a couple more weeks of rest. Limiting myself to ultra-short emails. From alexander.belopolsky at gmail.com Mon Oct 26 17:54:37 2009 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 26 Oct 2009 12:54:37 -0400 Subject: [Python-ideas] One obvious way to do interning [Was: Retrieve an arbitrary element from a set without removing it] Message-ID: Changing the subject to reflect branched discussion and forwarding to python-ideas where it probably belongs. On Mon, Oct 26, 2009 at 12:02 PM, Terry Reedy wrote: > Alexander Belopolsky wrote: > >> Here is an alternative idea on how storing interned objects in a set >> can be supported. Currently set.add method returns None and has no >> effect when set already has an object equal to the one being added. I >> propose to consider changing that behavior to make set.add return the >> added object or the set member that is equal to the object being >> added. It is unlikely that many programs rely on the return value >> being None (with doctests being a probable exception), so adding this >> feature is unlikely to cause much grief. > > I had exactly the same idea, but did not post because it violates the > general rule that mutators return None. Is there such a rule? What about set/dict pop? > On the other hand, the returned > value here would not be the mutated collection, so no chaining is possible. I assume you refer to chaining as in s.add(1).add(2) which would be enabled if s.add(..) returned s. My proposal would enable a different type of "chaining" which I would find useful, but ready to hear objections: v = compute_value() s.add(v) # use v can, with my proposal, be rewritten as v = s.add(compute_value()) with an added benefit that v that is used is the "canonical" value. > And 'add' is clearly intended to change something. > Is this an argument for or against the proposal? > On the other hand, frozensets do not have an add method. However, PySet_Add "works with frozenset instances (like PyTuple_SetItem() it can be used to fill-in the values of brand new frozensets before they are exposed to other code). " I will experiment with changing PySet_Add to see how much it would break and whether it will be helpful in implementing set-based interning of python strings. From fuzzyman at gmail.com Mon Oct 26 18:20:19 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 26 Oct 2009 18:20:19 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> Message-ID: <6f4025010910261020q1044d0d4r9321fa1e8e6ee8e3@mail.gmail.com> 2009/10/26 Guido van Rossum > On Mon, Oct 26, 2009 at 2:47 AM, Michael Foord wrote: > > Ironclad itself has binary compatibility with Python C extensions, they > > don't need to be recompiled. It uses the .NET FFI (P/Invoke) to work with > > these extensions and on the JVM would use its FFI. > > Ok, but that assumes the C extensions are already compiled for CPython > on that platform (presumably .NET). It uses the standard compiled binaries for CPython on Windows. Binary compatibility - so no need to compile a special version, just use the same binaries you use for CPython. > This sounds limiting, and hacky > (need lots of faith in the DLL loader). It needs an *understanding* of the DLL loader sure. You seem to be relying a lot on faith these days (well - two emails today anyway). > I also assume that calls back > into the CPython runtime are somehow intercepted and somewhat limited > to what is reasonable to support in the .NET world -- unless you > basically link all of CPython in and then you might as well just make > RPCs to CPython... > The CPython API is implemented so that calls to the API go through our code. This works in both directions (C extensions calling back into Python and Python calling into C extensions). Where the Python source code has C API functions that just call other functions we can reuse a lot of the CPython implementation, yes. This C code is running natively - it is *not* running on .NET, but where we need the bridge we use the FFI. The API functions that, for example, create a new type create an IronPython type instead of a CPython type though. This is the whole raison d'etre of Ironclad. It isn't limiting, what it gives you is the ability to use existing code that depends on C extensions without having to write your own wrapper layer or use RPC. So far there has been very little that has proved impossible with this approach. > > > My understanding is that Android now allows native code, > > Theoretically, yes. I don't think it's a good use of the platform > though, you get huge binaries, long startup times, and quite limited > functionality. Check out "Android Scripting Environment". > Right. *Most* Java code is not running on Android though (in the same way as most .NET code isn't running on .NET CE), so it could be useful if people wanted it. > > > so if Dalvik has > > the same FFI APIs and you can compile the Python extensions for it *and* > > Jython runs on Dalvik (not currently the case I believe?) > > Not yet, and not easy (the bytecode generator would have to be > retargeted and probably a bunch of unsupported Java features would > have to be worked around). > > Phones are slow and small... > > A lot of modern phones have 500mhz ARM processors and 128Mbyte RAM. Python was running on desktop devices a lot slower and 'smaller' not very long ago. :-) [snip...] > > > The CoreCLR in Silverlight does run IronPython but because of the > sandboxing > > you can't call into native code, so you can't use Ironclad in the > browser. > > > Despite the places where Ironclad doesn't work, *most* .NET code runs on > > desktops or servers using the standard .NET framework - and this is where > > Ironclad is used. Despite the variety of different JVMs this is an > analogous > > situation and I don't see why a port wouldn't be just as useful. > > Why not just use CPython? At Google we mix Python and Java using RPCs > intead of mixing runtimes. It works quite well. > > The use case for Resolver Systems (who created the Ironclad project) was that having written a programmable spreadsheet (desktop application) written in IronPython we had a lot of customers who wanted to use Numpy with it. We could have told them to go away and write their own RPC layer. Instead we decided to solve the problem. :-) With the Ironclad compatibility layer C extensions can be used without having to write an RPC layer and the data marshalling that involves. It works well... Other architectures (multi-process and RPC) may be suitable to some problems but it is nice to have the choice. There are a couple of interesting aspects of Ironclad that might have wider applicability in the future. IronPython doesn't have a GIL and doesn't use reference counting for garbage collection, but Ironclad has to work with C extensions that use both. If (for example) Unladen Swallow were to (eventually) be successful in removing the GIL and moved away from reference counting then it would be possible to retain binary API (ABI) compatibility with extensions written for 'standard' CPython. The GIL is simply faked. IronPython code is not restricted by the GIL but only one code path into Ironclad can acquire the GIL at a time. For reference counting we have a hybrid system keeping 'bridge' objects alive whilst the C extension has a reference to them but they may or may not be in use from the IronPython side. All the best, Michael -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From debatem1 at gmail.com Mon Oct 26 18:27:56 2009 From: debatem1 at gmail.com (geremy condra) Date: Mon, 26 Oct 2009 13:27:56 -0400 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> Message-ID: On Mon, Oct 26, 2009 at 5:47 AM, Michael Foord wrote: > > > 2009/10/26 Guido van Rossum >> >> On Sun, Oct 25, 2009 at 5:12 PM, Michael Foord wrote: >> > >> > >> > 2009/10/25 Guido van Rossum >> >> >> >> On Sun, Oct 25, 2009 at 4:38 PM, Michael Foord >> >> wrote: >> >> > Well, you can use Python C extensions with Ironclad [1]. The >> >> > maintainer >> >> > hopes to port the core to Jython at some point as well. >> >> >> >> What do you personally think of Ironclad? And (separately) of those >> >> hopes? >> >> >> > >> > It seems like it ought to be an impossible task - reimplementing the >> > Python >> > C API for another platform. It actually runs surprisingly well >> > (astonishingly) and I know of at least one bank in London now using it >> > in >> > production. The Ironclad implementation reuses the Python C source >> > wherever >> > possible in order to minimise the core that actually needs implementing. >> > >> > A *large* number of the Numpy and SciPy tests pass with it (~1000 of >> > each >> > last time I checked) and *generally* performance is pretty good. I'd >> > like to >> > see Ironclad in wider use. >> > >> > The hopes of the Ironclad maintainer to reimplement the core for Jython >> > is >> > certainly *plausible*, but it of course depends on him finding time in >> > the >> > future. >> > >> > Personally when I write IronPython code I try to avoid a dependency on C >> > extension modules as it seems to me that the *point* of IronPython is to >> > make use of the .NET framework (otherwise you might as well just use >> > CPython). Where Ironclad is being used is where people want to interface >> > existing Python systems to existing .NET systems and that makes a lot of >> > sense (you'd rather avoid rewriting chunks of either if you can and >> > Ironclad >> > acts as a bridge). >> >> Hm. In the Java world, there are many target environments where >> depending on C extensions would not fly at all, since the only >> commonality between platforms is the JVM interface. (And sometimes not >> even that -- e.g. Android uses a different VM to run Java.) I'm >> guessing that in the .NET world this is much less of an issue, since >> (a) the platform is more homogeneous and under control of Microsoft >> (Mono notwithstanding) and (b) .NET explicitly targets other languages >> (though it prefers "managed C++", it supports plain C++). Have I got >> this right? I see the prospects for an IronClad-like thing in the Java >> world as pretty slim because of this. > > > Hehe, you actually raise several points in that short email. I'll address > them one by one and try not to ramble too much. > > Firstly, although you are correct that .NET supports a managed variant of > C++ (that runs 'on .NET') and it is the same set of tools that you also use > to compile native code (unmanaged C/C++) this has nothing to do with .NET. > Python for Windows is compiled with the Visual C++ compiler but it doesn't > run on .NET. .NET doesn't even use the MSVCRT that compiled native code > links against - something that causes Ironclad 'difficulties' when managed > and native code need to share file handles. > > Ironclad itself has binary compatibility with Python C extensions, they > don't need to be recompiled. It uses the .NET FFI (P/Invoke) to work with > these extensions and on the JVM would use its FFI. > > My understanding is that Android now allows native code, so if Dalvik has > the same FFI APIs and you can compile the Python extensions for it *and* > Jython runs on Dalvik (not currently the case I believe?) then it could > work... > No need. Java has the Java Native Interface, which is supported in the Android Native Development Kit. Geremy Condra From guido at python.org Mon Oct 26 18:29:12 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 26 Oct 2009 10:29:12 -0700 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> Message-ID: On Mon, Oct 26, 2009 at 10:27 AM, geremy condra wrote: > No need. Java has the Java Native Interface, which is supported in the > Android Native Development Kit. If it's that easy why aren't more people developing Android apps using Jython? -- --Guido van Rossum PS. My elbow needs a couple more weeks of rest. Limiting myself to ultra-short emails. From g.brandl at gmx.net Mon Oct 26 18:37:16 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 26 Oct 2009 18:37:16 +0100 Subject: [Python-ideas] One obvious way to do interning [Was: Retrieve an arbitrary element from a set without removing it] In-Reply-To: References: Message-ID: Alexander Belopolsky schrieb: > Changing the subject to reflect branched discussion and forwarding to > python-ideas where it probably belongs. > > On Mon, Oct 26, 2009 at 12:02 PM, Terry Reedy wrote: >> Alexander Belopolsky wrote: >> >>> Here is an alternative idea on how storing interned objects in a set >>> can be supported. Currently set.add method returns None and has no >>> effect when set already has an object equal to the one being added. I >>> propose to consider changing that behavior to make set.add return the >>> added object or the set member that is equal to the object being >>> added. It is unlikely that many programs rely on the return value >>> being None (with doctests being a probable exception), so adding this >>> feature is unlikely to cause much grief. >> >> I had exactly the same idea, but did not post because it violates the >> general rule that mutators return None. > > Is there such a rule? What about set/dict pop? The rule is about methods that do not have an obvious return value, where the choice is between returning self and None. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From python at rcn.com Mon Oct 26 18:40:19 2009 From: python at rcn.com (Raymond Hettinger) Date: Mon, 26 Oct 2009 10:40:19 -0700 Subject: [Python-ideas] [Python-Dev] One obvious way to do interning [Was: Retrieve anarbitrary element from a set without removing it] References: Message-ID: >> Here is an alternative idea on how storing interned objects in a set >> can be supported. FWIW, here is a recipe that can retrieve canonical objects from any container, letting look-up a preferred representative of an equivalence class: http://code.activestate.com/recipes/499299/ Raymond From debatem1 at gmail.com Mon Oct 26 18:51:33 2009 From: debatem1 at gmail.com (geremy condra) Date: Mon, 26 Oct 2009 13:51:33 -0400 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> Message-ID: On Mon, Oct 26, 2009 at 1:29 PM, Guido van Rossum wrote: > On Mon, Oct 26, 2009 at 10:27 AM, geremy condra wrote: >> No need. Java has the Java Native Interface, which is supported in the >> Android Native Development Kit. > > If it's that easy why aren't more people developing Android apps using Jython? > > -- > --Guido van Rossum Because Jython doesn't run on dalvik- dalvik bytecode and normal java bytecode are quite different, and apparently porting Jython to dalvik would be quite the task in its own right. So far, there have been two solutions to this: the first is, as you mention, ASE, which works by exposing a small handful of convenience functions to scripting languages via JSON. As you point out, that's quite slow and limiting. Secondly, there's the approach that I took- simply wrap the most of the JNI in the CPython interface. It requires more work to get multiple language support- which was their goal in pushing ASE- but it's quite doable, and frankly if I knew enough about the JNI to make it happen somebody else has almost certainly had better luck. It took me a while to get as far as I did on the problem at least partially because of my unfamiliarity with both the JNI and with the CPython C API, but here's a (working) example of how you use it: Here?s the Java class: public class Sample { private boolean booleanMethod(boolean bool) { return !bool; } public int intMethod(int n) { return n+n; } } And here?s how you map it to Python: from java import Java @Java class Sample: '''Sample Java class implementation''' def __init__(self): self.java_init() def booleanMethod(truth: "Z") -> "Z": '''Returns the inverse of what was passed in.''' return self.java_booleanMethod(truth) def intMethod(x: "I") -> "I": '''Returns double the passed-in value''' return self.java_intMethod(x) And from then on you can use it just like a Python class, because it is. There are limitations- you can't pass arrays filled with arbitrary objects ATM- but if there's any interest in the project I could fill those gaps in the period of a few weeks. I can't make any quantitative evaluations of how quickly it runs- my main goal was not speed but getting full access- but I can say that feel-wise it was possible for a nontechnical friend to repeatedly identify whether a given app was using ASE or Javalin based solely on the speed at which they responded. Geremy Condra From fuzzyman at gmail.com Mon Oct 26 19:07:58 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 26 Oct 2009 19:07:58 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> Message-ID: <6f4025010910261107k7e05aefbrb97d124186850f7b@mail.gmail.com> 2009/10/26 geremy condra > [snip...] > > Firstly, although you are correct that .NET supports a managed variant of > > C++ (that runs 'on .NET') and it is the same set of tools that you also > use > > to compile native code (unmanaged C/C++) this has nothing to do with > .NET. > > Python for Windows is compiled with the Visual C++ compiler but it > doesn't > > run on .NET. .NET doesn't even use the MSVCRT that compiled native code > > links against - something that causes Ironclad 'difficulties' when > managed > > and native code need to share file handles. > > > > Ironclad itself has binary compatibility with Python C extensions, they > > don't need to be recompiled. It uses the .NET FFI (P/Invoke) to work with > > these extensions and on the JVM would use its FFI. > > > > My understanding is that Android now allows native code, so if Dalvik has > > the same FFI APIs and you can compile the Python extensions for it *and* > > Jython runs on Dalvik (not currently the case I believe?) then it could > > work... > > > > No need. Java has the Java Native Interface, which is supported in the > Android Native Development Kit. > > No need for what? If you are using Jython *and* you want to use Python C extensions then something like Ironclad would be needed. If you aren't using Jython then no need - but there are lots of good reasons for *wanting* to use Jython. Michael > Geremy Condra > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From debatem1 at gmail.com Mon Oct 26 19:10:02 2009 From: debatem1 at gmail.com (geremy condra) Date: Mon, 26 Oct 2009 14:10:02 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <8763a2adkp.fsf@uwakimon.sk.tsukuba.ac.jp> References: <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <4AE3E315.5060500@gmail.com> <4AE3F7BD.3040301@gmail.com> <87hbtmal6b.fsf@uwakimon.sk.tsukuba.ac.jp> <8763a2adkp.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Mon, Oct 26, 2009 at 2:12 AM, Stephen J. Turnbull wrote: > geremy condra writes: > > ?> There is an enormous risk that they are wasting their time. ?The > ?> goal [of a sandbox] is to save the core devs' time, not those > ?> proposing features. > > The core devs' time will be saved by ignoring the sandbox (other than > their own features), Until a feature because quite mature, yes- afterwards, no. That's the point of keeping it separate. > and if it intrudes on them (eg, in the commit > notices), they will request that it forcibly be shut up. ?Cf. the > short exchange you had with Nick on the subject of commit notices. > I've already addressed that, see the messages you reference. > ISTM that rather the goal is to ensure that features that would > otherwise be overlooked or be blocked because the proponent is > unwilling to follow through to the extent required (ie, by a new > proposal at the time of ending the moratorium) be given more > consideration. Nope, not the goal. >?I don't think a sandbox will help with that, unless it > is designed to give incentives and help in producing *better* > proposals. ?Just collecting them in one place is not going to help. > No, it wouldn't. Because that's not the point. > ?> > It would. ?But consider the PEP for ipaddr. > ?> > ?> I honestly don't see what this has to do with this proposal. > > It's a common pattern of discussion, for both stdlib and core > syntax/builtins, and I chose it because it was close to the boundary > of approval. ?I didn't want to choose nonstarters like multiline > blocks or trivial suggestions like a warning for "breakless for-else". > > ?> 2) the situation was complicated by recriminations about > ?> the openness of python, the role of the mailinglists vs > ?> the bugtracker, and numerous other issues that we don't > ?> need to go into here. > > Are you sure you're not confusing it with the distutils thread(s)? > Pretty sure. > ?> 3) According to people who know a great deal more about > ?> netmasks than I ever will, the things the author was > ?> unaware of would fill several large books, and even Guido > ?> repeatedly emphasized his lack of knowledge on the subject. > ?> By contrast, the group of core Python developers is > ?> uniquely qualified to talk about changes to the language's > ?> syntax and behavior. > > But those are precisely the people whose time you plan to save. Yes it is. > ?The only way to save their time is to do less reviewing. Nope, pretty sure that coming in to do a code review at the end is less time consuming than being involved with a feature from the beginning. >?Having an official sandbox might do that by giving the dross a > place to collect, so the core devs *never* look at it. I like to think that the fact that a patch comes from outside of core will not impact its consideration by that group. > ?> We're talking about changes to the core language. I would suggest > ?> that the only "apps" possible under those conditions would be > ?> extensive test suites and real-life use. > > Yup. ?Precisely the kind of stuff that happens in real projects, not > in sandboxes. True, and irrelevant. The sandbox I propose is not the existing python sandbox. Choose a different term for it if you want, but anything in it that wants acceptance had better be damned good, and that, in my mind, includes thorough testing. > ?> > There will be way too many of them to be reviewed at all, let alone > ?> > easily, if all people have to do is post a branch to an official > ?> > sandbox. > ?> > ?> I doubt this. Experience suggests that the number of individuals > ?> proposing changes with the technical chops to pull them off is > ?> relatively small, > > Sure, but they generally also are core developers. ?I don't see why an > "official sandbox" will be of use to them. Some core devs may decide to use it to produce features that aren't going to come in under the moratorium, but people outside the core dev group can also use it. Don't assume that the only people with the experience and knowledge to implement such a change are part of the core dev group- especially a couple of years from now. Geremy Condra From alexander.belopolsky at gmail.com Mon Oct 26 19:10:24 2009 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 26 Oct 2009 14:10:24 -0400 Subject: [Python-ideas] [Python-Dev] One obvious way to do interning [Was: Retrieve anarbitrary element from a set without removing it] In-Reply-To: References: Message-ID: On Mon, Oct 26, 2009 at 1:40 PM, Raymond Hettinger wrote: .. > FWIW, here is a recipe that can retrieve canonical objects > from any container, letting look-up a preferred representative > of an equivalence class: http://code.activestate.com/recipes/499299/ This is very clever, but I am not sure whether existence of this recipe is an argument for or against adding functionality to sets. On one hand, it does provide a solution for O(1) retrieval of an equivalent item, but it is far from obvious. Overloading == with a mutating method is borderline black magic! Furthermore, it is not clear how to generalize this recipe to efficiently insert the value that is not found in the set. Finally, overloading both __eq__ and __getattr__ with python methods is likely to negate any performance gains of an O(1) over O(n) algorithm for typical sets of built-in types. The use case in this recipe is not addressed by the "return value from set.add(..)" proposal. One would need a different "get-like" method for that. From alexander.belopolsky at gmail.com Mon Oct 26 19:24:26 2009 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Mon, 26 Oct 2009 14:24:26 -0400 Subject: [Python-ideas] One obvious way to do interning [Was: Retrieve an arbitrary element from a set without removing it] In-Reply-To: <4AE5DC62.6000008@arbash-meinel.com> References: <4AE5DC62.6000008@arbash-meinel.com> Message-ID: On Mon, Oct 26, 2009 at 1:29 PM, John Arbash Meinel wrote: ... > Just to let you know that I created my own "SimpleSet" class for use in > the Bazaar code base. And I did exactly this, overloading "Add" to also > return the object added. It works pretty well. > Very interesting. Is there a public web viewer for your branch? I tried http://code.python.org/loggerhead/, but it gives me 503 Service Temporarily Unavailable. From fuzzyman at gmail.com Mon Oct 26 21:38:58 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 26 Oct 2009 21:38:58 +0100 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <1A472770E042064698CB5ADC83A12ACD04B114F3@TK5EX14MBXC116.redmond.corp.microsoft.com> References: <4222a8490910211043h361eedfeka28042106abb0420@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04B114F3@TK5EX14MBXC116.redmond.corp.microsoft.com> Message-ID: <6f4025010910261338r57fb8fbcgf9ffb6d5e8681e0b@mail.gmail.com> 2009/10/21 Dino Viehland > [snip...] > > For example consider multiprocessing - which IronPython still doesn't > support. That's a much bigger work item than everything that changed > related to parsing in 2.6. New built-in functions can also be a huge > pain - again compare float.fromhex to everything that happened to the > parser in 2.6. I'm pretty sure I spent more time on float.fromhex even > w/ the already existing awesome test suite. > > Was it a pain because it was on a builtin or because of the specific functionality? Michael > So I definitely think not adding significant new functionality to the core > interpreter would be great but from an alternate implementation perspective > I'd also be happy to see some wiggle room on small new features. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Mon Oct 26 22:50:56 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 27 Oct 2009 07:50:56 +1000 Subject: [Python-ideas] [Python-Dev] One obvious way to do interning [Was: Retrieve an arbitrary element from a set without removing it] In-Reply-To: References: Message-ID: <4AE619C0.803@gmail.com> Terry Reedy wrote: > Alexander Belopolsky wrote: >> Terry Reedy wrote: >>> I had exactly the same idea, but did not post because it violates the >>> general rule that mutators return None. >> >> Is there such a rule? What about set/dict pop? > > The rule perhaps should be restated as 'Collection mutators return None > or possible an item but not the collection.' And to clarify the rationale for that guideline: it is to make it clear that the mutator is changing the container in place and *not* creating a new container object. myset.pop() # No new container, returns popped object mylist.sort() # No new container, returns None sorted(mylist) # New container, so return it mystr.lower() # Creates new string, so return it Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Mon Oct 26 23:05:35 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 27 Oct 2009 08:05:35 +1000 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <200910261022.44281.steve@pearwood.info> Message-ID: <4AE61D2F.6060200@gmail.com> geremy condra wrote: > On Sun, Oct 25, 2009 at 8:51 PM, Guido van Rossum wrote: > > > >> Also to stop the pointless discussions about anonymous blocks (though >> that may have been naive :-). >> >> -- >> --Guido van Rossum > > Tell everybody that proposes it to show you the code. 99% of them will > STFU, and it's probably worth a little bit of time to see what the other > 1% come up with, even if its never going to get in. Worst case scenario, > you've spared yourself the headache of dealing with people who can't > (or won't) help you implement what they so urgently desire. Best case > scenario, maybe you even get a great feature out of it. There's no point telling people to produce code for a design that isn't acceptable in the first place. No anonymous block proposal has ever even made it past the first hurdle of being better in concept than a named function (which do have their downsides, but those have always been looked minor when compared to the downsides of the proposed additions). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Mon Oct 26 23:13:08 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 27 Oct 2009 08:13:08 +1000 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: <6f4025010910261020q1044d0d4r9321fa1e8e6ee8e3@mail.gmail.com> References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> <6f4025010910261020q1044d0d4r9321fa1e8e6ee8e3@mail.gmail.com> Message-ID: <4AE61EF4.60300@gmail.com> Michael Foord wrote: > If (for example) Unladen Swallow were to (eventually) be successful in > removing the GIL and moved away from reference counting then it would be > possible to retain binary API (ABI) compatibility with extensions > written for 'standard' CPython. > > The GIL is simply faked. IronPython code is not restricted by the GIL > but only one code path into Ironclad can acquire the GIL at a time. > > For reference counting we have a hybrid system keeping 'bridge' objects > alive whilst the C extension has a reference to them but they may or may > not be in use from the IronPython side. Something I've long been curious about (but never even come close to having the time to research) is the idea of using CPython extensions with PyPy. It sounds like that would involve an effort in a similar vein to porting Ironclad to Jython (i.e. porting the back end to PyPy instead). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From guido at python.org Mon Oct 26 23:18:01 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 26 Oct 2009 15:18:01 -0700 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: <4AE61EF4.60300@gmail.com> References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> <6f4025010910261020q1044d0d4r9321fa1e8e6ee8e3@mail.gmail.com> <4AE61EF4.60300@gmail.com> Message-ID: On Mon, Oct 26, 2009 at 3:13 PM, Nick Coghlan wrote: > Something I've long been curious about (but never even come close to > having the time to research) is the idea of using CPython extensions > with PyPy. It sounds like that would involve an effort in a similar vein > to porting Ironclad to Jython (i.e. porting the back end to PyPy instead). I'd love to see someone from PyPy comment on this too. IIUC they aren't particularly keen -- their approach so far has been based on porting existing C extensions to use ctypes, which they *do* support. -- --Guido van Rossum PS. My elbow needs a couple more weeks of rest. Limiting myself to ultra-short emails. From benjamin at python.org Mon Oct 26 23:43:52 2009 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 26 Oct 2009 22:43:52 +0000 (UTC) Subject: [Python-ideas] stdlib with its own release cycle ? References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> <6f4025010910261020q1044d0d4r9321fa1e8e6ee8e3@mail.gmail.com> <4AE61EF4.60300@gmail.com> Message-ID: Nick Coghlan writes: > Something I've long been curious about (but never even come close to > having the time to research) is the idea of using CPython extensions > with PyPy. It sounds like that would involve an effort in a similar vein > to porting Ironclad to Jython (i.e. porting the back end to PyPy instead). We say "it would be possible", but no one is really interested in doing the work which it would involve. At the moment, it's not even possible to have loadable RPython extension modules because of how the RPython translator works. Our answer right now, though, is that we want to eliminate the need for C extensions at all. ctypes can be used for binding to C libraries and our JIT can provide speed. From debatem1 at gmail.com Mon Oct 26 23:47:37 2009 From: debatem1 at gmail.com (geremy condra) Date: Mon, 26 Oct 2009 18:47:37 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <4AE61D2F.6060200@gmail.com> References: <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <200910261022.44281.steve@pearwood.info> <4AE61D2F.6060200@gmail.com> Message-ID: On Mon, Oct 26, 2009 at 6:05 PM, Nick Coghlan wrote: > geremy condra wrote: >> On Sun, Oct 25, 2009 at 8:51 PM, Guido van Rossum wrote: >> >> >> >>> Also to stop the pointless discussions about anonymous blocks (though >>> that may have been naive :-). >>> >>> -- >>> --Guido van Rossum >> >> Tell everybody that proposes it to show you the code. 99% of them will >> STFU, and it's probably worth a little bit of time to see what the other >> 1% come up with, even if its never going to get in. Worst case scenario, >> you've spared yourself the headache of dealing with people who can't >> (or won't) help you implement what they so urgently desire. Best case >> scenario, maybe you even get a great feature out of it. > > There's no point telling people to produce code for a design that isn't > acceptable in the first place. No anonymous block proposal has ever even > made it past the first hurdle of being better in concept than a named > function (which do have their downsides, but those have always been > looked minor when compared to the downsides of the proposed additions). > > Cheers, > Nick. The point is that almost none of them will do it, and if somebody is so determined that they'll actually jump through all these hoops to get the feature, maybe it really *is* needed for something they're doing. In any event, by the time they've done that they deserve to be taken more seriously than the person just randomly jawing off. Plus, not all features are as pointless as multiline lambda, but almost all proposals for language change are being sidelined by the moratorium. We should provide a facility for those ideas to be identified and developed (by third parties) during the moratorium so that we have time for a careful and thorough review- in effect giving ourselves a solid one to two year head start on any future language changes. I might even go so far as to say that under this model a kind of rolling moratorium might be a good idea- a mandatory period of substantial length during which a change to the core language has to be reviewed by the devs, the community, and the other implementors for its viability and usefulness. But that's probably something to be decided by higher-ups later on. Geremy Condra From debatem1 at gmail.com Mon Oct 26 23:51:02 2009 From: debatem1 at gmail.com (geremy condra) Date: Mon, 26 Oct 2009 18:51:02 -0400 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: <6f4025010910261107k7e05aefbrb97d124186850f7b@mail.gmail.com> References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> <6f4025010910261107k7e05aefbrb97d124186850f7b@mail.gmail.com> Message-ID: On Mon, Oct 26, 2009 at 2:07 PM, Michael Foord wrote: > > > 2009/10/26 geremy condra >> >> [snip...] >> > Firstly, although you are correct that .NET supports a managed variant >> > of >> > C++ (that runs 'on .NET') and it is the same set of tools that you also >> > use >> > to compile native code (unmanaged C/C++) this has nothing to do with >> > .NET. >> > Python for Windows is compiled with the Visual C++ compiler but it >> > doesn't >> > run on .NET. .NET doesn't even use the MSVCRT that compiled native code >> > links against - something that causes Ironclad 'difficulties' when >> > managed >> > and native code need to share file handles. >> > >> > Ironclad itself has binary compatibility with Python C extensions, they >> > don't need to be recompiled. It uses the .NET FFI (P/Invoke) to work >> > with >> > these extensions and on the JVM would use its FFI. >> > >> > My understanding is that Android now allows native code, so if Dalvik >> > has >> > the same FFI APIs and you can compile the Python extensions for it *and* >> > Jython runs on Dalvik (not currently the case I believe?) then it could >> > work... >> > >> >> No need. Java has the Java Native Interface, which is supported in the >> Android Native Development Kit. >> > > No need for what? If you are using Jython *and* you want to use Python C > extensions then something like Ironclad would be needed. > > If you aren't using Jython then no need - but there are lots of good reasons > for *wanting* to use Jython. > > Michael Sorry, I wasn't clear- there's no need to get Jython running on Android, since you can wrap the class libraries using JNI. Geremy Condra From guido at python.org Tue Oct 27 00:00:41 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 26 Oct 2009 16:00:41 -0700 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> <6f4025010910261107k7e05aefbrb97d124186850f7b@mail.gmail.com> Message-ID: On Mon, Oct 26, 2009 at 3:51 PM, geremy condra wrote: > Sorry, I wasn't clear- there's no need to get Jython running on Android, > since you can wrap the class libraries using JNI. So one would be running CPython and calling into the Android-specific Java library via JNI? That doesn't sound like a good use of the Android platform to me. -- --Guido van Rossum PS. My elbow needs a couple more weeks of rest. Limiting myself to ultra-short emails. From fuzzyman at gmail.com Tue Oct 27 00:03:07 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Tue, 27 Oct 2009 00:03:07 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> <6f4025010910261020q1044d0d4r9321fa1e8e6ee8e3@mail.gmail.com> <4AE61EF4.60300@gmail.com> Message-ID: <6f4025010910261603s68bdad84p65ade711d308cf5b@mail.gmail.com> 2009/10/26 Guido van Rossum > On Mon, Oct 26, 2009 at 3:13 PM, Nick Coghlan wrote: > > Something I've long been curious about (but never even come close to > > having the time to research) is the idea of using CPython extensions > > with PyPy. It sounds like that would involve an effort in a similar vein > > to porting Ironclad to Jython (i.e. porting the back end to PyPy > instead). > > I'd love to see someone from PyPy comment on this too. IIUC they > aren't particularly keen -- their approach so far has been based on > porting existing C extensions to use ctypes, which they *do* support. > > Well, IronPython now has ctypes too (as of IronPython 2.6) - so porting C extensions to ctypes is still a better approach for cross implementation portability. Michael > -- > --Guido van Rossum > > PS. My elbow needs a couple more weeks of rest. Limiting myself to > ultra-short emails. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at gmail.com Tue Oct 27 00:05:36 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Tue, 27 Oct 2009 00:05:36 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: <4AE61EF4.60300@gmail.com> References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> <6f4025010910261020q1044d0d4r9321fa1e8e6ee8e3@mail.gmail.com> <4AE61EF4.60300@gmail.com> Message-ID: <6f4025010910261605p37071922pf8d94127f42a0bcb@mail.gmail.com> 2009/10/26 Nick Coghlan > Michael Foord wrote: > > If (for example) Unladen Swallow were to (eventually) be successful in > > removing the GIL and moved away from reference counting then it would be > > possible to retain binary API (ABI) compatibility with extensions > > written for 'standard' CPython. > > > > The GIL is simply faked. IronPython code is not restricted by the GIL > > but only one code path into Ironclad can acquire the GIL at a time. > > > > For reference counting we have a hybrid system keeping 'bridge' objects > > alive whilst the C extension has a reference to them but they may or may > > not be in use from the IronPython side. > > Something I've long been curious about (but never even come close to > having the time to research) is the idea of using CPython extensions > with PyPy. It sounds like that would involve an effort in a similar vein > to porting Ironclad to Jython (i.e. porting the back end to PyPy instead). > > Porting the Ironclad core to RPython using the PyPy FFI would presumably be possible. Last time I spoke to the PyPy devs about it they said they had a different approach in mind that would give them source compatibility but not binary compatibility. I guess reinventing the wheel is the core motivation of PyPy so I guess they can be forgiven. ;-) Michael > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at gmail.com Tue Oct 27 00:08:57 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Tue, 27 Oct 2009 00:08:57 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> <6f4025010910261107k7e05aefbrb97d124186850f7b@mail.gmail.com> Message-ID: <6f4025010910261608r795011c6q73691b1ea1fa56c2@mail.gmail.com> 2009/10/26 geremy condra > On Mon, Oct 26, 2009 at 2:07 PM, Michael Foord wrote: > > > > > > 2009/10/26 geremy condra > >> > >> [snip...] > >> > Firstly, although you are correct that .NET supports a managed variant > >> > of > >> > C++ (that runs 'on .NET') and it is the same set of tools that you > also > >> > use > >> > to compile native code (unmanaged C/C++) this has nothing to do with > >> > .NET. > >> > Python for Windows is compiled with the Visual C++ compiler but it > >> > doesn't > >> > run on .NET. .NET doesn't even use the MSVCRT that compiled native > code > >> > links against - something that causes Ironclad 'difficulties' when > >> > managed > >> > and native code need to share file handles. > >> > > >> > Ironclad itself has binary compatibility with Python C extensions, > they > >> > don't need to be recompiled. It uses the .NET FFI (P/Invoke) to work > >> > with > >> > these extensions and on the JVM would use its FFI. > >> > > >> > My understanding is that Android now allows native code, so if Dalvik > >> > has > >> > the same FFI APIs and you can compile the Python extensions for it > *and* > >> > Jython runs on Dalvik (not currently the case I believe?) then it > could > >> > work... > >> > > >> > >> No need. Java has the Java Native Interface, which is supported in the > >> Android Native Development Kit. > >> > > > > No need for what? If you are using Jython *and* you want to use Python C > > extensions then something like Ironclad would be needed. > > > > If you aren't using Jython then no need - but there are lots of good > reasons > > for *wanting* to use Jython. > > > > Michael > > Sorry, I wasn't clear- there's no need to get Jython running on Android, > since you can wrap the class libraries using JNI. > There are all sorts of reasons to want to use Jython and being on a fundamentally Java oriented platform sounds like just about the best reason I can imagine. Michael > > Geremy Condra > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From debatem1 at gmail.com Tue Oct 27 00:17:11 2009 From: debatem1 at gmail.com (geremy condra) Date: Mon, 26 Oct 2009 19:17:11 -0400 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> <6f4025010910261107k7e05aefbrb97d124186850f7b@mail.gmail.com> Message-ID: On Mon, Oct 26, 2009 at 7:00 PM, Guido van Rossum wrote: > On Mon, Oct 26, 2009 at 3:51 PM, geremy condra wrote: >> Sorry, I wasn't clear- there's no need to get Jython running on Android, >> since you can wrap the class libraries using JNI. > > So one would be running CPython and calling into the Android-specific > Java library via JNI? That doesn't sound like a good use of the > Android platform to me. > > -- > --Guido van Rossum > > PS. My elbow needs a couple more weeks of rest. Limiting myself to > ultra-short emails. > beats being stuck with ASE- which basically means not being able to make apps for Python, since you can't make .apk's with it- or trying to port Jython. If you have a better alternative, I'm all ears. Geremy Condra From curt at hagenlocher.org Tue Oct 27 00:18:14 2009 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Mon, 26 Oct 2009 16:18:14 -0700 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: <6f4025010910261608r795011c6q73691b1ea1fa56c2@mail.gmail.com> References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> <6f4025010910261107k7e05aefbrb97d124186850f7b@mail.gmail.com> <6f4025010910261608r795011c6q73691b1ea1fa56c2@mail.gmail.com> Message-ID: On Mon, Oct 26, 2009 at 4:08 PM, Michael Foord wrote: > > There are all sorts of reasons to want to use Jython and being on a > fundamentally Java oriented platform sounds like just about the best > reason I can imagine. Plus, Charlie made JRuby run on Android -- where's the competitive spirit? :) (http://blog.headius.com/2009/08/return-of-ruboto.html) -- Curt Hagenlocher curt at hagenlocher.org From debatem1 at gmail.com Tue Oct 27 00:20:07 2009 From: debatem1 at gmail.com (geremy condra) Date: Mon, 26 Oct 2009 19:20:07 -0400 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: <6f4025010910261608r795011c6q73691b1ea1fa56c2@mail.gmail.com> References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251638m5ea0e61eq17361955f170c0b8@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> <6f4025010910261107k7e05aefbrb97d124186850f7b@mail.gmail.com> <6f4025010910261608r795011c6q73691b1ea1fa56c2@mail.gmail.com> Message-ID: On Mon, Oct 26, 2009 at 7:08 PM, Michael Foord wrote: > > > 2009/10/26 geremy condra >> >> On Mon, Oct 26, 2009 at 2:07 PM, Michael Foord wrote: >> > >> > >> > 2009/10/26 geremy condra >> >> >> >> [snip...] >> >> > Firstly, although you are correct that .NET supports a managed >> >> > variant >> >> > of >> >> > C++ (that runs 'on .NET') and it is the same set of tools that you >> >> > also >> >> > use >> >> > to compile native code (unmanaged C/C++) this has nothing to do with >> >> > .NET. >> >> > Python for Windows is compiled with the Visual C++ compiler but it >> >> > doesn't >> >> > run on .NET. .NET doesn't even use the MSVCRT that compiled native >> >> > code >> >> > links against - something that causes Ironclad 'difficulties' when >> >> > managed >> >> > and native code need to share file handles. >> >> > >> >> > Ironclad itself has binary compatibility with Python C extensions, >> >> > they >> >> > don't need to be recompiled. It uses the .NET FFI (P/Invoke) to work >> >> > with >> >> > these extensions and on the JVM would use its FFI. >> >> > >> >> > My understanding is that Android now allows native code, so if Dalvik >> >> > has >> >> > the same FFI APIs and you can compile the Python extensions for it >> >> > *and* >> >> > Jython runs on Dalvik (not currently the case I believe?) then it >> >> > could >> >> > work... >> >> > >> >> >> >> No need. Java has the Java Native Interface, which is supported in the >> >> Android Native Development Kit. >> >> >> > >> > No need for what? If you are using Jython *and* you want to use Python C >> > extensions then something like Ironclad would be needed. >> > >> > If you aren't using Jython then no need - but there are lots of good >> > reasons >> > for *wanting* to use Jython. >> > >> > Michael >> >> Sorry, I wasn't clear- there's no need to get Jython running on Android, >> since you can wrap the class libraries using JNI. > > > There are all sorts of reasons to want to use Jython and being on a > fundamentally Java oriented platform sounds like just about the best reason > I can imagine. > > Michael But Jython doesn't run on Dalvik, so its kind of a nonissue. Geremy Condra From fuzzyman at gmail.com Tue Oct 27 00:23:23 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Tue, 27 Oct 2009 00:23:23 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> <6f4025010910261107k7e05aefbrb97d124186850f7b@mail.gmail.com> <6f4025010910261608r795011c6q73691b1ea1fa56c2@mail.gmail.com> Message-ID: <6f4025010910261623pe991e9axfb209d112203fa66@mail.gmail.com> 2009/10/26 geremy condra > On Mon, Oct 26, 2009 at 7:08 PM, Michael Foord wrote: > > > > > > 2009/10/26 geremy condra > >> > >> On Mon, Oct 26, 2009 at 2:07 PM, Michael Foord > wrote: > >> > > >> > > >> > 2009/10/26 geremy condra > >> >> > >> >> [snip...] > >> >> > Firstly, although you are correct that .NET supports a managed > >> >> > variant > >> >> > of > >> >> > C++ (that runs 'on .NET') and it is the same set of tools that you > >> >> > also > >> >> > use > >> >> > to compile native code (unmanaged C/C++) this has nothing to do > with > >> >> > .NET. > >> >> > Python for Windows is compiled with the Visual C++ compiler but it > >> >> > doesn't > >> >> > run on .NET. .NET doesn't even use the MSVCRT that compiled native > >> >> > code > >> >> > links against - something that causes Ironclad 'difficulties' when > >> >> > managed > >> >> > and native code need to share file handles. > >> >> > > >> >> > Ironclad itself has binary compatibility with Python C extensions, > >> >> > they > >> >> > don't need to be recompiled. It uses the .NET FFI (P/Invoke) to > work > >> >> > with > >> >> > these extensions and on the JVM would use its FFI. > >> >> > > >> >> > My understanding is that Android now allows native code, so if > Dalvik > >> >> > has > >> >> > the same FFI APIs and you can compile the Python extensions for it > >> >> > *and* > >> >> > Jython runs on Dalvik (not currently the case I believe?) then it > >> >> > could > >> >> > work... > >> >> > > >> >> > >> >> No need. Java has the Java Native Interface, which is supported in > the > >> >> Android Native Development Kit. > >> >> > >> > > >> > No need for what? If you are using Jython *and* you want to use Python > C > >> > extensions then something like Ironclad would be needed. > >> > > >> > If you aren't using Jython then no need - but there are lots of good > >> > reasons > >> > for *wanting* to use Jython. > >> > > >> > Michael > >> > >> Sorry, I wasn't clear- there's no need to get Jython running on Android, > >> since you can wrap the class libraries using JNI. > > > > > > There are all sorts of reasons to want to use Jython and being on a > > fundamentally Java oriented platform sounds like just about the best > reason > > I can imagine. > > > > Michael > > But Jython doesn't run on Dalvik, so its kind of a nonissue. > I was responding to your comment that there was no need to port it. If there is a reason to use it then there is a reason to port it. Michael > > Geremy Condra > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From debatem1 at gmail.com Tue Oct 27 00:31:55 2009 From: debatem1 at gmail.com (geremy condra) Date: Mon, 26 Oct 2009 19:31:55 -0400 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: <6f4025010910261623pe991e9axfb209d112203fa66@mail.gmail.com> References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910251712t6edfee73p743c73db973820b2@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> <6f4025010910261107k7e05aefbrb97d124186850f7b@mail.gmail.com> <6f4025010910261608r795011c6q73691b1ea1fa56c2@mail.gmail.com> <6f4025010910261623pe991e9axfb209d112203fa66@mail.gmail.com> Message-ID: On Mon, Oct 26, 2009 at 7:23 PM, Michael Foord wrote: > > > 2009/10/26 geremy condra >> >> On Mon, Oct 26, 2009 at 7:08 PM, Michael Foord wrote: >> > >> > >> > 2009/10/26 geremy condra >> >> >> >> On Mon, Oct 26, 2009 at 2:07 PM, Michael Foord >> >> wrote: >> >> > >> >> > >> >> > 2009/10/26 geremy condra >> >> >> >> >> >> [snip...] >> >> >> > Firstly, although you are correct that .NET supports a managed >> >> >> > variant >> >> >> > of >> >> >> > C++ (that runs 'on .NET') and it is the same set of tools that you >> >> >> > also >> >> >> > use >> >> >> > to compile native code (unmanaged C/C++) this has nothing to do >> >> >> > with >> >> >> > .NET. >> >> >> > Python for Windows is compiled with the Visual C++ compiler but it >> >> >> > doesn't >> >> >> > run on .NET. .NET doesn't even use the MSVCRT that compiled native >> >> >> > code >> >> >> > links against - something that causes Ironclad 'difficulties' when >> >> >> > managed >> >> >> > and native code need to share file handles. >> >> >> > >> >> >> > Ironclad itself has binary compatibility with Python C extensions, >> >> >> > they >> >> >> > don't need to be recompiled. It uses the .NET FFI (P/Invoke) to >> >> >> > work >> >> >> > with >> >> >> > these extensions and on the JVM would use its FFI. >> >> >> > >> >> >> > My understanding is that Android now allows native code, so if >> >> >> > Dalvik >> >> >> > has >> >> >> > the same FFI APIs and you can compile the Python extensions for it >> >> >> > *and* >> >> >> > Jython runs on Dalvik (not currently the case I believe?) then it >> >> >> > could >> >> >> > work... >> >> >> > >> >> >> >> >> >> No need. Java has the Java Native Interface, which is supported in >> >> >> the >> >> >> Android Native Development Kit. >> >> >> >> >> > >> >> > No need for what? If you are using Jython *and* you want to use >> >> > Python C >> >> > extensions then something like Ironclad would be needed. >> >> > >> >> > If you aren't using Jython then no need - but there are lots of good >> >> > reasons >> >> > for *wanting* to use Jython. >> >> > >> >> > Michael >> >> >> >> Sorry, I wasn't clear- there's no need to get Jython running on >> >> Android, >> >> since you can wrap the class libraries using JNI. >> > >> > >> > There are all sorts of reasons to want to use Jython and being on a >> > fundamentally Java oriented platform sounds like just about the best >> > reason >> > I can imagine. >> > >> > Michael >> >> But Jython doesn't run on Dalvik, so its kind of a nonissue. > > I was responding to your comment that there was no need to port it. If there > is a reason to use it then there is a reason to port it. > > Michael Unfortunately, according to people who know Jython much better than I, the amount of work involved would be analogous to the effort involved to write Jython in the first place. Since I doubt strongly that I have either the technical or political skills to develop and maintain my own Python implementation, I prefer the escape hatch. Your mileage apparently does vary. Geremy Condra From thobes at gmail.com Tue Oct 27 07:24:49 2009 From: thobes at gmail.com (Tobias Ivarsson) Date: Tue, 27 Oct 2009 07:24:49 +0100 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> <6f4025010910261107k7e05aefbrb97d124186850f7b@mail.gmail.com> <6f4025010910261608r795011c6q73691b1ea1fa56c2@mail.gmail.com> <6f4025010910261623pe991e9axfb209d112203fa66@mail.gmail.com> Message-ID: <9997d5e60910262324m408872cek89f5b4c42ca4c510@mail.gmail.com> On Tue, Oct 27, 2009 at 12:31 AM, geremy condra wrote: > > Unfortunately, according to people who know Jython much better > than I, the amount of work involved would be analogous to the > effort involved to write Jython in the first place. Since I doubt > strongly that I have either the technical or political skills to > develop and maintain my own Python implementation, I prefer > the escape hatch. Your mileage apparently does vary. Sure... it is true that making Jython run on Dalvik/Android is a big task, but saying that it requires the same amount of work as reimplementing Jython from scratch is a huge overstatement. It is true that it would not be as easy as it was for Charles to get JRuby to run on Dalvik/Android. JRuby has always had an interpreter and then JITs to Java bytecode while Jython compiles all code to Java bytecode immediately when loaded. There are efforts underway to make future versions of Jython run on Android, but just like the core CPython developers most of us are doing Jython as a side project, so it takes time. Very early experiments are looking promising though, but there is still a long way to go. If someone sent me an Android dev. phone I might get more motivated to focus my efforts in that direction... ;) Cheers, Tobias -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Tue Oct 27 13:49:51 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 27 Oct 2009 21:49:51 +0900 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: <87iqe5bbpf.fsf@uwakimon.sk.tsukuba.ac.jp> <4AE3E315.5060500@gmail.com> <4AE3F7BD.3040301@gmail.com> <87hbtmal6b.fsf@uwakimon.sk.tsukuba.ac.jp> <8763a2adkp.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <878wex80i8.fsf@uwakimon.sk.tsukuba.ac.jp> geremy condra writes: > On Mon, Oct 26, 2009 at 2:12 AM, Stephen J. Turnbull wrote: > > The core devs' time will be saved by ignoring the sandbox (other than > > their own features), > Until a feature because quite mature, yes- afterwards, no. That's > the point of keeping it separate. I see what you're saying, my assessment is that it will not be as effective as you think. Your assessment is different. Only way to prove it one way or the other is to try it. > Don't assume that the only people with the experience and knowledge > to implement [a language change] are part of the core dev group- > especially a couple of years from now. I don't make that assumption, it's an empirical observation of current conditions, backed up by the observation that new people have been gaining committer status regularly over the time I've watched Python-Dev -- if you've got the Right Stuff, there are few barriers to joining the core. I agree with you that they won't be the *same* people that are around today, but they will most likely be in the core. From debatem1 at gmail.com Tue Oct 27 15:26:37 2009 From: debatem1 at gmail.com (geremy condra) Date: Tue, 27 Oct 2009 10:26:37 -0400 Subject: [Python-ideas] stdlib with its own release cycle ? In-Reply-To: <9997d5e60910262324m408872cek89f5b4c42ca4c510@mail.gmail.com> References: <94bdd2610910250312g531e0fcbs1557063728c1e18b@mail.gmail.com> <6f4025010910260247v22c138fcpd93d628fbafaa18d@mail.gmail.com> <6f4025010910261107k7e05aefbrb97d124186850f7b@mail.gmail.com> <6f4025010910261608r795011c6q73691b1ea1fa56c2@mail.gmail.com> <6f4025010910261623pe991e9axfb209d112203fa66@mail.gmail.com> <9997d5e60910262324m408872cek89f5b4c42ca4c510@mail.gmail.com> Message-ID: On Tue, Oct 27, 2009 at 2:24 AM, Tobias Ivarsson wrote: > On Tue, Oct 27, 2009 at 12:31 AM, geremy condra wrote: >> >> Unfortunately, according to people who know Jython much better >> than I, the amount of work involved would be analogous to the >> effort involved to write Jython in the first place. Since I doubt >> strongly that I have either the technical or political skills to >> develop and maintain my own Python implementation, I prefer >> the escape hatch. Your mileage apparently does vary. > > Sure... it is true that making Jython run on Dalvik/Android is a big task, > but saying that it requires the same amount of work as reimplementing Jython > from scratch is a huge overstatement. It is true that it would not be as > easy as it was for Charles to get JRuby to run on Dalvik/Android. JRuby has > always had an interpreter and then JITs to Java bytecode while Jython > compiles all code to Java bytecode immediately when loaded. I can pretty much guarantee that you know more about what you're talking about than I do. Having said that- several highly skilled developers, each certainly more experienced with both Java and Jython than myself, have tried their hands at this and nearly a year later none have succeeded. My project took me- a guy who hasn't touched Java, let alone the JNI, in nearly a decade- less than 8 weeks of intermittent effort. Sure, it's closer to what ctypes does than what Jython does, but working is working, and I'll take that. Plus it let me work in Python 3, which I prefer. > There are efforts underway to make future versions of Jython run on Android, > but just like the core CPython developers most of us are doing Jython as a > side project, so it takes time. Very early experiments are looking promising > though, but there is still a long way to go. > > If someone sent me an Android dev. phone I might get more motivated to focus > my efforts in that direction... ;) I can send you a shiny new emulator ;) Geremy Condra From debatem1 at gmail.com Tue Oct 27 15:41:23 2009 From: debatem1 at gmail.com (geremy condra) Date: Tue, 27 Oct 2009 10:41:23 -0400 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <878wex80i8.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4AE3E315.5060500@gmail.com> <4AE3F7BD.3040301@gmail.com> <87hbtmal6b.fsf@uwakimon.sk.tsukuba.ac.jp> <8763a2adkp.fsf@uwakimon.sk.tsukuba.ac.jp> <878wex80i8.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Tue, Oct 27, 2009 at 8:49 AM, Stephen J. Turnbull wrote: > geremy condra writes: > ?> On Mon, Oct 26, 2009 at 2:12 AM, Stephen J. Turnbull wrote: > > ?> > The core devs' time will be saved by ignoring the sandbox (other than > ?> > their own features), > > ?> Until a feature because quite mature, yes- afterwards, no. That's > ?> the point of keeping it separate. > > I see what you're saying, my assessment is that it will not be as > effective as you think. ?Your assessment is different. ?Only way to > prove it one way or the other is to try it. I suggest we do that then. I'd appreciate input on how to structure it- the CGI idea (I believe it was yours?) seems to me to be the best on the table right now, but I'd like to hear contrary points of view. > ?> Don't assume that the only people with the experience and knowledge > ?> to implement [a language change] are part of the core dev group- > ?> especially a couple of years from now. > > I don't make that assumption, it's an empirical observation of current > conditions, backed up by the observation that new people have been > gaining committer status regularly over the time I've watched > Python-Dev -- if you've got the Right Stuff, there are few barriers to > joining the core. ?I agree with you that they won't be the *same* > people that are around today, but they will most likely be in the > core. If they're in core, so be it- I'm fine with that. But especially as the language's user base gets larger I suspect we're likely to see proposals that reflect a pretty diverse range of views on what Python should look like. Python-dev isn't the right place for a lot of those, especially while under the moratorium, but we still ignore them at our peril. The sandbox gives people a place to experiment with possible future Pythons in a way thats low-impact on the parts of the community that aren't interested in it. We disagree on how big the interested group is and who it contains, but my vote (for whatever its worth) is that we should find out. Geremy Condra From rrr at ronadam.com Tue Oct 27 18:02:44 2009 From: rrr at ronadam.com (Ron Adam) Date: Tue, 27 Oct 2009 12:02:44 -0500 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: References: Message-ID: <4AE727B4.5060603@ronadam.com> Guido van Rossum wrote: > On Wed, Oct 21, 2009 at 11:52 AM, geremy condra wrote: >> Towards that end, I'd also like to propose a very public, very >> accessible 'sandbox' specifically for the development and testing of >> new language features while the moratorium is in effect. Its goal >> would be to keep interest in changes to core language design >> ongoing by keeping the barrier to entry low, while simultaneously >> separating it from core development. With any luck, it would mean >> that when the moratorium lifts, Python will be able to take its pick >> from the best of the language proposals, while still having given >> other implementations the opportunity to study their behavior >> "in the wild" for a period of months or years. > > I can't stop people from forking the language to do experiments, but > one of the goals I have for the moratorium is actually to *reduce* the > interest in core language changes, and to *raise* the barrier to > entry. Most language change proposals are just fluff, and they will be > just as unneeded three years from now as they are today. Once the > moratorium is lifted, users should be able expect the normal, slow, > conservative evolution of the language to continue -- not to see the > floodgates lifted for a barrage of new features. Here's a thought: How about directing the developement effort in a predictable pattern. An example of this might be... 3.2 - library development, and bugs 3.3 - bug fix's only, (long term stable release) 3.4 - core development, bugs 3.5 - library, bugs 3.6 - bugs only, (long term stable release) 3.7 - core, bugs 3.8 - library, bugs 3.9 - bugs only, (long term stable release) Also one of the things I like about Ubuntu's development, is the pre announcement of what areas of Ubuntu will be worked on. I think that has a really good effect on how well development efforts are focused. Cheers, Ron Adam From sturla at molden.no Tue Oct 27 23:41:17 2009 From: sturla at molden.no (Sturla Molden) Date: Tue, 27 Oct 2009 23:41:17 +0100 Subject: [Python-ideas] XOR Message-ID: <4AE7770D.5070303@molden.no> Why does Python have a bitwise but not a logical xor operator? It's even weirder because boolean objects do have a __xor__ method. Does Py3k have an xor keyword? (I am using 2.6 due to NumPy.) (Yes I know BDFL is planning a moratorium on syntax.) S.M. From g.brandl at gmx.net Tue Oct 27 23:46:31 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 27 Oct 2009 23:46:31 +0100 Subject: [Python-ideas] XOR In-Reply-To: <4AE7770D.5070303@molden.no> References: <4AE7770D.5070303@molden.no> Message-ID: Sturla Molden schrieb: > Why does Python have a bitwise but not a logical xor operator? How often do you need the xor operator? > It's even weirder because boolean objects do have a __xor__ method. The reason for that is that they inherit from integers. That method is the usual bitwise integer xor. > Does Py3k have an xor keyword? (I am using 2.6 due to NumPy.) No. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From sturla at molden.no Tue Oct 27 23:46:47 2009 From: sturla at molden.no (Sturla Molden) Date: Tue, 27 Oct 2009 23:46:47 +0100 Subject: [Python-ideas] XOR In-Reply-To: <4AE7770D.5070303@molden.no> References: <4AE7770D.5070303@molden.no> Message-ID: <4AE77857.50601@molden.no> Sturla Molden skrev: > Why does Python have a bitwise but not a logical xor operator? It's > even weirder because boolean objects do have a __xor__ method. >>> True.__xor__(False) True >>> True xor False SyntaxError: invalid syntax Doesn't this look like a case for new syntax before the moratorium? S.M. From pyideas at rebertia.com Tue Oct 27 23:48:53 2009 From: pyideas at rebertia.com (Chris Rebert) Date: Tue, 27 Oct 2009 15:48:53 -0700 Subject: [Python-ideas] XOR In-Reply-To: <4AE7770D.5070303@molden.no> References: <4AE7770D.5070303@molden.no> Message-ID: <50697b2c0910271548i3520166egc3fe493eb1d17295@mail.gmail.com> On Tue, Oct 27, 2009 at 3:41 PM, Sturla Molden wrote: > Why does Python have a bitwise but not a logical xor operator? It's even > weirder because boolean objects do have a __xor__ method. It would be nice for symmetry, but it would be infrequently used compared to the other operators and is not strictly necessary as logical XOR cannot and would not short-circuit, unlike logical AND and OR. Cheers, Chris -- http://blog.rebertia.com From ubershmekel at gmail.com Tue Oct 27 23:49:54 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Wed, 28 Oct 2009 00:49:54 +0200 Subject: [Python-ideas] Proposal: Moratorium on Python language changes In-Reply-To: <4AE727B4.5060603@ronadam.com> References: <4AE727B4.5060603@ronadam.com> Message-ID: <9d153b7c0910271549t2dd92ad5yf4176ec061c1f833@mail.gmail.com> On Tue, Oct 27, 2009 at 7:02 PM, Ron Adam wrote: > > Here's a thought: How about directing the developement effort in a > predictable pattern. An example of this might be... > [snip] > > Cheers, > Ron Adam > > I like this approach to the moratorium because roadmaps are better than roadblocks IMO. I touched it up a bit: 3.0 - New language features 3.1 - Ironing out language features (moratorium starts) 3.2 - library development 3.3 - library development 3.4 - library development 3.5 - library development 3.6 - Implementing features for easing transition to 4.0 3.7 - End of road for 3.x 4.0 - New language features .... etc etc --yuv -------------- next part -------------- An HTML attachment was scrubbed... URL: From dickinsm at gmail.com Tue Oct 27 23:53:36 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 27 Oct 2009 22:53:36 +0000 Subject: [Python-ideas] XOR In-Reply-To: <4AE7770D.5070303@molden.no> References: <4AE7770D.5070303@molden.no> Message-ID: <5c6f2a5d0910271553v45fdf0c6wb6d887539bbbf6b4@mail.gmail.com> On Tue, Oct 27, 2009 at 10:41 PM, Sturla Molden wrote: > Why does Python have a bitwise but not a logical xor operator? It's even > weirder because boolean objects do have a __xor__ method. Have you looked at the recent python-list thread starting at: http://mail.python.org/pipermail/python-list/2009-July/188589.html ? Mark From bruce at leapyear.org Tue Oct 27 23:56:20 2009 From: bruce at leapyear.org (Bruce Leban) Date: Tue, 27 Oct 2009 15:56:20 -0700 Subject: [Python-ideas] XOR In-Reply-To: <50697b2c0910271548i3520166egc3fe493eb1d17295@mail.gmail.com> References: <4AE7770D.5070303@molden.no> <50697b2c0910271548i3520166egc3fe493eb1d17295@mail.gmail.com> Message-ID: -1 It would be unnice in my opinion because it would falsely suggest some kind of parallelism with how 'and' and 'or' work. Just use bool(x) ^ bool(y) if you really want xor behavior: >>> bool(3) ^ bool(4) False >>> bool(3) ^ bool(None) True --- Bruce http://www.vroospeak.com On Tue, Oct 27, 2009 at 3:48 PM, Chris Rebert wrote: > On Tue, Oct 27, 2009 at 3:41 PM, Sturla Molden wrote: > > Why does Python have a bitwise but not a logical xor operator? It's even > > weirder because boolean objects do have a __xor__ method. > > It would be nice for symmetry, but it would be infrequently used > compared to the other operators and is not strictly necessary as > logical XOR cannot and would not short-circuit, unlike logical AND and > OR. > > Cheers, > Chris > -- > http://blog.rebertia.com > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From digitalxero at gmail.com Wed Oct 28 00:03:29 2009 From: digitalxero at gmail.com (Dj Gilcrease) Date: Tue, 27 Oct 2009 17:03:29 -0600 Subject: [Python-ideas] XOR In-Reply-To: <4AE77857.50601@molden.no> References: <4AE7770D.5070303@molden.no> <4AE77857.50601@molden.no> Message-ID: On Tue, Oct 27, 2009 at 4:46 PM, Sturla Molden wrote: > Sturla Molden skrev: >> >> Why does Python have a bitwise but not a logical xor operator? It's even >> weirder because boolean objects do have a __xor__ method. > >>>> True.__xor__(False) > True >>>> True xor False > SyntaxError: invalid syntax > > Doesn't this look like a case for new syntax before the moratorium? No the proper syntax is >>>False ^ False False >>>False ^ True True >>>True ^ False True >>>True ^ True False From debatem1 at gmail.com Wed Oct 28 00:07:23 2009 From: debatem1 at gmail.com (geremy condra) Date: Tue, 27 Oct 2009 19:07:23 -0400 Subject: [Python-ideas] XOR In-Reply-To: References: <4AE7770D.5070303@molden.no> Message-ID: On Tue, Oct 27, 2009 at 6:46 PM, Georg Brandl wrote: > Sturla Molden schrieb: >> Why does Python have a bitwise but not a logical xor operator? > > How often do you need the xor operator? > 1) Technically, an operator is *never* needed, as its just syntactic sugar. 2) It sure would make crypto code look prettier, as we rely on xor operations extensively. Geremy Condra From sturla at molden.no Wed Oct 28 00:13:45 2009 From: sturla at molden.no (Sturla Molden) Date: Wed, 28 Oct 2009 00:13:45 +0100 Subject: [Python-ideas] XOR In-Reply-To: References: <4AE7770D.5070303@molden.no> <4AE77857.50601@molden.no> Message-ID: <4AE77EA9.8090803@molden.no> Dj Gilcrease skrev: > No the proper syntax is > > >>>> False ^ False >>>> I think you are missing the point: ^ is bitwise or. Don't think we can always go from bitwise to boolean operator by casting to bool. Xor is a special case. >>> bool(not True) False >>> bool(~True) True S.M. From g.brandl at gmx.net Wed Oct 28 00:14:01 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 28 Oct 2009 00:14:01 +0100 Subject: [Python-ideas] XOR In-Reply-To: References: <4AE7770D.5070303@molden.no> Message-ID: geremy condra schrieb: > On Tue, Oct 27, 2009 at 6:46 PM, Georg Brandl wrote: >> Sturla Molden schrieb: >>> Why does Python have a bitwise but not a logical xor operator? >> >> How often do you need the xor operator? >> > > 1) Technically, an operator is *never* needed, as its just syntactic sugar. Exactly. Therefore it makes sense to select the most often used operations and add syntactic sugar for them. Boolean XOR is not one of them. (The thread linked by Mark enumerates a fair number of equivalent spellings, but the most important thing is that it's impossible to make the "xor" behave equivalent to "and" and "or" in terms of short-circuiting and returning one of the operands.) > 2) It sure would make crypto code look prettier, as we rely on xor > operations extensively. We *do* have a bitwise xor. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From robert.kern at gmail.com Wed Oct 28 00:18:08 2009 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 27 Oct 2009 18:18:08 -0500 Subject: [Python-ideas] XOR In-Reply-To: References: <4AE7770D.5070303@molden.no> Message-ID: On 2009-10-27 18:07 PM, geremy condra wrote: > On Tue, Oct 27, 2009 at 6:46 PM, Georg Brandl wrote: >> Sturla Molden schrieb: >>> Why does Python have a bitwise but not a logical xor operator? >> >> How often do you need the xor operator? > > 1) Technically, an operator is *never* needed, as its just syntactic sugar. > 2) It sure would make crypto code look prettier, as we rely on xor > operations extensively. No, it wouldn't. Crypto uses the bitwise xor which we already have an operator for: ^. As I stated in the referenced thread, to me, the most compelling reason there is no "xor" keyword to go with "and" and "or" is that one cannot make an xor that shares the same short-circuiting behavior. Or the behavior of returning one of the operand objects rather than a coerced bool. Without either of those behaviors, there is little benefit to having a keyword operator where a trivial one-liner will suffice. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From alexander.belopolsky at gmail.com Wed Oct 28 00:18:44 2009 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 27 Oct 2009 19:18:44 -0400 Subject: [Python-ideas] XOR In-Reply-To: <4AE7770D.5070303@molden.no> References: <4AE7770D.5070303@molden.no> Message-ID: On Tue, Oct 27, 2009 at 6:41 PM, Sturla Molden wrote: > Why does Python have a bitwise but not a logical xor operator? .. because it does: != >>> True != True False >>> True != False True >>> False != False False In 2.x you can even use <> if you like syntactic sugar. :-) On arbitrary types a xor b is arguably bool(a) != bool(b) rather than simple a != b, but it is rare enough to warrant additional syntax. I thought I've seen this answered in an FAQ list somewhere. From debatem1 at gmail.com Wed Oct 28 00:26:50 2009 From: debatem1 at gmail.com (geremy condra) Date: Tue, 27 Oct 2009 19:26:50 -0400 Subject: [Python-ideas] XOR In-Reply-To: References: <4AE7770D.5070303@molden.no> Message-ID: On Tue, Oct 27, 2009 at 7:18 PM, Robert Kern wrote: > On 2009-10-27 18:07 PM, geremy condra wrote: >> >> On Tue, Oct 27, 2009 at 6:46 PM, Georg Brandl ?wrote: >>> >>> Sturla Molden schrieb: >>>> >>>> Why does Python have a bitwise but not a logical xor operator? >>> >>> How often do you need the xor operator? >> >> 1) Technically, an operator is *never* needed, as its just syntactic >> sugar. >> 2) It sure would make crypto code look prettier, as we rely on xor >> ? ? operations extensively. > > No, it wouldn't. Crypto uses the bitwise xor which we already have an > operator for: ^. > Actually, I use it primarily in the public-key context, where bitwise comparison doesn't make a whole lot of sense. > As I stated in the referenced thread, to me, the most compelling reason > there is no "xor" keyword to go with "and" and "or" is that one cannot make > an xor that shares the same short-circuiting behavior. Or the behavior of > returning one of the operand objects rather than a coerced bool. Without > either of those behaviors, there is little benefit to having a keyword > operator where a trivial one-liner will suffice. I've always tried to avoid and/or in Python for exactly that behavior. You're right that it would be confusing, though. Geremy Condra From guido at python.org Wed Oct 28 00:39:32 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 27 Oct 2009 16:39:32 -0700 Subject: [Python-ideas] XOR In-Reply-To: <4AE77857.50601@molden.no> References: <4AE7770D.5070303@molden.no> <4AE77857.50601@molden.no> Message-ID: On Tue, Oct 27, 2009 at 3:46 PM, Sturla Molden wrote: > Doesn't this look like a case for new syntax before the moratorium? No, the moratorium (assuming it's accepted) starts retroactively the day 3.1 was released. -- --Guido van Rossum (python.org/~guido) From rob.cliffe at btinternet.com Wed Oct 28 01:42:08 2009 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Wed, 28 Oct 2009 00:42:08 -0000 Subject: [Python-ideas] XOR References: <4AE7770D.5070303@molden.no> Message-ID: <3393C37506B641C1998F4C110CC1B754@robslaptop> I'm not in favour of of adding an xor operator, but it seems to me it IS possible to make it behave somewhat analagously to 'and' and 'or' as far as what it returns: X xor Y evaluates to: X if bool(X)==True and bool(Y)==False Y if bool(Y)==True and bool(X)==False False if bool(X)==Bool(Y)==True Y if bool(X)==bool(Y)==False The last case is analagous to X or Y evaluating to Y when bool(X)==bool(Y)==False, e.g. 0 or [] == [] [] or 0 == 0 Admittedly there is an aestheically unpleasing asymmetry here. But it means you could write code such as Z = X xor Y if Z: # Z is known to be either X or Y Of course, there is no evaluation short-circuiting. Rob Cliffe ----- Original Message ----- From: "Robert Kern" To: Sent: Tuesday, October 27, 2009 11:18 PM Subject: Re: [Python-ideas] XOR > On 2009-10-27 18:07 PM, geremy condra wrote: >> On Tue, Oct 27, 2009 at 6:46 PM, Georg Brandl wrote: >>> Sturla Molden schrieb: >>>> Why does Python have a bitwise but not a logical xor operator? >>> >>> How often do you need the xor operator? >> >> 1) Technically, an operator is *never* needed, as its just syntactic >> sugar. >> 2) It sure would make crypto code look prettier, as we rely on xor >> operations extensively. > > No, it wouldn't. Crypto uses the bitwise xor which we already have an > operator for: ^. > > As I stated in the referenced thread, to me, the most compelling reason > there is no "xor" keyword to go with "and" and "or" is that one cannot > make an xor that shares the same short-circuiting behavior. Or the > behavior of returning one of the operand objects rather than a coerced > bool. Without either of those behaviors, there is little benefit to having > a keyword operator where a trivial one-liner will suffice. > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though it > had > an underlying truth." > -- Umberto Eco > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas From digitalxero at gmail.com Wed Oct 28 02:06:38 2009 From: digitalxero at gmail.com (Dj Gilcrease) Date: Tue, 27 Oct 2009 19:06:38 -0600 Subject: [Python-ideas] XOR In-Reply-To: <4AE77EA9.8090803@molden.no> References: <4AE7770D.5070303@molden.no> <4AE77857.50601@molden.no> <4AE77EA9.8090803@molden.no> Message-ID: On Tue, Oct 27, 2009 at 5:13 PM, Sturla Molden wrote: > I think you are missing the point: ^ is bitwise or. Don't think we can > always go from bitwise to boolean operator by casting to bool. Xor is a > special case. No you cannot always go from bitwise to logical, but in the xor case it works, which is the case you were wanting to add syntax for. Just because it is also bitwise syntax does not mean you cannot use it in logical operations where it is intended to work. The only benefit I see to adding xor is not having to cast items to a bool before doing the test, which could be solved with no syntax changes by adding the __xor__ method to the types you want to check, though I think adding this to builtins would fall under the moratorium. could define __xor__ for all objects, except where the bitwise xor is intended as def __xor__(self, other): return bool(self) ^ bool(other) From alexander.belopolsky at gmail.com Wed Oct 28 02:09:40 2009 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 27 Oct 2009 21:09:40 -0400 Subject: [Python-ideas] One obvious way to do interning [Was: Retrieve an arbitrary element from a set without removing it] In-Reply-To: <4AE5DC62.6000008@arbash-meinel.com> References: <4AE5DC62.6000008@arbash-meinel.com> Message-ID: On Mon, Oct 26, 2009 at 1:29 PM, John Arbash Meinel wrote: .. > Anyway, I've had pretty good results, and some significant memory > savings at times. (The string interned() dict is often the largest > single object. Once you get 500k strings in there, it is something like > 12MB all by itself. I've certainly seen it in the 24MB+ range.) Please see http://bugs.python.org/issue7224 . It would be interesting to see if it make a difference to any of those 24MB+ interned string dictionary applications. From ubershmekel at gmail.com Thu Oct 29 08:13:19 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Thu, 29 Oct 2009 09:13:19 +0200 Subject: [Python-ideas] XOR In-Reply-To: References: <4AE7770D.5070303@molden.no> <4AE77857.50601@molden.no> <4AE77EA9.8090803@molden.no> Message-ID: <9d153b7c0910290013j5a65df0cveed5a1d553a898a9@mail.gmail.com> Btw, >>> from operator import xor >>> help(xor) Help on built-in function xor in module operator: xor(...) xor(a, b) -- Same as a ^ b. >>> And just nosing around for code that uses the word xor you guys are talking about reserving: as a function (these are not examples that just do ^): http://www.google.com/codesearch/p?hl=en&sa=N&cd=4&ct=rc#y7s_SD3fSG4/Python-1.2/Demo/sgi/gl/glstdwin/glstdwin.py&q=xor%20file:%5C.py$&l=376 http://www.google.com/codesearch/p?hl=en&sa=N&cd=10&ct=rc#xFWT5g5T9_Y/Singular-3-0-2/Singular/cnf2ideal.py&q=xor%20file:%5C.py$&l=68 as a variable name: http://www.google.com/codesearch/p?hl=en&sa=N&cd=5&ct=rc#OmjbpIVasQU/Zope-3.2.0b1/Dependencies/twisted-Zope-3.2.0b1/twisted/words/protocols/toc.py&q=xor%20file:%5C.py$&l=71 http://www.google.com/codesearch/p?hl=en&sa=N&cd=20&ct=rc#ptHG4K-HqzI/lib/python2.4/site-packages/mixminion/test.py&q=xor%20file:%5C.py$&l=861 --yuv -------------- next part -------------- An HTML attachment was scrubbed... URL: